Jest does not report tests as failed if beforeAll hook is timed out

[paluchs-iohk]

I have a set of suites A,B,C each one of them consist of corresponding tests [a1,a2], [b1,b2], [c1,c2]. If the suite fails to run because of the timeout during beforeAll the report will not include any trace of failed tests. Everything looks like all tests Passed. I found it only by knowing the number of tests that should be executed.

[Ma11hewThomas]

Thanks for reporting this, I'm unable to recreate the issue. I've tried with two basic examples below but the CTRF report is as expected.

Can you share a simple example like those below or more information about the configuration you are using like the jest version and jest.config.js

Thanks

I created the following test cases to investigate:

Example 1:

describe('Example 1 Suite A', () => {
    beforeAll(() => {
        return new Promise((resolve) => {
            setTimeout(resolve, 20000); // Simulates a long-running setup
        });
    }, 15000); // 15-second timeout
    test('Example 1 a1', () => {
        expect(true).toBe(true);
    });
    test('Example 1 a2', () => {
        expect(true).toBe(true);
    });
});
describe('Example 1 Suite B', () => {
    beforeAll(() => {
        return new Promise((_, reject) => {
            setTimeout(() => reject(new Error('Timeout in shared beforeAll')), 5000); // Timeout after 5 seconds
        });
    });
    test('Example 1 b1', () => {
        expect(true).toBe(true);
    });
    test('Example 1 b2', () => {
        expect(true).toBe(true);
    });
});
describe('Example 1 Suite C', () => {
    test('Example 1 c1', () => {
        expect(true).toBe(true);
    });
    test('Example 1 c2', () => {
        expect(true).toBe(true);
    });
});

Example 2:

beforeAll(() => {
    return new Promise((_, reject) => {
        setTimeout(() => reject(new Error('Timeout in shared beforeAll')), 5000); // Timeout after 5 seconds
    });
});
describe('Example 2 Suite A', () => {
    test('Example 2 a1', () => {
        expect(true).toBe(true);
    });
    test('Example 2 a2', () => {
        expect(true).toBe(true);
    });
});
describe('Example 2 Suite B', () => {
    test('Example 2 b1', () => {
        expect(true).toBe(true);
    });
    test('Example 2 b2', () => {
        expect(true).toBe(true);
    });
});
describe('Example 2 Suite C', () => {
    test('Example 2 c1', () => {
        expect(true).toBe(true);
    });
    test('Example 2 c2', () => {
        expect(true).toBe(true);
    });
});

CTRF report:

{
  "results": {
    "tool": {
      "name": "jest"
    },
    "summary": {
      "tests": 12,
      "passed": 2,
      "failed": 10,
      "pending": 0,
      "skipped": 0,
      "other": 0,
      "start": 1731692596610,
      "stop": 1731692607006
    },
    "tests": [
      {
        "name": "Example 2 Suite A Example 2 a1",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 2 Suite A Example 2 a2",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 2 Suite B Example 2 b1",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 2 Suite B Example 2 b2",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 2 Suite C Example 2 c1",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 2 Suite C Example 2 c2",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 1 Suite A Example 1 a1",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 1 Suite A Example 1 a2",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 1 Suite B Example 1 b1",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 1 Suite B Example 1 b2",
        "duration": 0,
        "status": "failed"
      },
      {
        "name": "Example 1 Suite C Example 1 c1",
        "duration": 0,
        "status": "passed"
      },
      {
        "name": "Example 1 Suite C Example 1 c2",
        "duration": 0,
        "status": "passed"
      }
    ]
  }
}