Jest test sometimes fails (DynamoDB integration tests) - docker

Question

I have Serverless application (node:14.19.1-bullseye-slim) with almost 400 tests. There are mostly functional tests with using of local DynamoDb. The problem is Bitbucket pipeline sometimes fail with message:
thrown: "Exceeded timeout of 10000 ms for a test.
Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."
The bad thing of this the issue that is not reproducible on local machine. It's green in 9/10 runs. Also on Bitbucket pipeline not every run fails and not in the same test suite.
Here is my configuration:
package.json
"devDependencies": {
"#aws-sdk/client-lambda": "3.58.0",
"#aws-sdk/client-s3": "3.58.0",
"#aws-sdk/client-ssm": "3.58.0",
"#aws-sdk/node-http-handler": "3.58.0",
"#fast-csv/format": "4.3.5",
"aws-sdk": "2.1001.0",
"axios-curlirize": "1.3.7",
"axios-mock-adapter": "1.20.0",
"chalk": "4.1.2",
"eslint": "8.12.0",
"eslint-config-airbnb-base": "15.0.0",
"eslint-plugin-import": "2.25.4",
"eslint-plugin-jest": "26.1.3",
"ion-js": "4.2.2",
"jest": "27.5.1",
"jest-junit": "13.0.0",
"js-yaml": "4.1.0",
"jsbi": "4.2.0",
"prettier": "2.6.1"
},
"scripts": {
"ci": "npx jest --coverage --colors --ci"
}
jest.config.js
module.exports = {
collectCoverageFrom: [
'src/**',
],
coverageReporters: [
'text',
'html',
],
maxWorkers: 1,
testEnvironment: 'node',
testTimeout: 10000,
verbose: true,
};
docker-compose.yml
dynamo:
image: amazon/dynamodb-local:1.18.0
command: '-jar DynamoDBLocal.jar -inMemory -sharedDb'
ports:
- "8000:8000"
More logs didn't help
I tried to get more information about failing test. I prepared "custom" DynamoDB docker image and turned-on AWS-SDK logs but it didn't help me a lot. I also tried latest LTS version of Node and AWS-SDK. I also found Jest issue and tried "guaranteed workarounds" but without chance.
Questions
Does someone resolved similar problem?
What I can do more for finding the problem?
The last thing what I have is rewrite tests to not directly using of dockerized DynamoDB but this will be the last try.

Answer 1

I suppose the problem is not on any DynamoDB side.
I have made some research on the problem and found the next thread with a discussion of the error like you. And the error has been appeared after upgrading jest to v.27
https://github.com/facebook/jest/issues/11607
They have assumptions about some bugs in the jest timer. So the solutions can be is:
to increase the timer value above 10 000.
or use jest.useFakeTimers('legacy') setting if possible.
if applicable switch the jest version back to v26.

Answer 2

Assuming Jest is configured correctly there could be a few reasons why this is happening:
Stateful tests whereby the order of the tests matters, and tests are not being executed in the same order each time, or between runs tables have changed
Throughput Capacity problems, as many tests are executed in a very quick manor this may mean that your table does not have enough WCU RCU to handle the requests
Tables not being cleaned correctly before each run
I have had a lot of problems when running tests with DynamoDB because I'd typically be using a test Table which is not provisioned with a high capacity because it is not needed normally. But when you run many tests really quickly even if it is only on your machine this can happen.