Some days ago I upgraded my codebase from jest 26 to 27.
Running tests in my local environment worked like a charm but, when I tried to run them on my CI machine, the tests "never" stop.
Actually, in CI, the process exited correctly when trying to run tests sequentially but not in parallel, eg with --runInBand --detectOpenHandles --forceExit.
I tried to build and run the same docker locally... and it worked.
Changing randomly some docker's CPU / RAM configs, I achieved the same result: the process hangs.
Tests hanging:
Running top in docker:
As you can see, it's not a problem of CPU / RAM considering what top says.
Do you have any hints?
Do you need more information?
After some days debugging, I figured out that the problem was the amount of RAM and some memory leaks in the tests.
I advise you to use the node --expose-gc ./node_modules/.bin/jest --logHeapUsage command to dive into the problems you can have. https://jestjs.io/docs/cli#--logheapusage
This post helped me out as well: https://chanind.github.io/javascript/2019/10/12/jest-tests-memory-leak.html
TL;DR:
I fixed memory leaks by downgrading node from 16.13.2 to 16.10.0.
Details:
Based on Mattia Larentis answer:
I had the same problem, code and tests were written for client app, used packages:
node: 16.13.2
jest: 27.4.7
ts-jest: 27.1.3
typescript: 4.5.5
After debugging and analysing jest tests, I detected memory leaks too, using:
node --expose-gc ./node_modules/.bin/jest --logHeapUsage
Then I analysed node heap snapshots (for more info see Your Jest Tests are Leaking Memory).
I found a lot of strings, containing transpiled TS code.
I went to ts-jest & jest repos to look for issues about memory leaks.
I found these:
ts-jest: Module caching memory leak
jest: [Bug]: Memory consumption issues on Node JS 16.11.0+
node: vm Script memory leak in Node.js 14 / 16
Then, I downgraded node version from 16.13.2 to 16.10.0, checked memory leaks using
node --expose-gc ./node_modules/.bin/jest --logHeapUsage
And found that problem was gone.
Related
I am trying to build ORO Platform js assets, using a non-docker environment, it works like a charm, but in Docker (either during Docker Build, or container execution) the building process stop and hangs with 100% CPU.
67% [0] building 1416/1470 modules 54 active ... ndles/orotask/sidebar_widgets/assigned_tasks/css/styles.scss
The building process does not necessarily hang on the exact same file. And also, the build seems to succeed on some occasion.
I've try to reduce to a minimum the process by removing Happy, tested with --max-old-space-size=4096, but no luck.
Sources : https://github.com/oroinc/platform/tree/master/build
How would you recommend debugging this ?
Thanks
There is a known issue when a NodeJs process hangs while you run it from the root user. As I know, there is no workaround for now. Consider using another user to build the assets.
If it's not the case, please review the Troubleshooting section in OroAssetBundle, that might help.
I am using the docker version of Gambit (The Global and Modular Beyond-the-Standard-Model Inference Tool) basically a software on docker platform. And I am working with OS X Sierra, and when I compile it, and I run the following commands:
($ mkdir build
cd build
cmake ..
make)
then I get error
c++: internal compiler error: Killed (program cc1plus)
(I have attached the whole thing, and there is no such file as //usr/share/doc/gcc-6/README.Bugs)
I am wondering if anyone can help me with this issue.
The MobyVM in which docker run has 2GB memory limit by default. Sometime these compilations require decent amount of memory to be available
You should try and increase the memory to 4GB or even higher and then try your docker run statement again
I'm building an elixir/phoenix application using a docker container.
This has been working for some time now, but recently it stopped working, with the error always being associated with a lack of memory.
For instance, the most frequent point of failure is during the mix compile task of Elixir (the most time heavy task in the Dockerfile), which crashes with the error:
eheap_alloc: Cannot allocate 147852528 bytes of memory (of type "old_heap").
Crash dump is being written to: erl_crash.dump...done
Sometimes it might be able to get through that step, but will again fail at a later step, like brunch build which compiles the frontend code. Sometimes it just fails at some other step with no specific error message, just saying:
Killed
While this is happening, I can easily check htop and see that I'm using 3 or 4GB of RAM, out of 16GB total, so there's no lack of physical RAM at all.
After some digging, I found that sudo sysctl vm.overcommit_memory=1 could help, but no luck there either.
The exact same build runs fine on my other computer, which runs Arch Linux, while this one runs Ubuntu 16.04
I'm baffled. I have a VM running Ubuntu 14.04. I've followed procedures here: http://clang.llvm.org/docs/LibASTMatchersTutorial.html and am at the step to run ninja. This builds llvm and clang. Now, my VM is no slouch, I gave it 6GB of RAM and 4 CPUs and a 20GB swap file. The biggest problem comes at link time - it seems to start a large number of ld processes, each using at least 3-4GB or virtual memory, and at some point a lot of CPU each. But the the swap file grew to over 12GB and the processes are all IO bound, but I don't know if they are doing something useful, or thrashing. All I know is the disk is getting hammered and the jobs run forever. I've actually just dropped the CPU count to the VM to 1, to see if it might be more efficient with less parallelism, as I surmised the issue may be thrashing.
I suppose my disk could be slow... Any ideas? Should I be using make instead of ninja? My expertise is not Linux (although I'm getting there :-) ) So I'm following the tutorial but perhaps it is not recommended the "best" way to build the clang / llvm programs.
I have been there, It's happening with the latest svn release (but not if you get clang 3.8 or older releases). What is happening is that since during development a lot of debug information is also being generated for each compilation unit the file sizes are becoming big.
The solution is to turn off all the debug info that's been attached by default. You probably are not going to debug clang, so won't need it. SO instead of just doing this
cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON
What you should do is
cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release
All the other steps remain the same. Now I have not tested this with ninja, but have verified it with make on ubuntu (this tutorial, I modified the same thing in step 7). This should owkr as weel.
I'm just starting to learn Grails and Groovy. Did a fresh install of Grails 1.3.8 today and it crawls - I must have something in my environment causing a problem. I ran create-app (following a tutorial), then run-app. Both took quite a while but hey, first time thru. Ran run-app again, took almost 5 minutes, apparently int eh compiling phase. Windows 7 machine with 4 GB memory, Intel i5 CPU. Don't know where to start looking, but know this isn't normal.
When I've run into this normally I'm out of heap or permGen space. Fire up visualvm and check both of those memory limits. It'll slowdown considerably before crashing if your memory limits are too small
I wonder how it happens for you. I have 2gb ram and I run Ubuntu on pc and it's so fast. First check your memory limits.
And for your question, you can find a good disscussion over here.