I have a Rails app that occasionally has Passenger instances using massive amounts of memory. For example, most instances of the app consume 150-250MB of RAM, but there is currently one that is using 2GB - huge difference. What is the recommended method to track down the cause of this? Thanks.
Most of the time it's not memory leaking, it's bloat: https://blog.engineyard.com/2009/thats-not-a-memory-leak-its-bloat
Related
I'm using Puma server and DelayedJob.
It seems that the memory taken by each job isn't released and I slowly get a bloat causing me to restart my dyno (Heroku).
Any reason why the dyno won't return to the same memory usage figure before the job was performed?
Any way to force releasing it? I tried calling GC but it doesn't seem to help.
You can have one of the following problems. Or actually all of them:
Number 1. This is not an actual problem, but a misconception about how Ruby releases memory to operating system. Short answer: it doesn't. Long answer: Ruby manages an internal list of free objects. Whenever your program needs to allocate new objects, it will get those objects from this free list. If there are no more objects there, Ruby will allocate new memory from operating system. When objects are garbage collected they go back to the free list. So Ruby still have the allocated memory. To illustrate it better, imagine that your program is normally using 100 MB. When at some point program will allocate 1 GB, it will hold this memory until you restart it.
There are some good resource to learn more about it here and here.
What you should do is to increase your dyno size and monitor your memory usage over time. It should stabilize at some level. This will show you your normal memory usage.
Number 2. You can have an actual memory leak. It can be in your code or in some gem. Check out this repository, it contains information about well known memory leaks and other memory issues in popular gems. delayed_job is actually listed there.
Number 3. You may have unoptimized code that is using more memory than needed and you should try to investigate memory usage and try to decrease it. If you are processing large files, maybe you should do it in smaller batches etc.
I have a Rails application which uses ~550 mb after 3-4 hours of use and stays there for the rest of the day, which leads me to thought that it's it normal way how Rails use memory.
Here is screenshot of my memory usage from Heroku and Gemfile.lock
My goal - reduce app memory usage. For this I need to understand what parts add to this memory usage. I looked through various articles in Interenet on Ruby\Rails memory usage topic.
I used derailed_benchmarks 'mem' command to track down what is large at boot. Basically, I have two questions:
Why my memory usage goes upto some N mb value and not more or less? How I can calculate that value?
Assume we have some gem which consumes 20mb. How much times does it added to overall usage?
If there any information that can help my issue, I would be very thankful to you.
I'm using have a server which running on 4gb ram...
Each uploaded picture will be watermark, so i decided to put in background process. However, when there are a lot of requests of uploading pictures..the server will facing high memory issue, and the memory don't deallocate themselves.
My Questions:
- why sidekiq worker terminate?
- is rmagick memory leak?
- how to handle this situation?
You've provided nowhere near the amount of details needed for us to give you informed advice, but I'll try something general: How many Sidekiq workers are you running? Consider reducing the #, then queue up tons of request to simulate a heavy load; keep doing that until you have few enough workers that Sidekiq can comfortably handle the worst load. (Or until you confirm that the issue appears the same even when there's only 1 Sidekiq worker!)
Once you've done that, then you'll have a better feel for the contours of the problem: something that looks like an Rmagick memory leak issue when your server is overloaded, may look different (or give you more ideas on how to address it) when you reduce the workload.
Also take a look at this similar SO question about Rmagick and memory leaks; it might be worth forcing garbage collection to limit how much damage any given leak can cause.
I've got a Rails 4.2.1 app that has a memory leak. I'm hosting on Heroku and when in production my memory continues to grow until the server starts paging. I'm trying to source out the leak - and wanted to know if there is a way I can debug the memory allocations still active after a request / response. If I can get that, I can curl my pages a few times to warm any globals then siege to see what memory is leaking. Any way to do this?
The rack-mini-profiler gem allows you to get a count of objects in memory by class (and allocated by the current request). It also dumps some of the most frequent objects such as strings - I've found it very helpful for diagnosing memory leaks.
I have an Jruby On Rails app which uses several WS's to gather data. The app processes the data displays it to the user, the user make his changes and then is sent back to the WS.
The problem here is that i store everything in the cache (session based) which is using memory store. But from time to time with no clear reason (for me at least) this error pops up:
ActionView::Template::Error (GC overhead limit exceeded)
I read what I could find about it and apparently this means that the Garbage Collector spends to much time in trying to free memory and no real progress is being made in that direction. My guess is that since everything is stored cache like into memory the GC tries to free it and can't do it and throws this error.
So here are the questions.
How can I get around this ?
If I switch from memory store to Redis, if my assumptions are correct, will this problem still appear.
Will the GC try to free Redis's memory area ? (Might be a stupid question but ... please help nonetheless :) )
Thank you.
Redis is a separate process, so your app garbage collector won't touch it. However, it is possible that redis is using up all the memory that would otherwise be available for the app, or that you are actually using a process memory cache and not redis, which is a different type of memory store.