Hello there I have a script that generate visits from google to websites.
I run it on my VPS but it only stays 15 minutes before it closes. I noticed that it takes too much ram (aroun 90%) then shut down after.
My question is , how can I reduce the phantomjs consumption.
PS: I can share codes just let me know what you need from me
thanks in advance.
Just do Drive.close() and Drive.quit() after loops and reopen driver.
Also consider gc.collect()
Related
I have used CEF4Delphi to embed CEF / Chromium in my Delphi XE 10.1 application. I am currently at CEF Version 3.2987.1597.gffc5773 (Chromium Version 57.0.2987.110), 32 bit.
I can see the render process in the Windows Task Manager normally staying at 0% CPU usage when customers are not browsing, but then sometimes the render process seems to get stuck and grabs all the CPU it can get (100% of a single CPU, 16% of total since I have 6 available), even when the user is doing nothing and the browser window is not visible. When stuck it seems to remain stuck until I end the process, several hours has been observed. I have turned on verbose logging but nothing gets logged as long as the user is not doing anything. I do get lots of logging at startup and when the user is actively browsing.
Any tips on how to resolve this? I only see this issue on my Terminal Server and have not been able to reproduce the behaviour myself either on dev machines or the TS. Last time I saw this behaviour was about two hours after start up of the application.
Any help appreciated.
Thanks,
Sjur
Sorry for the late answer.
Close your app and clear the cache and cookies directories.
We are running a big server (12 threads, 6 cores, 64Gb ram, 2 SDDs raid-0) for our rails app deployed with nginx/passenger.
Unfortunately, pages are taken forever to load something like between 10 and 40 seconds. However, the server is under a really light load, with a load average of 0.61 0.56 0.53. We have ram used weirdly, free -ml reporting 57Gb (of 64Gb) usage whereas htop reporting only 4Gb (of 64Gb).
We have checked our production log, and rails request takes something like 100/200ms to be completed, so almost nothing.
How can we identify the bottleneck?
This question is fairly vague, but I'll see if I can give you some pointers.
My first guess is that your app is spending a lot of time doing database related stuff, see below for my advice.
As for the odd memory usage, are you looking at the correct part of the free -ml output? Just to clarify, you want to be looking at the -/+ buffers/cache: line to get the accurate output.
You might also check to see if any of your passenger workers are hanging, as that is a fairly common issue with passenger. You can do this by running strace -p $pid on your passenger workers. If its hanging, it will have an obvious lack of "doing anything"
As for troubleshooting response time within rails itself, I would highly suggest looking into using newrelic(http://newrelic.com/). You can often times see exactly which part of your app is causing the bad response time that way by breaking down how much time is spent in each part of your app. It's a simple gem to install and once you get reporting working, its pretty invaluable for issues like this.
Finally, the bottleneck was passenger, passenger-status is pretty useful showing up the queue left.
Our server is pretty precent so we just increase the number of passenger processes in nginx.conf to 600, resulting:
passenger_root /usr/local/rvm/gems/ruby-2.0.0-p195/gems/passenger-4.0.5;
passenger_ruby /usr/local/rvm/wrappers/ruby-2.0.0-p195/ruby;
passenger_max_pool_size 600;
There is a delay between bus.send and the handler picking it up.
2 second delay with default config.
3 second delay with structuremap used for the builder.
Is this normal?
Can I expect the speeds to be significantly faster in a production environment on servers?
Method:
The solution I'm using is built with the code first example that pops up when you pull down NSB from nuget.
I'm testing by adding a UTCNow to the message at the Bus.Send and then appending another UTCNow at the handler.
I'm running the tests on a Windows 8 i7-3517U # 1.90Ghz 2.40 Ghz with 8GB of Ram, not sure about the disk speed. Dell Laptop.
At the time of the tests the CPU is running 5-10% with 4GB of free ram.
Andreas Öhlund responded in a comment to the OP: "is that for the first message? what about following messages, they should be quick?".
Turns out every message after that was near instant, so it looks like it just takes awhile for the process to start. I encountered a similar issue with Mass Transit. However, if you try to send a message right after starting with MT, then nothing looks like it happens at all. Apparently MSMQ needs a warmup period.
So I have tried this out on multiple computers with multiple setups (servers/apps) and I seem to consistently get Rails completing 8-15 requests per second even for doing selects on empty tables with 1 field. I think I'm doing something wrong here because I've read a lot of stats online where people are getting 60-200 with mongrel. So being down at 8 seems just awful. The first app I tested this on was a little more involved and had 2 queries in 1 controller but they were just selecting a few rows, not a big deal.
Is there some trick to this I don't realize? Ruby.exe is taking up nearly 50% of my cpu cycles but still this is pretty bad. I feel like I've tried this when messing with rails last year and have gotten something like 50 requests per second. Is it possible that routing is screwed up some how?
Any advice would be greatly appreciated. Even info as far as profiling tools go so I could at least figure out WHERE the problem is occurring.
Thanks ahead of time.
If you're on windows then that seems about right. Rails runs terribly slow on windows. Try running it on a linux box, or a mac if you have one. You could also try heroku. They have a free starter plan you can use for development.
If you must run in a windows environment you could try jruby for some extra speed.
I'm running a Rails app through Phusion Passenger (mod_rails) which will run smoothly for a while, then suddenly slow to a crawl (one or two requests per hour) and become unresponsive. CPU usage is low throughout the whole ordeal, although I'm not sure about memory.
Does anyone know where I should start to diagnose/fix the problem?
Update: restarting the app every now and then does fix the problem, although I'm looking for a more long-term solution. Memory usage gradually increases (initially ~30mb per instance, becomes 40mb after an hour, gets to 60 or 70mb by the time it crashes).
New Relic can show you combined memory usage. Engine Yard recommends tools like Rack::Bug, MemoryLogic or Oink. Here's a nice article on something similar that you might find useful.
If restarting the app cures the problem, looking at its resource usage would be a good place to start.
Sounds like you have a memory leak of some sort. If you'd like to bandaid the issue you can try setting the PassengerMaxRequests to something a bit lower until you figure out what's going on.
http://www.modrails.com/documentation/Users%20guide%20Apache.html#PassengerMaxRequests
This will restart your instances, individually, after they've served a set number of requests. You may have to fiddle with it to find the sweet spot where they are restarting automatically before they lock up.
Other tips are:
-Go through your plugins/gems and make sure they are up to date
-Check for heavy actions and requests where there is a lot of memory consumption (NewRelic is great for this)
-You may want to consider switching to REE as it has better garbage collection
Finally, you may want to set a cron job that looks at your currently running passenger instances and kills them if they are over a certain memory threshold. Passenger will handle restarting them.