I'm suddenly experiencing that rake and rails sort of bail out with some strange output. It most of all looks like a process list but it's clearly Ruby/Rails related. Also, it's several pages long - I actually had to increase the scrollback setting in my terminal to get to see what was going on before this output started.
Here's a short excerpt:
0.00 48.12 0.00 1 0.00 0.00 Rails::Rack::LogTailer#tail!
0.00 48.12 0.00 3 0.00 0.00 WEBrick::HTTPResponse#[]
0.00 48.12 0.00 1 0.00 0.00 Rack::Utils::HeaderHash#each
0.00 48.12 0.00 2 0.00 0.00 Range#begin
0.00 48.12 0.00 1 0.00 0.00 Range#end
0.00 48.12 0.00 1 0.00 10.00 Rack::File#each
0.00 48.12 0.00 1 0.00 0.00 WEBrick::HTTPRequest#fixup
0.00 48.12 0.00 1 0.00 0.00 Kernel.raise
0.00 48.12 0.00 1 0.00 0.00 Exception#to_s
0.00 48.12 0.00 1 0.00 0.00 WEBrick::GenericServer#stop
0.00 48.12 0.00 1 0.00 0.00 WEBrick::BasicLog#debug?
This particular output came after I killed WebRick (ctrl+c). I also experience this when running tests (it seems to show up after each test/file) and when running rake db:migrate (shows up when migration is done).
I'm currently running Rails 3.1.0 (upgraded from 3.0.5 hoping that would solve this) and ruby 1.9.2p180 installed through RVM.
Any ideas why this is happening?
You had a model named Profile, when you removed that Rails attempted to load it, though as that file no longer existed it would have looked elsewhere in the load path and got the profiler out of the standard libraries. Thus what you are seeing is your application being profiled (accidentally).
Related
I want to get the CPU and GPU utilisation of my cuda program and plot them like this.
What's the best way?
Here is my script:
### [1] Running my cuda program in background
./my_cuda_program &
PID_MY_CUDA_PROGRAM=$!
### [2] Getting CPU & GPU utilization in background
sar 1 | sed --unbuffered -e 's/^/SYSSTAT:/' &
PID_SYSSTAT=$!
nvidia-smi --format=csv --query-gpu=timestamp,utilization.gpu -l 1 \
| sed --unbuffered -e 's/^/NVIDIA_SMI:/' &
PID_NVIDIA_SMI=$!
### [3] waiting for the [1] process to finish,
### and then kill [2] processes
wait ${PID_MY_CUDA_PROGRAM}
kill ${PID_SYSSTAT}
kill ${PID_NVIDIA_SMI}
exit
That output:
SYSSTAT:Linux 4.15.0-176-generic (ubuntu00) 05/06/22 _x86_64_ (4 CPU)
NVIDIA_SMI:timestamp, utilization.gpu [%]
NVIDIA_SMI:2022/05/06 23:57:00.245, 7 %
SYSSTAT:
SYSSTAT:23:57:00 CPU %user %nice %system %iowait %steal %idle
SYSSTAT:23:57:01 all 8.73 0.00 5.74 7.48 0.00 78.05
NVIDIA_SMI:2022/05/06 23:57:01.246, 1 %
SYSSTAT:23:57:02 all 23.31 0.00 6.02 0.00 0.00 70.68
NVIDIA_SMI:2022/05/06 23:57:02.246, 16 %
SYSSTAT:23:57:03 all 25.56 0.00 3.76 0.00 0.00 70.68
NVIDIA_SMI:2022/05/06 23:57:03.246, 15 %
SYSSTAT:23:57:04 all 22.69 0.00 6.48 0.00 0.00 70.82
NVIDIA_SMI:2022/05/06 23:57:04.246, 21 %
SYSSTAT:23:57:05 all 25.81 0.00 3.26 0.00 0.00 70.93
it's a bit annoying to parse the log above.
When running training using Tensorflow, the root filesytem, which is on two SSDs, is seeing extremely high disk utilization, with a wait as high as 39min. Trying to figure out what is causing it.
11:42:43 PM DEV tps rd_sec/s wr_sec/s avgrq-sz avgqu-sz await svctm %util
11:42:45 PM sda 2.00 0.00 4096.00 2048.00 143.11 57598.00 500.00 100.00
11:42:45 PM md0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
11:42:45 PM sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
11:42:45 PM sdc 2.00 0.00 4096.00 2048.00 143.10 57594.00 500.00 100.00
All,
I'm learning to do benchmark testing using the Siege tool against our Rails app. I'm running Siege on my OS X box against a website hosted on another server. When I run it I always get zero hits and zero failed transactions no matter which site I run it against.
Because of the limitations of OS X, I've configured more ports with sudo sysctl -w net.inet.tcp.msl=1000 and more open files with launchctl limit maxfiles 10000 10000. I've also configured proxy variables in .siegerc since I'm running behind a proxy. Sample command line is:
siege -c1 -b -t10S 'www.google.com'.
But no matter which website I hit, I always get zero hits, zero failures and zero successful transactions. The Siege log file does not show any errors. What am I doing wrong?
siege -c1 -b -t10S 'www.google.com'
** SIEGE 3.0.5
** Preparing 1 concurrent users for battle.
The server is now under siege...
Lifting the server siege... done.
Transactions: 0 hits
Availability: 0.00 %
Elapsed time: 9.84 secs
Data transferred: 0.00 MB
Response time: 0.00 secs
Transaction rate: 0.00 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 0.00
Successful transactions: 0
Failed transactions: 0
Longest transaction: 0.00
Shortest transaction: 0.00
We've been trying to figure out how to reduce the boot-up memory footprint of our rails app by identifying memory-hungry gems and finding alternatives or solutions.
But there's one behavior on OS X that I find baffling.
With a brand new generated rails app (rails new memoryusage), with no Gemfile, no models, no data and no transactions, upon starting up rails c the memory OSX displays for the corresponding ruby process vill vary each time it is started up, from as low as 60MB to as high as 65MB, with no discernible pattern as to why the same app might be requiring less or more memory per execution.
I imagine this has to do in some way with how Ruby allocates memory, but I'm not completely clear on why its memory allocation would vary so wildly for the same code and no variable processing.
We have similarly unpredictable behavior when we try to calculate the memory consumed by the process after each gem in the Gemfile is required. We load up a vanilla rails process, then in rails c we run a script that parses the Gemfile and requires each Gem individually, logging the memory pre- and post-require, and what we notice is that not only does the memory footprint not have a consistent starting point, but also the incremental 'steps' in our memory consumption vary wildly.
We booted up our process three times, one after the other, and measured the start up memory and the incremental memory required by each gem. Not only did the startup book memory footprints bounce between 60MB and 92MB, but the points at which we saw memory jumps on loading each gem were inconsistent -- sometimes loading SASS would eat up an additional 5MB, sometimes it wouldn't, sometimes active_merchant would demand 10MB additional, others it wouldn't.
: BOOT UP #1 : BOOT UP #2 : BOOT UP #3
gem : increment | total : increment | total : increment | total
rails : 0.00 | 59.71 : 0.00 | 92.54 : 0.18 | 67.76
unicorn : 0.52 | 60.24 : 0.52 | 93.06 : 3.35 | 71.12
haml : 8.77 | 69.02 : 1.88 | 94.94 : 9.45 | 80.57
haml-rails : 0.00 | 69.02 : 0.00 | 94.94 : 0.00 | 80.57
sass : 4.36 | 73.38 : 6.95 | 101.89 : 0.99 | 81.55
mongoid : 0.00 | 73.38 : 0.00 | 101.89 : 0.00 | 81.55
compass : 11.56 | 84.93 : 3.23 | 105.12 : 8.41 | 89.96
compass-rails : 0.00 | 84.93 : 0.08 | 105.20 : 0.00 | 89.96
compass_twitter_bootstrap: 0.00 | 84.93 : 0.00 | 105.20 : 0.00 | 89.96
profanalyzer : 0.59 | 85.52 : 0.46 | 105.66 : 0.64 | 90.60
simple_form : 0.34 | 85.87 : 0.35 | 106.01 : 0.00 | 90.60
sorcery : 0.00 | 85.87 : 0.25 | 106.26 : 1.07 | 91.67
validates_timeliness: 1.47 | 87.34 : 1.82 | 108.07 : 1.62 | 93.29
mongoid_token : 0.00 | 87.34 : 0.00 | 108.07 : 0.00 | 93.29
nested_form : 0.00 | 87.34 : 0.00 | 108.07 : 0.01 | 93.30
nokogiri : 0.86 | 88.20 : 1.16 | 109.24 : 1.37 | 94.67
carmen : 0.00 | 88.20 : 0.07 | 109.30 : 0.00 | 94.67
carrierwave/mongoid : 2.78 | 90.98 : 0.38 | 109.69 : 0.13 | 94.80
yajl : 0.04 | 91.02 : 0.04 | 109.73 : 0.04 | 94.84
multi_json : 0.00 | 91.02 : 0.00 | 109.73 : 0.00 | 94.84
uuid : 0.00 | 91.03 : 0.00 | 109.73 : 0.41 | 95.25
tilt : 0.00 | 91.03 : 0.00 | 109.73 : 0.00 | 95.25
dynamic_form : 0.00 | 91.04 : 0.00 | 109.73 : 0.00 | 95.25
forem : 0.03 | 91.07 : 0.00 | 109.73 : 0.00 | 95.25
browser : 0.00 | 91.07 : 0.00 | 109.73 : 0.00 | 95.25
activemerchant : 2.17 | 93.24 : 1.18 | 110.92 : 10.58 | 105.83
kaminari : 0.00 | 93.24 : 0.00 | 110.92 : 0.00 | 105.83
merit : 0.00 | 93.24 : 0.00 | 110.92 : 0.00 | 105.83
memcachier : 0.00 | 93.24 : 0.00 | 110.92 : 0.00 | 105.83
dalli : 0.01 | 93.25 : 0.05 | 110.96 : 0.34 | 106.17
bitly : 2.47 | 95.72 : 9.43 | 120.40 : 1.53 | 107.70
em-synchrony : 1.00 | 96.72 : 0.18 | 120.57 : 0.55 | 108.24
em-http-request : 5.56 | 102.28 : 2.15 | 122.72 : 1.40 | 109.64
httparty : 0.00 | 102.28 : 0.00 | 122.72 : 0.00 | 109.64
rack-block : 0.00 | 102.28 : 0.00 | 122.72 : 0.00 | 109.64
resque/server : 1.21 | 103.49 : 1.73 | 124.45 : 1.68 | 111.32
resque_mailer : 0.00 | 103.49 : 0.00 | 124.45 : 0.00 | 111.32
rack-timeout : 0.00 | 103.49 : 0.00 | 124.45 : 0.00 | 111.32
chronic : 1.66 | 105.15 : 0.67 | 125.12 : 0.64 | 111.96
oink : 0.00 | 105.15 : 0.00 | 125.12 : 0.00 | 111.96
dotenv-rails : 0.00 | 105.15 : 0.00 | 125.12 : 0.00 | 111.96
jquery-rails : 0.00 | 105.15 : 0.03 | 125.15 : 0.00 | 111.96
jquery-ui-rails : 0.00 | 105.15 : 0.00 | 125.15 : 0.00 | 111.96
It's clear to me that there's something very basic that I'm missing and don't understand about how memory is allocated to Ruby processes, but I'm having a hard time figuring out why it could be this seemingly stochastic. Anyone have any thoughts?
I'm going to take a wild guess and say this is caused by address space layout randomization and by interactions with shared libraries whose footprints are influenced by running programs that are not in your test case.
OS X has received increasing support for ASLR starting with 10.5 and as of 10.8 even the kernel is randomly relocated.
In some cases, ASLR can cause a program segment to use extra pages depending on whether the offset causes a page boundary to be crossed. Since there are many segments and many libraries, this effect is difficult to predict.
I'm also wondering if (given the huge differences you are seeing) perhaps this is a reporting issue in OS X. I wonder if the overhead for shared objects is being charged unfairly depending on load order.
You can test this by consulting this Stack Overflow question: Disabling ASLR in Mac OS X Snow Leopard.
~# curl -L get.rvm.io | bash -s stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 185 100 185 0 0 11 0 0:00:16 0:00:15 0:00:01 1057
100 9979 100 9979 0 0 317 0 0:00:31 0:00:31 --:--:-- 20235
Downloading RVM from wayneeseguin branch stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 125 100 125 0 0 8 0 0:00:15 0:00:15 --:--:-- 384
100 125 100 125 0 0 1 0 0:02:05 0:01:33 0:00:32 0curl: (7) couldn't connect to host
Could not download 'https://github.com/wayneeseguin/rvm/tarball/stable'.
curl returned status '7'.
I think your answer is here:
https://github.com/wayneeseguin/rvm/issues/804