I have a project (Rails 4.0.2) that I'm currently running inside of Vagrant (1.3.5) running VirtualBox (4.3.4). The Guest OS is Debian 6.0. When I run the application on the Host OS, or I start up the Virtualbox manually, I see a dramatic improvement in responsiveness. As soon as I use 'vagrant up', performance seems to become really poor. Here are the relevant Apache Bench results:
Apache Bench Command
ab -n 10 -c 1 http://127.0.0.1:3000/application.js
Host OS
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 27 44 28.3 33 118
Waiting: 25 41 28.6 31 116
Total: 27 44 28.3 33 118
Virtualbox
min mean[+/-sd] median max
Connect: 0 0 0.4 0 1
Processing: 57 71 19.1 67 119
Waiting: 46 59 19.3 57 110
Total: 57 71 19.1 68 119
Vagrant
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 849 916 76.2 901 1115
Waiting: 831 892 72.6 883 1081
Total: 849 916 76.2 901 1115
I would expect a slowdown running the application in Virtualbox, but not an order of magnitude. I'm also not doing anything fancy with my Vagrantfile:
Vagrantfile
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "squeeze"
config.vm.network :forwarded_port, guest: 3000, host: 3000
end
I've tried the fixes specified in this github issue and this HackerNews comment but to no avail.
Make sure you don't place your project in synced folder (by default it uses vboxsf which has known performance issues with large numbers of files/directories).
This may also be related to "Webrick Reverse DNS Lookup", take a look at https://stackoverflow.com/a/19284483/1801697
Hope it helps.
Related
SI attempted to run a script to generate a table in my Informix database, but the script was missing a newline at EOF, so I think Informix had problems to read it and hence the script got blocked doing nothing. I had to kill the script and add the new line to the file so now the script works fine, except it does not create the table due to a lockecreated when I killed the script abruptly.
I am new to this, so sorry for the dumb question. IBM page does not have a clear and simple explanation of how to clean this now.
So, my question is: How do I unlock the locks so I can continue working in my script?
admin_proyecto#li1106-217 # onstat -k
IBM Informix Dynamic Server Version 12.10.FC9DE -- On-Line (CKPT REQ) -- Up 9 ds
Blocked:CKPT
Locks
address wtlist owner lklist type tbz
44199028 0 44ca6830 0 HDR+S
44199138 0 44cac0a0 0 HDR+S
441991c0 0 44cac0a0 4419b6f0 HDR+IX
44199358 0 44ca44d0 0 S
441993e0 0 44ca44d0 44199358 HDR+S
4419ac50 0 44cac0a0 441991c0 HDR+X
4419aef8 0 44ca44d0 441993e0 HDR+IX
4419b2b0 0 44ca79e0 0 S
4419b3c0 0 44ca82b8 0 S
4419b6f0 0 44cac0a0 44199138 HDR+X
4419b998 0 44ca8b90 0 S
4419bdd8 0 44ca44d0 4419aef8 HDR+X
12 active, 20000 total, 16384 hash buckets, 0 lock table overflows
On my "toy" systems i usually point LTAPEDEV to a directory:
LTAPEDEV /usr/informix/dumps/motor_003/backups
Then, when Informix blocks due to having all of it's logical logs full, i manually do an ontape -a to backup to files the used logical logs and free them to be reused.
For example, here I have an Informix instance blocked due to no more logical logs available:
$ onstat -l
IBM Informix Dynamic Server Version 12.10.FC8DE -- On-Line (CKPT REQ) -- Up 00:18:58 -- 213588 Kbytes
Blocked:CKPT
Physical Logging
Buffer bufused bufsize numpages numwrits pages/io
P-1 0 64 1043 21 49.67
phybegin physize phypos phyused %used
2:53 51147 28085 240 0.47
Logical Logging
Buffer bufused bufsize numrecs numpages numwrits recs/pages pages/io
L-1 13 64 191473 12472 6933 15.4 1.8
Subsystem numrecs Log Space used
OLDRSAM 191470 15247376
HA 3 132
Buffer Waiting
Buffer ioproc flags
L-1 0 0x21 0
address number flags uniqid begin size used %used
44d75f88 1 U------ 47 3:15053 5000 5 0.10
44b6df68 2 U---C-L 48 3:20053 5000 4986 99.72
44c28f38 3 U------ 41 3:25053 5000 5000 100.00
44c28fa0 4 U------ 42 3:53 5000 2843 56.86
44d59850 5 U------ 43 3:5053 5000 5 0.10
44d598b8 6 U------ 44 3:10053 5000 5 0.10
44d59920 7 U------ 45 3:30053 5000 5 0.10
44d59988 8 U------ 46 3:35053 5000 5 0.10
8 active, 8 total
On the online log I have:
$ onstat -m
04/23/18 18:20:42 Logical Log Files are Full -- Backup is Needed
So I manually issue the command:
$ ontape -a
Performing automatic backup of logical logs.
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000041
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000042
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000043
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000044
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000045
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000046
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000047
File created: /usr/informix/dumps/motor_003/backups/informix003.ifx.marqueslocal_3_Log0000000048
Do you want to back up the current logical log? (y/n) n
Program over.
If I check again the status of the logical logs:
$ onstat -l
IBM Informix Dynamic Server Version 12.10.FC8DE -- On-Line -- Up 00:23:42 -- 213588 Kbytes
Physical Logging
Buffer bufused bufsize numpages numwrits pages/io
P-2 33 64 1090 24 45.42
phybegin physize phypos phyused %used
2:53 51147 28091 36 0.07
Logical Logging
Buffer bufused bufsize numrecs numpages numwrits recs/pages pages/io
L-1 0 64 291335 15878 7023 18.3 2.3
Subsystem numrecs Log Space used
OLDRSAM 291331 22046456
HA 4 176
address number flags uniqid begin size used %used
44d75f88 1 U-B---- 47 3:15053 5000 5 0.10
44b6df68 2 U-B---- 48 3:20053 5000 5000 100.00
44c28f38 3 U---C-L 49 3:25053 5000 3392 67.84
44c28fa0 4 U-B---- 42 3:53 5000 2843 56.86
44d59850 5 U-B---- 43 3:5053 5000 5 0.10
44d598b8 6 U-B---- 44 3:10053 5000 5 0.10
44d59920 7 U-B---- 45 3:30053 5000 5 0.10
44d59988 8 U-B---- 46 3:35053 5000 5 0.10
8 active, 8 total
The logical logs are now marked as "Backed Up" and can be reused and the Informix instance is no longer blocked on Blocked:CKPT .
there, I ran into a 'Segmentation fault' error when using travis-ci for my project : IPython-Dashboard
there is no error msg and it works fine on local, I feel a little confusing. any one can give any idea on fixing this, thanks.
here is the travis build log on cloud:
travis-log
$ nosetests --with-coverage --cover-package=dashboard
../home/travis/build.sh: line 45: 3187 Segmentation fault (core dumped)
nosetests --with-coverage --cover-package=dashboard
The command "nosetests --with-coverage --cover-package=dashboard" exited with 139.
here is the build log on local [osx]
taotao#mac007:~/Desktop/github/IPython-Dashboard$sudo nosetests --with-coverage --cover-package=dashboard
.../Users/chenshan/Desktop/github/IPython-Dashboard/dashboard/tests/testCreateData.py:78: Warning: Can't create database 'IPD_data'; database exists
conn.cursor().execute('CREATE DATABASE IF NOT EXISTS {};'.format(config.sql_db))
/Library/Python/2.7/site-packages/pandas/io/sql.py:599: FutureWarning: The 'mysql' flavor with DBAPI connection is deprecated and will be removed in future versions. MySQL will be further supported with SQLAlchemy engines.
warnings.warn(_MYSQL_WARNING, FutureWarning)
...
Name Stmts Miss Cover Missing
---------------------------------------------------------------------
dashboard.py 13 0 100%
dashboard/client.py 1 0 100%
dashboard/client/sender.py 11 3 73% 26-27, 33
dashboard/conf.py 0 0 100%
dashboard/conf/config.py 29 0 100%
dashboard/server.py 0 0 100%
dashboard/server/resources.py 0 0 100%
dashboard/server/resources/dash.py 35 10 71% 36, 55-56, 67-69, 86-89
dashboard/server/resources/home.py 40 12 70% 25, 28-30, 83-91
dashboard/server/resources/sql.py 27 11 59% 30, 52-75
dashboard/server/resources/status.py 8 1 88% 19
dashboard/server/resources/storage.py 13 5 62% 26-28, 43-47
dashboard/server/utils.py 79 18 77% 20-24, 78-80, 82-83, 86, 96, 99-100, 126-127, 140-142
dashboard/server/views.py 21 1 95% 16
---------------------------------------------------------------------
TOTAL 277 61 78%
----------------------------------------------------------------------
Ran 6 tests in 4.600s
OK
taotao#mac007:~/Desktop/github/IPython-Dashboard$
Here is my problem:
top - 11:32:47 up 22:20, 2 users, load average: 0.03, 0.72, 1.27
Tasks: 112 total, 1 running, 110 sleeping, 1 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8193844k total, 7508292k used, 685552k free, 80636k buffers
Swap: 2102456k total, 15472k used, 2086984k free, 7070220k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
28555 root 20 0 57424 38m 1492 S 0 0.5 0:06.38 bash
28900 root 20 0 39488 7732 3176 T 0 0.1 0:03.12 python
28553 root 20 0 72132 5052 2600 S 0 0.1 0:00.22 sshd
28859 root 20 0 70588 3424 2584 S 0 0.0 0:00.06 sshd
29404 root 20 0 70448 3320 2600 S 0 0.0 0:00.06 sshd
28863 root 20 0 42624 2188 1472 S 0 0.0 0:00.02 sftp-server
29406 root 20 0 19176 1984 1424 S 0 0.0 0:00.00 bash
2854 root 20 0 115m 1760 488 S 0 0.0 5:37.02 rsyslogd
29410 root 20 0 19064 1400 1016 R 0 0.0 0:05.14 top
3111 ntp 20 0 22484 604 460 S 0 0.0 10:26.79 ntpd
3134 proftpd 20 0 64344 452 280 S 0 0.0 6:29.16 proftpd
2892 root 20 0 49168 356 232 S 0 0.0 0:31.58 sshd
1 root 20 0 27388 284 132 S 0 0.0 0:01.38 init
3121 root 20 0 4308 248 172 S 0 0.0 0:16.48 mdadm
As you can see 7.5 GB of memory is used, but there is no process that use it.
How it can be, and how to fix this?
Thanks for answer.
www.linuxatemyram.com
It's too good of a site to ruin by copy/pasting the entire contents here.
in order to see all process you can use that command:
ps aux
and then try to sort with different filters
ps faux
Hope that helps.
If your system starts using the swap file - then you have high memory load. Depends on the file system, programs that you use - linux system may allocate all of your system memory - but that doesn't mean that they are using it.
Lots of ubuntu and debian servers that we use have free memory 32 or 64 mb but don't use swap.
I'm not Linux-gure however, so please someone to correct me if I'm wrong :)
I don't have a Linux box handy to experiment, but it looks like you can sort top's output with interactive commands, so you could bring the biggest memory users to the top. Check the man page and experiment.
Update: In the version of top I have (procps 3.2.7), you can hit "<" and ">" to change the field it's sorting by. Doesn't actually say what field it is, you have to look at how the display is changing. It's not hard once you experiment a little.
However, Arrowmaster's point (that it's probably being used for cache) is a better answer. Use "free" to see how much is being used.
I had a similar problem. I was running Raspbian on a Pi B+ with a TP-Link USB Wireless LAN stick connected. The stick caused a problem which resulted in nearly all memory being consumed on system start (around 430 of 445 MB). Just like in your case, the running processes did not consume that much memory. When I removed the stick and rebooted everything was fine, just 50 MB memory consumption.
I have a Rails 2.3.8 app hosted and running on slicehost (256M). I am not familiar at all with the back-end, I basically followed the steps from the slicehost tutorials to install Apache. The memory usage being very high, I then changed my Apache conf file to reduce the MaxClient number to 10... but my slice is still swapping.
Here is what the memory usage I get after just a few clicks on my site:
top - 23:57:12 up 28 min, 2 users, load average: 0.43, 0.54, 0.30
Tasks: 79 total, 1 running, 78 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.1%us, 0.1%sy, 0.0%ni, 97.8%id, 0.1%wa, 0.0%hi, 0.0%si, 2.0%st
Mem: 262364k total, 258656k used, 3708k free, 260k buffers
Swap: 524280k total, 262772k used, 261508k free, 6328k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4004 web-app 20 0 178m 72m 1888 S 0 28.4 0:04.38 ruby1.8
4001 web-app 20 0 172m 61m 1932 S 0 24.2 0:02.72 ruby1.8
3941 root 20 0 164m 57m 1672 S 0 22.5 0:21.44 ruby
3990 web-app 20 0 209m 21m 1696 S 0 8.4 0:18.00 ruby1.8
3950 web-app 20 0 165m 7464 1548 S 0 2.8 0:20.40 ruby1.8
3684 mysql 20 0 224m 6504 2084 S 0 2.5 0:14.34 mysqld
3938 root 20 0 53632 3048 1036 S 1 1.2 0:01.50 starling
3839 root 20 0 243m 1456 1248 S 0 0.6 0:00.34 apache2
3897 www-data 20 0 243m 1452 1072 S 0 0.6 0:00.04 apache2
3894 www-data 20 0 243m 1368 1008 S 0 0.5 0:00.04 apache2
3895 www-data 20 0 243m 1220 960 S 0 0.5 0:00.02 apache2
3888 root 20 0 46520 1204 1100 S 0 0.5 0:02.29 ruby1.8
3866 root 20 0 17648 1184 896 S 0 0.5 0:00.08 bash
3896 www-data 20 0 243m 1180 952 S 0 0.4 0:00.00 apache2
3964 www-data 20 0 243m 1164 956 S 0 0.4 0:00.02 apache2
3892 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3948 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3962 www-data 20 0 243m 1132 956 S 0 0.4 0:00.02 apache2
3963 www-data 20 0 243m 1132 956 S 0 0.4 0:00.00 apache2
3965 www-data 20 0 243m 1080 888 S 0 0.4 0:00.00 apache2
3887 root 20 0 89008 960 796 S 0 0.4 0:00.00 ApplicationPool
I'm not sure what to do next... I could upgrade to a larger slice but for now I have almost no traffic on this app, so I think it's more a problem with my configuration or maybe my code?
Any concrete recommendations would be welcome!
Thanks
It looks like your rails app is using all your available memory. I would recommend three things:
Upgrade the memory on your server. 256MB is not very much for a Rails app. Going to 512 may alleviate your problem. If that solves it, you then need to consider the additional cost ($18/mo) vs how much time it will take to track down performance issues.
Profile your application to figure out which requests are consuming the most memory. This is likely going to be places where you're finding a lot of records and possibly including some associated tables too. There are a couple of tools out there to help you narrow down possible trouble areas. I've used oink but there are definitely others. Once you figure out where the problems are, you can make some tweaks to try and reduce the memory usage.
Assuming you're using Passenger with Apache, you can reduce the number of concurrent requests in the Passenger config file. This might be useful for that https://serverfault.com/questions/15350/running-ruby-on-rails-app-on-apache-passenger-to-much-memory
In short, 256MB is tight for a Rails application. You did not really give any specifics on how you are running rails, but I assume you are using Apache with the Passenger module. The Passenger module can be configured on how many instances it keeps running. You have 4 ruby instances running under the web-app account. I guess those come from Passenger. In the configuration, you can limit how many instances Passenger starts. This will reduce the memory requirements.
On the other hand, when working with only 256MB, and when you are only hosting 1 rails application, it might be better to go for another setup. The setup that I used myself before was an Nginx web server, and a mongrel cluster with 2 mongrels (on 192MB, and application was only for testing purposes). Basically that means that at any one time, you can process 2 (and only 2) rails requests in parallel. The setup is maybe a bit harder than Apache+Passenger, but definitely not difficult. I think that is a more performant solution when you stick with the 256MB.
I have a rails app (2.3.5) running on a VPS with 4 cores # 2 GHz and 4GB memory. I am running nginx (0.7.61) and phusion passenger(2.2.14) on Ruby Enterprise (1.8.7-2010.01) with the max pool size set at 30. My problem is that it seems as if every ruby process that is executing a rails request runs at near 100% cpu. If I run TOP they drop off every time the display refreshes so they are not getting hung, but they are still running at 100%.
Is there any way I can bring this down? Or at least figure out what portion of code is spiking the CPU? Is this a normal behavior?
Here is the TOP output:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2427 psadmin 25 0 91904 76m 2696 R 100 1.9 739:05.96 Rails: /var/www/apps/main_rails_app/current
3457 psadmin 25 0 98180 82m 2532 R 100 2.0 711:21.91 Rails: /var/www/apps/main_rails_app/current
2415 psadmin 25 0 93952 77m 2708 R 99 1.9 727:49.31 Rails: /var/www/apps/main_rails_app/current
3455 psadmin 25 0 99204 83m 2528 R 69 2.0 726:04.70 Rails: /var/www/apps/main_rails_app/current
2791 psadmin 16 0 98044 81m 2492 S 31 2.0 0:10.16 Rails: /var/www/apps/main_rails_app/current
8034 psadmin 15 0 8160 3656 1772 S 1 0.1 0:35.39 nginx: worker process
8035 psadmin 15 0 8324 3696 1732 S 0 0.1 0:31.34 nginx: worker process
2588 psadmin 15 0 197m 183m 2712 S 0 4.5 1:02.16 Rails: /var/www/apps/main_rails_app/current
Thanks!
Edit: Tried strace with follow forks as mentioned below. This is the output that is dumped over and over:
sudo strace -f -p 3455
clock_gettime(CLOCK_MONOTONIC, {394577, 508326476}) = 0
select(0, [], [], [], {0, 0}) = 0 (Timeout)
--- SIGVTALRM (Virtual timer expired) # 0 (0) ---
sigreturn()
check your logs for suspicious behavior. In general rails does suck a bunch of cpu though...you could also try pointing strace at the offending pids.