I have a Rails application running on jruby-9.0.4.0 and using Puma as the web server.
I am trying to increase the memory limit of puma; the current flags are -Xmx500m -Xss2048k. I added _JAVA_OPTIONS=-Xss4096k -Xmx2048m to my env and all the jruby processes use it like rake assets:precompile for example but the puma instance itself does not.
Part of capistrano trace
DEBUG [0aec947c] Command: cd
/dummy/production/releases/20160707071111 && (
export RBENV_ROOT="/usr/local/rbenv" RBENV_VERSION="jruby-9.0.4.0"
RAILS_ENV="production" ; /usr/local/rbenv/bin/rbenv exec bundle exec
rake assets:precompile )
DEBUG [0aec947c] Picked up _JAVA_OPTIONS: -Xss4096k -Xmx2048m
config/puma.rb
# Min and Max threads per worker
threads 8, 512
# Default to production
rails_env = ENV['RAILS_ENV'] || 'production'
environment rails_env
app_dir = "/dummy/#{rails_env}/current"
# Set up socket location
bind "tcp://localhost:3000"
# Logging
stdout_redirect "#{app_dir}/log/puma.stdout.log", "#{app_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{app_dir}/pids/puma.pid"
state_path "#{app_dir}/pids/puma.state"
activate_control_app
ENV variables
JAVA_OPTS=-Xss4096k -Xmx2048m
_JAVA_OPTIONS=-Xss4096k -Xmx2048m
JAVA_HOME=/usr/lib/jvm/java-7-oracle-amd64
Output of cat /proc/<pid>/environ
RBENV_ROOT=/usr/local/rbenvprevious=NUPSTART_JOB=rcPATH=/dummy/production/shared/bundle/jruby/2.2.0/bin:/usr/local/rbenv/versions/jruby-9.0.4.0/bin:/usr/local/rbenv/libexec:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binPWD=/dummy/production/releases/20160707133222RBENV_DIR=/dummy/production/releases/20160707133222SUPERVISOR_SERVER_URL=unix:///var/run/supervisor.sockUPSTART_EVENTS=runlevelRUNLEVEL=2NLSPATH=/usr/dt/lib/nls/msg/%L/%N.catSUPERVISOR_PROCESS_NAME=dummyprocessSUPERVISOR_ENABLED=1XFILESEARCHPATH=/usr/dt/app-defaults/%L/DtSHLVL=0UPSTART_INSTANCE=PREVLEVEL=NRBENV_VERSION=jruby-9.0.4.0RBENV_HOOK_PATH=:/usr/local/rbenv/rbenv.d:/usr/local/etc/rbenv.d:/etc/rbenv.d:/usr/lib/rbenv/hooksrunlevel=2SUPERVISOR_GROUP_NAME=dummyprocessTERM=linuxRUBYOPT=-rbundler/setupRUBYLIB=/usr/local/rbenv/versions/jruby-9.0.4.0/lib/ruby/gems/shared/gems/bundler-1.11.2/libroot
Machine is an 8-core with 24GB of RAM.
How can I let the puma instance also pick up the Java options?
As mentionned in this blog post, you can put the JVM options in JRUBY_OPTS environment variable.
In your case, you could set the variable before starting Puma daemon :
export JRUBY_OPTS="-J-Xss4096k -J-Xmx2048m"
You may also try configuring the JVM directly by setting JAVA_OPTS environment variable :
export JAVA_OPTS="-Xss4096k -Xmx2048m"
Also check how the puma daemon is started
cat /proc/<pid>/environ
Your environment is not propagated to the Puma daemon. You need to find out how Puma is started. It may be as an init.d service or an upstart service.
Update:
It seems you can set your environment with rbenv-vars.
Create a .rbenv-vars file in your Rails project with the following
JAVA_OPTS='-Xss4096k -Xmx2048m'
Then your environment should be propagated to the puma daemon.
https://devcenter.heroku.com/articles/optimizing-dyno-usage#basic-methodology-for-optimizing-memory
JRuby
JRuby servers like Puma make good use of concurrency without the need
for multiple processes. However, you will need to tune the amount of
memory allocated to the JVM, depending on the dyno type. The Ruby
buildpack defines sensible defaults, which can be overridden by
setting either JAVA_OPTS or JRUBY_OPTS.
Related
I have a stack in Elastic Beanstalk running the "Passenger with Ruby 2.5 running on 64bit Amazon Linux/2.8.3" image. Rails is v5.2.1. The desire is to make it a staging environment i.e., have 'Rails.env' return 'staging' and run off of a staging.rb configuration. The problem is, it seems to be running as 'production' no matter what I do.
I set up RACK_ENV and RAILS_ENV as EB configuration variables, both set to 'staging'. I confirmed their existence on the server with 'printenv' as ec2-user, webapp, and root.
I tried changing passenger_app_env in the Passenger config to 'staging'. I've confirmed that Passenger Standalone is using the correct config by looking at the process with 'ps aux | grep passenger'.
I've tried switching to the root server and manually doing '/etc/init.d/passenger stop' and then 'start', and the printout confirms Passenger is launching with its 'environment' set to 'staging':
=============== Phusion Passenger Standalone web server started ===============
PID file: /var/app/support/pids/passenger.pid
Log file: /var/app/support/logs/passenger.log
Environment: staging
Accessible via: http://0.0.0.0/
Serving in the background as a daemon.
Problems? Check https://www.phusionpassenger.com/documentation/Users%20guide%20Standalone.html#troubleshooting
I put this into environment.rb and added an EB config var for STAGING to be 'true'.:
if ENV['STAGING']
%w(RACK_ENV RAILS_ENV).each do |key|
ENV[key] = 'staging'
end
end
However, the test page I made in my Rails app still says 'Rails.env' is 'production', and is not using values from 'staging.rb'. And yet, that same test page says that 'ENV['RACK_ENV']' and 'ENV['RAILS_ENV'] are both set to 'staging'.
At this point, I'm out of ideas on how to force the environment in any other way.
After much hacking, I discovered that Passenger was launching with a passenger_app_env of 'production' (its default) and then switching over to 'staging'. Rails.env would get the the
'production' env and use 'production.rb', then RACK_ENV and RAILS_ENV would be overwritten to 'staging', creating the confusing duality.
The solution was moving the passenger_app_env directive higher up in the Passenger Standalone template that Passenger uses; we had it inside of a server directive inside of an http directive. Moving it up out of the server directive and into the http directive itself solved the issue.
In my Ruby on Rails dev environment, I am starting Rails and Unicorn via Foreman in the typical way:
(Procfile:)
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
I am also running Pow. But not as a webserver. I'm just using Pow to direct http requests from mydomain.dev (port 80) to the port Unicorn is listening on.
You can do this by creating a pow file (mydomain.dev) containing the port number Unicorn is running on.
Given this setup, is it possible in my rails code to know what the port I started Unicorn on?
I'm only wanting to know this in my dev environment, it's not a production issue.
In my Rails code, I've tried a few different things, but none of them work:
Unicorn::Configurator::RACKUP[:port] - returned nothing
Rails::Server.new.options[:Port] - doesn't exist in Rails 4
Rack::Server.new.options[:Port] - returns default rack port (9292) not the one configured for this rack instance.
Is there a way to get the current rack instance from rails?
request.port - returns 80, which is the port that Pow is listening on. Pow is routing http traffic to Unicorn, which is on a different port.
None of these give me the port that Unicorn is running on.
Any ideas?
EDIT If you're wondering why I want to know this, it's because in my dev environment, I'm trying to dynamically create configuration files for Pow, so I can route http requests to Unicorn on the correct port.
If you're responding to a request in a controller or view, use the request object - this should work, although you say it does not:
request.port
If you're in an initialiser :
Rails::Server.new.options[:Port]
How to find the local port a rails instance is running on?
You should just be able to access it via ENV['PORT'], given it's value has been set to the $PORT environment variable.
I've sort of found a way to do this.
Create separate config files for Unicorn. unicorn.development.rb and unicorn.test.rb
Install the dotenv-rails gem
Inside my unicorn config files, do something like this:
# unicorn.development.rb:
require "dotenv"
Dotenv.load(
"./.env.local",
"./.env.development",
"./.env"
)
if ENV['UNICORN_PORT'].nil?
throw 'UNICORN_PORT not set in environment!'
end
worker_processes 3
timeout 30
preload_app true
listen ENV['UNICORN_PORT'], backlog: 64
... rest of unicorn config...
# unicorn.testing.rb:
require "dotenv"
Dotenv.load(
"./.env.local",
"./.env.testing",
"./.env"
)
if ENV['UNICORN_PORT'].nil?
throw 'UNICORN_PORT not set in environment!'
end
worker_processes 3
timeout 30
preload_app true
listen ENV['UNICORN_PORT'], backlog: 64
... rest of unicorn config...
In my .env.development and .env.testing environment files, set the UNICORN_PORT environment variable
Make sure you use the correct Unicorn config file to start the app. This can be done by using separate Procfiles for dev and testing.
# Procfile.dev
web: bundle exec unicorn -c ./config/unicorn.development.rb
# Procfile.testing
web: bundle exec unicorn -c ./config/unicorn.testing.rb
This appears to mostly work, but is not without it's issues...
Probably a bad idea, but whatever:
uport = `netstat -n --listening --tcp -p | grep unicorn | sed 's/.*:\([0-9]*\) .*/\1/'`
I've a rails service which I am starting up with Puma. I am using
bundle exec puma -C config/puma.rb -p 3000. However I've to start a new service on a different port let's say 3001. So, I've a different puma2.rb which I am starting with bundle exec puma -C config/puma2.rb -p 3001.
Both of these services have a common log file which is development.log. I want to seperate these log files say development-3000.log & development-3001.log.
I've tried
stdout_redirect "#{Dir.pwd}/log/puma.stdout.log", "#{Dir.pwd}/log/puma.stderr.log"
for individual puma files but this logs only the requests. I want the log to be of log_level : debug. How can I achieve this?
Here is my puma config file:
app_path=File.expand_path('../', _ _FILE_ _)
tmp_dir= "#{app_path}/../tmp"
pidfile "#{tmp_dir}/pid"
state_path "#{tmp_dir}/state"
threads 8,32
workers 2
activate_control_app
One way to get separate logs would be to setup a separate rails environment. You could make a copy of the config/development.rb file and name it development2.rb, and then set the environment in the puma2.rb file to development2.
# puma2.rb
environment "development2"
That environment would write to development2.log
My puma config:
path = Dir.pwd + "/tmp/puma/"
threads 0,20
environment "production"
daemonize true
drain_on_shutdown true
bind "unix://" + path + "socket/puma.sock"
pidfile path + "pid/puma.pid"
state_path path + "pid/puma.state"
My environments/production.rb
MyApp::Application.configure do
config.log_level = :debug
end
I start my server:
starkers#ubuntu:~/Desktop/myspp$ pumactl -F config/puma.rb start
=> Booting Puma
=> Rails 4.0.2 application starting in production on http://0.0.0.0:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
config.eager_load is set to nil. Please update your config/environments/*.rb files accordingly:
* development - set it to false
* test - set it to false (unless you use a tool that preloads your test environment)
* production - set it to true
Puma 2.8.2 starting...
* Min threads: 0, max threads: 16
* Environment: production
* Listening on tcp://0.0.0.0:3000
I browse about my app. And my log/production.log is blank. Not sure why?
Directory access is 0777 throughout my app.
No idea what is causing this. Really need logs (obviously). Happening locally and remotely so it's something to do with my configuration. However I'm not sure what configuration. Is there anything in puma/ubuntu/rails that could be causing this?
development.log works perfectly. I've copy pasted my development.rb to my production.rb file. Literally identical. Okay? Identical development.rb and production .rb And yet:
RAILS_ENV=development rails s
populates development.log
and
RAILS_ENV=production rails s
leaves production.log as empty as Kim Kardashian's head.
Set bind at the end of config file:
path = Dir.pwd + "/tmp/puma/"
threads 0,20
environment "production"
daemonize true
drain_on_shutdown true
pidfile path + "pid/puma.pid"
state_path path + "pid/puma.state"
bind "unix://" + path + "socket/puma.sock"
I used command pumactl -F config/puma.rb start to start server (i guess there is no difference, but anyway).
And i would recommend to use #{} for path:
pidfile "#{path}pid/puma.pid"
state_path "#{path}pid/puma.state"
bind "unix://#{path}socket/puma.sock"
but it's your choice.
Hope it helps (for me you config didn't work too).
you can also add Puma logs:
stdout_redirect "#{Dir.pwd}/log/puma.stdout.log", "#{Dir.pwd}/log/puma.stderr.log"
Add this line before bind.
If you want to add the output of the server to a log, the easiest way to do this is by telling your system to do exactly that. Running your server start command like:
pumactl -F config/puma.rb start >> log/development.log
Will append each line of output from your server to the development log. Though to make things easier to debug, you may want to give each server its own log such as log/puma.log. If you do, you may wish to rewrite the file from scratch every time you start the server instead of keeping a cumulative log, if that's the case just turn the >> into a > such as:
pumactl -F config/puma.rb start > log/puma.log
However, if you have your system set up to automatically restart the server if it fails, using > will overwrite the log for what might have caused the crash when the server restarts.
Similarly, you can get your production.log working by starting your rails server like:
RAILS_ENV=production rails s >> log/production.log
If you want to run your server in the background like you might in your production environment, you can add a & character to the end like:
pumactl -F config/puma.rb start > log/puma.log &
If you do this you'll probably want to store the process identifier so you can kill the server later as ^C doesn't work for background processes. To store the process id, create another empty file somewhere like lib/pids/puma.pid and then export the process id of that puma server to the empty file like:
pumactl -F config/puma.rb start > log/puma.log &
echo $! > lib/pids/puma.pid
You would then be able to kill the server with:
kill `cat lib/pids/puma.pid`
It is important to remember that even if you append the output of the server to your development.log file, it will not show up in the output of your development rails server. If you want a live view of your log for debugging, you can use the tailf command such as:
tailf log/puma.log
For more information on the command line interface, the Command Line Crash Course is a good resource.
I was following this guide it documents the puma.rb file that is stored inside the app's config directory.
The guide is a bit flakey, but here's what I assume the puma.rb file does. Instead of running crazy commands such as this to get puma running on a specified socket:
bundle exec puma -e production -b unix:///var/run/my_app.sock
You can just specify the port, pid, session and other parameters in the puma.rb file like this:
rails_env = ENV['RAILS_ENV'] || 'production'
threads 4,4
bind "/home/starkers/Documents/alpha/tmp/socket"
pidfile "/home/starkers/Documents/alpha/tmp/pid"
state_path "/home/starkers/Documents/alpha/tmp/state"
activate_control_app
And then you could cd into the app's root and run a simple command like
'puma'
and the parameters set in puma.rb would be followed. Unfortunately that doesn't seem to work for me.
At least, I ran puma inside the root of a tiny test app, and no .sock file appeared in
/home/starkers/Documents/alpha/tmp/sockets so does that mean it isn't working?
How do I get this working? I am on a local development machine, so could that cause this error somehow? Is there a parameter I need to pass in when running
puma ?
I was also stuck trying to find documentation on the config file for puma but I did find the all-in-one config.ru file useful. I've formatted it here for future reference:
# The directory to operate out of.
# The default is the current directory.
directory '/u/apps/lolcat'
# Load “path” as a rackup file.
# The default is “config.ru”.
rackup '/u/apps/lolcat/config.ru'
# Set the environment in which the rack's app will run. The value must be a string.
# The default is “development”.
environment 'production'
# Daemonize the server into the background. Highly suggest that
# this be combined with “pidfile” and “stdout_redirect”.
# The default is “false”.
daemonize
daemonize false
# Store the pid of the server in the file at “path”.
pidfile '/u/apps/lolcat/tmp/pids/puma.pid'
# Use “path” as the file to store the server info state. This is
# used by “pumactl” to query and control the server.
state_path '/u/apps/lolcat/tmp/pids/puma.state'
# Redirect STDOUT and STDERR to files specified. The 3rd parameter
# (“append”) specifies whether the output is appended, the default is
# “false”.
stdout_redirect '/u/apps/lolcat/log/stdout', '/u/apps/lolcat/log/stderr'
stdout_redirect '/u/apps/lolcat/log/stdout', '/u/apps/lolcat/log/stderr', true
# Disable request logging.
# The default is “false”.
quiet
# Configure “min” to be the minimum number of threads to use to answer
# requests and “max” the maximum.
# The default is “0, 16”.
threads 0, 16
# Bind the server to “url”. “tcp://”, “unix://” and “ssl://” are the only
# accepted protocols.
# The default is “tcp://0.0.0.0:9292”.
bind 'tcp://0.0.0.0:9292'
bind 'unix:///var/run/puma.sock'
bind 'unix:///var/run/puma.sock?umask=0777'
bind 'ssl://127.0.0.1:9292?key=path_to_key&cert=path_to_cert'
# Listens on port 7001
# The default is 9292
port 7001
# Instead of “bind 'ssl://127.0.0.1:9292?key=path_to_key&cert=path_to_cert'” you
# can also use the “ssl_bind” option.
ssl_bind '127.0.0.1', '9292', { key: path_to_key, cert: path_to_cert }
# Code to run before doing a restart. This code should
# close log files, database connections, etc.
# This can be called multiple times to add code each time.
on_restart do
puts 'On restart...'
end
# Command to use to restart puma. This should be just how to
# load puma itself (ie. 'ruby -Ilib bin/puma'), not the arguments
# to puma, as those are the same as the original process.
restart_command '/u/app/lolcat/bin/restart_puma'
# === Cluster mode ===
# How many worker processes to run.
# The default is “0”.
workers 2
# Code to run when a worker boots to setup the process before booting
# the app.
# This can be called multiple times to add hooks.
on_worker_boot do
puts 'On worker boot...'
end
# === Puma control rack application ===
# Start the puma control rack application on “url”. This application can
# be communicated with to control the main server. Additionally, you can
# provide an authentication token, so all requests to the control server
# will need to include that token as a query parameter. This allows for
# simple authentication.
# Check out https://github.com/puma/puma/blob/master/lib/puma/app/status.rb
# to see what the app has available.
activate_control_app 'unix:///var/run/pumactl.sock'
activate_control_app 'unix:///var/run/pumactl.sock', { auth_token: '12345' }
activate_control_app 'unix:///var/run/pumactl.sock', { no_token: true }
Those settings would then go in a ruby file (e.g. config/puma.rb) and then as Starkers says, you can run it with
puma -C config/puma.rb
Update: The original answer is no longer correct for Puma versions since 2019: Puma added a fallback mechanism, so both locations are checked now. ( https://github.com/puma/puma/pull/1885)
Puma first looks for configuration at config/puma/<environment_name>.rb, and then falls back to config/puma.rb.
Outdated answer:
If there is an environment defined - which is the case in your example - the configuration file is read from config/puma/[environment].rb and not config/puma.rb.
Just move your config/puma.rb to config/puma/production.rb and it should work.
Read the Puma documentation for more details: Configuration file
This will work:
puma -C config/puma.rb
You need to tell puma where to find your rackup file you can do it by putting this in your config:
rackup DefaultRackup
It looks like a fix for this is merged into master: https://github.com/puma/puma/pull/271