I'm trying to start puma with 2 workers, but when it starts the logs say:
Puma 2.11.3 starting...
* Min threads: 0, max threads: 16
* Environment: production
* Listening on tcp://0.0.0.0:3000
config/puma.rb:
workers Integer(ENV['PUMA_WORKERS'] || 2)
threads Integer(ENV['PUMA_MIN_THREADS'] || 1), Integer(ENV['PUMA_MAX_THREADS'] || 6)
Any idea why puma is launched with 0 min thread rather than 1?
this is one of those RTFM questions. The puma documentation states:
when using the RACK_ENV environment variable, the default file location will be config/puma/environment_name.rb.
so the fix is to relocate config/puma.rb into config/puma/production.rb
Related
My Rails application doesn't output the result of the system tests (no failures, no assertions, nothing). The funny thing is that I can see chrome doing what it was expected to do (filling fields, etc...), but when the test is finished, no output on the shell, just the following:
francesco.mari#MB68D:DemographicsMapper (inline_mapping_editing *) $ bin/rails test:system
Puma starting in single mode...
* Version 4.3.8 (ruby 2.6.6-p146), codename: Mysterious Traveller
* Min threads: 0, max threads: 4
* Environment: test
* Listening on tcp://127.0.0.1:49330
Use Ctrl-C to stop
francesco.mari#MB68D:DemographicsMapper (inline_mapping_editing *)
Apparently the issue was on the file test_helper.rb. After I added the line parallelize(workers: :number_of_processors) it worked as it should.
class ActiveSupport::TestCase
#...
parallelize(workers: :number_of_processors) # added this line
#...
end
I succefully installed a rails app on my FreeBSD server but when I test rails s -e production or rails s -e development I get Read: #<NameError: uninitialized constant Puma::Server::UNPACK_TCP_STATE_FROM_TCP_INFO> from the Puma server after sending request
I missed a step somewhere ?
PS. I use Rails6 with SqlLite3
config/puma.rb
# Puma can serve each request in a thread from an internal thread pool.
# The `threads` method setting takes two numbers: a minimum and maximum.
# Any libraries that use thread pools should be configured to match
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port ENV.fetch("PORT") { 3000 }
# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch("RAILS_ENV") { "development" }
# Specifies the `pidfile` that Puma will use.
pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }
# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together
# the concurrency of the application would be max `threads` * `workers`.
# Workers do not work on JRuby or Windows (both of which do not support
# processes).
#
# workers ENV.fetch("WEB_CONCURRENCY") { 2 }
# Use the `preload_app!` method when specifying a `workers` number.
# This directive tells Puma to first boot the application and load code
# before forking the application. This takes advantage of Copy On Write
# process behavior so workers use less memory.
#
# preload_app!
# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
I just made monkey patch that seems to work.
(This problem happens on FreeBSD, not on Mac OS X)
Place this content in an initializer file. For example: config/initializers/puma_missing_constant_monkey_patch.rb.
Rails.application.config.after_initialize do
if defined?(::Puma) && !Object.const_defined?('Puma::Server::UNPACK_TCP_STATE_FROM_TCP_INFO')
::Puma::Server::UNPACK_TCP_STATE_FROM_TCP_INFO = "C".freeze
end
end
It just defines the missing constant. I've got no clue if it breaks something else. On the other hand Puma uses a constant that isn't defined. The define of this constant in Puma (lib/puma/server.rb) is conditional.
I am getting error messages as pasted below:
% USE_SERVER=puma bundle exec derailed exec perf:mem_over_time
Booting: production
docking_dev already exists
Endpoint: "/"
Port: 3857
Server: "puma"
[4990] Puma starting in cluster mode...
[4990] * Version 3.7.0 (ruby 2.3.3-p222), codename: Snowy Sagebrush
[4990] * Min threads: 5, max threads: 5
[4990] * Environment: production
[4990] * Process workers: 2
[4990] * Preloading application
[4990] * Listening on tcp://0.0.0.0:3000
[4990] Use Ctrl-C to stop
[4990] - Worker 0 (pid: 5013) booted, phase: 0
[4990] - Worker 1 (pid: 5014) booted, phase: 0
PID: 4990
149.67578125
Couldn't call app.
Bad request to "curl 'http://localhost:3857/' -s --fail 2>&1"
***RESPONSE***:
""
[5014] ! Detected parent died, dying
[5013] ! Detected parent died, dying
I checked RAILS_ENV=production rails server and RAILS_ENV=production rails console both working as expected. What else I need to check to make it working. Is this because my http://localhost:3000/ url has authentication enabled. I checked that I turned force_ssl to false. I checked this post, what it suggested not helping.
I also don't know why it is picking some random ports every time I run it, like in this pasted one it is 3857. But my app runs using 3000 port locally. Is there something I need to do to so that it uses same port 3000?
P.S. Why random port I got to know from gem code.
Ok I fixed it. In my case, root url redirects to login page when users are not signed in. So this is causing the gem bad requests, it seems not able to handle 302 correctly at this moment. So I fixed it..
PATH_TO_HIT="/login" bundle exec derailed exec perf:mem_over_time
Yes I also had to removed the USE_SERVER=puma as it was causing errors too.
I am trying to use MidiSmtpServer to receive email in a Heroku application, and have been using the code on one of the examples that the documents show. However, I don't know where to put that code for the SMTP server to start after Puma, or where to put it for it to start at all. Using on_worker_boot in puma.rb doesnt work.
puma.rb:
# Puma can serve each request in a thread from an internal thread pool.
# The `threads` method setting takes two numbers: a minimum and maximum.
# Any libraries that use thread pools should be configured to match
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port ENV.fetch("PORT") { 3000 }
# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch("RAILS_ENV") { "development" }
# Specifies the `pidfile` that Puma will use.
pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }
# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together
# the concurrency of the application would be max `threads` * `workers`.
# Workers do not work on JRuby or Windows (both of which do not support
# processes).
#
# workers ENV.fetch("WEB_CONCURRENCY") { 2 }
require "midi-smtp-server"
require "mail"
on_worker_boot do
class MySmtpd < MidiSmtpServer::Smtpd
def on_message_data_event(ctx)
puts "[#{ctx[:envelope][:from]}] for recipient(s): [#{ctx[:envelope][:to]}]..."
# Just decode message ones to make sure, that this message ist readable
#mail = Mail.read_from_string(ctx[:message][:data])
# handle incoming mail, just show the message source
puts #mail.to_s
end
end
# try to gracefully shutdown on Ctrl-C
trap("INT") do
puts "Interrupted, exit now..."
exit 0
end
# Output for debug
puts "#{Time.now}: Starting MySmtpd..."
# Create a new server instance listening at localhost interfaces 127.0.0.1:2525
# and accepting a maximum of 4 simultaneous connections
server = MySmtpd.new(2525, "0.0.0.0", 4)
# setup exit code
at_exit do
# check to shutdown connection
if server # Output for debug
puts "#{Time.now}: Shutdown MySmtpd..." # stop all threads and connections gracefully
server.stop
end # Output for debug
puts "#{Time.now}: MySmtpd down!\n"
end
# Start the server
server.start
# Run on server forever
server.join
end
# Use the `preload_app!` method when specifying a `workers` number.
# This directive tells Puma to first boot the application and load code
# before forking the application. This takes advantage of Copy On Write
# process behavior so workers use less memory.
#
# preload_app!
# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
Applications running on Heroku are containerized, and running an SMTP server in or with the web process is not possible.
You need to instead look at services that provide inbound mail delivery. If you're using Rails 6, follow the documentation to set up ActionMailbox.
We have a Rails app on Heroku with Sidekiq and are running out of database connections.
ActiveRecord::ConnectionTimeoutError: could not obtain a database
connection within 5.000 seconds (waited 5.000 seconds)
Heroku stuff:
Database plan: Standard0 (120 connections)
Web dynos: 2 Standard-2X
Worker dynos: 1 Standard-2X
heroku config:
MAX_THREADS: 5
(DB_POOL not set)
(WEB_CONCURRENCY not set)
Procfile:
web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq
database.yml:
...
production:
url: <%= ENV["DATABASE_URL"] %>
pool: <%= ENV["DB_POOL"] || ENV['MAX_THREADS'] || 5 %>
puma.rb:
# https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#adding-puma-to-your-application
workers Integer(ENV['WEB_CONCURRENCY'] || 2)
threads_count = Integer(ENV['MAX_THREADS'] || 2)
threads threads_count, threads_count
preload_app!
rackup DefaultRackup
port ENV['PORT'] || 3000
environment ENV['RACK_ENV'] || 'development'
on_worker_boot do
# Worker specific setup for Rails 4.1+
# See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot
ActiveRecord::Base.establish_connection
end
sidekiq.yml:
---
:concurrency: 25
:queues:
- [default]
We also have a couple of rake tasks that fire every 10 minutes, and they finish within a second or two.
The problem seems to happen when we do a lot of message processing in sidekiq. We do something like:
get article headlines from a 3rd party web service
insert each headline into the db inside a single transaction
create a message in sidekiq for each headline (worker.perform_async)
each message is processed, hits an endpoint to get the body and updates the body (can take .5 - 3 seconds)
While number 4 is happening we see the connection issue.
My understanding is we are way, way, way below the connection limit with our configuration above, but did we do something incorrectly? Is something just consuming the pool? Any help would be great, thanks.
Sources:
https://devcenter.heroku.com/articles/concurrency-and-database-connections
https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server
https://github.com/mperham/sidekiq/wiki/Advanced-Options
You are sharing 5 DB connections among 25 Sidekiq threads. Set DB_POOL to 25 or Sidekiq's concurrency to 5.