If I add the following lines in application.rb
puts 'in application.rb'
pid = spawn('rake jobs:work')
Process.detach pid
I see the following output
in application.rb
=> Booting Puma
...
* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
in application.rb
[Worker(host:local pid:9966)] Starting job worker
in application.rb
[Worker(host:local pid:9998)] Starting job worker
in application.rb
[Worker(host:local pid:10032)] Starting job worker
If I remove the spawn call in application.rb is displayed only once at the beginning, as expected.
This output is written at about once per second. All those processes are healthy and will be stoped when I killed puma.
I can't figure out what is happening. Why is this code getting executed every second?
... rake is also requiring the file, that's why.
To get this to work as expected:
if $0 =~ /rails/
pid = spawn('rake jobs:work')
Process.detach pid
end
Related
I am trying to use MidiSmtpServer to receive email in a Heroku application, and have been using the code on one of the examples that the documents show. However, I don't know where to put that code for the SMTP server to start after Puma, or where to put it for it to start at all. Using on_worker_boot in puma.rb doesnt work.
puma.rb:
# Puma can serve each request in a thread from an internal thread pool.
# The `threads` method setting takes two numbers: a minimum and maximum.
# Any libraries that use thread pools should be configured to match
# the maximum value specified for Puma. Default is set to 5 threads for minimum
# and maximum; this matches the default thread size of Active Record.
#
max_threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
min_threads_count = ENV.fetch("RAILS_MIN_THREADS") { max_threads_count }
threads min_threads_count, max_threads_count
# Specifies the `port` that Puma will listen on to receive requests; default is 3000.
#
port ENV.fetch("PORT") { 3000 }
# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch("RAILS_ENV") { "development" }
# Specifies the `pidfile` that Puma will use.
pidfile ENV.fetch("PIDFILE") { "tmp/pids/server.pid" }
# Specifies the number of `workers` to boot in clustered mode.
# Workers are forked web server processes. If using threads and workers together
# the concurrency of the application would be max `threads` * `workers`.
# Workers do not work on JRuby or Windows (both of which do not support
# processes).
#
# workers ENV.fetch("WEB_CONCURRENCY") { 2 }
require "midi-smtp-server"
require "mail"
on_worker_boot do
class MySmtpd < MidiSmtpServer::Smtpd
def on_message_data_event(ctx)
puts "[#{ctx[:envelope][:from]}] for recipient(s): [#{ctx[:envelope][:to]}]..."
# Just decode message ones to make sure, that this message ist readable
#mail = Mail.read_from_string(ctx[:message][:data])
# handle incoming mail, just show the message source
puts #mail.to_s
end
end
# try to gracefully shutdown on Ctrl-C
trap("INT") do
puts "Interrupted, exit now..."
exit 0
end
# Output for debug
puts "#{Time.now}: Starting MySmtpd..."
# Create a new server instance listening at localhost interfaces 127.0.0.1:2525
# and accepting a maximum of 4 simultaneous connections
server = MySmtpd.new(2525, "0.0.0.0", 4)
# setup exit code
at_exit do
# check to shutdown connection
if server # Output for debug
puts "#{Time.now}: Shutdown MySmtpd..." # stop all threads and connections gracefully
server.stop
end # Output for debug
puts "#{Time.now}: MySmtpd down!\n"
end
# Start the server
server.start
# Run on server forever
server.join
end
# Use the `preload_app!` method when specifying a `workers` number.
# This directive tells Puma to first boot the application and load code
# before forking the application. This takes advantage of Copy On Write
# process behavior so workers use less memory.
#
# preload_app!
# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
Applications running on Heroku are containerized, and running an SMTP server in or with the web process is not possible.
You need to instead look at services that provide inbound mail delivery. If you're using Rails 6, follow the documentation to set up ActionMailbox.
I am working on importing data from web based CSV into database, so I created a rake task that imports data into database. However I tried to make running my rails app a little seamless and I integrated the import rake task and running rails server into foreman.
However, when I run foreman start, the processes start but terminates after the rake task finishes. I also will like that rake task to start first before running rails s
Here is what I have done below:
lib/tasks/web_import.rake
require 'open-uri'
require 'csv'
namespace :web_import do
desc 'Import users from csv'
task users: :environment do
url = 'http://blablabla.com/content/people.csv'
# I forced encoding so avoid UndefinedConversionError "\xC3" from ASCII-8BIT to UTF-8
csv_string = open(url).read.force_encoding('UTF-8')
counter = 0
duplicate_counter = 0
user = []
CSV.parse(csv_string, headers: true, header_converters: :symbol) do |row|
next unless row[:name].present? && row[:email_address].present?
user = CsvImporter::User.create row.to_h
if user.persisted?
counter += 1
else
duplicate_counter += 1
end
end
p "Email duplicate record: #{user.email_address} - #{user.errors.full_messages.join(',')}" if user.errors.any?
p "Imported #{counter} users, #{duplicate_counter} duplicate rows ain't added in total"
end
end
Procfile
rake: rake web_import:users
server: rails s
when I run forman start, the image below shows the process
I will like the rake task in the foreman to run first before running rails s command. I also don't want it to terminate by itself. I don't know what am doing wrong.
Any help is appreciated.
I solved this by refactoring the Procfile. Instead of having two tasks, I merged it to just one command using && so I can determine which command takes the prefix and which one takes the suffix.
So I changed the profile to:
tasks: rake web_import:users && rails s -p 3000
With this I have my import run first and server command being the last.
If you noticed, I added port with -p flap so as not make sure server is listening on port 3000. Note adding port is optional.
I hope this helps someone as well.
I'm using websocket_rails to provide an API for a JS client. Locally it works great, but the exact same setup in production will (seemingly randomly) decide to stop working.
My production.log yields RuntimeError (eventmachine not initialized: evma_install_oneshot_timer)
At first I thought this was the root issue, but my Puma error log yields this when restart the server and try again: RuntimeError: async response must have empty headers and body
I added some logging in the puma gem, and indeed, it's receiving rails session headers when doing GET /websocket
Sometimes there is no issue at all, and everything works fine for a few days, and then, not. And no matter what I do it just refuses to work again.
Thanks in advance. I've wasted days on this problem!
Puma config:
# Change to match your CPU core count
workers 1
# Min and Max threads per worker
threads 1, 6
app_dir = File.expand_path("../..", __FILE__)
shared_dir = "#{app_dir}/shared"
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env
# Set up socket location
bind "unix://#{shared_dir}/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require "active_record"
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[rails_env])
end
I'm trying to do enqueue a simple job using Resque 1.26.0 (and Redis-rb 3.3.1). The job doesn't seem to be processing the perform function because resque-web is processing each job and shows 0 failures. The jobs also are being processed instantly.
The jobs are enqueued from a controller action with
Resque.enqueue(TestJob, url)
The job itself looks like
class TestJob < ApplicationJob
#queue = :tags_queue
Logger.new("log/resque_worker_QUEUE.log").fatal("thing")
def self.perform(url)
Logger.new("log/resque_worker_QUEUE.log").fatal("other thing")
logger.fatal("more errors please")
myDivideByZeroVar= 1/0
raise "error"
Logger.new("log/resque_worker_QUEUE.log").fatal("other thing")
logger.fatal("more errors please")
end
end
A rake task is also set up:
require 'resque/tasks'
task "resque:setup" => :environment
The redis-server is running.
The worker is started with rake resque:work QUEUE=*. Using verbose logging doesn't show anything useful.
The log file only shows the first fatal error string "thing". None of the other errors are logged that are inside perform.
What am I doing wrong here?
Solved this. The job needed to be called using ActiveJob instead of using Resque.enqueue. TestJob.perform_later(url) worked just fine.
I am calling function with LoadData.send_later(:test).
LoadData is my class and test is my method.
It's working fine while i am running rake jobs:work.
But when i am running script/delayed_job start or run that time delayed_job.log shows error like
TEastern Daylight Time: *** Starting job worker delayed_job host:KShah pid:5968
TEastern Daylight Time: * [Worker(delayed_job host:KShah pid:5968)] acquired lock on LoadData.load_test_data_with_delayed_job
Could not load object for job: uninitialized constant LoadData
TEastern Daylight Time: * [JOB] delayed_job host:KShah pid:5968 completed after 0.0310
TEastern Daylight Time: 1 jobs processed at 10.6383 j/s, 0 failed ...
Any solution??
Try putting include LoadData in an initializer. I seem to remember DelayedJob including activerecord classes, notifiers etc, but not custom classes. Personally I'd put the class in your models directory. It's still dealing with data, even if it's not activerecord.
Try doing this:
Delayed::Job.enqueue LoadData.test
Also, a big gotcha that took me while to realize... if you make changes to the code restart rake jobs:work!