I'm using ActiveJob in development (with delayed_job as a queuing backend but I think that is irrelevant) to do things like sending emails asynchronously or logging some events.
In development, in a terminal, I execute rake jobs:work to start the worker process, I use CTRL+C to kill it, and I use rake jobs:clear to empty the queue.
How do I do this in production (a droplet on DigitalOcean) and how do I know that my worker is running even though I'm not logged in on the server? I usually deploy with mina, can I script that in my deploy.rb?
Related
Hi I m new to Rails and was wondering how to configure Heroku in Rails 4 with Puma and sidekiq to run on a single dyno?
Currently my sidekiq jobs are queued but not executed..
Each dyno is a separate process. So you can run only sidekiq or puma on one dyno.
If you want to use free plan, then it is not possible.
However, there was a workaround, when you can create another one app, and run sidekiq there. So your puma and sidekiq will share same redis db to syncronize.
Check this link:
https://coderwall.com/p/fprnhg/free-background-jobs-on-heroku
Let's say my process will run 3 workers, and I want to dedicate 1 of them to process web requests, and 2 to handle Sidekiq background jobs, each process potentially being multi-threaded. Is there an easy or best-practices way to handle this? I've tried a few different configurations, but they either give me an error, or just don't process the jobs at all.
I'm using Rails 4 and ActiveJob, but I don't think those points are relevant.
You don't have to take care of sidekiq worker scheduling from you rails application. Sidekiq runs a separate process with the whole rails environment for managing the background workers. Each Worker is a separate Thread, backed by the Celluloid Actor framework. So for your setup you just do the following:
# start a single puma web server process with min 4 and max 16 Threads
$ bundle exec puma -t 4:16
# start a single multithreaded sidekiq process,
# with councurrency settings defined in sidekiq.yml
$ bundle exec sidekiq --pidfile tmp/pids/sidekiq_1.pid
# start another sidekiq process (if really necessary)
$ bundle exec sidekiq --pidfile tmp/pids/sidekiq_2.pid
I have an application running on Heroku and I have a messaging system in place. I've used the Faye gem, which has a Faye server to handle live messaging notifications.
How can i get this Faye server to run in production on Heroku?
It has it's own port, and ENV variable.
Please advise.
You have to run it as a background job if it is its own server.
https://devcenter.heroku.com/articles/background-jobs-queueing
You may be able to start it from a rake task.
https://devcenter.heroku.com/articles/rake
You can run it on a schedule, even once a month if it runs a long time.
https://devcenter.heroku.com/articles/scheduler
Use heroku config to see environment variables. heroku config:set to assign.
Use heroku run bash to get a shell.
You may have to use an additional worker thread from the Heroku dashboard.
New to Rails and very new to Delayed Jobs.
Got one that's supposed to be triggered after 5 minutes. I finally got it to work so that if I run
rake jobs:work
in my terminal, the job starts up and works correctly. If I CTRL-C and exit that action in my terminal, the delayed job stops working correctly. This is one thing on my local server and another on Heroku, where I have to start up the delayed job using
heroku run rake jobs:work
I looked into the new Heroku toolbelt and downloaded the gem they suggest for worker maintenance, foreman, but when I run "foreman start", I get this error
ERROR: procfile does not exist
I don't know what a procfile is, I'm afraid of breaking things after spending pretty much a day debugging my delayed_jobs actions, and I want to do this right to make sure it works instead of figuring out some hacky fix that breaks down later -- so I figured I should ask this question, however obnoxiously vague it may be.
Should I be using foreman for this? Or workless? (Saw that in another SO question). Where's my procfile? Should I do anything with it?
Thanks,
Sasha
You should be using a procfile to set up your Heroku processes, this is the standard method that Heroku uses to define and control the processes.
If you haven't utilised a procfile to this point everything will probably still work as Heroku adds some default processes when you push a Rails app, including both the web and worker processes. The default worker process is set to delayed job.
Foreman was developed in order to set up your local machine to use the same approach but, unlike the Heroku service, Foreman actually requires a procfile to be present to control the services that are started when Foreman is started as it doesn't know how to setup defaults.
I would suggest creating a procfile, placed in the root directory of your project, to ensure that your processes are set up and operating in the same manner on your local machine as on Heroku. If you want to mimic what Heroku sets up automatically you add the following to the procfile depending on whether you are using the Thin web server (which Heroku recommends) or not.
With Thin in your gemfile:
web: bundle exec thin start -R config.ru -e $RACK_ENV -p $PORT
worker: bundle exec rake jobs:work
Without a special web server (eg you are using webrick, the rails default):
web: bundle exec rails server -p $PORT
worker: bundle exec rake jobs:work
Once this file is in place you can run foreman on your local machine and it will start your web server and delayed_job workers automatically.
Running through this process will only impact starting delayed_job on the local machine. As you are running the exact same command bundle exec rake jobs:work as you are currently using there should be no impact on your dj actions in either locally or on Heroku. Obviously some testing is required to make suer this is actually the case.
Workless is designed to scale workers on Heroku so that you don't have to pay for them when there is no work available. It has no bearing on the procfile or defining how to actually start a dj worker process.
as far as I know, there are 2 version of delayed_job:
original(tobi's) https://github.com/tobi/delayed_job
collectiveidea's fork: https://github.com/collectiveidea/delayed_job
when using the collectiveidea version, you should start it as below:
# Runs two workers in separate processes.
$ RAILS_ENV=production script/delayed_job -n 2 start
I am not familiar with delayed_job on Heroku, please follow its instructions.
I have an app on Heroku running delayed jobs. However at the moment I have to start the job queue running with the terminal command:
heroku rake jobs:work
...but this means when I shut down my terminal the app's delayed job queue shuts down too.
Is there a way I can get Heroku to just always start and run delayed job in the background when the app starts up? Without having to run the command each time and without having it directly linked to my terminal shell?
Thanks very much.
Edit:
It's on the bamboo stack. Upping workers or running rake jobs:work , the delayed jobs runs for a while, but then the queue seems to just stop getting processed. There are no errors in the delayed jobs queue, the workers just stop processing the jobs. It has to explicitly restarted every 5 or 10 mins.
From the docs:
On Heroku's Aspen or Bamboo stack, use heroku workers 1
On the Cedar stack, you put this line in your Procfile:
worker: bundle exec rake jobs:work
And then do heroku scale worker=1.
we use the workless gem with our heroku stack. it starts worker when the delayed_job queue > 0 and quits the worker when delayed_job queue goes to 0.
It turns out that I was using the wrong rake gem.
The following was causing an issues with rails 3 on Heroku:
gem 'rake', '0.9.2'
Updating the gem fixed the issues, even though there were not errors in the log:
gem "rake", "0.8.7"