I have a rails app with the whenever gem installed to setup cron jobs which invoke various rake tasks. For reasons unbeknownst to me, each rake task gets invoked twice at precisely the same time. So my db backup task backs up the db twice at 4:00am.
Inspecting crontab reveals correct syntax for all of the cron jobs, so I don't think this is an issue with the whenever gem not correctly configuring the cron jobs. Also confusing is that in both staging and production environments and can invoke tasks on the command line and they only run once.
Any thoughts on what would cause this? I'm at a complete loss troubleshooting wise.
The number of cron jobs that run depends on the number of application instances running in the server box. Are you have two instances of rails application running in the same server box?
Related
It's nice to see Rails 4.2 come with Active Job as a common interface for background jobs. But I can't find how to start a worker in the document. It seems that the document is still immature (e.g. the right version of Sneakers is only referred to in Rails' Gemfile), so I'm not sure if the "running workers" part is not in Active Job or just not mentioned in docs.
So with Active Job, do I still need to manually start the job watcher threads like sidekiq or in my case, rake sneakers:run? If so, where should I put these commands to let rails server run these parallel tasks automatically in a develop environment?
ActiveJob is just a common interface. You still need the backend gem, and you still need to launch it separately from your server (it is a separated process, which is the objective).
Sample using resque:
In the Gemfile:
gem 'resque'
In the terminal, launching a worker:
bin/resque work
The case is similary when using sidekick, delayed job or something else.
If you want to launch the server & worker in a single command, you can create a short bash script for it, but I would advise not doing so: having two separated console helps to watch what is happening on each side (web app & worker).
A better solution would be to use the foreman gem to manage starting & stopping your process.
You can create a simple Procfile with the processes to start:
web: bundle exec rails s
job: bundle exec resque work
And then just start both using foreman:
foreman start
By default, foreman will interleave the logs of the process in the console, but this can be configured.
You still have to run the job thread watcher.
I usually just run "heroku run rake jobs:work" from the command line. The great thing about this apporach is that I get intimidate feedback on whether a job failed or not and what jobs are currently processing.
However, now I need to run "heroku ps:scale worker=1"
Is there a way to see what the worker is processing just like with the rake task via the command line?
You can use the heroku command like to access the log of the worker.
heroku logs -t --ps worker will show you what is currently being executed on your worker.
I also recommend to use gem workless which scales your worker up only when needed. This can save you a lot of money
You didn't say specifically what queue you are running, but it sounds like delayed_job. If so, you can install the delayed_job_web gem. Basic setup is as simple as adding it you your Gemfile and adding a route for it. Then you can browser to your site /delayed_job and manage the jobs.
I had developed an application that has delayed job and cron job and I need to deploy it to Heroku. I realized my delayed job need Heroku adds-on, "Heroku Scheduler Standard" and it costs $34.50.
1) Does that mean I need to buy that so that my delayed and cron job will run automatically?I wonder if delayed job and cron job can be done by using Heroku schedular then why we still need javan/whenever and collectiveidea/delayed_job? Can I use these in Heroku preferably in free condition?
2) My cron job does not worked in Heroku. How do I run my whenever gem in heroku?
There appear to be a couple of thoughts intermixed in your question. I'll do my best to separate them:
a) DelayedJob on Heroku
One way to process jobs queued in DelayedJob (i.e. records sitting in the delayed_jobs table in your database) is to run the following rake task:
$ rake jobs:work
On heroku, this rake task is commonly run via a 'worker' process.
By default for rails apps, your heroku app should already come with a slider for 'workers' (initially set to 0) that will run the rake task above. Merely sliding this to '1' (which will cost you around $34.00 per month) will launch the DelayedJob dequeue process in the background on a dedicated heroku instance, with your codebase, independent of your web dynos.
Note that DelayedJob does NOT require Heroku's "Scheduler" add-on.
Also note that heroku charges by hours of processing per month. So, if you slide your 'worker' to '1' for one hour a month, and then slide it back to '0' again when you're done, you will be paying far, far less than the ~$34 charge. There is a way to slide workers on and off programatically too.
b) Cron on Heroku
As it stands, managing your own, customized cron file is not possible on heroku or other cloud-based app service providers. Everything is ephemeral, and deployed files cannot be altered by you or your code.
Heroku's Scheduler is the appropriate add-on in this case to replace cron. You should be able to set tasks in your heroku scheduler to do what your cron tasks are currently doing.
Last I checked, heroku's scheduler was free (see: https://addons.heroku.com/scheduler), so I'm not sure why you state it is $34.50.
Hope this helps.
I have a few long-running restarts of processes in my deploy.rb like:
rake assets:precompile
script/delayed_job restart
rake sunspot:solr:stop, rake sunspot:solr:start
All of these processes have to occur, but not necessarily one after another.
I was wondering if I can run the assets:precompile and the delayed_job restart simultaneously, as they don't need to happen one after another, and I could speed up my deploy time by doing them asynchronously.
I've run some Google searches but I can't find anything about it.
This is not a feature that capistrano supports.
I have been looking around for a solution and found something on the Capistrano google groups. The suggestion was to use Capistrano to run a ruby script that runs the jobs in parallel using Ruby's own threading support.
If you read the post one of the authors does ask why do these tasks need to run in parallel because you can introduce race conditions and other non-deterministic behaviour which can make the deployment process more brittle.
I would like my development environment for Rails to automatically start redis and resque (and potentially in other projects, mongod, mysql-server etc.) for me, in the following cases:
When starting up the development server rails server.
Additionally, it would be nice if the following cases detect already running services, and, if not running start them up too:
Rake rspec, rspec /spec, when running tests.
When starting up a rails console.
When shutting down the rails server, the started child-services should be shut down too.
What is the correct place for such additional startup scripts?
And how to avoid them being started in production too (where I run everything trough /etc/init.d services)?
A lot of these built-in tasks are available as rake tasks already.
You can create a master rake task that does it all.
For example, with resque, you get "rake resque:start" "rake resque:scheduler:start", etc.
You can create a generic "start" task that depends on the rest. Similarly, a "stop" task would shut everything down.
So you would do:
rake start # starts all associated processes
rake stop # stops them all
This is also very use to use from Capistrano, when you end up deploying your code somewhere else. Rake tasks are very easy to call from Capistrano.
I think it's really better to do that in some external script. Do it in your rails server command can be really annoying to anyone to try your code.
By example, in one year, a nez developper come to your project. He can be desoriented if your rails server commande launch a such of other application in background.
In same idea, if you do that you need maintain your code in your rails env. Can be a little tricky. Maintain an independant script can be more usefull.
You can add your script in script directory. That be a good pratice. But not when you launch a command with a manual who do not that.