I was watching the RailsCast on Sidekiq and had some questions:
1) Sidekiq handles tasks through threads instead of processes. What does this mean? Why does it save on memory?
2) Does the method inside the worker class need to have a "perform" method?
3) On the Sidekiq docs, it says:
Start sidekiq from the root of your Rails application so the jobs
will be processed:
bundle exec sidekiq
So if I'm running this on localhost, I can run bundle exec sidekiq. If I pushed up to Heroku, what do I do now? How do I run Sidekiq on Heroku?
4) I am not sure if my Sidekiq is working. I have this code:
def set_defaults
self.clicks = 0 if clicks.blank?
self.title = TitleWorker.perform_async(orig_url)
end
But TitleWorker.perform_asynch(orig_url) in testing just seems to return a string of numbers. What is going on? How do I fix this?
While I'm not super clear about the first question, I might be able to answer the other questions you have:
Yes. It needs a Perform method to start the worker process.
Heroku is just amazing. You don't have to do any special configuration on Heroku for running Sidekiq. All you have to do is, just make sure RedisToGo is installed, and add this to the Procfile:
worker: bundle exec rake jobs:start
sidekiq: bundle exec sidekiq -c 15 -v
Related
I am newbie to Ruby on Rails development
Can someone please give explain to me what this command line do bundle exec rake jobs:work
I don't understand what is worker and what the command line can do.
Can someone gives me some examples.
Thank you
In ruby due to GIL(global interpreter lock), you can run only one ruby thread (multithreading is supported, but it works only if you do IO) at a time. To work around this issue and make things asynchronous, people use sidekiq, delayedjob, etc.
Worker in this terminology, is a separate background ruby process that processes jobs a.k.a task you put in it. And if you use DelayedJob the bundle exec rake jobs:work will start this processes (other gems for background jobs use other commands)
I am using rails 5 and sidekiq (4.1.2) on Heroku
and calling delay on a class method written in my User model
like:
delay.mass_invite_through_csv(mass_invitation.id, current_user, data)
here mass_invitation is a object of MassInvitation Class and current_user is current_user, and data is a hash params.
now this method is getting executed infinitely.
In my Procfile:
web: bundle exec rails server -p $PORT
worker: bundle exec sidekiq -C config/sidekiq.yml
in my config/sidekiq.yml
concurrency: 3
everything is working correctly locally, please help.
Sidekiq retries failed jobs automatically, therefore you have to understand why it is failing first. 2 possible solutions here: tail into sidekiq output either mount sidekiq dashboard to your app and look whats wrong there.
Guide to install the dashboard
Whenever I run heroku run bundle exec sidekiq, I see all my background jobs being done, however, I want them to be able to go without me needing to be there. When I exit out of that terminal tab, sidekiq stops working. How would I mitigate that?
Also, I've read something about procfiles and increasing workers. I don't know what procfiles are and I don't know how to increase workers either.
Basically, I'm a newbie trying to get sidekiq set up to run on Heroku for my Rails app. I want it to be running at all times.
Create a file named ./Procfile with the following in it:
web: bundle exec rails server -p $PORT
worker: bundle exec sidekiq
sidekiq on Heroku
more on Procfiles
foreman gem
I have been operating under 0 workers, but need to increase the speed at which I'm processing background tasks. I am using Sidekiq for all of my background workers.
When I increase the worker dyno count to 1, I keep getting this error in my heroku logs:
dont know how to build task 'jobs:work'
From researching this, it seems like the issue is that heroku worker dynos are reliant on delayed_job and I am not using delayed_job anywhere.
If I install delayed_job, what will I have to change to get sidekiq to work? Or do I even need delayed_job?
Update your projects Procfile to specify sidekiq for the worker:
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec sidekiq
Then redeploy your application.
I think Heroku defaults to trying to run delayed_job if you don't specify the worker in your Procfile.
New to Rails and very new to Delayed Jobs.
Got one that's supposed to be triggered after 5 minutes. I finally got it to work so that if I run
rake jobs:work
in my terminal, the job starts up and works correctly. If I CTRL-C and exit that action in my terminal, the delayed job stops working correctly. This is one thing on my local server and another on Heroku, where I have to start up the delayed job using
heroku run rake jobs:work
I looked into the new Heroku toolbelt and downloaded the gem they suggest for worker maintenance, foreman, but when I run "foreman start", I get this error
ERROR: procfile does not exist
I don't know what a procfile is, I'm afraid of breaking things after spending pretty much a day debugging my delayed_jobs actions, and I want to do this right to make sure it works instead of figuring out some hacky fix that breaks down later -- so I figured I should ask this question, however obnoxiously vague it may be.
Should I be using foreman for this? Or workless? (Saw that in another SO question). Where's my procfile? Should I do anything with it?
Thanks,
Sasha
You should be using a procfile to set up your Heroku processes, this is the standard method that Heroku uses to define and control the processes.
If you haven't utilised a procfile to this point everything will probably still work as Heroku adds some default processes when you push a Rails app, including both the web and worker processes. The default worker process is set to delayed job.
Foreman was developed in order to set up your local machine to use the same approach but, unlike the Heroku service, Foreman actually requires a procfile to be present to control the services that are started when Foreman is started as it doesn't know how to setup defaults.
I would suggest creating a procfile, placed in the root directory of your project, to ensure that your processes are set up and operating in the same manner on your local machine as on Heroku. If you want to mimic what Heroku sets up automatically you add the following to the procfile depending on whether you are using the Thin web server (which Heroku recommends) or not.
With Thin in your gemfile:
web: bundle exec thin start -R config.ru -e $RACK_ENV -p $PORT
worker: bundle exec rake jobs:work
Without a special web server (eg you are using webrick, the rails default):
web: bundle exec rails server -p $PORT
worker: bundle exec rake jobs:work
Once this file is in place you can run foreman on your local machine and it will start your web server and delayed_job workers automatically.
Running through this process will only impact starting delayed_job on the local machine. As you are running the exact same command bundle exec rake jobs:work as you are currently using there should be no impact on your dj actions in either locally or on Heroku. Obviously some testing is required to make suer this is actually the case.
Workless is designed to scale workers on Heroku so that you don't have to pay for them when there is no work available. It has no bearing on the procfile or defining how to actually start a dj worker process.
as far as I know, there are 2 version of delayed_job:
original(tobi's) https://github.com/tobi/delayed_job
collectiveidea's fork: https://github.com/collectiveidea/delayed_job
when using the collectiveidea version, you should start it as below:
# Runs two workers in separate processes.
$ RAILS_ENV=production script/delayed_job -n 2 start
I am not familiar with delayed_job on Heroku, please follow its instructions.