I am working on rails 3.1.10. Ruby 1.9.2.
I want to trigger a method after creation of every order(I am using spree). I want to do it in such a way that the rendering is not stalled and application is not slowed down.
I know I can use delayed_job, but I have found it to be unreliable. In past I have seen jobs not getting added to the delayed_job queue.
So, I have decided to use SQS by AWS. I want to push the order details to SQS as soon as order gets created. What is the best way of doing this?
Thanks
Related
I am developing a website using Ruby on Rails and I am doing a bit of rough planning. I can and have deployed rails websites before, just adding to the database and retrieving from database based on my use case, but this time around, its a bit different. I am adding to database but i will need the data to be processed on the server before the data is being sent back to the user or when he decides to retrieve it. What i do not get is how i am going to process the data on the server. I know this doesnt follow the normal pattern for asking questions, i would search for it with google except I dont know what I am looking for. A nudge in the right direction will do.
What I want to do exactly is have users register and click a button (request) which puts the users id in an array , what I need to do on the server is to randomly or not randomly connect two users based on some qualities, this program keeps running infinitely, such that the user can come back later to check if he has been connected with someone already.
This kind of logic typically belongs in the controller, or perhaps on the models. You should read the Rails docs, particularly on controllers: http://guides.rubyonrails.org/action_controller_overview.html
I think you may find a lot of benefit from running a background job for this that is constantly looking for matches. You could have an infinitely running Sidekiq process that is queued up with users. Then once one finishes, just fire it up again.
Or you could create a rake task that does a User.find_each and have it run again when the task finishes. But this would make things blocking if you end up having a lot of users. I'd recommend one job per user and just bloat the system with them. This way you can scale out both horizontally and vertically.
You'll want to learn about ActiveJob and Sidekiq to help achieve what you're looking for :). Sidekiq requires Redis which you'll also have to setup as well. I'd recommend the redis-rails gem to help with the integration.
To go off BenMorganIO's answer, I think this is a job for a background worker. This is a job that is processed in the background, so it doesn't slow up your app. A good example of this is firing off an email in the background.
There are primarily 3 gems I've seen for this:
delayed_job
Resque
Sidekiq (just celebrated 5 years!)
Those should point you in the right direction.
I am trying to find out the best way to run scripts in the background. I have been looking around and found plenty of options, but many/most seem to have become inactive in the past few years. Let me describe my needs.
The rails app is basically a front-end to configure when and how these scripts will be run. The scripts run and generate reports and send email alerts. So the user must be able to configure the start times and how often these scripts will run dynamically. The scripts themselves should have access to the rails environment in order to save the resulting reports in the DB.
Just trying to figure out the best method from the myriad of options.
I think you're looking for a background job queuing system.
For that, you're either looking for resque or delayed_job. Both support scheduling tasks at some point in the future -- delayed_job does this natively, whereas resque has a plugin for it called resque_scheduler.
You would enqueue jobs in the background with parameters that you specify, and then at the time you selected they'll be executed. You can set jobs to recur indefinitely or a fixed number of times (at least with resque-scheduler, not sure about delayed_job).
delayed_job is easier to set up since it saves everything in the database. resque is more robust but requires you to have redis in your stack -- but if you do already it's pretty much the ideal solution for your problem.
I recently learned about Sidekiq, and I think it is really great.
There's also a RailsCast about it - Sidekiq.
Take a look at the gem whenever at https://github.com/javan/whenever.
It allows you to schedule tasks like cron jobs.
Works very well under linux, and the last commit was 14 days ago. A friend of mine used it in a project and was pretty satisfied with it.
edit: take a look at the gem delayed_job as well, it is good for executing long tasks in the background. Useful when creating a cron job only to start other tasks.
I am writing a Web app that will need to run a background process that will poll a web service every minute or so and then query my Rails db and send out alerts to users of the app via Twitter. I have researched this a lot but I feel I am just going around in circles. I have come across delayed_job, background_job and a few other options like creating a custom daemon suggested in a Railscast. Does anyone have any suggestions for the best way to do this? The process will have to run constantly in the background and won't be triggered by an event in the front end. Any help or guidance would be appreciated.
Why don't you just create a rake task and add it to your CRON execution?
You can even use Whenever to configure this for you.
I used Beanstalkd for this and can recommend it.
http://railscasts.com/episodes/243-beanstalkd-and-stalker
You can simply use cron for tasks that has to be executed every X minutes, hours etc.
gem whenever is usefull to setup this with rails: https://github.com/javan/whenever
I don't know much about delayed_job. But you can check out some tutorials, for example this article on heroku: http://devcenter.heroku.com/articles/delayed-job
I used delayed_job for our application.
While working on this, we researched many sites and finally we are able to apply it.
We apply our experiences in the following link
http://www.kyybaventures.com/blog/rails-delayed-job#more-2916
Hope this will help to get started with background process in rails 3.
We can either use backgroundrb or unix crontab.
Crontab will do the job if you don't want to send any heavy loaded process to run asynchronously during the request process cycle of the application.
Backgroundrb consumes lot of memory and cpu in production environment if any of the process hangs out. Also we need to configure a monitor tool to make sure that the background process is running.
I have a long running operation that I activate by placing a message on a ruby resque queue. The endpoint does the work which might take many minutes.
I was going to periodically poll the database every few second but I think the world has moved on from polling. Is there a better way using ruby on rails to get the results as a push, perhaps using something like web sockets or comet?
Can anyone suggest anything to start my research?
Websockets is the thing to research, and Pusher is a good way to get started with it.
em-websocket..or websocket-rails
I need to build a background job that goes through a list of RSS feeds and analyze them say every 10 minutes.
I have been using delayed_job for handling background jobs and I liked it a lot. I believe though that it's not built for recurring background jobs. I guess I can auto-schedule background job at the end of everyone (maybe with begin..rescue just to ensure it gets executes). Or preschedule say a month of advance worth of jobs and have another one that reschedule the every month..etc
This raised some concerned to me as I started asking myself: what if the server goes down in the middle of execution and the jobs didn't get scheduled?
I have also looked at Daemons gems which seemed the like it runs simple Ruby scripts with start/stop commands. I like the way delayed_job schedules and handles retries.
What do you recommend using in this case? What do you think the best way to design such a system with recurring background jobs? Also do you know a way I can monitor that background process and get notified if it stops?
I just implemented delayed_job for a similar task (using :run_at => 2.days.from_now) and found it to be a perfect fit. The easiest way to handle your concern about a process failing is to make the first step of the job to create the next job. Also, you can create a has_many relationship to the delayed_job model which would allow you to access the :last_error. Or, look at the "Hooks" section of readme and it has a perfect example for failure.
I think that this was a similar question: A cron job for rails: best practices? - not only are there answers, but also links to railscasts about background jobs in rails.
I used cron + delayed_job, but scheduled tasks were supposed to run few times a day, mostly just once.
Take a look at SimpleWorker. It's an elastic scheduling and background processing worker queue. It's cloud based and has persistence and redundancy so you don't need to worry if your servers go down or are restarted.
Very flexible in terms of scheduling, provides great introspection of jobs in the queue as well as notifications on status and errors.
Full disclosure: I work at SimpleWorker.