Is there a way to automate and schedule an API request with Rails? I'd like to make a request and save that information into my database, however I'd probably only need it to make a request every few hours for the most up to date info. It's a good amount of data, so I'd like to have it stored as opposed to making a request every time a user visits.
Is there some kind of rake task that I can set up to do this for me on a schedule (or an alternative for what I'm hoping to accomplish)?
Thank you!
The clockwork or whenever gems are made for running scheduled tasks.
If you're hosted on Heroku, you can also use the Heroku Scheduler add-on to execute tasks in your Rails app every 10 minutes, every hour or daily.
The Above solutions are good and may suit your needs. There are more sophisticated queuing and scheduling solutions based on Redis that support multiple retries, have monitoring interfaces etc.
I have had good experiences with Resque and https://github.com/resque/resque-scheduler
If you can be sure your code is threadsafe there is also Sidekiq https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs
I'd check out Sidekiq. It uses redis and therefore supports retries like errata mentioned.
https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs
Related
I want to send email batch at specific time like CRON.
I think whenever gem (https://github.com/javan/whenever) is not to fit in Cloud Foundry Environment. Because Cloud Foundry can't use crontab.
Please inform me what options are available to me.
There's a node.js app here that you could use to schedule a specific rake task.
I haven't worked with cloudfare so I'm not sure if it'll serve your needs, but you can also try some of the batch job processing tools rails has available: Delayed job and sidekiq. Those store data for recurring jobs either on your database (DJ) or in a separate redis database (Sidekiq) and both need keeping extra processes up and running, so review them deeply and the changes you'd need for your deployment process before using each one. There's also resque, and here's a tutorial to use it with rails for scheduling tasks.
There are multiple solutions here, but the short answer is that whatever you end up doing needs to implement its own scheduler. This is because there is no cron service available to your application when it runs on CF. This means there is nothing to trigger or schedule your actions. Any project or solution that depends on cron will not work when deploying to CF. Any project that implements it's own scheduler should work fine.
Some specific things I've seen people do successfully:
Use a web service that sends HTTP requests to your app on predefined intervals. The requests trigger your action. It's the services responsibility to let you define when to trigger and to send the HTTP requests. I'm intentionally avoiding mentioning any specific services, but you can find them by searching for "cron http service" or something like that.
Importing a library that has cron like functionality. I'm not familiar with Ruby, so I don't know the landscape there. #mlabarca has mentioned a couple that you might try out. Again, look to see that they implement the scheduling functionality and do not depend on cron. I'm more familiar with Java where you have Quartz and Spring, which has some scheduling functionality too.
Implement a "clock" process or scheduler. This would generally be a second app that you deploy on CF. It would be lightweight and probably not have a web interface. It could be as simple as do something, sleep, loop for ever repeating those two steps. It really depends on your needs. You could even get fancy and implement something like the first option above where you're sending some sort of request to your other apps to trigger the actual events.
There are probably other solutions as well, those are just some examples to get you started.
Probably also worth mentioning that the Cloud Controller v3 API will have first class features to run tasks. In this case, the "task" is some job that runs in a finite amount of time and exits (like a batch job). This is opposed to the standard "app" that when run on CF should continue executing forever (i.e. if it exits, it's cause of a crash). That said, I do not believe it will include a scheduler so you'd still need something to trigger the task.
My application (in Rails) should provide two main tasks:
create activity (for example post to Twitter) in given time in the future
periodically crawler some site or download Tweets
I'm thinking about using DelayedJob gem or Whenever gem for cron tasks. Which is better in these situations?
Thanks for any advice.
create activity (for example post to Twitter) in given time in the future
If this is in response to an event/user action, then a background task would be ideal, as it's not a regular schedule. Ruby toolbox seems to be favouring Resque and Sidekiq over delayed job though, so have a look at those before settling.
periodically crawler some site or download Tweets
Converse to the above, if this is a regularly scheduled event then cron jobs are ideal. Nothing wrong with using whenever for this, but make sure you have some form of monitoring on the jobs to alert you if something goes wrong.
For creating the activity, I would definitely use a background job to do that (i.e. PostToTwitterJob). I have a preference for Sidekiq instead DelayedJob, as delayed_job does not seem to be active anymore. If you want the job to be performed at a specific time in the future, check out sidekiq's documentation.
As for the crawling, I would use both. Your crawler would be a background job (i.e. TweetsCrawlerJob), and you could launch it every hour with whenever.
I am not sure if this question is of the correct format for SO.
I have a rails app with deployment on Heroku. In development I am using the "crono" gem to send a simple email out every week to remind users of various things. I can see how to use this in production with Heroku. There is almost nothing on this in a google search. Therefore my question is: "what is the best way to implement a simple weekly job in a rails app that is deployed on Heroku. Heroku has "scheduler" but this is a "best effort" add on that can claim to be reliable (according to their documentation)
Thanks
There's two ways to achieve what you want:
1. Use the Heroku scheduler (I would)
Honestly it's just so simple to set up, I would use this. It's also extremely cheap because you only pay for the dyno while the job is running. After it's run Heroku destroys the dyno (it's a one off dyno)
The best way to implement it is to have you background jobs callable by a rake task, and simply have the scheduler call that.
If you have a time interval Heroku doesn't support, simply handle that in your code. For example if you want to send e-mails once a week, have a timestamp to record when the last email was sent. Run the scheduler once a day and just check to see if it's ok to send another email, if not do nothing.
2. Use some kind of scheduler gem
There's a bunch of them out there. For example, rufus.
In this case, you'd write your rufus jobs. You would then need to have a worker dyno always running for rufus, this is much more expensive.
You'd tell the dyno to run rufus and keep running rufus by specifying the command rufus needs in your procfile. Something like:
scheduler: rake rufus:scheduler # Add rake task for rufus scheduler process
(credit for the above snippet How to avoid Rufus scheduler being called as many times as there are no.of dynos?)
I'm working on the billing system for my app, but I'm not sure how to go about setting up scripts that run daily, one to send out payment reminders (email), and another to downgrade subscriptions that have not been renewed.
Any tips on how to do this?
Any gotchas I need to watch out for?
Any gems that I might be able to use?
I've completed most of the purchasing side already, so am not looking for gems like paypal_recurring, or stripe - just need to handle the payment reminders and dealing of accounts that have expired.
I've done this a number of ways and there are a bunch of ways to do it. I think the current best practice is to use Resque to queue the jobs and Rescue Scheduler to schedule them.
Resque is a solid 'job scheduling' app that can handle all kinds of tasks for you. Resque Scheduler can be fed full cron expressions (in addition to other methods of scheduling tasks) to manage the jobs.
One of the advantages of this approach is that you get the Resque Web app that gives you a web app to use for monitoring the jobs (or launching them for one-off job runs).
I've used this approach with Heroku and it works well and has been reliable.
Whenever at https://github.com/javan/whenever is a nice way to define scheduled tasks using cron. Or you can use cron directly: http://www.ameravant.com/posts/recurring-tasks-in-ruby-on-rails-using-runner-and-cron-jobs
What is the preferred way to create a background task for a Rails application? I've heard of Starling/Workling and the good ol' script/runner, but I am curious which is becoming the defacto way to manage this need?
Thanks!
Clarification: I like the idea of Rake in Background, but the problem is, I need something that is running constantly or every 10 hours. I am not going to have the luxury of sitting on a web request, it will need to be started by the server asynchronous to the activities occurring on my site.
Ryan Bates created three great screencasts that might really help you:
Rake in Background
Starling and Workling
Custom Daemon
He talks about the various pros and cons for using each one. This should help you get started.
It depends on your needs.
Try out delayed_job, which was created by Tobi delayed_job (last updated 2011), a Shopify founder.
There are forks by DHH deleayed_job (last updated 2008), and collectiveidea delayed_job (last updated 20 days ago as of 6/28/2018).
I usually rely on cronjob scheduling as it gives the flexibility without having to write separate code to schedule it. Anything that can be executed from shell, can be scheduled! Be it any script (ruby / rake task / py / bash / any other you like), cronjob scheduling can be easily achieved.
If running on windows, one can use scheduled tasks
Hope this helps.
async_observer is the best. It doesn't do all kinds of dumb busy wait stuff or lose jobs on worker crashes like starling, no DB polling, etc... and it integrates into rails remarkably well.
I push tons of jobs through it and it pretty much doesn't care.
Most of the plugins that have been mentioned will do the job, but if all you need is a Rake task run on a set schedule, then there's really no need to start throwing more architecture at it.
Just add a cron job which executes
"cd /path/to/rails/app; RAILS_ENV=production rake run:my:task"
Why reinvent the wheel, when Unix like operating systems have been running tasks on a schedule for decades?
I have used the daemons plugin in the past.
While I don't know if it is becoming a standard, I have had great success with BackgroundRB. I have several workers, some are long running tasks triggered by a user action while others are started on a schedule.
Have a look at Taskr. It's basically like cron, but with a RESTful web interface. You can use it to schedule tasks to periodically connect to your Rails app and trigger arbitrary code (via the Taskr4rails plugin). It's meant to fit nicely into a system built around RESTful services, plus it can notify you if a task returns an error, fails to run, etc.