I have a Rails app set up with a model Account that should be updated every morning with data coming from an external API I'm calling (a CRM). Basically either I create new accounts in my app that I find in the CRM and some of the fields that are mapped with my columns, either I find the account if it already exists and I update it.
So far, I've been putting this code into the seeds.rb file and from Heroku, where the app is hosted, I set up a scheduler with the command : rails db:seed that runs periodically.
My issue is that I'm sure there is a better way of doing this. I've read about rake tasks but I did not quite understand how that applied to my case. Otherwise I thought of putting my method in the models/account.rb file as a self method. But I don't really know how I can invoke it in a rake command to allow me to set up a scheduler in Heroku.
Any idea on where would be the best place to put this method, and how to call it from command line?
Thanks in advance.
You can create a script directory in your project, and put your script from db/seeds.rb into this directory, maybe called update_accounts.rb. Then you can run it with
rails runner script/update_accounts.rb
and schedule that task in heroku. More info about rails runner here.
I would suggest using a background processor such as Sidekiq: https://github.com/mperham/sidekiq
Once using Sidekiq, you need a scheduler like https://github.com/moove-it/sidekiq-scheduler to make sure it happens periodically as you require.
This will become easier to maintain as your application grows and you need more workers. It also moves your scheduling into version control.
Related
I have a rails app that only allows users to search and view the contents of its (MySQL) database. To add records to the database, I'd like to run a ruby script automatically twice a day which generates the contents of the database and then updates the db. Is this possible, or does my script have to be a part of the rails web app?
Just to lead you in the right direction - you can use a Rake task for this. These are usually put in the lib/tasks directory.
It's a good idea to separate non-web related things such as seeding or updating a database. If this is a scheduled job, you can use whenever to hook the rake task to cron.
Solved this using rails runner command
I am building a script in on of my controllers to fill a database with an excel files data. I would build the function, then access it through a route. (That i guess i can protect with cancan) But i thought about it, and it doesn't seem very ... 'Railsy'.
I know the scripts folder exists, and it is probably for these kinds of tasks. I've tried googling stuff like 'rails execute script' and other stuff, but i can't find any good advice for what to do next.
I'm sorry if this seems kind of stupid, but in my apps i've been kind of hacking around stuff to make it work, so any advice on this task would be appreciated.
If you need to upload the file in the app and process it, it should probably go in the "lib"directory and be accessed like any other Ruby library/module/etc.
If it's something you need to run locally, "on demand", "scripts" is fine. If you need access to your rails environment when running it like any Rails models, you can run it from "rails console" or "rails runner".
As Aln said, there are a variety of ways it could be scheduled as well.
You could simply do
#!/usr/bin/env ruby
require 'rubygems'
# regular ruby code here
and have it running just like any other util. Of course you can always call any *.rb with simply
ruby somescript.rb
If you need some scheduled script, check into rufus-scheduler gem.
This may be a stupid question but I was just wondering where, or if its possible to run a ruby script which is kind of unrelated to the rails application I would like it to run in. To clarify, I am working on an automation test suite that is written mainly in bash, but I want to create a front end (my rails application) that allows other users to run automated tests not through the command line. So I guess basically I want a user to select certain parameters, from a database or form fields, then take those parameters and pass them to a ruby script which calls my bash automation script.
I hope this is clear. Thanks!
If you want to call a script from a rails app it gets complex. You would want to use a background job or some sort of queue to run these jobs because they do block the server and your users would be waiting for the call to complete and the results to load, most likely hitting a timeout.
See delayed_job
and you might want to try creating a small wrapper script in ruby that can interface with your application.
Good luck!
for short tasks you should use system or popen
when tasks are longer then they are still needed in case of delayed_job
You can add a script to your scripts folder in the root of your rails app. Start your script like this:
your script can be [name here].rb
The reason why we load in the environment is so we can use rails models and rails related things in your script:
#!/bin/env ruby
ENV['RAILS_ENV'] = "production" # Set to your desired Rails environment name
require '/[path to your rails app on your server]/config/environment.rb'
require 'active_record'
If you want to run this on your server, then you have to edit your crontab on your server. Or you can use the whenever gem (which I''m having trouble with, but the entire universe doesn't). Conversely, if you have heroku, then there's the heroku scheduler that makes running scripts easy.
You can run Ruby code with rails runner.
… let us suppose that you have a model called “Report”. The Report model has a class method called generate_rankings, which you can call from the command line using
$ rails runner 'Report.generate_rankings'
Since we have access to all of Rails, we can even use the Active Record finder method to extract data from our application.
$ rails runner 'User.pluck(:email).each { |e| puts e }'
charles.quinn#highgroove.com
me#seebq.com
bill.gates#microsoft.com
obie#obiefernandet.com
Example taken from The Rails 5 Way by Obie Fernandez.
I am currently trying to automate the deployment process of our rails app as much as possible, so that a clean build on the CI server can trigger an automated deployment on a test server.
But I have run into a bit of a snag with the following scenario:
I have added the friendly_id gem to the application. There's a migration that creates all the necessary tables. But to fill these tables, I need to call a rake task.
Now, this rake tasks only has to be called once, so adding it to the deployment script would be overkill.
Ideally, I am looking for something like migrations, but instead of the database, it should keep track of scripts that need to be called during a deployment. Does such a beast already exist?
Looks like after_party gem does exactly what you want.
I can't think of anything that does exactly what you want, but if you just need to be able to run tasks on remote servers in a one off fashion you could always use rake through capistrano.
There's an SO question for that here: How do I run a rake task from Capistrano?, which also links to this article http://ananelson.com/said/on/2007/12/30/remote-rake-tasks-with-capistrano/.
Edit: I wonder if it's possible to create a migration which doesn't do any database changes, but just invokes a rake task? Rake::Task["task:name"].invoke. Worth a try?
I would consider that running that rake task is part of the migration to using friendly_id. Sure, you've created the tables, but you're not done yet! You still have to do some data updates before you've truly migrated.
Call the rake task from your migration. It'll update the existing data and new records will be handled by your app logic in the future.
I am currently writing an application that pulls new information from RSS sources and has to update those RSS sources in a certain frequency. Currently I am pulling only when the user requests a feed but I want to change that behavior to automatic periodic fetching.
I was writing a shellscript that would interact with the database and gets started periodically via cron - but this is lots of double effort so I was wondering what would be the "Rails Way" or "Ruby Way" to do this. I am using Ubuntu, Apache and Passenger. Can you suggest better methods that are maybe even included in the application, so I can easily deploy the app to another machine without having to mingle with cron?
I would suggest doing something like a rake task and using the whenever gem to generate your cron job to run the rake task.
Check out, http://railscasts.com/episodes/164-cron-in-ruby, for more information on the whenver gem.
The main benefit of the whenever gem is that it keeps your application requirements (i.e. the cron job running every x hours, in the application) inside your application, increasing the portability of your application.
I recommend a combination of the two above. You want a rake task, even if you have a direct method already created. This is because server admin stuff that you'd want to run in cron, you might also want to run from the command line occasionally, and this is what rake tasks are good for.
The whenever plugin sounds cool, although I can't vouch for it. Of course, it's good to know how to do things from scratch, then use plugins to make your life easier. Here's the from-scratch way.
Create a new file, lib/tasks/admin.rake
Inside, create the task itself:
namespace :admin
desc "Updates all RSS feeds"
task :rss => :environment do
RssFeed.update_all
end
end
This assumes you have an RssFeed class, and the update_all method does what you'd expect. You can call this from the command line:
rake admin:rss
And you can add this to cron (by calling crontab -l as the web user) and adding this line:
10 0 * * * cd /path/to/rails/app && rake RAILS_ENV=production admin:rss
There are a variety of solutions. For the simplest setup, you can use script/runner in your crontab something like so:
10 0 * * * /home/myuser/myproject/script/runner -e production ModelName.methodname
Methodname must be a static method on your model. You need to reference the project by full path, otherwise it will not be found most likely in the cron environment. Check your crontab man page for info on the crontab syntax if you're not familiar. The above, for example, runs the script at the 10th minute of the 0th hour of every day (at 12:10am, in short).
If you need a more powerful solution, you could use BackgroundRB. BackgroundRB runs a daemon and supports tasks that schedule, and can put results in a database. They even have a simple communication protocol to allow your web processes to request a task be completed, and then have a way to retrieve the result. This allows you to control background jobs right from the web interface, rather than a crontab which just "happens".
There is a good bit more setup needed for BackroundRB to work, but it may be worth it if jobs need to be controlled.
Try using whenever. Eventhough in the end it will create a cron, but the scheduling definition will be written inside your application using Ruby DSL.
For small teams and personal projects, the whenever gem is great. But if your company has an ops team separate from the development team, it might not be ideal.
At my last job, the ops team needed to be able to see the cron we were installing so they could be confident it wouldn't have any side effects for the system. So a DSL solution wasn't going to work. But we (the developers) wanted the cron scripts in version control.
So to compromise, we checked text files with the raw cron, similar to this:
10 0 * * * cd /path/to/rails/app && rake RAILS_ENV=production admin:rss
And we added a step to the capistrano script that installed that to the crontab as part of the deploy.
Try webmin setup in your server. If your hosted site provide it. Go to the below mentioned URL. It's easy to set up and user freiendly.
URL is:
http://your_ip_address:10000/
I have used this in many of my application it's worked for me to schedule cron jobs.