I'm using Redis To Go on Heroku, and I'd like to use it to store some data alongside Resque jobs, much like I can store DelayedJobs in the same Postgres database as the rest of my model data.
Is this possible? Care to explain briefly how Resque stores jobs in Redis? I'm new to Redis, and though I understand on a higher level how it works, I'm unfamiliar with the implementation details.
Yes, you can. According to the Resque documentation:
Resque has a redis setter which can be given a string or a Redis object. This means if you're already using Redis in your app, Resque can re-use the existing connection.
Also, the documentation for Resque.redis= states that a namespace is accepted, to prevent clashes with your other apps running on the same instance:
A 'hostname:port/namespace' String (to set the Redis namespace)
Check out the "Configuration" section of the Resque readme for more information: https://github.com/resque/resque/blob/master/README.markdown
Redis is an advanced key value store which is used by Resque to store data about jobs. Exact details depend on your application however the two should be fine working together. As long as your code stays away from using the lists Resque uses then you will be golden.
yes, completely agree with other answers, the Redis can be used for other purposes, not only for background processing. By the way, Rescue provides easy interface to setup/config/use/browse Redis.
Setup/Run
$ git clone --depth=1 git://github.com/defunkt/resque.git
$ cd resque
$ rake redis:install dtach:install
$ vim config/resque.yml
"development: localhost:6379"
$ vim config/initializers/rescue.rb
"rails_root = ENV['RAILS_ROOT'] || File.dirname(__FILE__) + '/../..'
rails_env = ENV['RAILS_ENV'] || 'development'
resque_config = YAML.load_file(rails_root + '/config/resque.yml')
Resque.redis = resque_config[rails_env]"
$ rake redis:start
Use
# model
def user_activity
a = Resque.redis.lrange "log_" + self.id.to_s, 0, -1
a.map{|i| JSON.parse(i)}
end
Browse
$ resque-web
It allows you to browse Redis content, not only background processing
Related
I have Resque Cleaner set up with Redis and a Ruby on Rails framework. I noticed the cleaner fills with failed jobs up until the 1000 mark. I've been trying to find a way to increase the 1000 threshold with no luck yet, is there a way to do this that I'm missing?
Looks like this is covered in the readme here: https://github.com/ono/resque-cleaner
Add this to a resque-web config file:
# [app_dir]/config/resque-web.rb
require 'resque-cleaner'
module Resque::Plugins
ResqueCleaner::Limiter.default_maximum = 10_000
end
Then pass the file when you start resque-web
$ resque-web [app_dir]/config/resque-web.rb
I'm trying to use cron in my application to send mails every week but I think it doesn't work on Windows.
Does anybody knows any equivalent to cron solution that works on Windows?
Windows equivalent of Unix's cron is a "Task Scheduler". You can configure your periodical task there.
Purely Ruby solution
If you want a purely Ruby solution look into:
rufus-scheduler - it's Windows cron gem.
crono - it's a in-Rails cron scheduler, so it should work anywhere.
Web services - there are plenty of free online services that would make a request to a given URL in specific time periods. This is basically a poor man's cronjob.
I recommend taking a look at Resque and the extension Resque-scheduler gems. You will need to have a resque scheduler process running with bundle exec rake resque:scheduler and at least one worker process running with QUEUE=* bundle exec rake resque:work.
If you want these services to run in the background as a windows service, you can do it with srvany.exe as described in this SO question.
The above assumes you are ok with installing Redis - a key-value store that is very popular among the Rails community as it can be easily used to support other Rails components such as caching and ActionCable, and it is awesome by itself for many multi-process use cases.
Resque is a queue system on top of Redis that allows you to define jobs that can be executed asynchronously in the background. When you run QUEUE=* bundle exec rake resque:work, a worker process runs constantly and polls the queue. Once a job is enqueued, an available worker pops it from the queue and starts working on it. This architecture is quite scalable, as you can have multiple workers listening to the queues if you'd like.
To define a job, you do this:
class MyWeeklyEmailSenderJob
def self.perform
# Your code to send weekly emails
end
end
While you can enqueue this job to the queue yourself from anywhere (e.g. from a controller as a response to an action), in your case you want it to automatically be placed into the queue once a week. This is what Resque-scheduler is for. It allows you to configure a file such as app/config/resque_schedule.yml in which you can define which jobs should be enqueued in which time interval. For example:
send_weekly_emails:
cron: 0 8 * * Mon
class: MyWeeklyEmailSenderJob
queue: email_sender_queue
description: "Send weekly emails"
Remember that a scheduling process has to run in order for this to work with bundle exec rake resque:scheduler.
thanks guys , actually i tried rufus scheduler gem and it worked for me , i guess it's the best and easier solution
We're currently in transition between a Ruby/RoR 1.8/3.2 and a rewritten 2.0/4.0 app on the same server. Since I don't want to mess with the current resque configuration and risk updating any gems that might break production, I elected to setup a separate Redis server on a new port and use the newest version of Resque with it. Resque appears to be running fine; if I manually launch a worker rake resque:work QUEUE='*' and watch the process it runs a queued job from resque-scheduler. However, none of my workers, launched manually or via script, show up in the resque-web instance running. The stats page of resque-web shows it's looking at the second redis instance. Does anyone have experience with this or any ideas?
/config/initializers/resque.rb:
require 'resque_scheduler'
rails_root = ENV["RAILS_ROOT"] || File.dirname(__FILE__) + "/../.."
rails_env = ENV["RAILS_ENV"] || "demo"
resque_config = YAML.load_file(rails_root + "/config/resque.yml")
Resque.redis = resque_config[rails_env]
Resque.redis.namespace = "resque-reos:#{rails_env}"
/config/resque.yml
demo: localhost:6380
I'm using EventMachine and Monetarily to start e TCP server along with my rails application. This is started from config/initializers/momentarily.rb.
My problem is that it starts also when I run rake tasks, like db:migrate. I only want it to start when when I start the HTTP server. Environments won't help, since both the server start and rake tasks are under Development environment. Is there a way of knowing that the application is running the HTTP server as opposed to anything else? Note that is not only rake tasks, the EM starts also if I run the rails console, which is again something not desirable for my case.
unless File.basename($0) == "rake" && ARGV.include?("db:migrate")
# you are not in rake db:migrate
end
There's not a great way of doing this that I know of. You could copy newrelic's approach (check discover_dispatcher in local_environment.rb) which basically has a list of heuristics used to detect if it is running inside passenger, thin, etc.
For passenger it checks
defined?(::PhusionPassenger)
for thin it checks
if defined?(::Thin) && defined?(::Thin::Server)
Set an environment variable in config.ru file, and use it anywhere in the code to detect if it's executed using a rails server command only.
For e.g.
File: config.ru
ENV['server_mode'] = '1'
And using it somewhere as:
File: config/environment.rb
Thread.new { infinite_loop! }.join if ENV['server_mode'] = '1'
Reference: Answer
Maybe you can implement a switch in the initializer based on ARGV?
Something like:
if ARGV.join(' ').match /something/
# your initializer code here
end
Don't start that other server from an initializer. Create a daemon in script/momentarily and start it from within your app.
After your application launches, you could have it shell out to check ps. If ps shows that the HTTP server is running and the running HTTP server has the same pid as your application (check the pid by inspecting $$), then you could launch the TCP server.
In addition to a great answer by Frederick Cheung above, there can be some other "footprints" in actual process environment. Eg. Phusion Passenger adds certain variables to ENV such as:
PASSENGER_APP_ENV
IN_PASSENGER
PASSENGER_SPAWN_WORK_DIR
PASSENGER_USE_FEEDBACK_FD
Web servers typically can also set SERVER_SOFTWARE variable eg.:
SERVER_SOFTWARE=nginx/1.15.8 Phusion_Passenger/6.0.2
I'm using rufus-scheduler for handling cron jobs for a Rails 3.2.x app. The root worker.rb is being fired off by foreman (actually upstart on this particular server) and therefore when it starts off it does not have the Rails context in which to operate. Obviously when I attempt to call logger or Rails.logger it will fail each time.
I'm using log4r as a replacement for the default Rails Logger, which is quite nice, but I am wondering what the proper solution for this problem would be:
1) Do I need to give the script the Rails context at startup (it is simply running rake tasks right now so it ends up getting the Rails environment when the worker script hands off to the rake task and I fear giving the script the Rails env before running the timed task would be overkill since it takes so long to fire up the env)? or
2) Should I just set up log4r as one would do in a non-Rails Ruby script that simply reads in the same log4r.yml config that the Rails app is using and therefore participate in the same log structure?
3) Or some other option I'm not thinking of?
Additionally, I would appreciate either an example or the steps that I should consider with the recommended implementation.
For reference, I followed "How to configure Log4r with Rails 3.0.x?" and I think this could be helpful when integrated with the above: "Properly using Log4r in Ruby Application?"
I think this might be what you're looking for..
Use this within the worker itself, and create a custom named log file
require 'log4r'
logger = Log4r::Logger.new('test')
logger.outputters << Log4r::Outputter.stdout
logger.outputters << Log4r::FileOutputter.new('logtest', :filename => 'logtest.log')
logger.info('started script')
## You're actual worker methods go here
logger.info('finishing')