I'm trying to make a small example with wisper-sidekiq gem. The queue is created, but it does not start. Why? I will describe the steps that you are doing, I think it will be easier. I hope for your help.
A. controller:
class BooksController < ApplicationController
def create
service = CreateBook.new
service.subscribe(ActivityListener, async: true)
service.on(:reserver_item_successfull) { |book| redirect_to book_path(book.id) }
service.on(:reserver_item_failed) { |book| #book = Book.new(book_params); respond_with(#book) }
service.execute(current_user, book_params)
end
B. service:
require 'wisper/sidekiq'
require 'sidekiq/api'
class CreateBook
include Wisper::Publisher
def execute(performer, attributes)
book = Book.new(attributes)
if book.valid?
book.save
broadcast(:reserver_item_successfull, performer, book)
else
broadcast(:book_failed, performer, book)
end
end
end
C. listener:
class ActivityListener
def self.reserver_item_successfull(performer, book)
puts performer.name.to_s + ", book: " + book.title.to_s
end
end
When I save the book, then of course creates a queue. But:
sidekiq silent (the logs are empty, but the queue was created)
redis silent too
Maybe I'm wrong start redis (redis-server) or sidekiq (bundle exec sidekiq)? Please help me.
P.S. Try bundle exec sidekiq -d -e production sidekiq -q default -C config/sidekiq.yml, but not result. The sidekiq.rb empty.
I think you can get this fixed by the following steps:
require 'sidekiq/web' and then, mount Sidekiq::Web => '/sidekiq'
go to /sidekiq to see if there're workers/tasks/queues
if there aint
something may be wrong with your redis
or your code, put a binding.pry there
if there're, must be something wrong with your code, put a binding.pry there
Hope that helps :-)
The problem is solved. I had to run sidekiq:
bundle exec sidekiq-r ./server.rb-L log/sidekiq.log
also in server.rb
require 'sidekiq'
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://localhost:6379/0' }
end
Sidekiq.configure_client do |config|
config.redis = { url: 'redis://localhost:6379/0' }
end
Related
My main purpose is creating a twitter bot. Which is basically going to retweet tweets from certain accounts. Everything works properly except keeping the bot awake. In order to keep to bot awake, I need to leave my laptop and console open. How can I deploy an App that will do this for me? I want to put the server to work for all this. I tried with Heroku but it gives timeout as expected cause request is should be open to keep my method going.
Thanks in advance...
my method:
def self.start_bot
#client = Twitter::REST::Client.new do |config|
config.consumer_key = ENV["CONSUMER_KEY"]
config.consumer_secret = ENV["CONSUMER_SECRET_KEY"]
config.access_token = ENV["ACCESS_TOKEN"]
config.access_token_secret = ENV["ACCESS_SECRET_TOKEN"]
end
while true do
begin
#client.search("from:ExampleAccount1 OR from:ExampleAccount1 OR from:ExampleAccount1
min_faves:250 -rt", result_type: "recent").take(5).collect do |tweet|
begin
#client.fav(tweet)
#client.retweet(tweet)
sleep(2)
rescue
puts "Already retweeted."
sleep(3)
end
end
puts "one round completed."
sleep(120)
rescue
puts "Something Happened"
sleep(240)
end
end
end
I think for your requirement you can use #clockwork_gem
You need to do the following:
Firstly install the clockwork gem.
In your lib folder create a lib/clock.rb
### Example :-
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(1.day, 'Creating Cycle', :at => '12:00') {YourClass_name.start_bot}
end
Create a procfile
To your Procfile use like this
worker: bundle exec clockwork lib/clock.rb
it will start the server every time after Heroku push/web server restart.
Or you can run using console from your computer like this but in this case, you need to run every time after web server restarting
heroku run bundle exec clockwork lib/clock.rb
and from Heroku open console and type
bundle exec clockwork lib/clock.rb
Hope this will help :)
I followed the sideqik gem tutorial docs and heroku Redistogo addon docs
initializers/sidekiq.rb:
Sidekiq.configure_server do |config|
config.redis = { url: 'redis://redistogo:xxx.redistogo.com:10076/' }
end
Sidekiq.configure_client do |config|
config.redis = { url: 'redis://redistogo:xxx.redistogo.com:10076/' }
end
app/workers/hard_woker.rb:
class HardWorker
include Sidekiq::Worker
def perform(shop_domain, webhook)
#performing stuffs
end
Webhook I am putting into background job (trying to at least):
class OrdersCreateJob < ActiveJob::Base
def perform(shop_domain:, webhook:)
shop = Shop.find_by(shopify_domain: shop_domain)
shop.with_shopify_session do
HardWorker.perform_async(shop_domain, webhook)
end
end
end
Procfile:
hardworker: bundle exec sidekiq -t 25
There are no errors in console.
Is something wrong here, did I miss something?
My queue:
irb(main):003:0> Sidekiq::Queue.all
=> [#<Sidekiq::Queue:0x000055b53a2d0920 #name="default", #rname="queue:default">]
I assume this means nothing is in the queue?
My goal is to take all my my CreateOrderWebhook code (which is almost 500 lines) into a background job to put less strain on the app and allow webhooks /prevent webhooks from being blocked
I have a Rails application that runs some background jobs via ActiveJob and Sidekiq. The sidekiq logs in both the terminal and the log file show the following:
2016-10-18T06:17:01.911Z 3252 TID-oukzs4q3k ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper JID-97318b38b1391672d21feb93 INFO: start
Is there some way to show the class names of the jobs here similar to how logs work for a regular Sidekiq Worker?
Update:
Here is how a Sidekiq worker logs:
2016-10-18T11:05:39.690Z 13678 TID-or4o9w2o4 ClientJob JID-b3c71c9c63fe0c6d29fd2f21 INFO: start
Update 2:
My sidekiq version is 3.4.2
I'd like to replace ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper with Client Job
So I managed to do this by removing Sidekiq::Middleware::Server::Logging from the middleware configuration and adding a modified class that displays the arguments in the logs. The arguments themself contain the job and action names as well.
For latest version, currently 4.2.3, in sidekiq.rb
require 'sidekiq'
require 'sidekiq/middleware/server/logging'
class ParamsLogging < Sidekiq::Middleware::Server::Logging
def log_context(worker, item)
klass = item['wrapped'.freeze] || worker.class.to_s
"#{klass} (#{item['args'].try(:join, ' ')}) JID-#{item['jid'.freeze]}"
end
end
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.remove Sidekiq::Middleware::Server::Logging
chain.add ParamsLogging
end
end
For version 3.4.2, or similar, override the call method instead:
class ParamsLogging < Sidekiq::Middleware::Server::Logging
def call(worker, item, queue)
klass = item['wrapped'.freeze] || worker.class.to_s
Sidekiq::Logging.with_context("#{klass} (#{item['args'].try(:join, ' ')}) JID-#{item['jid'.freeze]}") do
begin
start = Time.now
logger.info { "start" }
yield
logger.info { "done: #{elapsed(start)} sec" }
rescue Exception
logger.info { "fail: #{elapsed(start)} sec" }
raise
end
end
end
end
You must be running some ancient version. Upgrade.
Sorry, looks like that's a Rails 5+ feature only. You'll need to upgrade Rails. https://github.com/rails/rails/commit/8d2b1406bc201d8705e931b6f043441930f2e8ac
I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.
My webapp needs to encrypt its session data. What I setup is:
config/initializers/encryptor.rb:
require 'openssl'
require 'myapp/encryptor'
MyApp::Encryptor.config[ :random_key ] = OpenSSL::Random.random_bytes( 128 )
Session.delete_all
app/models/session.rb:
require 'attr_encrypted'
class Session < ActiveRecord::Base
attr_accessible :session_id, :data
attr_encryptor :data, :key => proc { MyApp::Encryptor.config[ :random_key ] }, :marshal => true
# Rest of model stuff
end
That all works great, and keeps the session data secured. Here's the problem: when I run my custom rake tasks it loads the initializer and clears all the sessions. Not good!
What can I put in my initializer to make sure it ONLY runs for the webapp initialization? Or, what can I put in my initializer to make it NOT run for rake tasks?
Update: OK, what I've done for the moment is add MYAPP_IN_RAKE = true unless defined? MYAPP_IN_RAKE to my .rake file. And then in my initializer I do:
unless defined?( MYAPP_IN_RAKE ) && MYAPP_IN_RAKE
# Web only initialization
end
Seems to work. But I'm open to other suggestions.
You might make a modification to your application in `config/application.rb' like this:
module MyApp
def self.rake?
!!#rake
end
def self.rake=(value)
#rake = !!value
end
Then in your Rakefile you'd add this:
MyApp.rake = true
It's nice to use methods rather than constants since sometimes you'd prefer to change or redefine them later. Plus, they don't pollute the root namespace.
Here's a sample config/initializers/rake_environment_test.rb script:
if (MyApp.rake?)
puts "In rake"
else
puts "Not in rake"
end
The programmable nature of the Rakefile affords you significant flexibility.
There is another work around:
unless ENV["RAILS_ENV"].nil? || ENV["RAILS_ENV"] == 'test'
When you launch with rake your ENV["RAILS_ENV"] will be nil. The test for 'test' is to avoid to run when using rspec.
I know that is reckon to use Rails.env but it return "development" when it is not initialised.
http://apidock.com/rails/Rails/env/class
# File railties/lib/rails.rb, line 55
def env
#_env ||= ActiveSupport::StringInquirer.new(ENV["RAILS_ENV"]
|| ENV["RACK_ENV"] || "development")
end