sneaker is not receiving messages on heroku - RabbitMQ Bigwig - ruby-on-rails

I am trying to run message queues on heroku. For this I am using RabbitMQ Bigwig plugin.
I am publishing messages using bunny gem and trying to receive messages with sneakers gem. This whole setup works smoothly on local machine.
I take following steps to setup queue
I run this rake on server to setup queue:
namespace :rabbitmq do
desc 'Setup routing'
task :setup_test_commands_queue do
require 'bunny'
conn = Bunny.new(ENV['SYNC_AMQP'], read_timeout: 10, heartbeat: 10)
conn.start
ch = conn.create_channel
# get or create exchange
x = ch.direct('testsync.pcc', :durable => true)
# get or create queue (note the durable setting)
queue = ch.queue('test.commands', :durable => true, :ack => true, :routing_key => 'test_cmd')
# bind queue to exchange
queue.bind(x, :routing_key => 'test_cmd')
conn.close
end
end
I am able to see this queue in rabbitmq management plugin with mentioned binding.
class TestPublisher
def self.publish(test)
x = channel.direct("testsync.pcc", :durable => true)
puts "publishing this = #{Test}"
x.publish(Test, :persistent => true, :routing_key => 'pcc_cmd')
end
def self.channel
#channel ||= connection.create_channel
end
def self.connection
#conn = Bunny.new(ENV['RABBITMQ_BIGWIG_TX_URL'], read_timeout: 10, heartbeat: 10) # getting configuration from rabbitmq.yml
#conn.start
end
end
I am calling TestPublisher.publish() to publish message.
I have sneaker worker like this:
require 'test_sync'
class TestsWorker
include Sneakers::Worker
from_queue "test.commands", env: nil
def work(raw_event)
puts "^"*100
puts raw_event
# o = CaseNote.create!(content: raw_event, creator_id: 1)
# puts "#########{o}"
test = Oj.load raw_event
test.execute
# event_params = JSON.parse(raw_event)
# SomeWiseService.build.call(event_params)
ack!
end
end
My Procfile
web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec rake jobs:work
sneaker: WORKERS=TestsWorker bundle exec rake sneakers:run
My Rakefile
require File.expand_path('../config/application', __FILE__)
require 'rake/dsl_definition'
require 'rake'
require 'sneakers/tasks'
Test::Application.load_tasks
My sneaker configuration
require 'sneakers'
Sneakers.configure amqp: ENV['RABBITMQ_BIGWIG_RX_URL'],
log: "log/sneakers.log",
threads: 1,
workers: 1
puts "configuring sneaker"
I am sure that message gets published. I am able to get message on rabbitmq management plugin. But sneaker does not work. There is nothing in sneakers.log that can help.
sneakers.log on heroku :
# Logfile created on 2016-04-05 14:40:59 +0530 by logger.rb/41212

Sorry for this late response. I was able to get this working on heroku. When I faced this error after hours of debugging I was not able to fix it. So I rewrote all above code and I did not check what was wrong with my previous code.
The only problem with this code and correct code is queue binding.
I had two queues on same exchange. pcc.commands with routing key pcc_cmd and test.commands with routing key test_cmd.
I was working with test_cmd but as per following line in TestPublisher
x.publish(Test, :persistent => true, :routing_key => 'pcc_cmd')
I was publishing to different queue(pcc.commands). Hence I was not able to recieve the message on test.commands queue.
In TestWorker
from_queue "test.commands", env: nil
This states that fetch messages only from test.commands queue.
Regarding sneakers.log file:
Above setup was not able to give me logs in sneakers.log file. Yes this setup works on your local development machine, but it was not working on heroku. Now days to debug such issue I ommit log attribute from configuration. like this:
require 'sneakers'
Sneakers.configure amqp: ENV['RABBITMQ_BIGWIG_RX_URL'],
# log: "log/sneakers.log",
threads: 1,
workers: 1
This way you will get sneaker logs (even heartbeat logs) in heroku logs which can be seen by running command heroku logs -a app_name --tail.

Related

How can I keep my Twitter Bot awake with Rails?

My main purpose is creating a twitter bot. Which is basically going to retweet tweets from certain accounts. Everything works properly except keeping the bot awake. In order to keep to bot awake, I need to leave my laptop and console open. How can I deploy an App that will do this for me? I want to put the server to work for all this. I tried with Heroku but it gives timeout as expected cause request is should be open to keep my method going.
Thanks in advance...
my method:
def self.start_bot
#client = Twitter::REST::Client.new do |config|
config.consumer_key = ENV["CONSUMER_KEY"]
config.consumer_secret = ENV["CONSUMER_SECRET_KEY"]
config.access_token = ENV["ACCESS_TOKEN"]
config.access_token_secret = ENV["ACCESS_SECRET_TOKEN"]
end
while true do
begin
#client.search("from:ExampleAccount1 OR from:ExampleAccount1 OR from:ExampleAccount1
min_faves:250 -rt", result_type: "recent").take(5).collect do |tweet|
begin
#client.fav(tweet)
#client.retweet(tweet)
sleep(2)
rescue
puts "Already retweeted."
sleep(3)
end
end
puts "one round completed."
sleep(120)
rescue
puts "Something Happened"
sleep(240)
end
end
end
I think for your requirement you can use #clockwork_gem
You need to do the following:
Firstly install the clockwork gem.
In your lib folder create a lib/clock.rb
### Example :-
require 'clockwork'
require './config/boot'
require './config/environment'
module Clockwork
handler do |job|
puts "Running #{job}"
end
every(1.day, 'Creating Cycle', :at => '12:00') {YourClass_name.start_bot}
end
Create a procfile
To your Procfile use like this
worker: bundle exec clockwork lib/clock.rb
it will start the server every time after Heroku push/web server restart.
Or you can run using console from your computer like this but in this case, you need to run every time after web server restarting
heroku run bundle exec clockwork lib/clock.rb
and from Heroku open console and type
bundle exec clockwork lib/clock.rb
Hope this will help :)

how setup active_job and rescue scheduler?

I'm trying to create background jobs for email notification and scraper.
I use resque-scheduler (4.0.0), resque (1.25.2) and rails 4.2.1.
My config.ru file:
# This file is used by Rack-based servers to start the application.
require ::File.expand_path('../config/environment', __FILE__)
run Rails.application
require 'resque/server'
run Rack::URLMap.new "/" => AppName::Application, "/resque" => Resque::Server.new
My /lib/tasks/resque.rake:
require 'resque/tasks'
require 'resque/scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque-scheduler'
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
end
end
My /config/resque_scheduler.yml:
CheckFsUpdatesJob:
queue: fs_updates
every:
- '1h'
- :first_in: '10s'
class: CheckFsUpdatesJob
args:
description: scrape page
My /config/initializer/active_job.rb
ActiveJob::Base.queue_adapter = :resque
My /config/initializer/resque.rb:
#config/initializers/resque.rb
require 'resque-scheduler'
require 'resque/scheduler/server'
uri = URI.parse("redis://localhost:6379/")
Resque.redis = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password)
Resque.after_fork = Proc.new { ActiveRecord::Base.establish_connection }
Dir["#{Rails.root}/app/jobs/*.rb"].each { |file| require file }
Resque.schedule = YAML.load_file(Rails.root.join('config', 'resque_schedule.yml'))
Resque::Server.use(Rack::Auth::Basic) do |user, password|
user = 'admin'
password = 'admin'
end
My first job for emails notifications:
class EmailNotificationJob < ActiveJob::Base
queue_as :email_notifications
def perform(episode_id, email)
NotificationMailer.new_record_appears(record_id, email).deliver_now
end
end
My second job for scheduled runs:
class CheckFsUpdatesJob < ActiveJob::Base
queue_as :fs_updates
def perform()
FsStrategy.new.check_for_updates
end
end
So I have to jobs:
1. emails notifications - should sends email when new record in DB appears
2. scrape a page - should runs every hour
How I run it:
redis-server
rake environment resque:work QUEUE=fs_updates
rake environment resque:work QUEUE=email_notifications
rake environment resque:scheduler
rails s
After running these commands I see in Resque Dashboard two workers and two queues, as it is expected.
But!
After clicking on 'queue now' button at resque Schedule tab, I see that task was created and wroted to "fs_updates" queue. But it's not running and in a few second it dissapears.
When I run a job for emails sending from rails console - it does not work at all.
Please, help me to fix my configurations.
Thanks kindly!
As I understood: rails and active_job developers is not responsible for resque plugins. Maybe this problem will be fixed in new gem versions, but now it does not work (active_job does not work fine with resque-scheduler).
Currently I use gem 'active_scheduler' to fix current problem.
I had the same issue trying to configure Sucker Punch on rails 4.2.1 In the end I moved my custom initialiser logic into application.rb, not great but it got me up and running.
Not sure if there is an issue with the initialisers in this release. Try moving your code from active_job.rb and resque.rb into application.rb or the appropriate environment file. Obviously this isn't a long term solution but it will at least help you you identify whether it's an initialiser issue or Resque config problem.

Sidekiq worker enqueues but doesn't complete when paired with Sidetiq

I have a Sidekiq worker that syncs the data of a Rails 3.2 application with a remote database. When I just execute it from the rails console, everything works as expected. I can verify that it connects to the remote DB and pulls in the data I am expecting.
The moment I try to set up the following:
recurrence {daily.hour_of_day(20).minute_of_hour(02)}
I see the following in Sidekiq but no signs of completion/execution:
2014-05-14T00:02:18Z 6747 TID-dvc2c INFO: [Sidetiq] Enqueue: BackendWorker (at: 1400112120.0) (last: -1.0)
I've got retries set to false (which I would like to change) currently, and I have attempted to clear out anything in redis before setting up another test run again.
Is there a way to squeeze some more information about what is going on? Is there a Sidekiq flag I can use to get some additional debug information?
Thanks in advance
Environment:
Rails 3.2
Ruby 2.1
Sidetiq 0.5.0
Sidekiq 2.17.7 (due to an issue with using 3.0)
UPDATE (5/14/2014):
I decided to test a simple example with Sidetiq and just have a worker that creates a DB entry with ActiveRecord:
class BackendTest
include Sidekiq::Worker
include Sidetiq::Schedulable
sidekiq_options :retry => false, :backtrace => true
recurrence {
daily.hour_of_day(11).minute_of_hour(05)
}
def perform
Book.create(title: "test", author: "testing", price: 9.99)
end
end
Another experiment I did was just trying to get Sidetiq and Sidekiq to just show something with logger.info:
class BackendTest
include Sidekiq::Worker
include Sidetiq::Schedulable
sidekiq_options :retry => false, :backtrace => true
recurrence {
daily.hour_of_day(16).minute_of_hour(00)
}
def perform
logger.info "test"
logger.info "test"
end
end
The jobs just still enters the default queue without being started or completed. Everything still works if I just fire it off from the rails console with perform_async, Sidekiq::Client.push, and Sidekiq::Client.enqueue still work from console.
I am a thoroughly confused newbie at this point.
Output:
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: start
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: test
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: test
2014-05-14T20:08:41Z 2149 TID-dfcfk BackendTest JID-6bc7ba08955f8340903f9063 INFO: done: 0.002 sec
After poking around other people's projects on github, I saw something interesting and tried it out and it seemed to solve my problem with Sidetiq.
I created /config/initializers/sidekiq.rb with the following content inside of it:
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://localhost:6379/3', :namespace => 'sidekiq' }
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://localhost:6379/3', :namespace => 'sidekiq' }
end
Job tasks enqueue and execute as expected.

Mailman saves Mails multiple times

I've got the mailman gem integrated into my rails project. It fetches emails from gmail successfully. In my app there is a model Message for my emails. The emails are properly saved as Message model.
The problem is that the emails are saved multiple times sometimes and I can't recognize a pattern. Some emails are saved once, some two times and some are saved three times.
But I can't find the failure in my code.
Here is my mailman_server script:
script/mailman_server
#!/usr/bin/env ruby
# encoding: UTF-8
require "rubygems"
require "bundler/setup"
require File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))
require 'mailman'
Mailman.config.ignore_stdin = true
#Mailman.config.logger = Logger.new File.expand_path("../../log/mailman_#{Rails.env}.log", __FILE__)
if Rails.env == 'test'
Mailman.config.maildir = File.expand_path("../../tmp/test_maildir", __FILE__)
else
Mailman.config.logger = Logger.new File.expand_path("../../log/mailman_#{Rails.env}.log", __FILE__)
Mailman.config.poll_interval = 15
Mailman.config.imap = {
server: 'imap.gmail.com',
port: 993, # usually 995, 993 for gmail
ssl: true,
username: 'my#email.com',
password: 'my_password'
}
end
Mailman::Application.run do
default do
begin
Message.receive_message(message)
rescue Exception => e
Mailman.logger.error "Exception occurred while receiving message:\n#{message}"
Mailman.logger.error [e, *e.backtrace].join("\n")
end
end
end
The email is processed inside my Message class:
def self.receive_message(message)
if message.from.first == "my#email.com"
Message.save_bcc_mail(message)
else
Message.save_incoming_mail(message)
end
end
def self.save_incoming_mail(message)
part_to_use = message.html_part || message.text_part || message
if Kontakt.where(:email => message.from.first).empty?
encoding = part_to_use.content_type_parameters['charset']
Message.create topic: message.subject, message: part_to_use.body.decoded.force_encoding(encoding).encode('UTF-8'), communication_partner: message.from.first, inbound: true, time: message.date
else
encoding = part_to_use.content_type_parameters['charset']
Message.create topic: message.subject, message: part_to_use.body.decoded.force_encoding(encoding).encode('UTF-8'), communication_partner: message.from.first, inbound: true, time: message.date, messageable_type: 'Company', messageable_id: Kontakt.where(:email => message.from.first).first.year.id
end
end
def self.save_bcc_mail(message)
part_to_use = message.html_part || message.text_part || message
if Kontakt.where(:email => message.to.first).empty?
encoding = part_to_use.content_type_parameters['charset']
Message.create topic: message.subject, message: part_to_use.body.decoded.force_encoding(encoding).encode('UTF-8'), communication_partner: message.to.first, inbound: false, time: message.date
else
encoding = part_to_use.content_type_parameters['charset']
Message.create topic: message.subject, message: part_to_use.body.decoded.force_encoding(encoding).encode('UTF-8'), communication_partner: message.to.first, inbound: false, time: message.date, messageable_type: 'Company', messageable_id: Kontakt.where(:email => message.to.first).first.year.id
end
end
I have daemonized the mailman_server with this script:
script/mailman_daemon
#!/usr/bin/env ruby
require 'rubygems'
require "bundler/setup"
require 'daemons'
Daemons.run('script/mailman_server')
I deploy with capistrano.
This are the parts which are responsible for stopping, starting and restarting my mailman_server:
script/deploy.rb
set :rails_env, "production" #added for delayed job
after "deploy:stop", "delayed_job:stop"
after "deploy:start", "delayed_job:start"
after "deploy:restart", "delayed_job:restart"
after "deploy:stop", "mailman:stop"
after "deploy:start", "mailman:start"
after "deploy:restart", "mailman:restart"
namespace :deploy do
desc "mailman script ausfuehrbar machen"
task :mailman_executable, :roles => :app do
run "chmod +x #{current_path}/script/mailman_server"
end
desc "mailman daemon ausfuehrbar machen"
task :mailman_daemon_executable, :roles => :app do
run "chmod +x #{current_path}/script/mailman_daemon"
end
end
namespace :mailman do
desc "Mailman::Start"
task :start, :roles => [:app] do
run "cd #{current_path};RAILS_ENV=#{fetch(:rails_env)} bundle exec script/mailman_daemon start"
end
desc "Mailman::Stop"
task :stop, :roles => [:app] do
run "cd #{current_path};RAILS_ENV=#{fetch(:rails_env)} bundle exec script/mailman_daemon stop"
end
desc "Mailman::Restart"
task :restart, :roles => [:app] do
mailman.stop
mailman.start
end
end
Could it be that multiple instances of the mailman server are started during my deploy at nearly the same time and then each instance polls nearly at the same time? The second and third instance pools before the first instance marks the email as read and polls and processes the email as well?
Update 30.01.
I had set the polling intervall to 60 seconds. but that changes nothing.
I checked the folder where the mailman pid file is stored. there is only one mailman pid file. So there is definitely only one mailman server running. I checked the logfile and can see, that the messages are fetched multiple times:
Mailman v0.7.0 started
IMAP receiver enabled (my#email.com).
Polling enabled. Checking every 60 seconds.
Got new message from 'my.other#email.com' with subject 'Test nr 0'.
Got new message from 'my.other#email.com' with subject 'Test nr 1'.
Got new message from 'my.other#email.com' with subject 'test nr 2'.
Got new message from 'my.other#email.com' with subject 'test nr 2'.
Got new message from 'my.other#email.com' with subject 'test nr 3'.
Got new message from 'my.other#email.com' with subject 'test nr 4'.
Got new message from 'my.other#email.com' with subject 'test nr 4'.
Got new message from 'my.other#email.com' with subject 'test nr 4'.
So that seems to me, that the problem is definitely in my mailman server code.
Update 31.1.
Seems to me, that is has something to do with my production machine. when I'm testing this in development with the exact same configuration (changed my local database from sqlite to mysql this morning to test it) as on the production machine I don't get duplicates. Probably is everything ok with my code, but there is a problem with the production machine. Will ask my hoster if they could see a solution for this. To fix this I will go with Ariejan'S suggestion.
The solution:
I found the problem. I deploy to a machine where the tmp directory is a shared one between all releases. I forgot to define the path where the pid file of the mailman_daemon should be saved. So it was saved in the script directory instead of the /tmp/pids directory. Because of this the old mailman_daemon could not be stopped after a new deploy. That had led to an army of working mailman_daemons which were polling my mailaccount... After killing all these processes all went well! No more duplicates!
This may be some concurrency/timing issue. E.g. new mails are imported before the ones currently processing have been saved.
Edit: Just noticed you have Mailman.config.poll_interval set to 15. This means it will check for new messages every 15 seconds. Try increasing this value to the default 60 seconds. Regardless of this setting, it might be a good idea to add the deduplication code I mentioned below.
My tip would be to also store the message_id from each email, so you can easily spot duplicates.
Instead of:
Message.create(...)
do:
# This makes sure you have the latest pulled version.
message = Message.find_or_create(message_id: message.message_id)
message.update_attributes(...)
# This makes sure you only import it once, then ignore further duplicates.
if !Message.where(message_id: message.message_id).exists?
Message.create(...)
end
For more info on message_id: http://rdoc.info/github/mikel/mail/Mail/Message#message_id-instance_method
Remember that email and imap are not meant to be consistent data stores like you'd expect Postgres or Mysql to be. Hope this helps you sort out the duplicate mails.
I found the problem. I deploy to a machine where the tmp directory is a shared one between all releases. I forgot to define the path where the pid file of the mailman_daemon should be saved. So it was saved in the script directory instead of the /tmp/pids directory. Because of this the old mailman_daemon could not be stopped after a new deploy. That had led to an army of working mailman_daemons which were polling my mailaccount... After killing all these processes all went well! No more duplicates!

Resque workers fail immediately: undefined method `write' for nil:NilClass

I am using Resque (and resque-scheduler) in my Rails app to run a recurring job. This was working fine for me, until today. I made some code changes, which I thought were unrelated, but now every worker fails before the perform method is even entered (checked with a debug statement). The same worker method works fine when I run it in the rails console. It only fails via resque on the development localhost (Postgres DB).
The error shown in the resque console for the failed worker is:
Exception
NoMethodError
Error
undefined method `write' for nil:NilClass
There is no additional stack trace for the error. Any idea why this is failing?
Additional info:
lib/tasks/resque.rake
# Resque tasks
require 'resque/tasks'
require 'resque_scheduler/tasks'
namespace :resque do
task :setup do
require 'resque'
require 'resque_scheduler'
require 'resque/scheduler'
# you probably already have this somewhere
Resque.redis = 'localhost:6379'
# If you want to be able to dynamically change the schedule,
# uncomment this line. A dynamic schedule can be updated via the
# Resque::Scheduler.set_schedule (and remove_schedule) methods.
# When dynamic is set to true, the scheduler process looks for
# schedule changes and applies them on the fly.
# Note: This feature is only available in >=2.0.0.
#Resque::Scheduler.dynamic = true
# The schedule doesn't need to be stored in a YAML, it just needs to
# be a hash. YAML is usually the easiest.
Resque.schedule = YAML.load_file("#{Rails.root}/config/resque_schedule.yml")
# If your schedule already has +queue+ set for each job, you don't
# need to require your jobs. This can be an advantage since it's
# less code that resque-scheduler needs to know about. But in a small
# project, it's usually easier to just include you job classes here.
# So, something like this:
# require 'jobs'
end
end
task "resque:setup" => :environment do
#ENV['QUEUE'] = '*'
Resque.before_fork = Proc.new { ActiveRecord::Base.establish_connection }
end
config/resque.yml
development: localhost:6379
test: localhost:6379:1
staging: redis1.se.github.com:6379
fi: localhost:6379
production: redis1.ae.github.com:6379
initializers/resque.rb
rails_root = Rails.root || File.dirname(__FILE__) + '/../..'
rails_env = Rails.env || 'development'
resque_config = YAML.load_file(rails_root.to_s + '/config/resque.yml')
Resque.redis = resque_config[rails_env]
# This will make the tabs show up.
require 'resque_scheduler'
require 'resque_scheduler/server'
config/resque_schedule.yml
populate_game_data:
# you can use rufus-scheduler "every" syntax in place of cron if you prefer
every: 1m
# By default the job name (hash key) will be taken as worker class name.
# If you want to have a different job name and class name, provide the 'class' option
class: PopulateDataWorker
queue: high
args:
description: "This job populates the game and data"
Should note that the above files were not changed between working and non-working state.
We had the same issue this morning, and we pinned it down to a gem update by New Relic.
Version 3.5.6.46 of newrelic_rpm was yanked on rubygems, but it was somehow installed by bundle update.
They are still on the beta track for 3.5.6 and had some issues with Resque. See https://github.com/newrelic/rpm/commit/e81889c2bce97574ec682dafee12015e13ccb2e1
The fix was to add '~> 3.5.5.38' in our Gemfile for newrelic_rpm

Resources