I'm parsing remote JSON data into MongoDB, Actually i'm parsing dynamic JSON data,but i want to update MongoDB for every 30 Sec with dynamic data.
parsing JSON data like this
require 'open-uri'
require 'json'
result = JSON.parse(open("url_of_json_service").read)
how i can update MongoDB for every 30sec?
Using rufus-schedular gem and it's working fine.
in Gemfile
gem 'rufus-scheduler', :require => "rufus/scheduler"
in config/initializers/reminder_sheduler.rb
scheduler = Rufus::Scheduler.start_new
scheduler.cron("0 5 * * *") do
Model.send_reminder_email
end
Cron is a common solution for recurring jobs, from the cron-job you can start either a script/runner Rails script or a rake task.
I should mention that with Cron the finest granularity is 1 minute.
Another solution would be to create a background job, which runs as a Daemon and basically runs a continuous loop: loading the remote JSON, sleeping for 30 seconds, loading the remote JSON, sleeping, ...
Check out these Railscasts:
http://railscasts.com/episodes/164-cron-in-ruby-revised
http://railscasts.com/episodes/164-cron-in-ruby
http://railscasts.com/episodes/129-custom-daemon
Cron:
http://en.wikipedia.org/wiki/Cron
Related
I'm trying to lock a part of my code using Redis, with Redlock library.
I've implemented it here:
def perform(task_id, task_type)
lock_manager = Redlock::Client.new([ "redis://127.0.0.1:6379" ])
lock_key = "task_runnuer_job_#{task_type}_#{task_id}"
puts("locking! #{task_id}")
lock_task = lock_manager.lock(lock_key , 6 * 60 * 1000)
if lock_task.present?
begin
# Exec task...
Services::ExecTasks::Run.call task.id
ensure
puts("unlocking! #{task_id}")
lock_manager.unlock(lock_task)
end
else
puts("Resource is locked! #{lock_key}")
end
end
What I get when running multiple Sidekiq jobs at the same time, is the following logs:
"locking! 520"
"locking! 520"
"unlocking! 520"
"unlocking! 520"
This happens when both of my 520 task, which should not be executed together, are being called with 1ms diffrence.
Yet sometimes the lock works as expected.
I've checked Redis and it works just fine.
Any ideas? Thanks!
Currently, I'm trying to migrate a large data set from an IBM DB2 database. I found a quick way of doing queries using imb_db2 gem (version 4.0.0).
The main goal is to have a script that runs in parallel and imports independent data.
My import script works fine sequential, but when trying to execute the script into multiple background processes, it crashes the process.
I tried Sidekiq with multiple processes, each process with its own queue.
For example, I try the following approach
DemoWorker.set(queue: :que_1).perform_async(1, 10)
Then I do
DemoWorker.set(queue: :que_2).perform_async(11, 20)
Each queue has its independent Sidekiq process, with concurrency 1. E.g.
bundle exec sidekiq -e development -C config/sidekiq.yml -q que_1
And I execute the following code with a query as a parameter
def get_by_sql(sql)
data = []
stmt = IBM_DB.exec(self.conn, sql.squish)
while row = IBM_DB.fetch_assoc(stmt)
data << row
end
data
end
And from time to time I get the following crash error.
[BUG] object allocation during garbage collection phase
I have no idea why it fails, could you advise? Thank you very much!
And here is the crash report: https://ufile.io/fqmfh0y8
EDIT:
Crash Report uploaded to GIST: https://gist.github.com/RaresSalcudean/db8467bc4cd3ea2164bcaab5da86780e
Crash 2 Report: https://gist.github.com/RaresSalcudean/6f7bb7f74e34a537b687c776d806bf04
I am trying to run a cron job in every 10 seconds that runs a piece of code. I have used an approach which requires running a code and making it sleep for 10 seconds, but it seems to make drastically degrading the app performance. I am using whenever gem, which run every minute and sleeps for 10 seconds. How can I achieve the same w/o using sleep method. Following is my code.
every 1.minute do
runner "DailyNotificationChecker.send_notifications"
end
class DailyNotificationChecker
def self.send_notifications
puts "Triggered send_notifications"
expiry_time = Time.now + 57
while (Time.now < expiry_time)
if RUN_SCHEDULER == "true" || RUN_SCHEDULER == true
process_notes
end
sleep 10 #seconds
end
def self.process_notes
notes = nil
time = Benchmark.measure do
Note.uncached do
notes = Note.where(status: false)
notes.update_all(status: true)
end
end
puts "time #{time}"
end
end
Objective of my code is to change the boolean status of objects to true which gets checked every 10 seconds. This table has 2 million records.
I suggest using a Sidekiq background jobs for this. With the sidekiq-scheduler gem you can run ordinary sidekiq jobs schedules in whatever internal you need. Bonus points for having a web-interface to handle and monitor the jobs via the Sidekiq gem.
You would use the clockwork gem. It runs in a separate process. The configuration is pretty simple.
require 'clockwork'
include Clockwork
every(10.seconds, 'frequent.job') { DailyNotificationChecker.process_notes }
I am using 'Sidekiq' to schedule reminder about any task on given time. Its working perfect. Now i want to append it to notify on my navbar, for it i am using 'Private Pub' to publish the reminder message.
Here is code of Sidekiq Worker.
class ReminderWorker
include Sidekiq::Worker
def perform(args)
reminder = Reminder.find(args['id'])
reminder.activate = true
PrivatePub.publish_to("reminder", message: reminder)
reminder.save
end
end
and inside "application.js" i am using alert for testing but it is not working.
PrivatePub.subscribe("reminder", function(data, channel) {
return alert('Remarks ='+ data.message.remarks);
});
Am i missing some thing? As it is possible to publish data through Private Pub in rb file accourding to Ryan http://railscasts.com/episodes/316-private-pub?autoplay=true
This gem has been updated last time three years ago and seems not maintained.
Maybe it's better to use newest rails solution like ActionCabble? https://github.com/rails/rails/tree/master/actioncable
I found this issue, that proofs that ActionCabble can be integrated to ActiveJob or other background workers:
https://github.com/rails/rails/issues/22897
I'm using Rufus-scheduler gem in my ROR application to send emails in the background. My setup is like:
# config/initializers/rufus_scheduler.rb
scheduler = Rufus::Scheduler.new(lockfile: '.rufus-scheduler.lock')
scheduler.cron '0 2 * * fri' do
UserMailer.send_some_emails
end
Any changes I make in the .send_some_email class method isn't reflected in the Rufus-scheduler task, how can I fix this? I don't want to restart the server every time I make a change!
Let's assume UserMailer.send_some_emails is defined in whatever/user_mailer.rb
scheduler = Rufus::Scheduler.new(:lockfile => '.rufus-scheduler.lock')
scheduler.cron '0 2 * * fri' do
load 'whatever/user_mailer.rb'
UserMailer.send_some_emails
end