Here is my scenario, i'm using resque to queue a job in redis, the usual way its done in ROR. The format of my key looks something like this (as per my namespace convention)
"resque:lock:Jobs::XYZ::SomeCreator-{:my_ids=>[101]}"
The job runs successfully to completition. But the key still exists in redis. For a certain flow, i need to queue and execute the job again for the same parameters (the key will essentially be same). But seems like the job does not get queued.
My guess is that since the key already exists in Redis, it does not queue the job again.
Questions:
Is this behavior of resque normal (not removing the key after successful completition)?
If Yes, how should i tackle this scenario (as per best practices)?
If No, can you help me understand what is going wrong?
After a couple of hours of debugging, finally this is the observed behavior:
I was creating the job and passing the options (parameters) with symbolized keys which when created the Redis key for the same job with symbolized param in the key.
Example:
Jobs::Abc::SomeJobCreator.create({:some_ids => [101]}) would create the "redis key" as "resque:lock:Jobs::Abc::SomeJobCreator.create({:some_ids => [101]})" (Notice the key being a symbol in the key)
Now when the after_perform_hook executes, it tries to remove the Redis Key but it searches the key with Stringified keys: "resque:lock:Jobs::Abc::SomeJobCreator-({\"some_ids\"=>[101]}" Which obviously won't be found, as the key in Redis has symbolized params in key.
To fix this issue i had to change the calls to job creation in the code and use stringified params like this: Jobs::Abc::SomeJobCreator.create({'some_ids' => [101]}). This works fine.
Not sure if this has anything to do with the version of Resque. Since its a old codebase i haven't yet updated the version. Its currently at Resque v1.25.2
Related
According to the redis-rb ruby gem docs;
redis.watch("key") do
if redis.get("key") == "some value"
redis.multi do |multi|
multi.set("key", "other value")
multi.incr("counter")
end
else
redis.unwatch
end
end
I have two questions about this;
Why is the unwatch line required? My thought is that if key == "some value", set "key" to other value and increase counter if "key" has not changed by the time EXEC is called. In my mind, the else statement never gets executed in this scenario. If you run this code without the else/unwatch, nil is returned.
If redis.watch "key" is entered without a block (redis.watch "key"); Why am I unable to unwatch a specific key? I get (wrong number of arguments (given 1, expected 0))??
Here is a link to the ruby gem https://github.com/redis/redis-rb
I am not a Ruby nor Redis expert.
Though by reading the Redis documentation about transaction here https://redis.io/topics/transactions we learn that a watched key is unwatched in two ways : either after EXEC is used. (then the transactions completes) or either by UNWATCH.
So there is indeed a reason to use UNWATCH. If the multi block is not called then EXEC is not called either. Then the key is never flushed.
Also I think the conditionnal bit containing redis.unwatch can indeed happen to be called. As redis.get("key") == "some value" seems to be purely arbitrary and basically a code design.
It could well be redis.get("key") > "some value" for example.
(Also it seem having a block passed to watched does not UNWATCH the key automatically as per documentation, as the block is optionnal)
For your second question you are correct, you cannot unwatch a specific key it seems. Though redis.unwatch seems to work fine.
This does not seem to be a Ruby design. Watching a key is only useful in order to process an atomic transaction, there is no point in leaving watched keys behind us when transaction is either successful or aborted.
I don't see any example of using UNWATCH in the Redis documentation https://redis.io/topics/transactions but it feels natural to flush every key after the transaction block.
So I've been looking for the simplest way to send an e-mail when X column of Payments table in the database is == 'condition'. Basically what I want is to add a payment and set a date like 6 months. When 6 months have passed I want to send the mail. I've seen many solutions like using Whenever cron jobs and others but I want to know the absolute simplest way (perhaps using Rails only without relying on outside source) to keep my application light and clean. I was thinking I could use the auto generated created_at to evaluate when x time has passed.
Since you have a column in your db for the time to send email, make it a datetime datatype and you can set the email date as soon as the event payment event is created. Then, you can have a rake task where,
range = Time.now.beginning_of_day..Time.now.end_of_day
Payment.where(your_datetime_custom_column: range).each do |payment|
payment.user.send_email
end
and you can run this task everyday from the scheduler.
The "easiest" way is to use Active Job in conjunction with a state machine:
EmailJob.set(wait: 6.months).perform_later(user.id) if user.X_changed?
The problem with this is that the queue will accumulate jobs since jobs don't get handled right away. This may lead to other performance issues since there are now more jobs to scan and they're taking up more memory.
Cron jobs are well suited for this kind of thing. Depending on your hosting platform, there may be various other ways to handle this; for example, Heroku has Heroku Scheduler.
There are likely other ways to schedule repeating tasks without cron, such as this SO answer.
edit: I did use a gem once called 'fist_of_fury', but it's not currently maintained and I'm not sure how it would perform in a production environment. Below are some snippets for how I used it in a rails project:
in Gemfile
gem 'fist_of_fury'
in config/initializers/fist_of_fury.rb
# Ensure the jobs run only in a web server.
if defined?(Rails::Server)
FistOfFury.attack! do
ObserveAllJob.recurs { minutely(1) }
end
end
in app/jobs/observe_all_job.rb
class ObserveAllJob
include SuckerPunch::Job
include FistOfFury::Recurrent
def perform
::Task.all.each(&:observe)
end
end
To test that my mails are being sent I'm running heroku run rails c -a my_app. Then I enqueue the job and it is enqueued fine. However, when I go to Redis and see queued jobs the job is not there. Instead, it is on "retry".
This is what I see:
{"retry":true,"queue":"default","class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","args":[{"job_class":"SendMailJob","job_id":"4b4ba46f-94d7-45cd-b923-ec1678c73076","queue_name":"default","arguments":["any_help",{"_aj_globalid":"gid://gemfeedapi/User/546641393834330002000000"}]}],"jid":"f89235d7ab19f605ed0461a1","enqueued_at":1424175756.9351726,"error_message":"Error while trying to deserialize arguments: \nProblem:\n Document(s) not found for class User with id(s) 546641393834330002000000.\nSummary:\n When calling User.find with an id or array of ids, each parameter must match a document in the database or this error will be raised. The search was for the id(s): 546641393834330002000000 ... (1 total) and the following ids were not found: 546641393834330002000000.\nResolution:\n Search for an id that is in the database or set the Mongoid.raise_not_found_error configuration option to false, which will cause a nil to be returned instead of raising this error when searching for a single id, or only the matched documents when searching for multiples.","error_class":"ActiveJob::DeserializationError","failed_at":1424175773.317896,"retry_count":0}
However, object is in the Database.
I've tried to add after_create callback (Mongoid) but doesn't make any difference.
Any idea on what is happening?
Thanks.
Sidekiq is so fast that it executes your job before the database has committed the transaction. Use after_commit to create the job.
Ok, my fault. You need to start a new Heroku Worker Dyno in order to make Sidekiq work (it doesn't do it automatically).
I have around 10 workers that performs a job that includes the following:
user = User.find_or_initialize_by(email: 'some-email#address.com')
if user.new_record?
# ... some code here that does something taking around 5 seconds or so
elsif user.persisted?
# ... some code here that does something taking around 5 seconds or so
end
user.save
The problem is that at certain times, two or more workers run this code at the exact time, and thus I later found out that two or more Users have the same email, in which I should always end up only unique emails.
It is not possible for my situation to create DB Unique Indexes for email as unique emails are conditional -- some Users should have unique email, some do not.
It is noteworthy to mention that my User model has uniqueness validations, but it still doesn't help me because, between .find_or_initialize_by and .save, there is a code that is dependent if the user object is already created or not.
I tried Pessimistic and Optimistic locking, but it didn't help me, or maybe I just didn't implement it properly... should you have some suggestions regarding this.
The solution I can only think of is to lock the other threads (Sidekiq jobs) whenever these lines of codes get executed, but I am not too sure how to implement this nor do I know if this is even a suggestable approach.
I would appreciate any help.
EDIT
In my specific case, it is gonna be hard to put email parameter in the job, as this job is a little more complex than what was just said above. The job is actually an export script in which a section of the job is the code above. I don't think it's also possible to separate the functionality above into another separate worker... as the whole job flow should be serial and that no parts should be processed parallely / asynchronously. This job is just one of the jobs that are managed by another job, in which ultimately is managed by the master job.
Pessimistic locking is what you want but only works on a record that exists - you can't use it with new_record? because there's nothing to lock in the DB yet.
I managed to solve my problem with the following:
I found out that I can actually add a where clause in Rails DB Uniqueness Partial Index, and thus I can now set up uniqueness conditions for different types of Users on the database-level in which other concurrent jobs will now raise an ActiveRecord::RecordNotUnique error if already created.
The only problem now then is the code in between .find_or_initialize_by and .save, since those are time-dependent on the User objects in which always only one concurrent job should always get a .new_record? == true, and other concurrent jobs should then trigger the .persisted? == true as one job would always be first to create it, but... all of these doesn't work yet because it is only at the line .save where the db uniqueness index validation gets called. Therefore, I managed to solve this problem by putting .save before those conditions, and at the same time I added a rescue block for .save which then adds another job to the queue of itself should it trigger the ActiveRecord::RecordNotUnique error, to make sure that async jobs won't get conflicts. The code now looks like below.
user = User.find_or_initialize_by(email: 'some-email#address.com')
begin
user.save
is_new_record = user.new_record?
is_persisted = user.persisted?
rescue ActiveRecord::RecordNotUnique => exception
MyJob.perform_later(params_hash)
end
if is_new_record
# do something if not yet created
elsif is_persisted
# do something if already created
end
I would suggest a different architecture to bypass the problem.
How about a producer-worker model, where one master Sidekiq process gets a list of email addresses, and then spawns a worker Sidekiq process for each email? Sidekiq makes this easy with a dedicated queue for master and workers to communicate.
Doing so, the email address becomes an input parameter of workers, so we know by construction that workers will not stump on each other data.
As per the resque-status home page on GitHub I should be able to pass back data from a job. For some reason this does not seem to be working for me. If anyone else has encountered this problem and worked around it I would like to know how.
I am using resque-status with JRuby 1.6.5 in a Rails 3.2.3 application.
Passing back data from the job
You may want to save data from inside the job to access it from outside the job.
A common use-case is web-triggered jobs that create files, later available for download by the user.
A Status is actually just a hash, so inside a job you can do:
status['filename'] = '/myfilename'
Also, all the status setting methods take any number of hash arguments. So you could do:
complete('filename' => '/myfilename')
Apparently such functionality is not implemented, as read on
https://github.com/quirkey/resque-status/issues/66
we've found a work around using the function set_status to add the required data to the status hash:
set_status({"my variable" => "my value" })
hope this helps!