In Rails 5.2.2.4, I'm getting this inexplicable behavior where a thread_mattr_accessor is becoming nil randomly.
Given this module:
module Test
thread_mattr_accessor :max_installments
self.max_installments = 12
end
Out of 6 pager refreshes (using Puma in development), I get a comparison of Integer with nil failed when comparing an integer with that variable.
On the Rails red exception page, I can use the bottom controller to confirm it's actually nil!
However, simply refreshing the page some times, I will eventually get a normal page load with that variable being set.
What might be setting that to nil? There's no code at all that sets that variable anywhere else other than the self.max_installments= in the class definition above.
Module Test
module Test
thread_mattr_accessor :max_installments
self.max_installments = 12
end
Normal use
puts "max_installments is: #{Test.max_installments.inspect}"
# max_installments is: 12
In Thread use
Thread.new { puts "max_installments in thread is #{Test.max_installments.inspect}" }
# max_installments in thread is nil
Set value inside the thread
Thread.new do
Test.max_installments = 99
puts "max_installments in thread now is #{Test.max_installments.inspect}"
end
# max_installments in thread now is 99
Thread.new { puts "max_installments in another thread is #{Test.max_installments.inspect}" }
# max_installments in another thread is nil
puts "max_installments is: #{Test.max_installments}"
# max_installments is: 12
ASAP this commit must be released and we can set a default value to use in threads like this
module Test
thread_mattr_accessor :max_installments, default: 12
self.max_installments = 12
end
For now, if you go use inside threads, you need set value before to avoid get nil.
Related
In a Rails 6.x app, I have a controller method which backgrounds queries that take longer than 2 minutes (to avoid a browser timeout), advises the user, stores the results, and sends a link that can retrieve them to generate a live page (with Highcharts charts). This works fine.
Now, I'm trying to implement the same logic with a method that backgrounds the creation of a report, via a Tempfile, and attaches the contents to an email, if the query runs too long. This code works just fine if the 2-minute timeout is NOT reached, but the Tempfile is empty at the commented line if the timeout IS reached.
I've tried wrapping the second part in another thread, and wrapping the internals of each thread with a mutex, but this is all getting above my head. I haven't done a lot of multithreading, and every time I do, I feel like I stumble around till I get it. This time, I can't even seem to stumble into it.
I don't know if the problem is with my thread(s), or a race condition with the Tempfile object. I've had trouble using Tempfiles before, because they seem to disappear quicker than I can close them. Is this one getting cleaned up before it can be sent? The file handle actually still exists on the file system at the commented point, even though it's empty, so I'm not clear on what's happening.
def report
queue = Queue.new
file = Tempfile.new('report')
thr = Thread.new do
query = %Q(blah blah blah)
#calibrations = ActiveRecord::Base.connection.exec_query query
query = %Q(blah blah blah)
#tunings = ActiveRecord::Base.connection.exec_query query
if queue.empty?
unless #tunings.empty?
CSV.open(file.path, 'wb') do |csv|
csv << ["headers...", #parameters].flatten
#calibrations.each do |c|
line = [c["h1"], c["h2"], c["h3"], c["h4"], c["h5"], c["h6"], c["h7"], c["h8"]]
t = #tunings.select { |t| t["code"] == c["code"] }.first
#parameters.each do |parameter|
line << t[parameter.downcase]
end
csv << line
end
end
send_data file.read, :type => 'text/csv; charset=iso-8859-1; header=present', :disposition => "attachment; filename=\"report.csv\""
end
else
# When "timed out", `file` is empty here
NotificationMailer.report_ready(current_user, file.read).deliver_later
end
end
give_up_at = Time.now + 120.seconds
while Time.now < give_up_at do
if !thr.alive?
break
end
sleep 1
end
if thr.alive?
queue << "Timeout"
render html: "Your report is taking longer than 2 minutes to generate. To avoid a browser timeout, it will finish in the background, and the report will be sent to you in email."
end
end
The reason the file is empty is because you are giving the query 120 seconds to complete. If after 120 seconds that has not happened you add "Timeout" to the queue. The query is still running inside the thread and has not reached the point where you check if the queue is empty or not. When the query does complete, since the queue is now not empty, you skip the part where you write the csv file and go to the Notification.report line. At that point the file is still empty because you never wrote anything into it.
In the end I think you need to rethink the overall logic of what you are trying to accomplish and there needs to be more communication between the threads and the top level.
Each thread needs to tell the top level if it has already sent the result, and the top level needs to let the thread know that its past time to directly send the result, and instead should email the result.
Here is some code that I think / hope will give some insight into how to approach this problem.
timeout_limit = 10
query_times = [5, 15, 1, 15]
timeout = []
sent_response = []
send_via_email = []
puts "time out is set to #{timeout_limit} seconds"
query_times.each_with_index do |query_time, query_id|
puts "starting query #{query_id} that will take #{query_time} seconds"
timeout[query_id] = false
sent_response[query_id] = false
send_via_email[query_id] = false
Thread.new do
## do query
sleep query_time
unless timeout[query_id]
puts "query #{query_id} has completed, displaying results now"
sent_response[query_id] = true
else
puts "query #{query_id} has completed, emailing result now"
send_via_email[query_id] = true
end
end
give_up_at = Time.now + timeout_limit
while Time.now < give_up_at
break if sent_response[query_id]
sleep 1
end
unless sent_response[query_id]
puts "query #{query_id} timed out, we will email the result of your query when it is completed"
timeout[query_id] = true
end
end
# simulate server environment
loop { }
=>
time out is set to 10 seconds
starting query 0 that will take 5 seconds
query 0 has completed, displaying results now
starting query 1 that will take 15 seconds
query 1 timed out, we will email the result of your query when it is completed
starting query 2 that will take 1 seconds
query 2 has completed, displaying results now
starting query 3 that will take 15 seconds
query 1 has completed, emailing result now
query 3 timed out, we will email the result of your query when it is completed
query 3 has completed, emailing result now
I am using beaneater/beanstalk in my app for maintaining the job queues.
https://github.com/nesquena/backburner
My global config file for backburner look like -
Backburner.configure do |config|
config.beanstalk_url = ["beanstalk://#{CONFIG['beanstalk']['host']}:#{CONFIG['beanstalk']['port']}"]
config.tube_namespace = CONFIG['beanstalk']['tube_name']
config.on_error = lambda { |e| puts e }
config.max_job_retries = 5 # default 0 retries
config.retry_delay = 30 # default 5 seconds
config.default_priority = 65536
config.respond_timeout = 120
config.default_worker = Backburner::Workers::Simple
config.logger = Logger.new('log/backburner.log')
config.priority_labels = { :custom => 50, :useless => 1000 }
config.reserve_timeout = nil
end
I want to set different retry limit and retry delay for different jobs.
I was looking at rubydoc for corresponding variable/function. As per this rubydoc link, I tried configuring retry_limit locally in a worker as:
One specific worker look like -
class AbcJob
include Backburner::Queue
queue "abc_job" # defaults to 'backburner-jobs' tube
queue_priority 10 # most urgent priority is 0
queue_respond_timeout 300 # number of seconds before job times out
queue_retry_limit 2
def self.perform(abc_id)
.....Task to be done.....
end
end
However, it is still picking up the retry limit from global config file and retrying it 5 times instead of 2. Any thing that I am missing here?
How can I over write retry limit and retry delay locally?
I could not find right way to do it but I found a solution.
I am putting entire body of perform in begin-rescue block and in case of failure I am re-enqueing it with custom delay. Also, to keep the track of number of retries I made it an argument which I am enqueueing.
class AbcJob
include Backburner::Queue
queue "abc_job" # defaults to 'backburner-jobs' tube
queue_priority 10 # most urgent priority is 0
queue_respond_timeout 300 # number of seconds before job times out
def self.perform(abc_id, attempt = 1)
begin
.....Task to be done.....
rescue StandardError => e
# Any notification method so that you can know about failure reason and fix it before next retry
# I am using NotificationMailer with e.message as body to debug
# Any function you want your retry delay to be, I am using quadratic
delay = attempt * attempt
if attempt + 1 < GlobalConstant::MaxRetryCount
Backburner::Worker.enqueue(AbcJob, [abc_id, attempt + 1], delay: delay.minute)
else
raise # if you want your jobs to be buried eventually
end
end
end
I have kept the default value of attempt to 1 so that magic nuber 1 do not appear in code which might raise question about why are we passing a constant. For enqueueing from other places in code you can use simple enqueue
Backburner::Worker.enqueue(AbcJob, abc_id)
When i am using win32ole as stand alone application at that time everything seems working fine, as soon as i put into my rails application which is running on mongrel server it goes into infinite loop.
I am trying to access "https://microsoft/sharepoint/document.doc"
def generatertm(issue)
begin
word = WIN32OLE.new('word.application')
logger.debug("Word Initialized...")
word.visible = true
myDocLink = "https://microsoft/sharepoint/url.doc"
myFile = word.documents.open(myDocLink)
logger.debug("File Opened...")
puts "Started Reading bookmarks..."
myBookMarks = myFile.Bookmarks puts "bookmarks fetched working background task..."
print ("Bookmakr Count : " + myBookMarks.Count.to_s + "\n")
myBookMarks.each do |i|
logger.warn ("Bookmark Name : " + i.Name + "\n")
end
rescue WIN32OLERuntimeError => e
puts e.message
puts e.backtrace.inspect
else
ensure
word.activedocument.close( true ) # presents save dialog box
#word.activedocument.close(false) # no save dialog, just close it
word.quit
end
end
When i run this code stand alone at that time one Pop up come for Microsoft Share point credentials. however in mongrel rails it goes into infinite loop.
Do i need to handle this pop up to appear through Rails?
Have you looked into patching the win32ole.rb file?
Basically, here's the reason for the patch:
t turns out that win32ole.rb patches the thread to call the windows
OleInitialize() & OleUninitialize() functions around the yield to the
block. However, the MS documentation for CoInitialize (which
OleInitialize calls internally) state that: "the first thread in the
application that calls CoInitialize with 0 (or CoInitializeEx with
COINIT_APARTMENTTHREADED) must be the last thread to call
CoUninitialize. Otherwise, subsequent calls to CoInitialize on the STA
will fail and the application will not work."
http://msdn.microsoft.com/en-us/library/ms678543(v=VS.85).aspx
And here's the modified win32ole.rb file to fix the threading issue:
require 'win32ole.so'
# Fail if not required by main thread.
# Call OleInitialize and OleUninitialize for main thread to satisfy the following:
#
# The first thread in the application that calls CoInitialize with 0 (or CoInitializeEx with COINIT_APARTMENTTHREADED)
# must be the last thread to call CoUninitialize. Otherwise, subsequent calls to CoInitialize on the STA will fail and the
# application will not work.
#
# See http://msdn.microsoft.com/en-us/library/ms678543(v=VS.85).aspx
if Thread.main != Thread.current
raise "Require win32ole.rb from the main application thread to satisfy CoInitialize requirements."
else
WIN32OLE.ole_initialize
at_exit { WIN32OLE.ole_uninitialize }
end
# re-define Thread#initialize
# bug #2618(ruby-core:27634)
class Thread
alias :org_initialize :initialize
def initialize(*arg, &block)
if block
org_initialize(*arg) {
WIN32OLE.ole_initialize
begin
block.call(*arg)
ensure
WIN32OLE.ole_uninitialize
end
}
else
org_initialize(*arg)
end
end
end
http://cowlibob.co.uk/ruby-threads-win32ole-coinitialize-and-counin
I'm trying to call two lengthy commands in a when statement, but for some reason, because of its syntax, it performs two of the commands twice when it is called :
#email = Email.find(params[:id])
delivery = case #email.mail_type
# when "magic_email" these two delayed_jobs perform 2x instead of 1x. Why is that?
when "magic_email" then Delayed::Job.enqueue MagicEmail.new(#email.subject, #email.body)
Delayed::Job.enqueue ReferredEmail.new(#email.subject, #email.body)
when "org_magic_email" then Delayed::Job.enqueue OrgMagicEmail.new(#email.subject, #email.body)
when "all_orgs" then Delayed::Job.enqueue OrgBlast.new(#email.subject, #email.body)
when "all_card_holders" then Delayed::Job.enqueue MassEmail.new(#email.subject, #email.body)
end
return delivery
How can I make it so that when I hit when "magic_email", it only renders both those delayed jobs once ?
I have tried this with following example:
q = []
a = case 1
when 1 then q.push 'ashish'
q.push 'kumar'
when 2 then q.push 'test'
when 4 then q.push 'another test'
end
puts a.inspect #######["ashish", "kumar"]
This is working fine. It means your case-when syntax is ok. It might be you have aome other problem.
You are calling return delivery and delivery varible may be having the value to call the delayed job again. It depends on what the then statement returns, so try not to return anything if possible. I believe you want to do the delayed job and not return anything by using the function.
Perhaps you should just have the case and dont store it in any variable. I mean delivery variable has no purpose here.
Greetings,
I want to tinker with the global memcache object, and I found the following problems.
Cache is a constant
Cache is a module
I only want to modify the behavior of Cache globally for a small section of code for a possible major performance gain.
Since Cache is a module, I can't re-assign it, or encapsulate it.
I Would Like To Do This:
Deep in a controller method...
code code code...
old_cache = Cache
Cache = MyCache.new
code code code...
Cache = old_cache
code code code...
However, since Cache is a constant I'm forbidden to change it. Threading is not an issue at the moment. :)
Would it be "good manners" for me to just alias_method the special code I need
just for a small section of code and then later unalias it again? That doesn't
pass the smell test IMHO.
Does anyone have any ideas?
TIA,
-daniel
But you can overwrite constants in Ruby (regardless of whether it's a module or class or simple other object):
MyConst = 1
# do stuff...
old_my_const = MyConst
MyConst = 5
puts "MyConst is temporarily #{MyConst}"
MyConst = old_my_const
puts "MyConst is back to #{MyConst}"
Output:
a.rb:6: warning: already initialized constant MyConst
MyConst is temporarily 5
a.rb:8: warning: already initialized constant MyConst
MyConst is back to 1
The warnings are simply that: warnings. Your code will continue to run the same.
Okay, maybe the warnings are unacceptable in your situation for some reason. Use this suppress_all_warnings method I've written. Example includes reassigning a module.
def suppress_all_warnings
old_verbose = $VERBOSE
begin
$VERBOSE = nil
yield if block_given?
ensure
# always re-set to old value, even if block raises an exception
$VERBOSE = old_verbose
end
end
module OriginalModule
MyConst = 1
end
module OtherModule
MyConst = 5
end
def print_const
puts OriginalModule::MyConst
end
print_const
suppress_all_warnings do
old_module = OriginalModule
OriginalModule = OtherModule
print_const
OriginalModule = old_module
end
print_const
Now you get the correct output, but without the warnings:
1
5
1