Ruby please give a simple NON-thread safe example - ruby-on-rails

Can someone please give a concrete example demonstrating non-thread safety? (in a similar manner to a functioning version of mine below if possible)
I need an example class that demonstrates a non-thread safe operation such that I can assert on the failure, and then enforce a Mutex such that I can test that my code is then thread safe.
I have tried the following with no success, as the threads do not appear to run in parallel. Assuming the ruby += operator is not threadsafe, this test always passes when it should not:
class TestLock
attr_reader :sequence
def initialize
#sequence = 0
end
def increment
#sequence += 1
end
end
#RSpec test
it 'does not allow parallel calls to increment' do
test_lock = TestLock.new
threads = []
list1 = []
list2 = []
start_time = Time.now + 2
threads << Thread.new do
loop do
if Time.now > start_time
5000.times { list1 << test_lock.increment }
break
end
end
end
threads << Thread.new do
loop do
if Time.now > start_time
5000.times { list2 << test_lock.increment }
break
end
end
end
threads.each(&:join) # wait for all threads to finish
expect(list1 & list2).to eq([])
end

Here is an example which instead of find a race condition with addition, concatenation, or something like that, uses a blocking file write.
To summarize the parts:
file_write method performs a blocking write for 2 seconds.
file_read reads the file and assigns it to a global variable to be referenced elsewhere.
NonThreadsafe#test calls these methods in succession, in their own threads, without a mutex. sleep 0.2 is inserted between the calls to ensure that the blocking file write has begun by the time the read is attempted. join is called on the second thread, so we be sure it's set the read value to a global variable. It returns the read-value from the global variable.
Threadsafe#test does the same thing, but wraps each method call in a mutex.
Here it is:
module FileMethods
def file_write(text)
File.open("asd", "w") do |f|
f.write text
sleep 2
end
end
def file_read
$read_val = File.read "asd"
end
end
class NonThreadsafe
include FileMethods
def test
`rm asd`
`touch asd`
Thread.new { file_write("hello") }
sleep 0.2
Thread.new { file_read }.join
$read_val
end
end
class Threadsafe
include FileMethods
def test
`rm asd`
`touch asd`
semaphore = Mutex.new
Thread.new { semaphore.synchronize { file_write "hello" } }
sleep 0.2
Thread.new { semaphore.synchronize { file_read } }.join
$read_val
end
end
And tests:
expect(NonThreadsafe.new.test).to be_empty
expect(Threadsafe.new.test).to eq("hello")
As for an explanation. The reason the non-threadsafe shows the file's read val as empty is because the blocking writing operation is still happening when the read takes place. When you use synchronize the Mutex, though, the write will complete before the read. Note also that the .join in the threadsafe example takes longer than in the non-threadsafe value - that's because it's sleeping for the full duration specified in the write thread.

Related

Speed up rake task by using typhoeus

So i stumbled across this: https://github.com/typhoeus/typhoeus
I'm wondering if this is what i need to speed up my rake task
Event.all.each do |row|
begin
url = urlhere + row.first + row.second
doc = Nokogiri::HTML(open(url))
doc.css('.table__row--event').each do |tablerow|
table = tablerow.css('.table__cell__body--location').css('h4').text
next unless table == row.eventvenuename
tablerow.css('.table__cell__body--availability').each do |button|
buttonurl = button.css('a')[0]['href']
if buttonurl.include? '/checkout/external'
else
row.update(row: buttonurl)
end
end
end
rescue Faraday::ConnectionFailed
puts "connection failed"
next
end
end
I'm wondering if this would speed it up, Or because i'm doing a .each it wouldn't?
If it would could you provide an example?
Sam
If you set up Typhoeus::Hydra to run parallel requests, you might be able to speed up your code, assuming that the Kernel#open calls are what's slowing you down. Before you optimize, you might want to run benchmarks to validate this assumption.
If it is true, and parallel requests would speed it up, you would need to restructure your code to load events in batches, build a queue of parallel requests for each batch, and then handle them after they execute. Here's some sketch code.
class YourBatchProcessingClass
def initialize(batch_size: 200)
#batch_size = batch_size
#hydra = Typhoeus::Hydra.new(max_concurrency: #batch_size)
end
def perform
# Get an array of records
Event.find_in_batches(batch_size: #batch_size) do |batch|
# Store all the requests so we can access their responses later.
requests = batch.map do |record|
request = Typhoeus::Request.new(your_url_build_logic(record))
#hydra.queue request
request
end
#hydra.run # Run requests in parallel
# Process responses from each request
requests.each do |request|
your_response_processing(request.response.body)
end
end
rescue WhateverError => e
puts e.message
end
private
def your_url_build_logic(event)
# TODO
end
def your_response_processing(response_body)
# TODO
end
end
# Run the service by calling this in your Rake task definition
YourBatchProcessingClass.new.perform
Ruby can be used for pure scripting, but it functions best as an object-oriented language. Decomposing your processing work into clear methods can help clarify your code and help you catch things like Tom Lord mentioned in the comments on your question. Also, instead of wrapping your whole script in a begin..rescue block, you can use method-level rescues as in #perform above, or just wrap #hydra.run.
As a note, .all.each is a memory hog, and is thus considered a bad solution to iterating over records: .all loads all of the records into memory before iterating over them with .each. To save memory, it's better to use .find_each or .find_in_batches, depending on your use case. See: http://api.rubyonrails.org/classes/ActiveRecord/Batches.html

How to DRY a list of functions in ruby that are differ only by a single line of code?

I have a User model in a ROR application that has multiple methods like this
#getClient() returns an object that knows how to find certain info for a date
#processHeaders() is a function that processes output and updates some values in the database
#refreshToken() is function that is called when an error occurs when requesting data from the object returned by getClient()
def transactions_on_date(date)
if blocked?
# do something
else
begin
output = getClient().transactions(date)
processHeaders(output)
return output
rescue UnauthorizedError => ex
refresh_token()
output = getClient().transactions(date)
process_fitbit_rate_headers(output)
return output
end
end
end
def events_on_date(date)
if blocked?
# do something
else
begin
output = getClient().events(date)
processHeaders(output)
return output
rescue UnauthorizedError => ex
refresh_token()
output = getClient().events(date)
processHeaders(output)
return output
end
end
end
I have several functions in my User class that look exactly the same. The only difference among these functions is the line output = getClient().something(date). Is there a way that I can make this code look cleaner so that I do not have a repetitive list of functions.
The answer is usually passing in a block and doing it functional style:
def handle_blocking(date)
if blocked?
# do something
else
begin
output = yield(date)
processHeaders(output)
output
rescue UnauthorizedError => ex
refresh_token
output = yield(date)
process_fitbit_rate_headers(output)
output
end
end
end
Then you call it this way:
handle_blocking(date) do |date|
getClient.something(date)
end
That allows a lot of customization. The yield call executes the block of code you've supplied and passes in the date argument to it.
The process of DRYing up your code often involves looking for patterns and boiling them down to useful methods like this. Using a functional approach can keep things clean.
Yes, you can use Object#send: getClient().send(:method_name, date).
BTW, getClient is not a proper Ruby method name. It should be get_client.
How about a combination of both answers:
class User
def method_missing sym, *args
m_name = sym.to_s
if m_name.end_with? '_on_date'
prop = m_name.split('_').first.to_sym
handle_blocking(args.first) { getClient().send(prop, args.first) }
else
super(sym, *args)
end
end
def respond_to? sym, private=false
m_name.end_with?('_on_date') || super(sym, private)
end
def handle_blocking date
# see other answer
end
end
Then you can call "transaction_on_date", "events_on_date", "foo_on_date" and it would work.

blocks in silly blocks rspec testing

I had the following tests given to me as an exercise:
require "silly_blocks"
describe "some silly block functions" do
describe "reverser" do
it "reverses the string returned by the default block" do
result = reverser do
"hello"
end
result.should == "olleh"
end
it "reverses each word in the string returned by the default block" do
result = reverser do
"hello dolly"
end
result.should == "olleh yllod"
end
end
describe "adder" do
it "adds one to the value returned by the default block" do
adder do
5
end.should == 6
end
it "adds 3 to the value returned by the default block" do
adder(3) do
5
end.should == 8
end
end
describe "repeater" do
it "executes the default block" do
block_was_executed = false
repeater do
block_was_executed = true
end
block_was_executed.should == true
end
it "executes the default block 3 times" do
n = 0
repeater(3) do
n += 1
end
n.should == 3
end
it "executes the default block 10 times" do
n = 0
repeater(10) do
n += 1
end
n.should == 10
end
end
end
I was able to solve them with the following code:
def reverser
k = []
x = yield.split(" ")
x.each do |y|
n = y.reverse
k.push(n)
end
m = k.join(" ")
m
end
def adder(num=1, &block)
block.call + num
end
def repeater(num=1, &block)
for i in (1..num) do
block.call
end
end
However I some of these concepts I do not understand all that well. For example:
What exactly does the & symbol in the &block parameter mean?
Similarly what is block.call and where is the actual block object I am assuming its calling?
Could I theoretically use another method on block if I wanted to achieve something else?
Also where can I learn a bit more about blocks
This exercise was a bit above my current knowledge.
It means "this is the block parameter". You are not bound to calling it &block, so there needs to be a way to separate it from the other arguments. The same notation is used to pass arguments to a function as block as opposed to normal arguments (see below)
block.call is exactly the same thing as yield. The difference is that you can use block to access the block itself without calling it immediately. For example, you could store the block for later execution. This is a common pattern known as lazy evaluation.
Yes, you can also pass different things than a do/end block as the &block parameter. See below for some examples.
#UriAgassi gave you an excellent link.
Here are some other things you can pass as block argument. First, just a simple method that takes a block for demonstration:
def reverser(&block)
block.call.reverse
end
You can now pass a standard block
reverser do
"hello"
end
#=> "olleh"
Or, in alternative block syntax, used for inline style
reverser { "hello" }
#=> olleh
You can also pass a lambda or proc, which is similar to a block.
By using the &block notation you can pass a variable as block argument:
my_block = lambda { "hello world!" }
reverser(&my_block)
#=> "!dlrow olleh"
Or, in alternative lambda syntax
my_block = -> { "hello world!" }
reverser(&my_block)
#=> "!dlrow olleh"
You can even take an existing method and pass it as block argument
here you can see the great advantage of blocks: They are evaluated
when block.call is executed, not when the code is loaded. Here this
means that the string will change every time accordingly.
def foobar
"foobar at #{Time.now}"
end
reverser(&method(:foobar))
#=> "0020+ 15:42:90 02-50-4102 ta raboof"
#=> "0020+ 31:52:90 02-50-4102 ta raboof"
You can do cool stuff with this, for example:
[1, 2, 3].each(&method(:puts))
1
2
3
#=> [1, 2, 3]
But remember not to overdo it, Ruby is all about expressive and readable code. Use these techniques when they enhance your code, but use simpler ways if possible.
Finally, here is also an example of lazy evaluation:
class LazyReverser
def initialize(&block)
#block = block
end
def reverse
#block.call.reverse
end
end
reverser = LazyReverser.new do
# some very expensive computation going on here,
# maybe we do not even need it, so lets use the
# lazy reverser!
"hello dolly"
end
# now go and do some other stuff
# it is not until later in the program, that we can decide
# whether or not we even need to call the block at all
if some_condition
reverser.reverse
#=> "yllod olleh"
else
# we did not need the result, so we saved ourselves
# the expensive computation in the block altogether!
end

Ruby Thread.new doing weird thing to class call

I have a piece of code:
config_item_relation = OTRS::Relation.new
config_item_threads = []
config_items.each do |ci|
config_item_threads << Thread.new do
config_item_relation << Tracker::ConfigItem.object_preprocessor(ci.first)
end
end
Which is causing this error:
LoadError: Expected app/models/tracker/config_item.rb to define Tracker::ConfigItem
If I comment out the thread creation as such:
config_item_relation = self.superclass::Relation.new
config_item_threads = []
config_items.each do |ci|
#config_item_threads << Thread.new do
config_item_relation << Tracker::ConfigItem.object_preprocessor(ci.first)
#end
end
The code runs just fine, except of course it won't do it in separate threads.
The referenced file in the error is indeed defining Tracker::ConfigItem.
class Tracker::ConfigItem < OTRS::ConfigItem
It's class I use many many places elsewhere with no issue until I use it with Thread here.
I have the same Thread usage against a different, but extremely similar class (same inheritance) in the same code chunk that works perfectly fine:
ticket_threads = []
if tickets
ticket_relation = self.superclass::Relation.new
tickets.each do |t|
ticket_threads << Thread.new do
ticket_relation << Tracker::Ticket.object_preprocessor(t)
end
end
end
Am I missing something with these threads?
It looks like Tracker::ConfigItem isn't loaded at this point so multiple threads start to try to load the same file.
Require is very much thread dangerous in current versions of ruby and can give rise to a variety of race conditions. I'd stick a
require_dependency 'tracker/config_item'
The file above so that you are sure that the requiring happens on the main thread rather than your child threads fighting it out

Writing an around_each filter in ruby for every method within a block.

I need a method that takes a block, and performs something similar to an around_each filter for every method within the block.
For instance:
def method_that_takes_block
(#threads ||= Array.new) << Thread.new {yield if block.given?}
end
method_that_takes_a_block do
method_one
method_two
method_three
end
In this instance I would like my method that takes a block to Thread each method within the block and pushes that thread to the #threads array. Essentially I'm just looking for a DRY way to wrap a thread around every method called within a block.
You can't directly wrap a thread around each statement in the block body, if they can be arbitrary statements; there's no way to get that sort of control over the execution of a block body in Ruby. If you can restrict what goes in the block body, you have some more flexibility.
If each of the statements you are executing is simply a method call, as you imply in your example, you can use instance_exec to execute that block on a proxy object, which uses method_missing to spawn a new thread and then forward the method call on to the real object (or do whatever wrapper you're interested in; for the sake of example, I'll just wrap with some print statements):
class Proxy
def initialize obj
#obj = obj
end
def method_missing method, *args
puts "<wrapper>"
#obj.send method, *args
puts "</wrapper>"
end
end
class MethodWrapper
def tell_me_a_joke
puts "Knock, knock?"
end
def whos_there
puts "Orange"
end
def orange_who
puts "Orange you glad I didnt say banana?"
end
def wrap_around &blk
Proxy.new(self).instance_exec &blk
end
end
And here's how you can use it:
>> MethodWrapper.new.wrap_around { tell_me_a_joke; whos_there; orange_who }
<wrapper>
Knock, knock?
</wrapper>
<wrapper>
Orange
</wrapper>
<wrapper>
Orange you glad I didnt say banana?
</wrapper>
=> nil
The previous follows the pattern that you gave in your question, but it's less than ideal as only methods that are forwarded to the underlying object get wrapped:
>> MethodWrapper.new.wrap_around { tell_me_a_joke; puts "something" }
<wrapper>
Knock, knock?
</wrapper>
something
=> nil
You could instead just use instance_exec directly, and call the a wrapper method that takes a block, to get almost the same effect, though slightly less DRY as you need to call your wrapper method each time:
class SimpleWrapper
def tell_me_a_joke
puts "Knock, knock?"
end
def whos_there
puts "Interrupting cow"
end
def interrupting_co
puts "Moooooo!"
end
def wrap
puts "<wrap>"
yield
puts "</wrap>"
end
end
And in use:
>> SimpleWrapper.new.instance_exec do
wrap { tell_me_a_joke }
wrap { whos_there }
wrap { interrupting_co }
wrap { puts "Something" }
end
<wrap>
Knock, knock?
</wrap>
<wrap>
Interrupting cow
</wrap>
<wrap>
Moooooo!
</wrap>
<wrap>
Something
</wrap>
=> nil
You can also do it like this:
%w(method_one method_two method_three).each do |method|
(#threads ||= Array.new) << Thread.new { self.send(method) }
end

Resources