I am trying to cache rails request to an external service which takes a proc.
For me to be able to differentiate 2 requests, its important that the proc is part of the Rails cache key.
However multiple requests with the same proc and the values in the proc code, would still evaluate to a different proc object each time.
Eg:
#<Proc:0x007ff6f675dd08#/Users/app/development/work/runner.rb:10>
#<Proc:0x007ff6ffb50790#/Users/app/development/work/runner.rb:10>
Hence , I get cache miss even for same requests.
How do I use the block/proc in the cache key that evaluates to same if the procs are same in terms of code and variable values.
The service call is something like
def some_method
call_service do
a= 3;
b=6
end
end
def call_service(block)
Rails.cache.fetch(Date.today, block) {
external_service(&block)
}
end
I want to be able to compare blocks :
eg
{a=3, a*a} == {a=3, a*a} => True
{a=3, a*a} == {a=4, a*a} => false
{a=3, a*a} == {a=4, a*a*a} => False
I tried to use,
block.source
RubyVM::InstructionSequence.of(block).disasm
But none of them captures the state of the block, i.e the values of the variables (a=3 etc)
What the best way to achieve this type of caching in rails ?
P.S:
Using Rails4 and reddis as the cache
Each time you define an inline block with do ... end you will get a different Proc object. You will have to go out of your way to be absolutely certain you're sending through exactly the same Proc:
def some_method
#my_call ||= lambda {
a = 3
b = 6
}
call_service(&#my_call)
end
This is probably a bad idea from the outset. What you'd be better off doing is passing in a cache key:
call_service("3/6") do
# ...
end
Then you cache against this consistent key, not the arbitrary Proc.
You might get some mileage this way. This function returns a simple Proc object. The object will have a source location and a binding that contains the state of arguments a, b and c when the Proc was built.
def build_proc(a, b, c)
Proc.new { puts "I was built with params #{a} #{b} #{c}!" }
end
So we can build our Proc objects--
2.0.0-p195 :121 > p1 = build_proc(1, 2, 3)
=> #<Proc:0x98849e8#(irb):118>
2.0.0-p195 :122 > p2 = build_proc(2, 4, 6)
=> #<Proc:0x985e57c#(irb):118>
To get the state of an argument you can call binding.eval('argument') against a Proc object. This runs code in the context of that Proc's binding. So you can say--
p1.binding.eval('[a, b, c]') == p2.binding.eval('[a, b, c]')
2.0.0-p195 :127 > p1.binding.eval('[a, b, c]') == p2.binding.eval('[a, b, c]')
=> false
2.0.0-p195 :128 > p2 = build_proc(1, 2, 3)
=> #<Proc:0x97ae4b0#(irb):118>
2.0.0-p195 :129 > p1.binding.eval('[a, b, c]') == p2.binding.eval('[a, b, c]')
=> true
And obviously you can compare source locations--
p1.source_location == p2.source_location &&
p1.binding.eval('[a, b, c]') == p2.binding.eval('[a, b, c]')
You'd need a way to extract the arguments out of the binding in a universal way and you'd want to combine their hashes rather than build an array for comparison (e.g. 'a.hash ^ b.hash ^ c.hash').
This does feel horrible though! At the very least, overriding :== on Procs is probably a bad idea.
Related
I am performing an operation on hash values, lets say :
hash = { a: true, b: false, c: nil }
I am executing an each loop on hash but I want to skip keys b and c. I don't want to delete these from hash.
I have tried:
hash = { a: true, b: false, c: nil}
hash.except(:c)
{ a: true, b: false, c: nil}
But it is not working. I am using ruby 2.4.2
Actually hash.except(:c) returns { a: true, b: false } as expected. Since you're using Rails it should work. The only subtle moment, that I want to take a note on is that:
hash.except([:b, :c])
won't work. You need to use
hash.except(:b, :c)
instead.
For general solution you need to use splat operator:
keys = [:b, :c]
hash.except(*keys)
If you simply need to skip it while looping over the hash pairs, I personally would steer clear of using except, and use the uglier next if key == whatever within the loop.
A equality check between symbols is cheap, about the the most low overhead thing that can be done in Ruby, the basic equivalent of comparing two integers or booleans.
except on the other hand is not, especially as the size of the hash grows. You are creating a new cloned hash, minus the specified values, every time you call it. Even with a small hash, you are creating a new object needlessly.
I understand that many Ruby users are forever in pursuit of the "one-liners" or absolute shortest amount of code possible, I am guilty of it myself, but it needs done mindfully of whats going on beneath the surface, or you are creating less efficient code, not more.
So although not as "pretty", this would be more efficient:
hash.each do |k, v|
next if k == :b || k == :c
# Do stuff
end
EDIT
I was curious of the performance difference between what I was stating, and the use of except, and the resulting differences are significant.
First, I added the source for except, I didn't have Rails installed. This is straight from the source code from activesupport/lib/active_support/core_ext/hash/except.rb.
class Hash
def except!(*keys)
keys.each { |key| delete(key) }
self
end
def except(*keys)
dup.except!(*keys)
end
end
Then I did some benchmarking. I figured a 1,000,000 samples was enough for one run.
require 'benchmark'
hash = { a: true, b: false, c: nil }
count = 1_000_000
Benchmark.bm do |bm|
bm.report("except") do
count.times do
hash.except(:b, :c).each do |k, v|
# Do nothing
end
end
end
bm.report("next") do
count.times do
hash.each do |k, v|
next if k == :b || k == :c
# Do nothing
end
end
end
end
Running various times, including changing to bmbm to confirm the GC isn't skewing anything:
user system total real
except 1.282000 0.000000 1.282000 ( 1.276943)
next 0.250000 0.000000 0.250000 ( 0.246193)
On average, the use of next resulting in over 5x faster code. This difference grows even more the larger the hash becomes.
I'd probably just do something like this
my_hash = hash.reject { |k, _| %i[a b].include?(k) }
my_hash.each do |k, v|
# Do what you need to and be free from worry about those pesky :a and :b keys
end
I would like to compare two hashes and forces them to be equal:
one with Symbols on keys and values
the second with only strings.
e.g:
sym_hash = {:id=>58, :locale=>:"en-US"}
string_hash = {"id"=>58, "locale"=>"en-US"}
Try like this does not work:
> sym_hash == string_hash
=> false
I first tried to symbolized the string_hash:
> string_hash.deep_symbolize_keys
=> {:id=>58, :locale=>"en-US"}
But it is still false because sym_hash still has : in front of locale var.
Then I tried to stringified the sym_hash:
> sym_hash.with_indifferent_access
=> {"id"=>58, "locale"=>:"en-US"}
But when I test for equality it is still false for the same reasons.
EDIT
To answer many comments abouy why I wanted those hashes to be equal here, I'll explain what I'm trying to do.
I'm using Reque to manage my jobs. Now I wanted to do a class to avoid having the same* job running, or being enqueued twice in the same time.
(same: for me the same job is a job having the same parameters, I would like to be able to enqueu twice the same jobs having differents ids for instance.)
For that I'm a using the plugin resque-status, so far I'm able to know when a job is running or not. Beside, when I save the params using set I notice that the message written to Redis(because resque-status is using Redis to keep track of the job's status) is not properly saved with symbols.
Here is my class:
# This class is used to run thread-lock jobs with Resque.
#
# It will check if the job with the exact same params is already running or in the queue.
# If the job is not finished, it will returns false,
# otherwise, it will run and returns a the uuid of the job.
#
class JobLock
def self.run(obj, params = {})
# Get rid of completed jobs.
Resque::Plugins::Status::Hash.clear_completed
# Check if your job is currently running or is in the queue.
if !detect_processing_job(obj, params)
job_uuid = obj.create(params)
Resque::Plugins::Status::Hash.set(job_uuid,
job_name: obj.to_s,
params: params)
job_uuid
else
false
end
end
def self.detect_processing_job(obj, params = {})
Resque::Plugins::Status::Hash.statuses.detect do |job|
job['job_name'] == obj.to_s && compare_hashes(job['params'], params)
end
end
def self.compare_hashes(string_hash, sym_hash)
[sym_hash, string_hash].map do |h|
h.map { |kv| kv.map(&:to_s) }.sort
end.reduce :==
end
end
And here how I can use it:
JobLock.run(MyAwesomeJob, id: 58, locale: :"en-US")
As you can see I used #mudasobwa's answer but I hope there is a easier way to achieve what I am trying to do!
How about this?
require 'set'
def sorta_equal?(sym_hash, str_hash)
return false unless sym_hash.size == str_hash.size
sym_hash.to_a.to_set == str_hash.map { |pair|
pair.map { |o| o.is_a?(String) ? o.to_sym : o } }.to_set
end
sym_hash= {:id=>58, :locale=>:"en-US"}
sorta_equal?(sym_hash, {"id"=>58, "locale"=>"en-US"}) #=> true
sorta_equal?(sym_hash, {"locale"=>"en-US", "id"=>58 }) #=> true
sorta_equal?(sym_hash, {"id"=>58, "local"=>"en-US", "a"=>"b" }) #=> false
sorta_equal?(sym_hash, {"id"=>58, "lacole"=>"en-US"}) #=> false
sorta_equal?(sym_hash, {"id"=>58, [1,2,3]=>"en-US"}) #=> false
sorta_equal?({}, {}) #=> true
class A; end
a = A.new
sorta_equal?({:id=>a, :local=>:b}, {"id"=>a, "local"=>"b"}) #=> true
You could try to convert both hashes to JSON, and then compare them:
require 'json'
# => true
sym_hash = {:id=>58, :locale=>:"en-US"}
# => {:id=>58, :locale=>:"en-US"}
string_hash = {"id"=>58, "locale"=>"en-US"}
# => {"id"=>58, "locale"=>"en-US"}
sym_hash.to_json == string_hash.to_json
# => true
The version below works as PHP force-coercion equality:
[sym_hash, string_hash].map do |h|
h.map { |kv| kv.map(&:to_s) }.sort
end.reduce :==
BTW, it’s not a one-liner only because I respect people with smartphones. On terminals of width 80 it’s a perfect oneliner.
To coerce only symbols to strings, preserving numerics to be distinguished from their string representations:
[sym_hash, string_hash].map do |h|
h.map { |kv| kv.map { |e| e.is_a?(Symbol) ? e.to_s : e } }.sort
end.reduce :==
The value of locale in sym_hash is a Symbol :"en-US",
while the value of locale in string_hash is a String.
So they are not equal.
Now if you do:
sym_hash = {:id=>58, :locale=>"en-US"}
string_hash = {"id"=>58, "locale"=>"en-US"}
string_hash.symbolize_keys!
sym_hash == string_hash
=> true
Finaly, to answer my problem I didn't need to force comparaison between hashes. I use Marshal to avoid the problem
class JobLock
def self.run(obj, params = {})
# Get rid of completed jobs.
Resque::Plugins::Status::Hash.clear_completed
# Check if your job is currently running or is in the queue.
if !detect_processing_job(obj, params)
job_uuid = obj.create(params)
Resque::Plugins::Status::Hash.set(job_uuid,
job_name: obj.to_s,
params: Marshal.dump(params))
job_uuid
else
false
end
end
def self.detect_processing_job(obj, params = {})
Resque::Plugins::Status::Hash.statuses.detect do |job|
job['job_name'] == obj.to_s && Marshal.load(job['params']) == params
end
end
end
Anyway, I let this question here because maybe it will help some people in the future...
This is a slight deviation from the original question and an adaptation of some of the suggestions above. If the values can also be String / Symbol agnostic, then may I suggest:
def flat_hash_to_sorted_string_hash(hash)
hash.map { |key_value| key_value.map(&:to_s) }.sort.to_h.to_json
end
this helper function can then be used to assert two hashes have effectively the same values without being type sensitive
assert_equal flat_hash_to_sorted_string_hash({ 'b' => 2, a: '1' }), flat_hash_to_sorted_string_hash({ b: '2', 'a' => 1 }) #=> true
Breakdown:
by mapping a hash, the result is an array
by making the keys / values a consistent type, we can leverage the Array#sort method without raising an error: ArgumentError: comparison of Array with Array failed
sorting gets the keys in a common order
to_h return the object back to a hash
NOTE: this will not work for complex hashes with nested objects or for Float / Int, but as you can see the Int / String comparison works as well. This was inspired by the JSON approach already discussed, but without needing to use JSON, and felt like more than a comment was warranted here as this was the post I found the inspiration for the solution I was seeking.
I'm writing some code to evaluate the presence of a bunch of strings, and I want to ensure that only 1 is present. They're mutually exclusive.
class MyClass
include ActiveModel::Validations
attr_accessor :a, :b, :c
validate :mutex_values
def initialize(attr = {})
attr.each do |k, v|
send("#{k}=", v)
end
end
private
def mutex_values
# here, I want to do this:
# errors.add(:base, "specify only 1") unless a ^ b ^ c
# instead I do this
errors.add(:base, "specify only 1") unless a.present? ^ b.present? ^ c.present?
end
end
MyClass.new(:a => "A", :b => "B", :c => "C").valid?
=> false
Is there another way that doesn't require repetitive use of the .present?? Monkey patch String to define an operator ^? I'm just so used to being able to do if a that it feels unnatural to need what is basically an explicit cast to boolean. I feel like this use case would 'just work' in Ruby.
Enumerable actually defines a one? method which does exactly what you need.
# So either...
[a, b, c].one?(&:present?)
# or
[a, b, c].one?(&:presence)
# ... would do the trick in this case.
Unfortunately if you want two? or etcera, you're out of luck.
You could do something like this with count:
errors.add(:base 'specify exactly 1') unless [a, b, c].count(&:present?) == 1
If you want at most one instead of exactly one, then:
errors.add(:base 'specify at most 1') unless [a, b, c].count(&:present?) > 1
For example:
> ['',nil,'6',''].count(&:present?)
=> 1
> ['',nil,'6',11].count(&:present?)
=> 2
> ['',nil,''].count(&:present?)
=> 0
If you want to check whether exactly one of methods a, b, and c returns non-null value, you can use:
[a, b, c].compact.count == 1
or even, thanks to numbers,
[a, b, c].one?
I know that serializing an object is (to my knowledge) the only way to effectively deep-copy an object (as long as it isn't stateful like IO and whatnot), but is one way particularly more efficient than another?
For example, since I'm using Rails, I could always use ActiveSupport::JSON, to_xml - and from what I can tell marshalling the object is one of the most accepted ways to do this. I'd expect that marshalling is probably the most efficient of these since it's a Ruby internal, but am I missing anything?
Edit: note that its implementation is something I already have covered - I don't want to replace existing shallow copy methods (like dup and clone), so I'll just end up likely adding Object::deep_copy, the result of which being whichever of the above methods (or any suggestions you have :) that has the least overhead.
I was wondering the same thing, so I benchmarked a few different techniques against each other. I was primarily concerned with Arrays and Hashes - I didn't test any complex objects. Perhaps unsurprisingly, a custom deep-clone implementation proved to be the fastest. If you are looking for quick and easy implementation, Marshal appears to be the way to go.
I also benchmarked an XML solution with Rails 3.0.7, not shown below. It was much, much slower, ~10 seconds for only 1000 iterations (the solutions below all ran 10,000 times for the benchmark).
Two notes regarding my JSON solution. First, I used the C variant, version 1.4.3. Second, it doesn't actually work 100%, as symbols will be converted to Strings.
This was all run with ruby 1.9.2p180.
#!/usr/bin/env ruby
require 'benchmark'
require 'yaml'
require 'json/ext'
require 'msgpack'
def dc1(value)
Marshal.load(Marshal.dump(value))
end
def dc2(value)
YAML.load(YAML.dump(value))
end
def dc3(value)
JSON.load(JSON.dump(value))
end
def dc4(value)
if value.is_a?(Hash)
result = value.clone
value.each{|k, v| result[k] = dc4(v)}
result
elsif value.is_a?(Array)
result = value.clone
result.clear
value.each{|v| result << dc4(v)}
result
else
value
end
end
def dc5(value)
MessagePack.unpack(value.to_msgpack)
end
value = {'a' => {:x => [1, [nil, 'b'], {'a' => 1}]}, 'b' => ['z']}
Benchmark.bm do |x|
iterations = 10000
x.report {iterations.times {dc1(value)}}
x.report {iterations.times {dc2(value)}}
x.report {iterations.times {dc3(value)}}
x.report {iterations.times {dc4(value)}}
x.report {iterations.times {dc5(value)}}
end
results in:
user system total real
0.230000 0.000000 0.230000 ( 0.239257) (Marshal)
3.240000 0.030000 3.270000 ( 3.262255) (YAML)
0.590000 0.010000 0.600000 ( 0.601693) (JSON)
0.060000 0.000000 0.060000 ( 0.067661) (Custom)
0.090000 0.010000 0.100000 ( 0.097705) (MessagePack)
I think you need to add an initialize_copy method to the class you are copying. Then put the logic for the deep copy in there. Then when you call clone it will fire that method. I haven't done it but that's my understanding.
I think plan B would be just overriding the clone method:
class CopyMe
attr_accessor :var
def initialize var=''
#var = var
end
def clone deep= false
deep ? CopyMe.new(#var.clone) : CopyMe.new()
end
end
a = CopyMe.new("test")
puts "A: #{a.var}"
b = a.clone
puts "B: #{b.var}"
c = a.clone(true)
puts "C: #{c.var}"
Output
mike#sleepycat:~/projects$ ruby ~/Desktop/clone.rb
A: test
B:
C: test
I'm sure you could make that cooler with a little tinkering but for better or for worse that is probably how I would do it.
Probably the reason Ruby doesn't contain a deep clone has to do with the complexity of the problem. See the notes at the end.
To make a clone that will "deep copy," Hashes, Arrays, and elemental values, i.e., make a copy of each element in the original such that the copy will have the same values, but new objects, you can use this:
class Object
def deepclone
case
when self.class==Hash
hash = {}
self.each { |k,v| hash[k] = v.deepclone }
hash
when self.class==Array
array = []
self.each { |v| array << v.deepclone }
array
else
if defined?(self.class.new)
self.class.new(self)
else
self
end
end
end
end
If you want to redefine the behavior of Ruby's clone method , you can name it just clone instead of deepclone (in 3 places), but I have no idea how redefining Ruby's clone behavior will affect Ruby libraries, or Ruby on Rails, so Caveat Emptor. Personally, I can't recommend doing that.
For example:
a = {'a'=>'x','b'=>'y'} => {"a"=>"x", "b"=>"y"}
b = a.deepclone => {"a"=>"x", "b"=>"y"}
puts "#{a['a'].object_id} / #{b['a'].object_id}" => 15227640 / 15209520
If you want your classes to deepclone properly, their new method (initialize) must be able to deepclone an object of that class in the standard way, i.e., if the first parameter is given, it's assumed to be an object to be deepcloned.
Suppose we want a class M, for example. The first parameter must be an optional object of class M. Here we have a second optional argument z to pre-set the value of z in the new object.
class M
attr_accessor :z
def initialize(m=nil, z=nil)
if m
# deepclone all the variables in m to the new object
#z = m.z.deepclone
else
# default all the variables in M
#z = z # default is nil if not specified
end
end
end
The z pre-set is ignored during cloning here, but your method may have a different behavior. Objects of this class would be created like this:
# a new 'plain vanilla' object of M
m=M.new => #<M:0x0000000213fd88 #z=nil>
# a new object of M with m.z pre-set to 'g'
m=M.new(nil,'g') => #<M:0x00000002134ca8 #z="g">
# a deepclone of m in which the strings are the same value, but different objects
n=m.deepclone => #<M:0x00000002131d00 #z="g">
puts "#{m.z.object_id} / #{n.z.object_id}" => 17409660 / 17403500
Where objects of class M are part of an array:
a = {'a'=>M.new(nil,'g'),'b'=>'y'} => {"a"=>#<M:0x00000001f8bf78 #z="g">, "b"=>"y"}
b = a.deepclone => {"a"=>#<M:0x00000001766f28 #z="g">, "b"=>"y"}
puts "#{a['a'].object_id} / #{b['a'].object_id}" => 12303600 / 12269460
puts "#{a['b'].object_id} / #{b['b'].object_id}" => 16811400 / 17802280
Notes:
If deepclone tries to clone an object which doesn't clone itself in the standard way, it may fail.
If deepclone tries to clone an object which can clone itself in the standard way, and if it is a complex structure, it may (and probably will) make a shallow clone of itself.
deepclone doesn't deep copy the keys in the Hashes. The reason is that they are not usually treated as data, but if you change hash[k] to hash[k.deepclone] they will also be deep copied also.
Certain elemental values have no new method, such as Fixnum. These objects always have the same object ID, and are copied, not cloned.
Be careful because when you deep copy, two parts of your Hash or Array that contained the same object in the original will contain different objects in the deepclone.
I know these are the basics of rails but i still don't know the full difference between = sign and => and the difference between #some_variable, ##some_variable and :some_variable in rails.
Thanks.
OK.
The difference between the = and the => operators is that the first is assignment, the second represents an association in a hash (associative array). So { :key => 'val' } is saying "create an associative array, with :key being the key, and 'val' being the value". If you want to sound like a Rubyist, we call this the "hashrocket". (Believe it or not, this isn't the most strange operator in Ruby; we also have the <=>, or "spaceship operator".)
You may be confused because there is a bit of a shortcut you can use in methods, if the last parameter is a hash, you can omit the squiggly brackets ({}). so calling render :partial => 'foo' is basically calling the render method, passing in a hash with a single key/value pair. Because of this, you often see a hash as the last parameter to sort of have a poor man's optional parameters (you see something similar done in JavaScript too).
In Ruby, any normal word is a local variable. So foo inside a method is a variable scoped to the method level. Prefixing a variable with # means scope the variable to the instance. So #foo in a method is an instance level.
## means a class variable, meaning that ## variables are in scope of the class, and all instances.
: means symbol. A symbol in Ruby is a special kind of string that implies that it will be used as a key. If you are coming from C#/Java, they are similar in use to the key part of an enum. There are some other differences too, but basically any time you are going to treat a string as any sort of key, you use a symbol instead.
Wow, a that's a lot of different concepts together.
1) = is plain old assignment.
a = 4;
puts a
2) => is used to declare hashes
hash = {'a' => 1, 'b' => 2, 'c' => 3}
puts hash['b'] # prints 2
3) #var lets you access object instance variable.
class MyObject
def set_x(x)
#x = x
end
def get_x
#x
end
end
o = MyObject.new
o.set_x 3
puts o.get_x # prints 3
4) ##var lets you access class ('static') variables.
class MyObject
def set_x(x)
##x = x # now you can access '##x' from other MyObject instance
end
def get_x
##x
end
end
o1 = MyObject.new
o1.set_x 3
o2 = MyObject.new
puts o2.get_x # prints 3, even though 'set_x' was invoked on different object
5) I usually think of :var as special 'label' class. Example 2 can be rephrased like this
hash = {:a => 1, :b => 2, :c => 3}
puts hash[:b] # prints 2