How do I cache a method with Ruby/Rails? - ruby-on-rails

I have an expensive (time-consuming) external request to another web service I need to make, and I'd like to cache it. So I attempted to use this idiom, by putting the following in the application controller:
def get_listings
cache(:get_listings!)
end
def get_listings!
return Hpricot.XML(open(xml_feed))
end
When I call get_listings! in my controller everything is cool, but when I call get_listings Rails complains that no block was given. And when I look up that method I see that it does indeed expect a block, and additionally it looks like that method is only for use in views? So I'm guessing that although it wasn't stated, that the example is just pseudocode.
So my question is, how do I cache something like this? I tried various other ways but couldn't figure it out. Thanks!

an in-code approach could look something like this:
def get_listings
#listings ||= get_listings!
end
def get_listings!
Hpricot.XML(open(xml_feed))
end
which will cache the result on a per-request basis (new controller instance per request), though you may like to look at the 'memoize' helpers as an api option.
If you want to share across requests don't save data on the class objects, as your app will not be threadsafe, unless you're good at concurrent programming & make sure the threads don't interfere with each other's data access to the shared variable.
The "rails way" to cache across requests is the Rails.cache store. Memcached gets used a lot, but you might find the file or memory stores fit your needs. It really depends on how you're deploying and whether you want to prioritise cache hits, response time, storage (RAM), or use a hosted solution e.g. a heroku addon.

As nruth suggests, Rails' built-in cache store is probably what you want.
Try:
def get_listings
Rails.cache.fetch(:listings) { get_listings! }
end
def get_listings!
Hpricot.XML(open(xml_feed))
end
fetch() retrieves the cached value for the specified key, or writes the result of the block to the cache if it doesn't exist.
By default, the Rails cache uses file store, but in a production environment, memcached is the preferred option.
See section 2 of http://guides.rubyonrails.org/caching_with_rails.html for more details.

You can use the cache_method gem:
gem install cache_method
require 'cache_method'
In your code:
def get_listings
Hpricot.XML(open(xml_feed))
end
cache_method :get_listings
You might notice I got rid of get_listings!. If you need a way to refresh the data manually, I suggest:
def refresh
clear_method_cache :get_listings
end
Here's another tidbit:
def get_listings
Hpricot.XML(open(xml_feed))
end
cache_method :get_listings, (60*60) # automatically expire cache after an hour

You can also use cachethod gem (https://github.com/reneklacan/cachethod)
gem 'cachethod'
Then it is deadly simple to cache method's result
class Dog
cache_method :some_method, expires_in: 1.minutes
def some_method arg1
..
end
end
It also supports argument level caching

There was suggested cache_method gem, though it's pretty heavy. If you need to call method without arguments, solution is very simple:
Object.class_eval do
def self.cache_method(method_name)
original_method_name = "_original_#{method_name}"
alias_method original_method_name, method_name
define_method method_name do
#cache ||= {}
#cache[method_name] = send original_method_name unless #cache.key?(method_name)
#cache[method_name]
end
end
end
then you can use it in any class:
def get_listings
Hpricot.XML(open(xml_feed))
end
cache_method :get_listings
Note - this will also cache nil, which is the only reason to use it instead of #cached_value ||=

Late to the party, but in case someone arrives here searching.
I use to carry this little module around from project to project, I find it convenient and extensible enough, without adding an extra gem. It uses the Rails.cache backend, so please use it only if you have one.
# lib/active_record/cache_method.rb
module ActiveRecord
module CacheMethod
extend ActiveSupport::Concern
module ClassMethods
# To be used with a block
def cache_method(args = {})
#caller = caller
caller_method_name = args.fetch(:method_name) { #caller[0][/`.*'/][1..-2] }
expires_in = args.fetch(:expires_in) { 24.hours }
cache_key = args.fetch(:cache_key) { "#{self.name.underscore}/methods/#{caller_method_name}" }
Rails.cache.fetch(cache_key, expires_in: expires_in) do
yield
end
end
end
# To be used with a block
def cache_method(args = {})
#caller = caller
caller_method_name = args.fetch(:method_name) { #caller[0][/`.*'/][1..-2] }
expires_in = args.fetch(:expires_in) { 24.hours }
cache_key = args.fetch(:cache_key) { "#{self.class.name.underscore}-#{id}-#{updated_at.to_i}/methods/#{caller_method_name}" }
Rails.cache.fetch(cache_key, expires_in: expires_in) do
yield
end
end
end
end
Then in an initializer:
# config/initializers/active_record.rb
require 'active_record/cache_method'
ActiveRecord::Base.send :include, ActiveRecord::CacheMethod
And then in a model:
# app/models/user.rb
class User < AR
def self.my_slow_class_method
cache_method do
# some slow things here
end
end
def this_is_also_slow(var)
custom_key_depending_on_var = ...
cache_method(key_name: custom_key_depending_on_var, expires_in: 10.seconds) do
# other slow things depending on var
end
end
end
At this point it only works with models, but can be easily generalized.

Other answers are excellent but if you want a simple hand-rolled approach you can do this. Define a method like the below one in your class...
def use_cache_if_available(method_name,&hard_way)
#cached_retvals ||= {} # or initialize in constructor
return #cached_retvals[method_name] if #cached_retvals.has_key?(method_name)
#cached_retvals[method_name] = hard_way.call
end
Thereafter, for each method you want to cache you can put wrap the method body in something like this...
def some_expensive_method(arg1, arg2, arg3)
use_cache_if_available(__method__) {
calculate_it_the_hard_way_here
}
end
One thing that this does better than the simplest method listed above is that it will cache a nil. It has the convenience that it doesn't require creating duplicate methods. Probably the gem approach is cleaner, though.

I'd like to suggest my own gem https://github.com/igorkasyanchuk/rails_cached_method
For example:
class A
def A.get_listings
....
end
end
Just call:
A.cached.get_listings

Related

Rails N+1 query : monkeypatching ActiveRecord::Relation#as_json

Situation
I have a model User:
def User
has_many :cars
def cars_count
cars.count
end
def as_json options = {}
super options.merge(methods: [:cars_count])
end
end
Problem
When I need to render to json a collection of users, I end up being exposed to the N+1 query problem. It is my understanding that including cars doesn't solve the problem for me.
Attempted Fix
What I would like to do is add a method to User:
def User
...
def self.as_json options = {}
cars_counts = Car.group(:user_id).count
self.map do |user|
user.define_singleton_method(:cars_count) do
cars_counts[user.id]
end
user.as_json options
end
end
end
That way all cars counts would be queried in a single query.
Remaining Issue
ActiveRecord::Relation already has a as_json method and therefore doesn't pick the class defined one. How can I make ActiveRecord::Relation use the as_json method from the class when it is defined? Is there a better way to do this?
Edits
1. Caching
I can cache my cars_count method:
def cars_count
Rails.cache.fetch("#{cache_key}/cars_count") do
cars.count
end
end
This is nice once the cache is warm, but if a lot of users are updated at the same time, it can cause request timeouts because a lot of queries have to be updated in a single request.
2. Dedicated method
Instead of calling my method as_json, I can call it my_dedicated_as_json_method and each time I need to render a collection of users, instead of
render json: users
write
render json: users.my_dedicated_as_json_method
However, I don't like this way of doing. I may forget to call this method somewhere, someone else might forget to call it, and I'm losing clarity of the code. Monkey patching seems a better route for these reasons.
Have you considered using a counter_cache for cars_count? It's a good fit for what you're wanting to do.
This blog article also offers up some other alternatives, e.g. if you want to manually build a hash.
If you really wanted to continue down the monkey patching route, then ensure that you are patching ActiveRecord::Relation rather than User, and override the instance method rather than creating a class method. Note that this will then affect every ActiveRecord::Relation, but you can use #klass to add a condition that only runs your logic for User
# Just an illustrative example - don't actually monkey patch this way
# use `ActiveSupport::Concern` instead and include the extension
class ActiveRecord::Relation
def as_json(options = nil)
puts #klass
end
end
Option 1
In your user model:
def get_cars_count
self.cars.count
end
And in your controller:
User.all.as_json(method: :get_cars_count)
Option 2
You can create a method which will get all the users and their car count. And then you can call the as_json method on that.
It would roughly look like:
#In Users Model:
def self.users_with_cars
User.left_outer_joins(:cars).group(users: {:id, :name}).select('users.id, users.name, COUNT(cars.id) as cars_count')
# OR may be something like this
User.all(:joins => :cars, :select => "users.*, count(cars.id) as cars_count", :group => "users.id")
end
And in your controller you can call as_json:
User.users_with_cars.as_json
Here is my solution in case someone else is interested.
# config/application.rb
config.autoload_paths += %W(#{config.root}/lib)
# config/initializers/core_extensions.rb
require 'core_extensions/active_record/relation/serialization'
ActiveRecord::Relation.include CoreExtensions::ActiveRecord::Relation::Serialization
# lib/core_extensions/active_record/relation/serialization.rb
require 'active_support/concern'
module CoreExtensions
module ActiveRecord
module Relation
module Serialization
extend ActiveSupport::Concern
included do
old_as_json = instance_method(:as_json)
define_method(:as_json) do |options = {}|
if #klass.respond_to? :collection_as_json
scoping do
#klass.collection_as_json options
end
else
old_as_json.bind(self).(options)
end
end
end
end
end
end
end
# app/models/user.rb
def User
...
def self.collection_as_json options = {}
cars_counts = Car.group(:user_id).count
self.map do |user|
user.define_singleton_method(:cars_count) do
cars_counts[user.id]
end
user.as_json options
end
end
end
Thanks #gwcodes for pointing me at ActiveSupport::Concern.

Ruby on Rails decorator for caching a result?

In Python, you can write a decorator for memoizing a function's response.
Is there something similar for Ruby on Rails? I have a model's method that makes a query, which I would like to cache.
I know I can do something inside the method, like:
def foo(param)
if self.cache[param].nil?
self.cache[param] = self.get_query_result(param)
else
self.cache[param]
end
end
However, given that I would do this often, I'd prefer a decorator syntax. It is clearer and better IMO.
Is there something like this for Ruby on Rails?
I usually do this using custom accessors, instance variables, and the ||= operator:
def foo
#foo ||= something_or_other
end
something_or_other could be a private method on the same class that returns the object that foo should be.
EDIT
Here's a slightly more complicated solution that lets you cache any method based on the arguments used to call them.
class MyClass
attr_reader :cache
def initialize
#cache = {}
end
class << self
def cacheable(symbol)
alias_method :"_#{symbol}_uncached", symbol
define_method(symbol) do |*args|
cache[[symbol, *args]] ||= (send :"_#{symbol}_uncached", *args)
end
end
end
end
How this works:
class MyClass
def foo(a, b)
a + b
end
cacheable :foo
end
First, the method is defined normally. Then the class method cacheable is called, which aliases the original method to a new name, then redefines it under the original name to be executed only if it's not already cached. It first checks the cache for anything using the same method and arguments, returns the value if present, and executes the original method if not.
http://martinfowler.com/bliki/TwoHardThings.html:
There are only two hard things in Computer Science: cache invalidation and naming things.
-- Phil Karlton
Rails has a lot of built in caching(including query caching). You might not need to do anything:
http://guides.rubyonrails.org/caching_with_rails.html
Here is a recent blog post about problems with roll your own caching:
http://cmme.org/tdumitrescu/blog/2014/01/careful-what-you-memoize/

How to avoid multiple Nokogiri requests inside a class?

I'm working on a gem.
Here is the homepage of it:
https://github.com/scaryguy/vakit
If you check out the source code, you can see that I'm parsing an external HTML page to filter some data from it.
The issue is, eventhough I fetch all data I want with one request, each time I call Vakit.sabah or Vakit.oglen a new request is done.
require "vakit/version"
require 'vakit/connect'
require 'Nokogiri'
require 'open-uri'
module Vakit
def self.today
Vakit::Connect.shaber
end
def self.imsak
Vakit::Connect.shaber[:imsak]
end
def self.sabah
Vakit::Connect.shaber[:sabah]
end
def self.oglen
Vakit::Connect.shaber[:oglen]
end
def self.ikindi
Vakit::Connect.shaber[:ikindi]
end
def self.aksam
Vakit::Connect.shaber[:aksam]
end
def self.yatsi
Vakit::Connect.shaber[:yatsi]
end
end
I don't think that it's an efficient way.
I should be able to access attributes of my hash without new request, shouldn't I?
module Vakit
class Connect
def initialize(opt={})
#path = opt[:path]
end
def self.shaber
doc = Nokogiri::HTML(open('http://www.samanyoluhaber.com/'))
x = doc.css('#hnmzT')
times = []
x.each do |vakit|
data = vakit.children.first.children.last.content
data_add = data.slice(0..data.length-2)
times.push(data_add)
end
times
vakit = {
imsak: times[0],
sabah: times[1],
oglen: times[2],
ikindi: times[3],
aksam: times[4],
yatsi: times[5]
}
end
end
end
I need some enlightment.
Every time you use shaber you explicitly open and reparse the content. You're not making an attempt to locally store the content or parsed DOM and check to see if you already have it.
Instead of doc = use ##doc ||= and change the occurrences of doc to ##doc.
The ||= operator will only assign when ##doc is empty. Once it's assigned to a non-nil or non-false value it won't trigger again so it's a poor-mans "memoize".
Because you are using a class-method I recommended using a class variable. ##doc could be an instance variable #doc instead if you have multiple instances of the class that are looking at different pages. As is, that won't make a difference because you've hard-coded only one page, but for future code-growth it might be useful.
The code you've written to access the page isn't very idiomatic Ruby. I'd write it more like the following, which doesn't work because the URL isn't returning a page that has enough time values:
require 'nokogiri'
require 'open-uri'
module Vakit
URL = 'http://www.samanyoluhaber.com/'
class Connect
def initialize(opt={})
#path = opt[:path]
#url = opt[:url] || URL
end
def shaber(url=nil)
doc = Nokogiri::HTML(open(url || #url))
doc.at_css('#hnmzT').to_html # => "<li id=\"hnmzT\" name=\"imsak\"><a id=\"at\"><span>\u0130msak:</span>4:22\u00A0</a></li>"
x = doc.at_css('li#hnmzT a')
x.to_html # => "<a id=\"at\"><span>\u0130msak:</span>4:22\u00A0</a>"
times = x.text.scan(/\d+:\d+/)
Hash[[:imsak, :sabah, :oglen, :ikindi, :aksam, :yatsi].zip(times)]
end
end
end
connection = Vakit::Connect.new
connection.shaber # => {:imsak=>"4:22", :sabah=>nil, :oglen=>nil, :ikindi=>nil, :aksam=>nil, :yatsi=>nil}
Connect isn't a good name for a class. A class is an object, a thing. Connect is a verb, something that occurs, or happens, to a thing. connect would be a good name for a method.
This line
doc = Nokogiri::HTML(open('http://www.samanyoluhaber.com/')) is what's making the requested call several times.
I tested this with your gem and this seems to work.
if #doc.nil?
#doc = Nokogiri::HTML(open('http://www.samanyoluhaber.com/'))
end
In addition in vakit.rb change
require 'Nokogiri' to require 'nokogiri' (simple n)
I will make the following suggestions:
Vakit::Connect should be an instance method inside a class. so it can be instantiated & saved in memory. this way each request will be different instances of the same object. if class doesn't exist. Do create it.
use memorization to cache costly operations.
If the operation takes long time, say greater than 3sec. convert it into a background job & use workers to process them.however, since you are authoring a gem. I think this responsibility should be delagated to developers using the gem.
so I would do it something like this:
the connect class:
module Vakit
class Connect
attr_accessor :path, :doc
def initialize(opt={}, url)
#path = opt[:path]
#doc = Nokogiri::HTML(open(url))
end
def shaber
x = #doc.css('#hnmzT')
#no changes here
end
end
the Vakit module
module Vakit
class Main #name it something more meaningful
def initialize(opt={})
#connected = Connect.new(opt[:path], 'http://www.samanyoluhaber.com/')
end
def today
#connected.shaber
end
def imsak
today[:imsak]
end
def sabah
today[:sabah]
end
def oglen
today[:oglen]
end
def ikindi
today[:ikindi]
end
def aksam
today[:aksam]
end
def yatsi
today[:yatsi]
end
end
then in code, you can use it like this:
#v = Vakit::Main.new
#v.aksam
This may not be 100% perfect because I really don't understand the purpose of your code, O understand what its doing but not why. But this will not make new request everytime you access your hash.

How to add attribute to existing Notifications payload?

In Rails notifications, I am subscribing to "process_action.action_controller", and would like to add more attributes to the payload. How can I do that?
I have tried using append_info_to_payload, but this seems to do nothing.
module AppendExceptionPayload
module ControllerRuntime
extend ActiveSupport::Concern
protected
def append_info_to_payload(payload)
super
payload[:happy] = "HAPPY"
end
end
end
The subscription and above code is in a Rails engine, so this is where I make the call to add it:
require 'append_exception_payload'
module Instrument
class Engine < ::Rails::Engine
ActiveSupport.on_load :action_controller do
include AppendExceptionPayload::ControllerRuntime
end
end
end
After putting up the bounty, I found a solution myself. Rails handles this really cleanly.
Basically, the append_info_to_payload method is meant exactly for this.
So to include session information and signed_in user information I added this to my application_controller.rb:
def append_info_to_payload(payload)
super
payload[:session] = request.session_options[:id] rescue ""
payload[:user_id] = session[:user_id] rescue "unknown"
end
So i jumped in and had a look at the api for the process_action method (private) and the append_info_to_payload instance method (public) and the proccess action method seems to call append_info_to_payload in its code like so:
ActiveSupport::Notifications.instrument("process_action.action_controller", raw_payload) do |payload|
result = super
payload[:status] = response.status
append_info_to_payload(payload)
result
end
and append_info_to_payload works something like this
def append_info_to_payload(payload) #:nodoc:
payload[:view_runtime] = view_runtime
end
I can suggest trying payload[:view_runtime] instead of payload[:happy] or trying to use payload[:status]
Let me know how you get on and I will try help more, unfortunately there is really no documentation for this stuff.

getting in-out from ruby methods

I am using jruby to run bunch of ruby scripts, though I am using pure ruby part of it.
It sometimes gets difficult to follow from output what exactly is happening or where exactly something went wrong.
I wanted to get something like this in my std out for every method:
entered in method A
out of method A
Now I can surely go and put those comments in every method ..which feels very wrong. Is there a way to run ruby in a little verbose more to get this information in my log. Hopefully I would avoid using a lot of gems etc .. since these are on some managed servers and I will have to spend some time to just get more s/f on it. Hoping something would be avail as part of jruby itself
Thanks!
You could use this code:
module InOutHook
module ClassMethods
def setup_hooks(*syms)
syms.each do |sym| # For each symbol
str_id = "__#{sym}__hooked__"
unless private_instance_methods.include?(str_id)
alias_method str_id, sym # Backup original method
private str_id # Make backup private
define_method sym do |*args| # Replace method
puts ">>> #{self.class}\##{sym} >>>"
ret = __send__ str_id, *args # Invoke backup
puts "<<< #{self.class}\##{sym} >>>"
ret
end
end
end
end
end
def InOutHook.included(base)
base.extend(ClassMethods)
end
end
class TestClass
def test1
puts "test!"
end
def test2(v)
puts "Value is #{v}"
end
include InOutHook
setup_hooks(:test1, :test2)
end
# works on existing classes too:
class Array
include InOutHook
setup_hooks(:[])
end
tc = TestClass.new
tc.test1
tc.test2(10)
ary = [1,2,3]
puts ary[1..2]
In case you want to add a hoot to every method, just add a splat asterisk:
setup_hooks(*[].methods)

Resources