I have a function, which returns a list of ID's, in the Rails caching guide I can see that an expiration can be set on the cached results, but I have implemented my caching somewhat differently.
def provide_book_ids(search_param)
#returned_ids ||= begin
search = client.search(query: search_param, :reload => true)
search.fetch
search.options[:query] = search_str
search.fetch(true)
search.map(&:id)
end
end
What is the recomennded way to set a 10 minute cache expiry, when written as above?
def provide_book_ids(search_param)
#returned_ids = Rails.cache.fetch("zendesk_ids", expires_in: 10.minutes) do
search = client.search(query: search_param, :reload => true)
search.fetch
search.options[:query] = search_str
search.fetch(true)
search.map(&:id)
end
end
I am assuming this code is part of some request-response cycle and not something else (for example a long running worker or some class that is initialized once in your app. In such a case you wouldn't want to use #returned_ids directly but instead call provide_book_ids to get the value, but from I understand that's not your scenario so provided approach above should work.
Related
Using Rails memory cache like this in one controller.
def form_config_cache
Rails.cache.fetch("form_config", :expires_in => 12.hours) do
puts 'Building cache...'
form_config = s3_read_object('form_config.js')
return JSON.parse(form_config)
end
end
This is working fine on the controller where it is defined. But when I try to read the value from another controller, it is returning as nil. Can anyone explain what might be going on? Here is how I am trying to read it in another controller.
form_config = Rails.cache.read('form_config')
Your code doesn't actually ever cache anything: return returns form the whole method, so the bit of fetch that stores values in the cache never executes and there is nothing for your call to read to return.
You could either use next or nothing at all:
def form_config_cache
Rails.cache.fetch("form_config", :expires_in => 12.hours) do
form_config = s3_read_object('form_config.js')
JSON.parse(form_config)
end
end
Right now I'm building a call tracking app to learn rails and twilio. The app has 2 relevant models ; The Plans model has_many users. The plans table also has the value max_minutes.
I want it to make it so that when a particular user goes over their max_minutes, their sub account is disabled, and I can also warn them to upgrade in the view.
To do this, here's a parameter I created in the User class
def at_max_minutes?
time_to_bill=0
start_time = Time.now - ( 30 * 24 * 60 * 60) #30 days
#subaccount = Twilio::REST::Client.new(#user.twilio_account_sid, #user.twilio_auth_token)
#subaccount.calls.list({:page => 0, :page_size => 1000, :start_time => ">#{start_time.strftime("%Y-%m-%d")}"}).each do |call|
time_to_bill += (call.duration.to_f/60).ceil
end
time_to_bill >= self.plan.max_minutes
end
This allows me to run if/else statements in the view to urge them to upgrade. However, I'd also like to make an if/else statement where, if at_max_minutes? than the user's twilio subaccount is disabled, else, it's enabled.
I'm not sure where I would put that though in rails.
It would look something like this
#client = Twilio::REST::Client.new(#user.twilio_account_sid, #user.twilio_auth_token)
#account = #client.account
if at_max_minutes?
#account = #account.create({:status => 'suspended'})
else
#account = #account.create({:status => 'active'})
end
BUT, I'm not sure where I would put this code, so that it's active all the time.
How would you implement this code, for the functionality to work?
Instead of constantly computing the total minutes used in at_max_minutes?, why not keep track of a user's used minutes, and set the status to "suspended" on the transition (when used minutes goes over max_minutes). Then your view and call code would only have to check status (you may also want to store status directly on user, to save API calls over to Twilio).
Add to User model:
used_minutes
When every call ends, update minutes:
def on_call_end( call )
self.used_minutes += call.duration_in_minutes # this assumes Twilio gives you a callback and has the length of the call)
save!
end
Add an after_save to User:
after_save :check_minutes_usage
def check_minutes_usage
if used_minutes >= plan.max_minutes
#account = #account.create({:status => 'suspended'})
else
#account = #account.create({:status => 'active'})
end
end
You're going to have to do some sort of scheduled background job for this check if you want it to be "active all the time". I'd recommend resque with resque-scheduler, which is a pretty good scheduling solution for Rails. Basically what you to do is to make a job, which executes that second block of code you specified, and have it run on a regular interval (maybe every 2 hours).
Let's suppose I have a method like this:
def foo
Rails.cache.fetch("cache_key", :expires_in => 60.minutes) do
return_something
end
end
return_something sometimes returns a nil value. When this happens, I don't want the nil value to be cached for 60 minutes. Instead, the next time I call foo, I want the block passed to fetch to be executed again.
Is Rails.cache.fetch working like this by default? Or do I have to implement this functionality?
Update (with Answer)
Turns out, the answer was no, at least when using Memcached.
it depends on the implementation of the cache-store that you are using. i would say that it should not cache nil values, but empty strings are ok to cache.
look at the dalli store implementation ie:
def fetch(name, options=nil)
options ||= {}
name = expanded_key name
if block_given?
unless options[:force]
entry = instrument(:read, name, options) do |payload|
payload[:super_operation] = :fetch if payload
read_entry(name, options)
end
end
if !entry.nil?
instrument(:fetch_hit, name, options) { |payload| }
entry
else
result = instrument(:generate, name, options) do |payload|
yield
end
write(name, result, options)
result
end
else
read(name, options)
end
end
The updated answer to this question is: By default fetch caches nil values, but using the dalli_store engine you can avoid it with cache_nils option:
Rails.cache.fetch("cache_key", expires_in: 60.minutes, cache_nils: false) do
return_something
end
Worth noting, the defaults for Dalli have changed in recent years - the flag for nil-caching is currently false by default. See https://github.com/petergoldstein/dalli
It's definitely worth adding a test to check that your setup does what you expect (especially for production mode)
I have a class with this method:
def telecom_info
Rails.cache.fetch("telecom_info_for_#{ref_num}", :expires_in=> 3.hours) do
info = Hash.new(0)
Telecom::SERVICES.each do |source|
results = TelecomUsage.find(:all,
:joins=>[:telecom_invoice=>{ :person=> :org_person}],
:conditions=>"dotted_ids like '%#{ref_num}%' and telecom_usages.ruby_type = '#{source}'",
:select=>"avg(charge) #{source.upcase}_AVG_CHARGE,
max(charge) #{source.upcase}_MAX_CHARGE,
min(charge) #{source.upcase}_MIN_CHARGE,
sum(charge) #{source.upcase}_CHARGE,
avg(volume) #{source.upcase}_AVG_VOLUME,
max(volume) #{source.upcase}_MAX_VOLUME,
min(volume) #{source.upcase}_MIN_VOLUME,
sum(volume) #{source.upcase}_VOLUME
")
results = results.first
['charge', 'volume'].each do |source_type|
info["#{source}_#{source_type}".to_sym] = results.send("#{source}_#{source_type}".downcase).to_i
info["#{source}_min_#{source_type}".to_sym] = results.send("#{source}_min_#{source_type}".downcase).to_i
info["#{source}_max_#{source_type}".to_sym] = results.send("#{source}_max_#{source_type}".downcase).to_i
info["#{source}_avg_#{source_type}".to_sym] = results.send("#{source}_avg_#{source_type}".downcase).to_i
end
end
return info
end
end
As you can see, this is an expensive call, and it is called ALOT for each request so I want to cache it. The problem is that memcached does not seem to work, in the log file, I am getting:
Cache read: telecom_info_for_60000000
Cache miss: telecom_info_for_60000000 ({})
The weird thing is that I know memcached is working since it does cache the results of some other functions I have in another model.
Any suggestions? I am running Rails 2.3.5 on REE 1.8.7
Replace return info with info.
Rails.cache.fetch("telecom_info_for_#{ref_num}", :expires_in=> 3.hours) do
# ...
info
end
The return keyword always returns from the current method, which means that info is never returned to your call to Rails.cache.fetch, nor is the rest of that method ever executed. When the last statement simply is info, this is the value that will be given to Rails.cache.fetch, and you will allow the method to finish its duty by storing this value in the cache.
Compare the following:
def my_method
1.upto(3) do |i|
# Calling return immediately causes Ruby to exit the current method.
return i
end
end
my_method
#=> 1
As a rule of thumb: always omit return unless you really mean to exit the current block and return from the current method.
We're interested in logging and computing the number of times an item comes up in search or on a list page. With 50k unique visitors a day, we're expecting we could produce 3-4 million 'impressions' per day, which isn't a terribly high amount, but one we'd like to architect well.
We don't need to read this data in real time, but would like to be able to generate daily totals and analyze trends, etc. Similar to a business analytics tool.
We're planning to do this with an Ajax post after the page is rendered - this will allow us to count results even if those results are cached. We can do this in a single post per page, to send a comma delimited list of ids and their positions on the page.
I am hoping there is some sort of design pattern/gem/blog post about this that would help me avoid the common first-timer mistakes that may come up. I also don't really have much experience logging or reading logs.
My current strategy - make something to write events to a log file, and a background job to tally up the results at the end of the day and put the results back into mysql.
Ok, I have three approaches for you:
1) Queues
In your AJAX Handler, write the simplest method possible (use a Rack Middleware or Rails Metal) to push the query params to a queue. Then, poll the queue and gather the messages.
Queue pushes from a rack middleware are blindingly fast. We use this on a very high traffic site for logging of similar data.
An example rack middleware is below (extracted from our app, can handle request in <2ms or so:
class TrackingMiddleware
CACHE_BUSTER = {"Cache-Control" => "no-cache, no-store, max-age=0, must-revalidate", "Pragma" => "no-cache", "Expires" => "Fri, 29 Aug 1997 02:14:00 EST"}
IMAGE_RESPONSE_HEADERS = CACHE_BUSTER.merge("Content-Type" => "image/gif").freeze
IMAGE_RESPONSE_BODY = [File.open(Rails.root + "public/images/tracker.gif").read].freeze
def initialize(app)
#app = app
end
def call(env)
if env["PATH_INFO"] =~ %r{^/track.gif}
request = Rack::Request.new(env)
YOUR_QUEUE.push([Time.now, request.GET.symbolize_keys])
[200, IMAGE_RESPONSE_BODY, IMAGE_RESPONSE_HEADERS]
else
#app.call(env)
end
end
end
For the queue I'd recommend starling, I've had nothing but good times with it.
On the parsing end, I would use the super-poller toolkit, but I would say that, I wrote it.
2) Logs
Pass all the params along as query params to a static file (/1x1.gif?foo=1&bar=2&baz=3).
This will not hit the rails stack and will be blindingly fast.
When you need the data, just parse the log files!
This is the best scaling home brew approach.
3) Google Analytics
Why handle the load when google will do it for you? You would be surprised at how good google analytics is, before you home brew anything, check it out!
This will scale infinitely, because google buys servers faster than you do.
I could rant on this for ages, but I have to go now. Hope this helps!
Depending no the action required to list items, you might be able to do it in the controller and save yourself a round trip. You can do it with an after_filter, to make the addition unobtrusive.
This only works if all actions that list items you want to log, require parameters. This is because page caching ignores GET requests with parameters.
Assuming you only want to log search data on the search action.
class ItemsController < ApplicationController
after_filter :log_searches, :only => :search
def log_searches
#items.each do |item|
# write to log here
end
end
...
# rest of controller remains unchanged
...
end
Otherwise you're right on track with the AJAX, and an onload remote function.
As for processing the you could use a rake task run by a cron job to collect statistics, and possibly update items for a popularity rating.
Either way you will want to read up on the Ruby Logging class. Learning about cron jobs and rake tasks wouldn't hurt either.
This is what I ultimately did - it was enough for our use for now, and with some simple benchmarking, I feel OK about it. We'll be watching to see how it does in production before we expose the results to our customers.
The components:
class EventsController < ApplicationController
def create
logger = Logger.new("#{RAILS_ROOT}/log/impressions/#{Date.today}.log")
logger.info "#{DateTime.now.strftime} #{params[:ids]}" unless params[:ids].blank?
render :nothing => true
end
end
This is called from an ajax call in the site layout...
<% javascript_tag do %>
var list = '';
$$('div.item').each(function(item) { list += item.id + ','; });
<%= remote_function(:url => { :controller => :events, :action => :create}, :with => "'ids=' + list" ) %>
<% end %>
Then I made a rake task to import these rows of comma delimited ids into the db. This is run the following day:
desc "Calculate impressions"
task :count_impressions => :environment do
date = ENV['DATE'] || (Date.today - 1).to_s # defaults to yesterday (yyyy-mm-dd)
file = File.new("log/impressions/#{date}.log", "r")
item_impressions = {}
while (line = file.gets)
ids_string = line.split(' ')[1]
next unless ids_string
ids = ids_string.split(',')
ids.each {|i| item_impressions[i] ||= 0; item_impressions[i] += 1 }
end
item_impressions.keys.each do |id|
ActiveRecord::Base.connection.execute "insert into item_stats(item_id, impression_count, collected_on) values('#{id}',#{item_impressions[id]},'#{date}')", 'Insert Item Stats'
end
file.close
end
One thing to note - the logger variable is declared in the controller action - not in environment.rb as you would normally do with a logger. I benchmarked this - 10000 writes took about 20 seconds. Averaging about 2 milliseconds a write. With the file name in the envirnment.rb, it took about 14 seconds. We made this trade-off so we could dynamically determine the file name - an easy way to switch files at midnight.
Our main concern at this point - we have no idea how many different items will be counted per day - ie. we don't know how long the tail is. This will determine how many rows are added to the db each day. We expect we'll need to limit how far back we keep daily reports and will role up results even further at that point.