I am storing database query results in memcache on heroku. I am using memcachier addon on heroku. For example if I have a cache User's tasks in memcache. I do something like this:
def cached_tasks
Rails.cache.fetch([:users, id, :tasks], :expires_in => 12.hours) { tasks.to_a }
end
This works perfectly fine but I want to use two different memcache instances to store data.
Why?
One that is used very frequently, basically data changes frequently and another for those which are big data objects and those will never change or very rarely.
How can I use two different instances and specify that cached_tasks should be stored in memcache_instance_1 and other like cached_images should be stored in memcache_instance_2
Why not to use the same one:
Because sometimes I need to flush the whole cache and that will flush the big data too which I don't want to.
Any suggestions?
Declare three new env variables for your heroku app
BIG_MEMCACHIER_SERVERS
BIG_MEMCACHIER_USERNAME
BIG_MEMCACHIER_PASSWORD
Add a file called big_cache.rb in the config\initializers directory:
module Rails
def self.big_cache
#big_cache ||= ActiveSupport::Cache::DalliStore.new(
(ENV["BIG_MEMCACHIER_SERVERS"] || "").split(","),
:username => ENV["BIG_MEMCACHIER_USERNAME"],
:password => ENV["BIG_MEMCACHIER_PASSWORD"])
end
end
Now you can access the 2nd cache as follows:
Rails.big_cache.fetch(..)
What i would do is to try and define what from the the things i am caching is actually something that requires cache, or it actually needs some life-limited persistency.
I would leave the Rails internal caching system to do what it does best: page, action and fragment caching, and use another persistency engine (such as Redis) to hold and maintain those large objects you talked about (implementation of this suggestion is a completely different question).
note that Redis also allows setting a TTL (time to live) on keys - so it can provide this life limited persistency that you want.
Related
We are already using cookie based sessions, and switching off them to file store sessions in not an option. However, I need a way to store larger amounts of session data (up to 10MG or so) -- beyond the limit of cookie session and, even it weren't, round-tripping that much data on multiple requests would be slow.
I am currently attempting to solve this by using (abusing?) Rails.cache. The basic setup is like this:
I post to a route, which has the following code:
# calculate some results...
Rails.cache.write('search_results' + session.id), search_results)
redirect_to '/results'
Inside GET /results, I read the cached data and send it to the client:
#results = Rails.cache.read('search_results' + session.id)
This works fine. However, if I subsequently make a request to another route like GET /results2 that also calls Rails.cache.read('search_results' + session.id), it will return nil. Even if all calls happen within a 5-10s span.
So my questions are:
Why does this happen? What determines when Rails.cache clean itself?
Is there a way to make this work?
Is there a better approach altogether that doesn't involve using a DB or redis?
Answer to your questions:
The problem with file cache store is that it stores file locally. Thus, if you have multiple servers, cache can be written to one server while cache is read on another server which will return ‘nil’. The solution is to use cache store that can be shared among multiple servers.
Using redis-store may be a solution: https://github.com/redis-store/redis-rails
I have an API endpoint that aggregates a bunch of data from code that leverages Rails.cache for small pieces of data here and there. There are times, however, when I want 100% up-to-date data, as if Rails.cache was empty. Obviously I could clear cache prior to aggregating the data, but that will affect unrelated data and requests.
Is there a way for me to have a request in rails act as if Rails.cache is empty, similar to if Rails.cache was configured to be :null_store?
The query cache in ActiveRecord has something like this - an "uncached" function that you can pass a block to, where the block will run w/o query cache enabled. I need something similar, but for Rails.cache in general.
Since it does not appear there is a solution to this out of the box, I coded a solution of my own by adding the following code as config/initializers/rails_cache.rb
module Rails
class << self
alias :default_rails_cache :cache
def cache
# Allow any thread to override Rails.cache with its own cache implementation.
RequestStore.store[:rails_cache] || default_rails_cache
end
end
end
This allows any thread to specify its own cache store, which will then be used for all fetches, reads, and writes. As such, it will not read from the default Rails.cache, nor will its values be written to the default Rails.cache.
If the thread is long-running and benefits from having caching enabled, you can easily set this to its own MemoryStore instance:
RequestStore.store[:rails_cache] = ActiveSupport::Cache.lookup_store(:memory_store)
And if you want caching completely off for this thread, you can :null_store instead of :memory_store.
If you are not using the request_store gem, "RequestStore.store" can be replaced with "Thread.current" for the same effect - just have to be more careful about thread reuse across requests.
I have a class method (placed in /app/lib/) which performs some heavy calculations and sub-http requests until a result is received.
The result isn't too dynamic, and requested by multiple users accessing a specific view in the app.
So, I want to schedule a periodic run of the method (using cron and Whenever gem), store the results somewhere in the server using JSON format and, by demand, read the results alone to the view.
How can this be achieved? what would be the correct way of doing that?
What I currently have:
def heavyMethod
response = {}
# some calculations, eventually building the response
File.open(File.expand_path('../../../tmp/cache/tests_queue.json', __FILE__), "w") do |f|
f.write(response.to_json)
end
end
and also a corresponding method to read this file.
I searched but couldn't find an example of achieving this using Rails cache convention (and not some private code that I wrote), on data which isn't related with ActiveRecord.
Thanks!
Your solution should work fine, but using Rails.cache should be cleaner and a bit faster. Rails guides provides enough information about Rails.cache and how to get it to work with memcached, let me summarize how I would use it in your case
Heavy method
def heavyMethod
response = {}
# some calculations, eventually building the response
Rails.cache.write("heavy_method_response", response)
end
Request
response = Rails.cache.fetch("heavy_method_response")
The only problem here is that when ur server starts for the first time, the cache will be empty. Also if/when memcache restarts.
One advantage is that somewhere on the flow, the data u pass in is marshalled into storage, and then unmartialled on the way out. Meaning u can pass in complex datastructures, and dont need to serialize to json manually.
Edit: memcached will clear your item if it runs out of memory. Will be very rare since its using a LRU (i think) algoritm to expire things, and I presume you will use this often.
To prevent this,
set expires_in larger than your cron period,
change your fetch code to call the heavy_method if ur fetch fails (like Rails.cache.fetch("heavy_method_response") {heavy_method}, and change heavy_method to just return the object.
Use something like redis which will not delete items.
I'm working on a mashup site and would like to limit the number of fetches to scrape the source sites. There is essentially one bit of data I need, an integer, and would like to cache it with a defined expiration period.
To clarify, I only want to cache the integer, not the entire page source.
Is there a ruby or rails feature or gem that already accomplishes this for me?
Yes, there is ActiveSupport::Cache::Store
An abstract cache store class. There are multiple cache store
implementations, each having its own additional features. See the
classes under the ActiveSupport::Cache module, e.g.
ActiveSupport::Cache::MemCacheStore. MemCacheStore is currently the
most popular cache store for large production websites.
Some implementations may not support all methods beyond the basic
cache methods of fetch, write, read, exist?, and delete.
ActiveSupport::Cache::Store can store any serializable Ruby object.
http://api.rubyonrails.org/classes/ActiveSupport/Cache/Store.html
cache = ActiveSupport::Cache::MemoryStore.new
cache.read('Chicago') # => nil
cache.write('Chicago', 2707000)
cache.read('Chicago') # => 2707000
Regarding the expiration time, this can be done by passing the time as a initialization parameter
cache = ActiveSupport::Cache::MemoryStore.new(expires_in: 5.minutes)
If you want to cache a value with a different expiration time, you can also set this when writing a value to the cache
cache.write(key, value, expires_in: 1.minute) # Set a lower value for one entry
See Caching with Rails, particularly the :expires_in option to ActiveSupport::Cache::Store.
For example, you might go:
value = Rails.cache.fetch('key', expires_in: 1.hour) do
expensive_operation_to_compute_value()
end
Is there a way to cache per-request data in Rails? For a given Rails/mongrel request I have the result of a semi-expensive operation that I'd like to access several times later in that request. Is there a hash where I can store and access such data?
It needs to be fairly global and accessible from views, controllers, and libs, like Rails.cache and I18n are.
I'm ok doing some monkey-patching if that's what it takes.
Memcached doesn't work because it'll be shared across requests, which I don't want.
A global variable similarly doesn't work because different requests would share the same data, which isn't what I want.
Instance variables don't work because I want to access the data from inside different classes.
There is also the request_store gem. From the documentation:
Add this line to your application's Gemfile:
gem 'request_store'
and use this code to store and retrieve data (confined to the request):
# Set
RequestStore.store[:foo] = 0
# Get
RequestStore.store[:foo]
Try PerRequestCache. I stole the design from the SQL Query Cache.
Configure it up in config/environment.rb with:
config.middleware.use PerRequestCache
then use it with:
PerRequestCache.fetch(:foo_cache){ some_expensive_foo }
One of the most popular options is to use the request_store gem, which allows you to access a global store that you from any part of your code. It uses Thread.current to store your data, and takes care of cleaning up the data after each request.
RequestStore[:items] = []
Be aware though, since it uses Thread.current, it won't work properly in a multi-threaded environment where you have more than one thread per request.
To circumvent this problem, I have implemented a store that can be shared between threads for the same request. It's called request_store_rails, it's thread-safe, and the usage is very similar:
RequestLocals[:items] = []
Have you considered flash? It uses Session but is automatically cleared.
Memoisation?
According to this railscast it's stored per request.
Global variables are evil. Work out how to cleanly pass the data you want to where you want to use it.
app/models/my_cacher.rb
class MyCacher
def self.result
##result ||= begin
# do expensive stuff
# and cache in ##result
end
end
end
The ||= syntax basically means "do the following if ##result is nil" (i.e. not set to anything yet). Just make sure the last line in the begin/end block is returning the result.
Then in your views/models/whatever you would just reference the function when you need it:
MyCacher.result
This will cache the expensive action for the duration of a request.