I'm writing a string to my memcached using rails (Dalli), and then using node.js (node-memcached) to read the value, and Rails is writing to memcache with these extra prepended stuff. I also checked memcache using command line.
Writing with rails:
Rails.cache.write("test", 'helloworld' )
Reading from node.js:
// output
I"helloworld:ET
What's happening is that Dalli is calling Marshal.dump('helloworld') before writing the value to the cache. To avoid this you'll need to interact with Dalli directly instead of going through Rails.cache then you can pass the :raw => true option to make Dalli store the exact value that you pass to it.
Something like this should do it:
dcache = Dalli::Client.new
dcache.set("test", 'helloworld', 0, :raw => true)
The third argument (0) is the ttl (time to live) value. 0 means that the value never expires. To expire values from the cache you can set a non-zero value which is the time measured in seconds. So to expire the value after 5 minutes you could pass 300.
Related
I am trying to understand the race_condition_ttl directive in Rails when using Rails.cache.fetch.
I have a controller action that looks like this:
def foo
#foo = Rails.cache.fetch("foo-testing", expires_in: 30.seconds, race_condition_ttl: 60.seconds) do
Time.now.to_s
end
#foo # this gets used in a view down the line...
end
Based on what I'm reading in the Rails docs, this value should expire after 30 seconds, but the stale value is allowed to be served for another 60 seconds. However, I can't figure out how to reproduce conditions that will show me this behavior working. Here is how I'm trying to test it.
100.times.map do
t = Thread.new { RestClient.get("http://myenvironment/foo") }
t
end.map {|t| t.join.value }.uniq
I have my Rails app running on a VM behind a standard nginx/unicorn setup. I am trying to spawn 100 threads hitting the site simultaneously to simulate the "dog pile effect". However, when I run my test code, all the threads report the same value back. What I would expect to see is that one thread gets the fresh value, while at least one other thread gets served some stale content.
Any pointers are welcome! Thanks so much.
You are setting race_condition_ttl to 60 seconds which means your threads will only start getting the new value after this time expires, even not taking into account the initial 30 seconds.
Your test doesn't look like it would take 1.5 minutes to run which would be required in order for the threads to start getting the new value. From the Rails Cache docs:
Yes, this process is extending the time for a stale value by another few seconds. Because of extended life of the previous cache, other processes will continue to use slightly stale data for a just a bit longer.
The text implies using a small race_condition_ttl and it makes sense both for its purpose and your test.
UPDATE
Also note that the life of stale cache is extended only if it expired recently. Otherwise a new value is generated and :race_condition_ttl does not play any role.
Without reading source it is not particularly clear how Rails decides when its server is getting hammered or what exactly recently means in the quote above. It seems clear though that the first process (of many) of those waiting to access the cache gets to set the new value while extending life of the previous one. The presence of waiting processes might be the condition Rails looks for. In any case the expected behaviour should be observed after both initial timeout and ttl expire and cache starts serving the updated value. The delay between initial timeout and the time new value starts showing up should be similar to the ttl. Of course the precondition is the server should be hammered around the moment of initial timeout expiration.
I set 10000 keys in memcache
for i in 1..10000
Rails.cache.write("short_key#{i}", i)
end
After ~500s (not benchmarked but happens around 10m), when I do
_random = rand(10000)
Rails.cache.read("short_key#{_random}")
returns nil. This is fine. Memcached LRU policy might have destroyed those keys.
But, issue is I see a lot of free memory on server.
Also, when I run following command in telnet session,
stats cachedump 1 10
I get some random keys which I set earlier in loop and even when I try to fetch them via rails or telnet/get, memcached is not able to read that value.
Those key/values are eating up memory but somehow getting destroyed.
I use dalli to connect with memcached.
How can I correct this?
At first glance, this seems possible if the default keep alive time value is low (10 minutes or 500 seconds are both possible default values).
Since you are not setting up the expires_in (or equivalent time_to_live field), the key will be setup for default time, after which the value will expire.
Referring here:
Setting :expires_in will set an expiration time on the cache. All caches support auto-expiring content after a specified number of seconds. This value can be specified as an option to the constructor (in which case all entries will be affected), or it can be supplied to the fetch or write method to effect just one entry.
I'm working on a mashup site and would like to limit the number of fetches to scrape the source sites. There is essentially one bit of data I need, an integer, and would like to cache it with a defined expiration period.
To clarify, I only want to cache the integer, not the entire page source.
Is there a ruby or rails feature or gem that already accomplishes this for me?
Yes, there is ActiveSupport::Cache::Store
An abstract cache store class. There are multiple cache store
implementations, each having its own additional features. See the
classes under the ActiveSupport::Cache module, e.g.
ActiveSupport::Cache::MemCacheStore. MemCacheStore is currently the
most popular cache store for large production websites.
Some implementations may not support all methods beyond the basic
cache methods of fetch, write, read, exist?, and delete.
ActiveSupport::Cache::Store can store any serializable Ruby object.
http://api.rubyonrails.org/classes/ActiveSupport/Cache/Store.html
cache = ActiveSupport::Cache::MemoryStore.new
cache.read('Chicago') # => nil
cache.write('Chicago', 2707000)
cache.read('Chicago') # => 2707000
Regarding the expiration time, this can be done by passing the time as a initialization parameter
cache = ActiveSupport::Cache::MemoryStore.new(expires_in: 5.minutes)
If you want to cache a value with a different expiration time, you can also set this when writing a value to the cache
cache.write(key, value, expires_in: 1.minute) # Set a lower value for one entry
See Caching with Rails, particularly the :expires_in option to ActiveSupport::Cache::Store.
For example, you might go:
value = Rails.cache.fetch('key', expires_in: 1.hour) do
expensive_operation_to_compute_value()
end
I've done some googling and couldn't find the answer to this question. Rails allows to specify expiry times for its cache like that:
Rails.cache.fetch("my_var", :expires_in => 10.seconds)
But what happens if I specify nothing:
Rails.cache.fetch("my_var")
It never expires? Is there a default value? How can I explicitly define something that never expires?
It really depends on which cache storage you're using. Rails provides several, one of them most popular is Memcached. One of key features of Memcached is that it automatically expires old unused records, so you can forget about :expire option.
Other Rails cache storages, like memory storage or redis storage will keep will not expire date unless you explicitly specify when to do that.
More about how cache key expiration works in Rails.
Using Dalli for memcached (who doesn't), the default expiry time is never, as #Rahul says. You don't have to worry about garbage collection, as #icem says, memcached throw out the old unused records.
See the official dalli documentation:
Expires_in default is 0, which means never
https://github.com/mperham/dalli#configuration
you can set the global expiry time for dalli
config.cache_store = :dalli_store, { expires_in: 1.day}
and for better individual control:
Rails.cache.write "some_cache_key", some_cachable_string, expires_in: 3.hours
the new doc http://apidock.com/rails/ActiveSupport/Cache/Store/write doesn't say much, but the old does:
http://apidock.com/rails/ActiveSupport/Cache/MemCacheStore/write
manually expire a cache (if some event occurred):
Rails.cache.delete "some_cache_key"
They never expires. (for FileStore based cache, which is default in Rails)
If they key is found in the cache store, the value would be used. Thus it is always recommended to add atleast any expiry time.
I have a rails application, in that I am using simple rails cache. My testing is as follows:
Rails.cache.write('temp',Date.today,:expires_in => 60.seconds)
I can read it through Rails.cache.read('temp') and Rails.cache.fetch('temp').
The problem is it doesn't expire. It will still be alive after 60 seconds. Can any one tell me what is missing here.
FYI: I have declared in my development.rb as follows :
config.action_controller.perform_caching = true
config.cache_store = :memory_store
Is there anything I missed out? I want to expires my cache.
After some search, I have found one possible reason why the cache is not cleaned after 60 seconds.
You call Rails.cache.write which is documented here.
It calls write_entry(namespaced_key(name, options), entry, options), where your option :expires_in is one part of the options argument.
The implementation of write_entry has the following condition:
if expires_in > 0 && !options[:raw]
# Set the memcache expire a few minutes in the future to support race condition ttls on read
expires_in += 5.minutes
end
So there are 5 minutes added to your 60 seconds. 2 possible solutions:
Just live with it :-)
Try to include the option :raw => true, perhaps this will skip the condition, so that your expiry works as suspected.
The :expires_in option only works with compatible stores (eg memcached) - not memory.
From http://guides.rubyonrails.org/caching_with_rails.html:
Finally, if you are using memcached or Ehcache, you can also pass
:expires_in. In fact, all parameters not used by caches_action are
sent to the underlying cache store.
You should use fetch method with do block instead of Rails.cache.write. Even though the name is "fetch", it writes to the cache if you put inside do block. You also do not need to clear cache entry manually, just use the "fetch" with expires_in and race_condition_ttl. Keep the ttl value under expiry time and keep it as small as possible.
output_json = Rails.cache.fetch('temp',expires_in: 1.minute,race_condition_ttl:3) do
my_expansive_operation().to_json
end