Heroku request timeout when Memcachier reaches cache limit - ruby-on-rails

I have a Rails app deployed to Heroku using Memcachier (Dalli as client). I'm using the free add-on (which offers a 25 MB cache).
We started to receive request timeouts from heroku and, after debugging, we found out that manually flushing Memcachier solved the problem.
Timeouts occur when Memcachier reaches levels near its limit, like 20 MB (when limit is 25 MB).
Why Memcachier doesn't free cache space with time? Is there any missing configuration to tell Memcachier to flush when cache reach certain size?
My conf:
application.rb
config.cache_store = :dalli_store
production.rb
client = Dalli::Client.new
config.action_dispatch.rack_cache = {
:metastore => client,
:entitystore => client,
:allow_reload => false
}

Perhaps try updating your production.rb to include socket_timeout and socket_failure_delay options.
require 'dalli'
cache = Dalli::Client.new((ENV["MEMCACHIER_SERVERS"] || "").split(","),
{:username => ENV["MEMCACHIER_USERNAME"],
:password => ENV["MEMCACHIER_PASSWORD"],
:failover => true,
:socket_timeout => 1.5,
:socket_failure_delay => 0.2
})
-- OR --
If you are using Puma, you need to have the connection_pool gem installed.
Gemfile
gem 'connection_pool'
And this configuration in your production.rb
config.cache_store = :dalli_store,
(ENV["MEMCACHIER_SERVERS"] || "").split(","),
{:username => ENV["MEMCACHIER_USERNAME"],
:password => ENV["MEMCACHIER_PASSWORD"],
:failover => true,
:socket_timeout => 1.5,
:socket_failure_delay => 0.2,
:pool_size => 5
}

Related

Rails cached value lost/nil despite expires_in 24.hours

I am using ruby 2.3.3 and Rails 4.2.8 with Puma (1 worker, 5 threads) and on my admin (i.e. not critical) page I want to show some stats (integer values) from my database. Some requests take quite a long time to perform so I decided to cache these values and use a rake task to re-write them every day.
Admin#index controller
require 'timeout'
begin
timeout(8) do
#products_a = Rails.cache.fetch('products_a', :expires_in => 24.hours) { Product.where(heavy_condition_a).size }
#products_b = Rails.cache.fetch('products_b', :expires_in => 24.hours) { Product.where(heavy_condition_b).size }
#products_c = Rails.cache.fetch('products_c', :expires_in => 24.hours) { Product.where(heavy_condition_c).size }
#products_d = Rails.cache.fetch('products_d', :expires_in => 24.hours) { Product.where(heavy_condition_d).size }
end
rescue Timeout::Error
#products_a = 999
#products_b = 999
#products_c = 999
#products_d = 999
end
Admin#index view
<li>Products A: <%= #products_a %></li>
<li>Products B: <%= #products_b %></li>
<li>Products C: <%= #products_c %></li>
<li>Products D: <%= #products_d %></li>
Rake task
task :set_admin_index_stats => :environment do
Rails.cache.write('products_a', Product.where(heavy_condition_a).size, :expires_in => 24.hours)
Rails.cache.write('products_b', Product.where(heavy_condition_b).size, :expires_in => 24.hours)
Rails.cache.write('products_c', Product.where(heavy_condition_c).size, :expires_in => 24.hours)
Rails.cache.write('products_d', Product.where(heavy_condition_d).size, :expires_in => 24.hours)
end
I am using this in production and use Memcachier (on Heroku) as a cache store. I also use it for page caching on the website and it works fine there. I have:
production.rb
config.cache_store = :dalli_store
The problem I am experiencing is that the cached values disappear almost instantly, and quite intermittently, from the cache. In the console I have tried:
I Rails.cache.write one value (e.g. product_a) and check it a minute later, it is still there. Although crude, I can see the "Set cmds" increments by one in Memcachier admin tool.
However, when I add the next value (e.g. product_b) the first one disappears (becomes nil)! Sometimes if I add all 4 values, 2 seems to stick. These are not always the same values. It is like whack-a-mole!
If I run the rake to write the values and then try to read the values typically only two values are left, whereas the others are lost.
I have seen a similar question related to this where the reason explained was the use of a multithread server. The cached value was saved in one thread and could not be reached in another, the solution was to use a memcache, which I do.
It is not only the console. If I just reload admin#index view to store the values or run the rake task, I experience the same problem. The values do not stick.
My suspicion is that I am either not using the Rails.cache-commands properly or that these commands do not in fact use Memcachier. I have not been able to determine whether my values are in fact stored in Memcachier but when I use my first command in the console I do get the following:
Rails.cache.read('products_a')
Dalli::Server#connect mc1.dev.eu.ec2.memcachier.com:11211
Dalli/SASL authenticating as abc123
Dalli/SASL authenticating as abc123
Dalli/SASL: abc123
Dalli/SASL: abc123
=> 0
but I do not get that for subsequent writes (which I assume is a matter of readability in the console and not a proof of Memcachier not being used.
What am I missing here? Why won't the values stick in my cache?
Heroku DevCenter states a little different cache config and gives some advice about threaded Rails app servers like Puma using connection_pool gem:
config.cache_store = :mem_cache_store,
(ENV["MEMCACHIER_SERVERS"] || "").split(","),
{ :username => ENV["MEMCACHIER_USERNAME"],
:password => ENV["MEMCACHIER_PASSWORD"],
:failover => true,
:socket_timeout => 1.5,
:socket_failure_delay => 0.2,
:down_retry_delay => 60,
:pool_size => 5
}

A/B Test with Rails resets data after version change

I'm using split gem on Rails and my experiments keeps incrementing it's version, and after each increment all the data are lost. I don't know the pattern for it to change thus don't know how to prevent it. For example:
Experiment: landing v19 Goal:view-wheels
ALTERNATIVE NAME PARTICIPANTS NON-FINISHED COMPLETED CONVERSION RATE
A 70 40 30 42.86%
B 70 27 43 61.43% +43.33%
After some event that I don't know, the version will change to v20 and all data will be lost. I want to prevent it, I need to keep counting until I actively say to stop.
My redis configuration:
On redis.rb:
uri = URI.parse(ENV['REDISTOGO_URL'])
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :username => uri.user)
On split.rb:
require "split/dashboard"
include Split::Helper
Split.redis = REDIS
Split.configure do |config|
config.experiments = {
"landing" => {
:alternatives => ["A", "B"],
:resettable => false,
:goals => ["opt-in", "view-wheels"]
}
}
config.db_failover = true # handle redis errors gracefully
config.db_failover_on_db_error = proc{|error| Rails.logger.error(error.message) }
end

What is the best Rails Logging Gem

what is the best way of configuring Logging features on a rails project? I'm looking for something like Log4J which is available to Rails. I have found log4r and it's conflicting built in Logger class and also tried 'Logging' gem and It has some issues configuring as a audit logger. Please let me know your suggestions on this topic since I'm a beginner on the subject.
I have used below code block in logging.rb and included in environment.rb
But I'm receiving a error on 'returning' keyword as it's deprecated on rails 2.8
config/environment.rb
# Logging
require File.join(File.dirname(__FILE__), 'logging')
Rails::Initializer.run do |config|
config/logging.rb
require 'logging'
# Logging.init is required to avoid
# unknown level was given 'info' (ArgumentError)
# or
# uninitialized constant Logging::MAX_LEVEL_LENGTH (NameError)
# when an Appender or Layout is created BEFORE any Logger is instantiated:
Logging.init :debug, :info, :warn, :error, :fatal
layout = Logging::Layouts::Pattern.new :pattern => "[%d] [%-5l] %m\n"
# Default logfile, history kept for 10 days
default_appender = Logging::Appenders::RollingFile.new 'default', \
:filename => 'log/default.log', :age => 'daily', :keep => 10, :safe => true, :layout => layout
# Audit logfile, history kept forever
audit_appender = Logging::Appenders::RollingFile.new 'audit', \
:filename => 'log/audit.log', :age => 'daily', :safe => true, :layout => layout
# Production logfile, history kept forever
prod_appender = Logging::Appenders::RollingFile.new 'prod', \
:filename => 'log/production.log', :age => 'daily', :safe => true, :layout => layout
DEFAULT_LOGGER = returning Logging::Logger['server'] do |l|
l.add_appenders default_appender
end
Have a look at the following threads:
Rails Logging API
logging in rails app
What's a good logging replacement for rails?
It should be like this:
config/logging.rb
require 'logging'
# Logging.init is required to avoid
# unknown level was given 'info' (ArgumentError)
# or
# uninitialized constant Logging::MAX_LEVEL_LENGTH (NameError)
# when an Appender or Layout is created BEFORE any Logger is instantiated:
Logging.init :debug, :info, :warn, :error, :fatal
layout = Logging::Layouts::Pattern.new :pattern => "[%d] [%-5l] %m\n"
# Default logfile, history kept for 30 days
default_appender = Logging::Appenders::RollingFile.new 'default', \
:filename => "log/#{Rails.env}.log", :age => 'daily', :keep => 30, :safe => true, :layout => layout
log = Logging::Logger[:root]
log.add_appenders default_appender
log.level = :info
Rails.logger = log

Rails 3 additional session configuration options (key, expires_after, secure)

Can someone point out what the new Rails 3.x session configuration options are?
I'm trying to duplicate the same configuration that I have in my Rails 2.3.x application.
This is the configuration that I used in the application:
#environment.rb
config.action_controller.session_store = :active_record_store
config.action_controller.session = {
:key => '_something', #non-secure for development
:secret => 'really long random string'
}
# production.rb - override environment.rb for production
config.action_controller.session = {
:key => '_something_secure',
:secret => 'really long random string',
:expire_after => 60*60,#time in seconds
:secure => true #The session will now not be sent or received on HTTP requests.
}
However, in Rails 3.x, I can only find mention of the following:
AppName::Application.config.session_store :active_record_store
AppName::Application.config.secret_token = 'really long random string'
AppName::Application.config.cookie_secret = 'another really long random string'
Are there other config settings to control the key, expire_after time, and secure option?
Regarding the latter, if "config.force_ssl = true" is set in production.rb, I assume the secure option is no longer required?
Thanks very much!
You now configure the Cookie-based session store through an initializer, probably in config/initializers/session_store.rb. In Rails 3 the session store is a piece of middleware, and the configuration options are passed in with a single call to config.session_store:
Your::Application.config.session_store :cookie_store, :key => '_session'
You can put any extra options you want in the hash with :key, e.g.
Your::Application.config.session_store :cookie_store, {
:key => '_session_id',
:path => '/',
:domain => nil,
:expire_after => nil,
:secure => false,
:httponly => true,
:cookie_only => true
}
(Those are just the standard defaults)
If you force SSL in production then setting secure on the cookie shouldn't really make a difference in practice, but you might want to set it just to be on the safe side...
Your::Application.config.session_store :cookie_store, {
:key => '_session_id',
:secure => Rails.env.production?
}

Failed to get session options?

I'm trying to switch the session store in Rails 3 by changing the
config/application.rb as following:
config/application.rb
#-----------------------------------
memcache_options = {
:compression => true,
:debug => false,
:namespace => "xx-cache",
:readonly => false,
:urlencode => false
}
CACHE = MemCache.new(memcache_options)
CACHE.servers = ['127.0.0.1:17898']
#check if CACHE is connected
#puts CACHE
config.action_dispatch.session = {
:session_key => '_xx_session',
:secret => 'xx',
:cache => CACHE,
:expires => 900
}
config.action_dispatch.session_store = :mem_cache_store
#-----------------------------------
Memcache server is running. However, when run rails s, i got this
message:
=> Booting WEBrick
=> Rails 3.0.0.beta3 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
Exiting
/usr/lib/ruby/gems/1.9.1/gems/actionpack-3.0.0.beta3/lib/
action_dispatch/middleware/session/mem_cache_store.rb:19:in
`initialize': #<ActionDispatch::Session::MemCacheStore:0xa302950>
unable to find server during initialization. (RuntimeError)
It seems the session options was not passed correctly. But i'm not
sure what's wrong here cause I'm new to Rails.
Any help will be appreciated.
Thanks
I'm not -entirely- sure, but I think you need to include "cached_model." Try adding
require 'cached_model' #(At the top)
Let me know if that worked.

Resources