Sidekiq Forbidden error with Redis sessions - ruby-on-rails

Seeing a Forbidden error when we attempt to retry / delete / etc from the sidekiq UI.
Read through a lot of the web traffic on people who experienced this issue, including: https://github.com/mperham/sidekiq/issues/1289
Running on
rails 4.1.11
ruby 2.2.3
sidekiq-pro 3.0.1
redis 2.8.21
puma
Heroku
redis-rails for storing sessions
relevant snippets from our app: https://gist.github.com/toddmetheny/4b511e364a4c91ad8187
We're using redis to store sessions. It seemed like a fair number of people having the problem were doing that but there were various problems sharing sessions w/ the sidekiq UI.
Tried most of the things I've seen suggested in the related closed issues without any luck. As described by others, it works well in development...and staging (which seems really weird). Was hoping someone might have something to add. Thanks in advance.

Read the comments just posted here:
https://github.com/mperham/sidekiq/issues/2487#issuecomment-179667037
Run bundle update sinatra to make sure you have the latest Sinatra and Rack stuff.
To properly share the session between Sinatra and Rails, you need to do something like this:
Sidekiq::Web.set :session_secret, Rails.application.secrets[:secret_token]
Sidekiq::Web.set :sessions, Rails.application.config.session_options

Related

Rails caching not working on server

In my application, I have used caching. This is the code, I have used. In after_filter, I called the method which include this one line code.
Rails.cache.write("properties", #properties.to_xml)
I try to get this in another action in before_filter like
#hotels = Rails.cache.fetch("properties")
this all working fine in development machine. But in server it returns null value. the application run in same development mode in server. Can you please anyone suggest me the right way. Thanks in advance.
It sounds like you haven't configured a backend for the store, so it will use ActiveSupport::Cache::MemoryStore
From the documentation:
If you're running multiple Ruby on Rails server processes (which is the case if you're using mongrel_cluster or Phusion Passenger), then this means that Rails server process instances won't be able to share cache data with each other.
This works in development since you are likely using a single server instance, so the cache is only stored in one process. For production you need to configure an alternative shared store. I'd recommend running a memcached instance, and installing and using the Dalli Gem as per the README.

How to Use Resque Web alongside a Sinatra app

I’ve got a Sinatra web app. I’m using Resque and Resque Scheduler, and now I’m looking into adding Resque Web to (hopefully) see what my Resque queue looks like. Now here’s my problem: the official Resque Web is a Rails app. I don’t know how to use a Rails app inside of Sinatra, or if it’s even possible.
So my question: What’s the best way to implement Resque Web into my Sinatra app? Can I use the rails app inside of Sinatra? I saw one person say that you should have a separate part of your app running Rails, but that seems really nasty. Is there a better way to do it?
Thanks!
I've not used the ResqueWeb, but Rails and Sinatra are both Rack compliant frameworks, so they should be able to run each other or alongside.
http://www.sinatrarb.com/intro.html#Rack%20Middleware
## my-amazing-app.rb
use MySlightlyLessAmazingRailsApp
# rest of Sinatra stuff…
or
# config.ru
map "/" do
# easiest to mount Sinatra apps this way if using the modular style
run MyAmazingSinatraApp
end
map "/resque" do
run MySlightlyLessAmazingRailsApp
end
I don't know how you'd do this with Rails, perhaps try this link http://m.onkey.org/rails-meets-sinatra or perhaps this:
RailsApp::Application.routes.draw do
mount MyAmazingApp, :at => "/more-amazed"
# rest of the rails routes
end
Here’s what I was looking for: http://asciicasts.com/episodes/271-resque
Resque-web can be stand alone, run from the command line. So I’m just starting it up with a shell command whenever I spin up an instance of my server. Not exactly what I was looking for, but it does the trick!
Thanks for the suggestions, all.

Rails extremely slow

Sorry I know this is a generic question, I'll try to provide as much detail as possible
I am running Bitnami Rubystack (3.2.7) on Amazon EC2 Medium instance. and some aspects of Rails are extremely slow, here are some of them:
when logging in (I am using devise gem), if you provide an invalid password, it would take a long time to tell you that the password is invalid.
Sign Up process takes extremely long, responds after about 2 minutes (when all it has to do is run a couple of queries agains the db?)
File uploads (on carrierwave) are so slow they are practically not working. (files are going to S3 via Fog on CarrierWave).
The code in the above instances is pretty straight forward and I don't see anything obviously wrong. In fact, most of the work gets performed by the gems (e.g. devise handles registrations and logins). any help would be greatly appreciated.
Try to use a analytics tool, like New Relic
This will help you locate te slowest code and/or the slowest db query
EDIT
On the comments below you have mentioned that you are using devise 0.5.8, this is very bad, considering that devise is, today, at version 2.1.2
Please update your devise and keep me posted.
NEW EDIT
Since the devise version is not the problem you could look into the views.
In the views check for http requests that could be inserted on a single request or a async request.
For instance Google Analytics.
If the load of their javascript files are blocking your view to load maybe putting them on head or making them async could help
I would suggest you to compile a Ruby 1.9.3-p194 with falcon patch, it increases ruby and rails speed dramatically.
falcon patch in rvm
You download ruby src and apply this patch if you do not want to use RVM.
It might also be a DNS issue if some parameter for reverse DNS lookup enabled in Apache configuration.
Just put gem rails tweak in your gem file and after that run
bundle install
I think it will solve your problem.
Thanks

How to enable output compression (gzip) on everything that goes out of my Ruby on rails on Mongrel/WEBrick Server?

I am not sure this is the right place to ask this question, as answer to this may be coding solution or a config solution :) any way,
I am using ruby 1.8.7 and rails 2.3.11 running on the Mongrel server. How do i enable the gzip compression on everything (views, data, JS, CSS etc.) that goes out of this server?
I tried google, and most responses say use apache:P but i donot have access to any other web server. There are no straight answers as to how to get webrick to do this.
Can anyone help me?
thanks
You should be able to do this using the Mongrel GZipFilter.

Rails - Invalid Authenticity Token After Deploy

We're using EngineYard Cloud to deploy our Ruby on Rails application. We are running Rails v2.3.3.
EngineYard Cloud deploys to AWS instances in a manner similar to Capistrano. After each deploy, we're running into Invalid Authenticity Token errors. Specifically, any user that has previously visited our application and then visits after the deploy and then tries to submit a form gets an invalid authenticity token error. This error persists until they reset their cookies for the site. After they reset their cookies, the site works as expected with no errors.
We are using ActiveRecord's session store and sessions are being saved to the database.
This is the error we are seeing:
ActionController::InvalidAuthenticityToken
/usr/lib/ruby/gems/1.8/gems/actionpack-2.3.3/lib/action_controller/request_forgery_protection.rb:79:in `verify_authenticity_token'
The session object is nil after the deploy, however, the session data still persists in the database and the session ID cookie still exists:
Session:
session id: nil
data: nil
We haven't been able to explain this one. Any thoughts on what could be the root cause?
Thanks for any suggestions!
EDIT: Just to update on this, we've been able to isolate an example of the error.
1) User loads form
2) Code is updated on server
3) User submits form
** Invalid Authenticity Token error occurs
It seems that when the environment changes, Rails is unable to handle this with the authenticity token.
We've tried several steps to resolve:
Resetting the session
Deleting the session cookie (both in JavaScript and Rails)
Wiping the session table in the database after deploying code
Nothing works. The only thing that works is having the user clear their cookies client-side.
(We've been Googling (even tried Binging!) for answers, but no dice. This seems to be a similar related issue: http://railsforum.com/viewtopic.php?id=21479)
Also: initially we thought this was isolated to our deployment to EngineYard, but we've also been able to reproduce it on our development server that we deploy to via Capistrano.
Any thoughts would be gratefully accepted.
Thanks!
ANSWER: After extensive work by EngineYard (they're awesome!) they were able to diagnose the issue. The root cause of this issue is a bug with mongrel clusters. Mongrel doesn't seem to see the first post request after being started. EngineYard did extensive work to diagnose this:
There doesn't appear to be anything in your code causing the issue and I have found people outside of our environment that have experienced the bug as well (http://www.thought-scope.com/2009/07/mongrelcluster-rails-23x-bad-post.html). I suppose a lot of people don't see it because the first request to a site generally isn't a post or they chalk it up to flukes.
[There is a potential workaround using CURL.] The curl work around would do a simple GET request to each of your mongrels on the server to prime them so to speak. You could do this with capistrano, but that won't work if you deploy via the dashboard. You can find a short section on deploy hooks we have built into the infrastructure here:
https://cloud-support.engineyard.com/faqs/overview/getting-started-with-engine-yard-cloud
Adding a simple run curl http://localhost:500x > /dev/null should work (where x is the port you have 5000-50005 on your current setup).
We have addressed the issue by switching our stack from Mongrel to Passenger, but apparently, a fix for Mongrel is in the works. Hopefully, this helps someone who sees this same strange issue.
The authenticity token is a hidden field on the form that rails checks when the form is submitted to ensure that the post data is coming from a live session.
It is there as a security measure to prevent malicious people from using a form submit on their site to say a delete action on someones account.
You can turn it off on your whole app by adding this to config/environment.rb
config.action_controller.allow_forgery_protection = false
You can turn it off a single controller using
skip_before_filter :verify_authenticity_token
or turn it on
protect_from_forgery :except => :index
check out the ActionController::RequestForgeryProtection::ClassMethods docs for more details
It sounds like the secret key used for authentication is changing when you redeploy, invalidating all existing sessions.
Do you have the configuration parameter config.action_controller.session set anywhere, and if you do, is there anything which would cause it to change when you redeploy?
One of my apps has it configured in config/environment.rb, and a more recent one (generated with Rails 2.3) has it set in config/initializers/session_store.rb. The setting looks like:
config.action_controller.session = {
:secret => 'long-string-of-hex-digits'
}
If you don't have this configured for some reason, rake secret will generate a key for you, which can then be inserted into your configuration.
(If it is — and it's not being changed by your deployment processes — then I have no idea what's going on.)
If it would only be there for mongrels! I'm getting the exact same error on passenger as well (user loads form, deploy, submit -> invalid authenticity token). It'd be interesting to know how you solved the issue by switching to passenger? Any further hints are highly welcome. I'll have a closer look as well...
Cheers!
Have encountered this same problem with Rails 2.3 and a Mongrel cluster where the session secret is definitely set in the session initializer. The problem occured even after clearing the client cookies on the client.
However the suggestion of doing a curl get request across all the mongrels after they restart appears to work - thank goodness someone figured this out because it appears to be pretty darned obscure.
The only added info I can supply we are using Apache mod_proxy_balancer along with https in front of our Mongrels, however this problem was occuring before we turned on SSL. Is anyone seeing this with haproxy as the balancer instead of Apache?
This solved this issue for me :-) :-) :-)
https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/4690-mongrel-doesnt-work-with-rails-238#ticket-4690-37 Posted by Mike Bethany
August 30th, 2010 # 06:43 PM.
I've never gone to any length to figure out the details, but for me, this is a client-side data rot issue. If I've been messing around with the way I store my sessions (and therefore, my authorization details,) I get this error from time to time. Clearing out the private browser data; cookies, authenticated sessions, the works, has always solved it for me.
Hope this helps.

Resources