Rails 4.2 InvalidAuthenticityToken error, but only in production - ruby-on-rails

I'm making this little app that uses (more or less) the sessions management code from https://www.railstutorial.org/book.
It works fine in development mode, and if running in production mode on my development machine. When I deploy to a machine running nginx and Phusion Passenger, I start getting InvalidAuthenticityToken on every request, where a token is used (forms and links with method: delete for example).
I have verified that the token is generated and is sent along with the request.
I have noticed one thing. The main area of the app at / is open to all and does not require any kind of login. The area needing login is at /admin. When running in development, one session cookie is generated with a path of /. When deployed, there are two session cookies, one for / and one for /admin. I suspect that the CSRF token is generated using one session and then validated using the other session.
Does this sound plausible? How would I go about investigating this further and fixing it?
Thank you in advance.

It seems this is linked to bugs regarding cookies in Phusion Passenger 5 beta:
Session being emptied on POST requests in 5.0.0-beta2
Downgrading to 4.0.57 fixed the problems.

Related

Active Admin requires a force refresh after records are updated

I'm using a rather vanilla implementation of Active Admin.
After the user updates a record using the Active Admin interface the changes are not reflected in the browser until the user performs a hard refresh (ctrl+F5).
This behavior is not observed in development or testing, but is seen in production. I believe it's a caching issue.
Is there a way I can force the newly updated information to be shown so the user doesn't have to force refresh after every record update?
The problem had no relation to the browser or rails cache, but was due to server side caching from Nginx.
Apparently, a hard refresh (ctrl+F5) will some how tell Nginx to not use cache. Nginx, under a hard refresh request, will actually request the page contents from rails and then serve up the new content, instead of the cached content.
For now I've disabled caching with Nginx to prove this theory resolved the problem, via adding to the Nginx configuration file:
expires off;
Then I restarted Nginx with:
service nginx restart
Now everything is working fine. It seems like black magic, how Nginx could know (server side) that a user requested a hard refresh.

Migrating to Cedar on Heroku and losing URL from heroku.com to herokuapp.com

I'm in the process of doing several major improvements to my main production app on Heroku
This includes : using PostgreSQL in development, upgrading to the latest Rails, move to a dedicated database with Crane, and using thin as a webserver. The "last" thing I wanted to do as a logical step was to upgrade my app to Cedar stack instead of Bamboo. I've followed most of the instructions and have an ok "clone" app.
I'd like to move forward and use this new cedar app instead of the bamboo one.
The problem is that this app main use is as a backend serving API requests to an iOS app. These requests are in the format : xxx.heroku.com/...
It was probably a bad idea to use this url in the first place, but this is it and can't be changed on all our current iOS users.
I can find a way to rename my cedar app xxx. The problem is that it will be xxx.herokuapp.com . I know there is an automatic redirect on heroku, but it seems to be only for GET requests. So all my API requests won't return the XML responses I need for the iOS app.
Any suggestions ? I thought I would be able to use xxx.heroku.app going forward and I'm a bit stuck now.
Simple thing is to replace your bamboo app with a simple app that redirects requests to your new URL.
I would suggest never using the heroku domains as production domains - always put your own domain in front of them.

Help debugging Apache, Passenger and Rails problem

We have an environment running that uses Apache, Passenger and rails. The system is handling most request normally, yet certain requests do not make it to the rails application. For instance, a request to /books is successful, but /books/1 hits apache and passenger, but does not even make it to rails.
We set the apache log level to debug and the passenger log level to 3 so that we could monitor all incoming requests. We could see each request coming through and even the /books/1 request is being handled by passenger. But it never gets to rails.
Is there any way to determine where the request goes between Passenger and rails or where debugging information might live? Has anyone ever seen any problems with passenger spawning or queuing? We have spawning set to conservative. Also, we have had some permission/ownership problems in the past, so I am not ruling this out yet.
Thanks in advance
First guess: is that it's reading from your cache at public/books/1.html. This fits all the symptoms. If there is a public/books/1.html file when you go to request books/1 Apache will serve the request by sending just that file.
Second guess: alternate configuration is mucking about with how Apache is serving the route.

App session cookie not being created in Rails, sporadically

This is an issue sporadically for very few users, however we haven't been able to replicate it. However I have now got a Chrome instance (Mac) which is reproducing the error (for some unknown reason), and I hope to not restart it until I have this nailed!
Rails application, using memcached for session store. While the bug manifests in the _app_session_id cookie not being created, our javascript-generated cookie test and app-generated language cookies are being created successfully. This means that InvalidAuthenticityToken errors are thrown for every form that is submitted by those afflicted - people can't log into the app.
The error occurs across all browsers - had reports for IE7 and Firefox (which most users use). Switching to another browser often fixes the issue (though not always), and standard cache-cookie-clear tactics do not.
So now that I have got Chrome open which is having the same issue - in development, staging and live environments (meaning http and https). All other browsers are fine.
I've restarted the servers and restarted memcached. I don't really want to restart Chrome - in the risk that the issue does go away with that (having said that, it hasn't worked for users).
I've been tcpdumping the requests - and although I'll keep digging, I'd love it if anyone had any suggestions, places to start looking, anything. This is really painful ;)
Thanks!

Rails - Invalid Authenticity Token After Deploy

We're using EngineYard Cloud to deploy our Ruby on Rails application. We are running Rails v2.3.3.
EngineYard Cloud deploys to AWS instances in a manner similar to Capistrano. After each deploy, we're running into Invalid Authenticity Token errors. Specifically, any user that has previously visited our application and then visits after the deploy and then tries to submit a form gets an invalid authenticity token error. This error persists until they reset their cookies for the site. After they reset their cookies, the site works as expected with no errors.
We are using ActiveRecord's session store and sessions are being saved to the database.
This is the error we are seeing:
ActionController::InvalidAuthenticityToken
/usr/lib/ruby/gems/1.8/gems/actionpack-2.3.3/lib/action_controller/request_forgery_protection.rb:79:in `verify_authenticity_token'
The session object is nil after the deploy, however, the session data still persists in the database and the session ID cookie still exists:
Session:
session id: nil
data: nil
We haven't been able to explain this one. Any thoughts on what could be the root cause?
Thanks for any suggestions!
EDIT: Just to update on this, we've been able to isolate an example of the error.
1) User loads form
2) Code is updated on server
3) User submits form
** Invalid Authenticity Token error occurs
It seems that when the environment changes, Rails is unable to handle this with the authenticity token.
We've tried several steps to resolve:
Resetting the session
Deleting the session cookie (both in JavaScript and Rails)
Wiping the session table in the database after deploying code
Nothing works. The only thing that works is having the user clear their cookies client-side.
(We've been Googling (even tried Binging!) for answers, but no dice. This seems to be a similar related issue: http://railsforum.com/viewtopic.php?id=21479)
Also: initially we thought this was isolated to our deployment to EngineYard, but we've also been able to reproduce it on our development server that we deploy to via Capistrano.
Any thoughts would be gratefully accepted.
Thanks!
ANSWER: After extensive work by EngineYard (they're awesome!) they were able to diagnose the issue. The root cause of this issue is a bug with mongrel clusters. Mongrel doesn't seem to see the first post request after being started. EngineYard did extensive work to diagnose this:
There doesn't appear to be anything in your code causing the issue and I have found people outside of our environment that have experienced the bug as well (http://www.thought-scope.com/2009/07/mongrelcluster-rails-23x-bad-post.html). I suppose a lot of people don't see it because the first request to a site generally isn't a post or they chalk it up to flukes.
[There is a potential workaround using CURL.] The curl work around would do a simple GET request to each of your mongrels on the server to prime them so to speak. You could do this with capistrano, but that won't work if you deploy via the dashboard. You can find a short section on deploy hooks we have built into the infrastructure here:
https://cloud-support.engineyard.com/faqs/overview/getting-started-with-engine-yard-cloud
Adding a simple run curl http://localhost:500x > /dev/null should work (where x is the port you have 5000-50005 on your current setup).
We have addressed the issue by switching our stack from Mongrel to Passenger, but apparently, a fix for Mongrel is in the works. Hopefully, this helps someone who sees this same strange issue.
The authenticity token is a hidden field on the form that rails checks when the form is submitted to ensure that the post data is coming from a live session.
It is there as a security measure to prevent malicious people from using a form submit on their site to say a delete action on someones account.
You can turn it off on your whole app by adding this to config/environment.rb
config.action_controller.allow_forgery_protection = false
You can turn it off a single controller using
skip_before_filter :verify_authenticity_token
or turn it on
protect_from_forgery :except => :index
check out the ActionController::RequestForgeryProtection::ClassMethods docs for more details
It sounds like the secret key used for authentication is changing when you redeploy, invalidating all existing sessions.
Do you have the configuration parameter config.action_controller.session set anywhere, and if you do, is there anything which would cause it to change when you redeploy?
One of my apps has it configured in config/environment.rb, and a more recent one (generated with Rails 2.3) has it set in config/initializers/session_store.rb. The setting looks like:
config.action_controller.session = {
:secret => 'long-string-of-hex-digits'
}
If you don't have this configured for some reason, rake secret will generate a key for you, which can then be inserted into your configuration.
(If it is — and it's not being changed by your deployment processes — then I have no idea what's going on.)
If it would only be there for mongrels! I'm getting the exact same error on passenger as well (user loads form, deploy, submit -> invalid authenticity token). It'd be interesting to know how you solved the issue by switching to passenger? Any further hints are highly welcome. I'll have a closer look as well...
Cheers!
Have encountered this same problem with Rails 2.3 and a Mongrel cluster where the session secret is definitely set in the session initializer. The problem occured even after clearing the client cookies on the client.
However the suggestion of doing a curl get request across all the mongrels after they restart appears to work - thank goodness someone figured this out because it appears to be pretty darned obscure.
The only added info I can supply we are using Apache mod_proxy_balancer along with https in front of our Mongrels, however this problem was occuring before we turned on SSL. Is anyone seeing this with haproxy as the balancer instead of Apache?
This solved this issue for me :-) :-) :-)
https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/4690-mongrel-doesnt-work-with-rails-238#ticket-4690-37 Posted by Mike Bethany
August 30th, 2010 # 06:43 PM.
I've never gone to any length to figure out the details, but for me, this is a client-side data rot issue. If I've been messing around with the way I store my sessions (and therefore, my authorization details,) I get this error from time to time. Clearing out the private browser data; cookies, authenticated sessions, the works, has always solved it for me.
Hope this helps.

Resources