We have a new web app constructed by another developer. My job was to implement a backend API for other programs to use to exchange XML data with the database. It was going pretty well, until the main app implemented devise to authenticate users. I figured I'd be able to tweak the curl command to log in then submit/request data, but it isn't going very well. I'm trying things like
curl --cookie-jar ~/Desktop/cjar --data "user[login]=<username>" --data "user[password]=<pwd>" --data "commit=Sign in" localhost:3000/users/sign_in
followed by
curl --cookie ~/Desktop/cjar --data "<root_node></root_node>" localhost:3000/my_update_method
my latest error message is
WARNING: Can't verify CSRF token authenticity
Completed 401 Unauthorized in 1ms
I guess that's better than the redirection messages I was getting, but I'm still not close to knowing what's going on.
I've found a few related posts that make me think this is partly related to devise, and partly related to automatic authentication done by Rails. When I look at the info being passed by the web page, I see authenticity_token and utf8 params that I don't know how to construct for my curl command.
I can keep trying stuff, but I'm new to both curl and authentication, so I'm sort of shooting in the dark, and hoping that someone can save me some time. I guess my questions are:
Can this be done?
Should this be done? If not, what other options do I have?
One post suggested a way around the authentication, but is is possible for only the curl commands to require a login/pwd, but bypass any other validation, without affecting the main web app?
UPDATE 1:
Thanks for all the help, I appreciate it. The response to the first curl command (listed above) is now just a redirection to the login page, so logging in doesn't seem to take. Does anyone have any suggestions about where to start debugging that? Nothing is ever getting into the application, so there's no logging there to look at. Here's the content of the cookie, if it means anything to anyone:
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.
#HttpOnly_localhost FALSE / FALSE 0 _glow_session BAh7B0kiCmZsYXNoBjoGRUZvOiVBY3Rpb25EaXNwxhc2g6OkZsYXNoSGFzaAk6CkB1c2VkbzoIU2V0BjoKQGhhc2h7ADoMQGNsb3NlZEY6DUBmbGFzaGVzewY6C25vdGljZUkiHFNpZ25lZCBpbiBzdWNjZXNzZnVsbHkuBjsAVDoJQG5vdzBJIg9zZXNzaW9uX2lkBjsARkkiJTIwODc1Y2VjM2ViNzlmZDE3ZjA4ZjVmMDAxNWMxMDU4BjsAVA%3D%3D--d90722b6da386630b33f57902447b440f30d0b2a
I've added skip_before_filter :verify_authenticity_token to the controller that handles the api requests, and that eliminated the error I was seeing before.
UPDATE 2:
After some troubleshooting and debugging, I think I may actually be getting logged in okay, but when I execute the second curl, the development.log file shows
Started POST "/upsert_experiment" for 127.0.0.1 at 2013-10-16 12:14:12 -0500
Processing by WebServiceController#upsert_experiment as */*
Parameters: {"experiment_xml"=>"<experiment></experiment>"}
Completed 401 Unauthorized in 1ms
which leads me to think that the authorization is still the problem.
UPDATE 3:
I think the problem is just that skip_before_filter :verify_authenticity_token isn't working.
I have
class ApplicationController < ActionController::Base
rescue_from DeviseLdapAuthenticatable::LdapException do |exception|
render :text => exception, :status => 500
end
protect_from_forgery
before_filter :authenticate_user!
end
and
class WebServiceController < ApplicationController
skip_before_filter :verify_authenticity_token
...
end
With this setup, I get "Completed 401 Unauthorized in 1ms". If I comment out the protect_from_forgery line in the superclass, it just works, so the forgery protection is definitely the problem. I just don't know how to solve it.
Rails CSRF protection is preventing you from making this request.
From rails documentation
CSRF protection automatically include a security token, calculated from the current session and the server-side secret, in all forms and Ajax requests generated by Rails. You won't need the secret, if you use CookieStorage as session storage. If the security token doesn't match what was expected, the session will be reset. Note: In Rails versions prior to 3.0.4, this raised an ActionController::InvalidAuthenticityToken error.
If you want to skip it, add this line to your controller
skip_before_action :verify_authenticity_token, :only => ["your_update_action"]
CSRF stands for Cross Site Request Forgery and is a security protection you can read more about here. Essentially rails generates a random token included in the form rendered on the page and it expects to get that token back on form submit. You don't need this protection for an api, but it is important for the web app side. See this question for methods to disable the CSRF token.
One approach you might take is to use the token_authenticable Devise module and authenticate your requests with auth_token=#{token}, then skip CSRF protection if an auth token is present.
You're missing a CSRF token. This is to prevent cross site request forgery.
Related
I started working on a Rails project, where it's not only API app, so it used CSRF verifications for each request. However, requests return JSON and could be theoretically tested using Postman (or similar Http client). The only thing that gets in the way in csrf token validation.
The only way that enabled me to bypass it is adding skip_before_filter for certain actions (temporarily for testing):
skip_before_action :verify_authenticity_token, only: %i[action_name]
Is there anyway to bypass this check with using special headers/credentials?
Cookie is fine, I copied it from UI request. However, CSRF cannot be copied since it is different for each request.
Thanks in advance
I've been implementing a Rails 4 application with an API. I want to be able to call the API from mobile phones and the webapp itself. I came across this note while researching protect_from_forgery:
It's important to remember that XML or JSON requests are also affected and if you're building an API you'll need something like:
class ApplicationController < ActionController::Base
protect_from_forgery
skip_before_action :verify_authenticity_token, if: :json_request?
protected
def json_request?
request.format.json?
end
end
I was thinking of doing this, but I have some reservations/questions:
This solution seems to leave the CSRF hole open because now an attacker could craft a link with an onclick javascript that posts JSON?
Would checking for an API token be a reasonable substitute? i.e., what if instead of skipping the authenticity check, allowing it to fail the check and recover in handle_unverified_request if the api token is present and correct for current user?
Or maybe I should just make the webapp and mobile devices send the CSRF token in the HTTP headers? Is that safe? How would the mobile phone even obtain the CSRF token, given that it isn't rendering HTML forms to begin with?
Edit for clarification:
I am more concerned about the webapp user clicking a crafted CSRF link. The mobile users are authenticated, authorized, an have an API key, so I am not concerned about them. But by enabling CSRF protection for the webapp users, the mobile users are blocked from using the protected API. I want to know the correct strategy for handling this, and I don't believe the Rails documentation gives the right answer.
An attacker could CURL at your controllers all they like, but if your API requires authentication, they wont get anywhere.
Making the API consumers send a CSRF is not really what CSRF does. To do this you'd need to implement a type of knocking mechanism where your client hits an authorization endpoint first to get the code (aka CSRF) and then submit it in the POST. this sucks for mobile clients because it uses their bandwidth, power, and is laggy.
And anyway, is it actually forgery (i.e. the F in CSRF) if its an authorized client hitting your controller after all?
Sending the CSRF token in an HTTP header is indeed a common approach. It ensures that the client has somehow obtained a valid token. For example, a crafted CSRF link will be sent with credential cookies but the header will not include the CSRF token. Your own javascript on the client will have access to domain cookies and will be able to copy the token from a cookie to the header on all XHR requests.
AngularJS follows this approach, as explained here.
As for your first two questions:
This solution seems to leave the CSRF hole open...
Indeed, which is why you should not disable the CSRF token also in your API.
Would checking for an API token be a reasonable substitute? ...
Probably not. Take into consideration the following (from OWASP):
CSRF tokens in GET requests are potentially leaked at several locations: browser history, HTTP log files, network appliances that make a point to log the first line of an HTTP request, and Referer headers if the protected site links to an external site.
General recommendation: Don't try to invent the wheel. OWASP has a page called REST Security Cheat Sheet as well as the one I linked to before. You can follow the Angular approach (copying the token from a cookie to a header on each XHR request) and for regular non-ajax forms, be sure to use only POST and a hidden field as is normally done in CSRF protection of static server forms.
I've been developing Rails app with REST API for access from mobile application.
It works quite well. When user logs in from mobile application, he gets auth_token that he uses in his future requests to API. The issue is that API is also accessible from web by going to path /api/v1/... and because of this, it has to be protected from CSRF.
I have BaseApiController class which inherits from ApplicationController that has protect_from_forgery "enabled". Here's example:
class Api::V1::BaseApiController < ApplicationController
# ...
end
class ApplicationController < ActionController::Base
protect_from_forgery
# ...
end
Now, when I do non-GET requests to my API, with auth_token, my request gets completed successfully, but in the logs I can see the famous WARNING: Can't verify CSRF token authenticity. If I remove protect_from_forgery from my BaseApiController, I don't get any warnings (obviously), but then my API is vulnerable to CSRF attacks (I made a simple HTML form that successfully changes the data across domains when there's no protect_from_forgery).
My question is: How to assure my API stays secure, but also remove the warning when doing non-GET requests?
Here's one of the solutions I've come up with, but it looks more like a hack and executes one extra DB query:
class Api::V1::BaseApiController < ApplicationController
# ...
def verified_request?
super || User.where(authentication_token: params['auth_token']).count > 0
end
end
More details about the project: Rails 3.2.14, Devise, AngularJS. The project's source code can be found here.
You may see people suggest that CSRF is not an issue for API requests (there is no state to begin with, so what is there to hijack anyhow?), so some suggest the following to simply eliminate the warning:
skip_before_filter :verify_authenticity_token, :only => [:your_method]
However, there was some commentary that it is possible to commit CSRF with text/plain using various Flash and Java-based methods. I believe that was the reason for the security patch in rails a while back: http://weblog.rubyonrails.org/2011/2/8/csrf-protection-bypass-in-ruby-on-rails/
In any event, a good solution that actually checks for an authenticity token can be found here: WARNING: Can't verify CSRF token authenticity rails
It involves actually setting the header in your request.
Good luck!
Back in February 2011, Rails was changed to require the CSRF token for all non-GET requests, even those for an API endpoint. I understand the explanation for why this is an important change for browser requests, but that blog post does not offer any advice for how an API should handle the change.
I am not interested in disabling CSRF protection for certain actions.
How are APIs supposed to deal with this change? Is the expectation that an API client makes a GET request to the API to get a CSRF token, then includes that token in every request during that session?
It appears that the token does not change from one POST to another. Is it safe to assume that the token will not change for the duration of the session?
I don't relish the extra error handling when the session expires, but I suppose it is better than having to GET a token before every POST/PUT/DELETE request.
Old question but security is important enough that I feel it deserves a complete answer. As discussed in this question there are still some risk of CSRF even with APIs. Yes browsers are supposed to guard against this by default, but as you don't have complete control of the browser and plugins the user has installed, it's should still be considered a best practice to protect against CSRF in your API.
The way I've seen it done sometimes is to parse the CSRF meta tag from the HTML page itself. I don't really like this though as it doesn't fit well with the way a lot of single page + API apps work today and I feel the CSRF token should be sent in every request regardless of whether it's HTML, JSON or XML.
So I'd suggest instead passing a CSRF token as a cookie or header value via an after filter for all requests. The API can simply re-submit that back as a header value of X-CSRF-Token which Rails already checks.
This is how I did it with AngularJS:
# In my ApplicationController
after_filter :set_csrf_cookie
def set_csrf_cookie
if protect_against_forgery?
cookies['XSRF-TOKEN'] = form_authenticity_token
end
end
AngularJS automatically looks for a cookie named XSRF-TOKEN but feel free to name it anything you want for your purposes. Then when you submit a POST/PUT/DELETE you should to set the header property X-CSRF-Token which Rails automatically looks for.
Unfortunately, AngualrJS already sends back the XSRF-TOKEN cookie in a header value of X-XSRF-TOKEN. It's easy to override Rails' default behaviour to accomodate this in ApplicationController like this:
protected
def verified_request?
super || form_authenticity_token == request.headers['X-XSRF-TOKEN']
end
For Rails 4.2 there is a built in helper now for validating CSRF that should be used.
protected
def verified_request?
super || valid_authenticity_token?(session, request.headers['X-XSRF-TOKEN'])
end
I hope that's helpful.
EDIT: In a discussion on this for a Rails pull-request I submitted it came out that passing the CSRF token through the API for login is a particularly bad practice (e.g., someone could create third-party login for your site that uses user credentials instead of tokens). So cavet emptor. It's up to you to decide how concerned you are about that for your application. In this case you could still use the above approach but only send back the CSRF cookie to a browser that already has an authenticated session and not for every request. This will prevent submitting a valid login without using the CSRF meta tag.
Rails works with the 'secure by default' convention. Cross-Site or Cross-Session Request Forgery requires a user to have a browser and another trusted website. This is not relevant for APIs, since they don't run in the browser and don't maintain any session. Therefore, you should disable CSRF for APIs.
Of course, you should protect your API by requiring HTTP Authentication or a custom implemented API token or OAuth solution.
My mini-web-appliance will submit data samples to a RoR app, which will add them to a MySQL table.
I figured out how to form the POST data packet, but what I don't get is how to avoid the authenticity-token problem.
Is there a way for my little dumb client to grab the right token and send it back? (I'm guessing not, or it wouldn't be much of a security feature).
This is not a highly security-sensitive application, so should I just tell this page to ignore the authentity-token altogether?
It will hopefully be authenticated by the fact that each client (web appliance) logs in with a unique user ID and password, so it would be protected by the session ID.
If I'm using "loose" language, please feel free to correct me. I'm new to deploying sites.
Keb'm
If each client is authenticated then it's ok to disable the authenticity token, that said you should only disable it for that one action.
skip_before_filter :verify_authenticity_token, :only => :create
If each client is authenticated then it's ok to disable the authenticity token
This is only true if you're using another authentication mechanism than http cookies. Because
you've mentioned 'session_id', i assume this is not the case.
With a standard rails session_id cookie, the user_id stored in a session and this action
being accessible by a webbrowser, it will be exposed to csrf attacks.
The best strategy for api's is implementing a custom authentication mechanism, some sort of authentication token, which is send with every http header.
Then either change the csrf protection to null_session or if you are less paranoid disable
csrf protection entirely for your api request as described here
If you still want to stick with cookie based authentication for your api, you should set
the csrf authenitcation token with the first GET request into an extra cookie. Then you read this cookie and send it's token as 'X-CSRF-Token' header. Rails will check for this header in the protect_from_forgery method and as cookies cannot be read by 3d parties an attacker will not be able to forge this request.
#application_controller.rb
protect_from_forgery with: :exception
after_action :set_csrf
def set_csrf
cookies['X-CSRF-Token'] = form_authenticity_token if protect_against_forgery?
end
# request session and x-csrf-toke
# the cookies will be stored into cookie.txt
curl -c cookie.txt http://example.com
#curl post command
curl -H "X-CSRF-Token: <token>" -b cookie.txt -d '{"item":{"title":"test"}}' "http://example.com/items.json"
See :verified_request? method to see how rails check for request forgery.