I'm writing some script using mechanize(ruby) to test my site, when I make a get request to the login page, I get the html which includes CSRF token in the login form that is different from the CSRF stored in rails session, so when submitting a post request with login data, an error is generated Can't verify CSRF token authenticity and I can't login. This doesn't happen when logging from a browser normally, so any thought ?
Note: The CSRF returned when using mechanize to fetch the login page, always has the same value over all my tests today and yesterday! I don't know if this is helpful or not.
My code:
agent = Mechanize.new
page = agent.get('http://localhost:3000')
form = page.forms.last
form['user[email]'] = 'my email'
form['user[password]'] = 'password'
form.submit
I faced this problem before, I asked on different Q and A sites but no useful answers I got it.
the only solution I found this :
skip_before_filter :verify_authenticity_token
it will skip the CSRF verification I think it will good for test environment for sure not for production.
I wish to find other solution.
okay, i know this topic is quite old, but I stumbled upon this and needed a working solution myself and here we go:
please set your credentials first and start your local server, then run the script.
require 'mechanize'
require 'nokogiri'
require 'open-uri'
# set global login credentials
$email = "email#emailprovider.com"
$password = "your-password"
# generate a mechanize agent object for persistent "browsing"
a = Mechanize.new { |agent| agent.user_agent_alias = 'Mac Safari' }
def form_login(a)
# get the desired page with the login form
a.get('http://localhost:3000/users/sign_in') do |page|
# search the current csrf-token in the head of the document
csrf_token = page.search('//meta[#name="csrf-token"]/#content')
# now let's dive into the form, that asks for email, password
# and for the authenticity_token in a hidden field
login_result = page.form_with(:id => 'new_user') do |login|
login.field_with(:name => 'user[email]').value = $email
login.field_with(:name => 'user[password]').value = $password
login.field_with(:name => 'authenticity_token').value = csrf_token
# check output in console
puts login.values
# submit the form
login.submit
end # of login block
end
end
form_login(a)
Related
I would like to implement the Gmail API into one of my projects. So I've followed the quickstart tutorial maid by google to do it, and it's working great.
require "google/apis/gmail_v1"
require "googleauth"
require "googleauth/stores/file_token_store"
require "fileutils"
OOB_URI = "urn:ietf:wg:oauth:2.0:oob".freeze
APPLICATION_NAME = "Gmail API Ruby Quickstart".freeze
CREDENTIALS_PATH = "credentials.json".freeze
# The file token.yaml stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
TOKEN_PATH = "token.yaml".freeze
SCOPE = Google::Apis::GmailV1::AUTH_GMAIL_MODIFY
##
# Ensure valid credentials, either by restoring from the saved credentials
# files or intitiating an OAuth2 authorization. If authorization is required,
# the user's default browser will be launched to approve the request.
#
# #return [Google::Auth::UserRefreshCredentials] OAuth2 credentials
def authorize
client_id = Google::Auth::ClientId.from_file CREDENTIALS_PATH
token_store = Google::Auth::Stores::FileTokenStore.new file: TOKEN_PATH
authorizer = Google::Auth::UserAuthorizer.new client_id, SCOPE, token_store
user_id = "default"
credentials = authorizer.get_credentials user_id
if credentials.nil?
url = authorizer.get_authorization_url base_url: OOB_URI
puts "Open the following URL in the browser and enter the " \
"resulting code after authorization:\n" + url
code = "XXXX"
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id, code: code, base_url: OOB_URI
)
end
credentials
end
# Initialize the API
service = Google::Apis::GmailV1::GmailService.new
service.client_options.application_name = APPLICATION_NAME
service.authorization = authorize
messages = []
next_page = nil
begin
result = service.list_user_messages('me', max_results: [500].min, page_token: next_page)
messages += result.messages
break if messages.size >= 500
next_page = result.next_page_token
end while next_page
puts "Found #{messages.size} messages"
messages.each do |message|
puts "- #{message.thread_id }"
end
In the project, we made a loop with the labels, and all is working correctly. Now, I would like to do the same with my emails. As you can see on the following script, I'm looping the id of the mails. This is working without problem, but when I'm trying to loop the content or any other attributes described on the documentation, it's rendering an array empty.
puts "- #{message.body }"
Do you have any leads that allow me to identify my mistake?
You can reproduce with the Try this API for listing messages that
the response contains only the thread Ids and message Ids.
To retrieve message.body you need to use the method Users.messages: get specifying as parameter the message Id you retrieved with Users.messages: list.
I'm trying to use iNaturalist's API via Ruby on Rails. I'm new to Ruby and iNaturalist's documentation is pretty sparse. As a first step, I need to figure out how to get authorization from their site.
iNaturalist provides the sample code below. I set up a project with iNaturalist and tried running the sample code in Rails Console with my credentials. #{url} in the following line is replaced with a url that the user is supposed to go to in order to log in to iNat:
puts "Go to #{url}, approve the app, and you should be redirected to your " +
"redirect_uri. Copy and paste the 'code' param here."
I went to the resulting url and logged in:
https://www.inaturalist.org/oauth/authorize?client_id=[my client id]&redirect_uri=https://ruby_on_rails--timrobinson41199691.codeanyapp.com/login/&response_type=code
iNaturalist responds with "The redirect uri included is not valid."
If I leave off &response_type=code, it responds with "The authorization server does not support this response type."
My website is on codeanywhere.com. The url of the main page is "https://ruby_on_rails--timrobinson41199691.codeanyapp.com/". Part of the problem is that I don't understand what kind of page I'm supposed to create for redirect_uri, since I'm still kind of new at this.
require 'rubygems'
require 'rest_client'
require 'json'
site = "https://www.inaturalist.org"
app_id = 'YOUR APP ID'
app_secret = 'YOUR APP SECRET'
redirect_uri = 'YOUR APP REDIRECT URI' # you can set this to some URL you control for testing
# REQUEST AN AUTHORIZATION CODE
# Your web app should redirect the user to this url. They should see a screen
# offering them the choice to authorize your app. If they aggree, they will be
# redirected to your redirect_uri with a "code" parameter
url = "#{site}/oauth/authorize?client_id=#{app_id}&redirect_uri=#{redirect_uri}&response_type=code"
# REQUEST AN AUTH TOKEN
# Once your app has that code parameter, you can exchange it for an access token:
puts "Go to #{url}, approve the app, and you should be redirected to your " +
"redirect_uri. Copy and paste the 'code' param here."
print "Code: "
auth_code = gets.strip
puts
payload = {
:client_id => app_id,
:client_secret => app_secret,
:code => auth_code,
:redirect_uri => redirect_uri,
:grant_type => "authorization_code"
}
puts "POST #{site}/oauth/token, payload: #{payload.inspect}"
puts response = RestClient.post("#{site}/oauth/token", payload)
puts
# response will be a chunk of JSON looking like
# {
# "access_token":"xxx",
# "token_type":"bearer",
# "expires_in":null,
# "refresh_token":null,
# "scope":"write"
# }
# Store the token (access_token) in your web app. You can now use it to make authorized
# requests on behalf of the user, like retrieving profile data:
token = JSON.parse(response)["access_token"]
headers = {"Authorization" => "Bearer #{token}"}
puts "GET /users/edit.json, headers: #{headers.inspect}"
puts RestClient.get("#{site}/users/edit.json", headers)
puts
After the user logs in to iNat, he should be redirected back to my website with the authorization code provided in the data. In routes.rb, my login route is set as:
post '/login', to: 'organisms#login'
I've tried using get, as well.
iNat is returned the error mentioned above and not redirecting back to my site.
OAuth can be a bit daunting at first. And that guide really just shows the equivalent of using cURL to test your API.
In an actual application redirect_uri is whatever endpoint in your application that handles the response when the provider redirects back from authorization.
So lets setup a minimal real rails app.
1. Register your app
Register a new application or edit your existing app.
Use http://localhost:3000/oauth/inaturalist/callback for the callback url (adjust the host as needed).
Keep the window open as you will need the client_id and secret in a moment.
2. Setup your routes
# /config/routes.rb
Rails.application.routes.draw do
# just make sure you have a root path defined.
root to: 'pages#home'
namespace :oauth do
namespace :inaturalist, controller: :callbacks do
# This is just a simple redirect route
get '/', action: :passthru, as: :authorize
# This is the route that handles the actual callback
get 'callback'
end
end
end
You can actually do this without the redirect route and just plant a link to the https://www.inaturalist.org/oauth/authorize... url in your view. But having it isolates your application against the craziness that is OAuth and its how OmniAuth does it.
3. Add your credentials to the Rails app.
In Rails 5 use the encrypted credentials to store your client_id and secret.
Run $ bin/rails credentials:edit from your shell.
inaturalist:
client_id: <from the inaturalist site>
secret: <from the inaturalist site>
In earlier versions use ENV vars instead.
4. Install the oauth2 gem
# Place this in your gemfile outside any groups
gem 'oauth2', '~> 1.4', '>= 1.4.1'
Then run bundle install.
4. Setup the controller
# app/controllers/oauth/inaturalist/callbacks_controller.rb
require 'oauth2'
module Oauth
module Inaturalist
class CallbacksController < ::ActionController::Base
# GET /oauth/inaturalist
def passthru
redirect_to client.auth_code.authorize_url
end
# This endpoint is where the provider redirects the user back to
# after authorization.
# GET /oauth/inaturalist/callback
def callback
# Fetch an auth token from the access code
token = client.auth_code.get_token(params[:code])
# Perform an authenticated request to get the users data
api_response = token.get("/users/edit.json")
#user_data = JSON.parse(api_response.body)
# This is just an example of how you can use the user data from
# the provider
#user = {
uid: #user_data["id"],
nickname: #user_data["nickname"]
}
session[:user_id] = #user[:uid]
session[:token] = token.to_hash
redirect_to root_path, success: "Hello #{#user[:nickname]}"
end
private
# Change this if you are not using Rails 5 credentials.
def client
OAuth2::Client.new(
credentials.fetch(:client_id),
credentials.fetch(:secret),
site: "https://www.inaturalist.org",
redirect_uri: oauth_inaturalist_callback_url
)
end
def credentials
Rails.application.credentials.fetch(:inaturalist)
end
end
end
end
token here is actually a new OAuth2::AccessToken instance that can be called to call endpoints with the fetched credentials.
This example stores the token in the session. You can retrieve it in subsequent requests with:
token = OAuth2::AccessToken.from_hash( session[:token] )
The docs kind of mention trading the oauth access token for an api token for api.inaturalist.org. But the details are kind of sparse.
5 Add a link to sign in:
<%= link_to 'Sign in to iNaturalist.org', oauth_inaturalist_authorize_path %>
I've installed the mechanize gem in rails app and to test it I'm just copying and pasting the code below into the irb console. It logs into the page and I can put Orange into the search field and submit but then the next page has no content with "Orange" nor any of the orange employees that I see in my browser. Does linkedin have some security features to stop this or am I doing something wrong?
require 'rubygems'
require 'mechanize'
require 'nokogiri'
require 'open-uri'
#create agent
agent = Mechanize.new { |agent|
agent.user_agent_alias = 'Mac Safari 4'
}
agent.follow_meta_refresh = true
#visit page
page = agent.get("https://www.linkedin.com/")
#login
login_form = page.form('login')
login_form.session_key = "email"
login_form.session_password = "password"
page = agent.submit(login_form, login_form.buttons.first)
# get the form
form = agent.page.form_with(:name => "commonSearch")
#fill form out
form.keywords = 'Orange France'
# get the button you want from the form
button = form.button_with(:value => "Search")
# submit the form using that button
agent.submit(form, button)
agent.page.link_with(:text => "Orange")
=> nil
The problem with Mechanize is it won't work directly with JavaScript loaded content, like the one found on this scenario using a LinkedIn search.
A solution for this is to look on the page's body and use regular expressions to get the desired content, and then parse the results as JSON.
For example:
url = "http://www.linkedin.com/vsearch/p?type=people&keywords=dario+barrionuevo"
results = agent.get(url).body.scan(/\{"person"\:\{.*?\}\}/)
person = results.first # You'd use an each here, but for the example we'll get the first
json = JSON.parse(person)
json['person']['firstName'] # => 'Dario'
json['person']['lastName'] # => 'Barrionuevo'
I'm trying to make an authenticated post request, and I need the CSRF. When I log in, it isn't generating the _csrf_token for some reason:
2.0.0p247 :126 > app.post '/community_members/login', {"refinery_user[login]"=>'chloe', 'refinery_user[password]'=>'test'}
=> 302
2.0.0p247 :127 > app.session
=> {"warden.user.refinery_user.key"=>[[56], "$2a$10$gr/rTcQfuXnes1Zml3qOPu"], "session_id"=>"f77d89cef9ff1710890f575b479bb690"}
I tried app.session[:_csrf_token] ||= SecureRandom.base64(32) before login, but it is always deleted. I also tried to get the login form first, but _csrf_token is still not set.
2.0.0p247 :133 > app.get '/community_members/sign_in'
2.0.0p247 :134 > app.response # authenticity_token is burried in the raw HTML
2.0.0p247 :136 > app.post '/community_members/login', {"refinery_user[login]"=>'chloe', 'refinery_user[password]'=>'test'}
2.0.0p247 :137 > app.session
=> {"warden.user.refinery_user.key"=>[[56], "$2a$10$gr/rTcQfuXnes1Zml3qOPu"], "session_id"=>"c2c564229e55b81ca788788558d7d11a"}
How do I manually generate the token to pass to the post request?
Oh ok, I think I need to submit the authenticity_token (that is set in the session after GETing the login form) to the login form, then it puts it in the session permanently! This worked:
app.post '/community_members/login', {'authenticity_token'=>'GfT5GtcUmYQ927oNQmh2MR0NKQucGSx8mtMg3Ph9kXw=', "refinery_user[login]"=>'chloe', 'refinery_user[password]'=>'test'}
Here is a full example: https://stackoverflow.com/a/23899701/148844
I felt like using Nokogiri to parse the response was a little heavy so I wrote a simple regex to pull the authenticity token out of the response.
app.get '/api_with_form'
authenticity_token = app.response.body.match(/<[^<]+authenticity_token[^>]+value="([^"]+)"[^>]+>/)[1]
I'm using this to log in
app.get '/users/sign_in'
authenticity_token = app.response.body.match(/<[^<]+authenticity_token[^>]+value="([^"]+)"[^>]+>/)[1]
app.post '/users/sign_in', 'user[email]' => 'my_username', 'user[password]' => 'my_password', authenticity_token: authenticity_token
Unless you load the page before you do app.post, there is no CSRF token generated to begin with. Manually generating a new one will not help because it won't match what is stored on the server, which is likely to be some null value.
You need to load the page, parse out the CSRF token, and then use that one.
Alternately, you can load the form, try to read the CSRF token out of app.session[:_csrf_token] and use that.
I use these commands in rails console:
app_controller = ActionController::Base::ApplicationController.new
app_controller.request = ActionDispatch::Request.new({})
app_controller.send(:form_authenticity_token)
here's my problem:
I need to post data from RoR server to remote PHP server, to a specific url, but before that I need to authenticate.. any help is much appreciated..
What I have done so far..
#sample data
postparams ={'id'=>1, 'name'=>'Test', 'phone'=>'123123123'}
#url - is in form http://domain.com/some/somemore
#user - contains username
#pass - contains password
require "uri"
require "net/http"
uri = URI(url)
req = Net::HTTP::Post.new(uri.path)
req.set_form_data(postparams)
res = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(req)
end
case res
when Net::HTTPSuccess, Net::HTTPRedirection
#all ok
else
res.value
end
Obviously I get 403.. because I'm not authorized? How do I authorize?
I also tried my luck with mechanize gem (below - using the same "sample" data\vars)
#when not logged in it renders login form
login_form = agent.get(url).forms.first
login_form.username = user
login_form.password = pass
# submit login form
agent.submit(login_form, login_form.buttons.first)
#not sure how to submit to url..
#note that accessing url will not render the from
#(I can't access it as I did with login form) - I simply need to post postparams
#to this url... and get the response code..
I think the mechanize gem is your best choice.
Here is an example showing how to post a file to flicker using mechanize.
Maybe you could easily adapt to your needs:
require 'rubygems'
require 'mechanize'
abort "#{$0} login passwd filename" if (ARGV.size != 3)
a = Mechanize.new { |agent|
# Flickr refreshes after login
agent.follow_meta_refresh = true
}
a.get('http://flickr.com/') do |home_page|
signin_page = a.click(home_page.link_with(:text => /Sign In/))
my_page = signin_page.form_with(:name => 'login_form') do |form|
form.login = ARGV[0]
form.passwd = ARGV[1]
end.submit
# Click the upload link
upload_page = a.click(my_page.link_with(:text => /Upload/))
# We want the basic upload page.
upload_page = a.click(upload_page.link_with(:text => /basic Uploader/))
# Upload the file
upload_page.form_with(:method => 'POST') do |upload_form|
upload_form.file_uploads.first.file_name = ARGV[2]
end.submit
end
I strongly suggest the use of ruby rest-client gem.