converting a curl call to a faraday request - ruby-on-rails

I have the following curl request -
`curl --socks5 #{proxy} --connect-timeout 10 -H "Accept: application/json" #{url}`
I want to write a faraday request instead of a curl call. I am doing this -
faraday = Faraday.new("#{url}", ssl:{verify: false} do |f|
f.proxy "#{proxy}"
end
faraday.get
I am getting a reponse but the response has no body or headers. Following is the response -
#<Faraday::Response:0x0056075d353ad0 #on_complete_callbacks=[], #env=#<Faraday::Env #method=:get #url=#<URI::HTTP:0x0056075d358328 URL:main_url> #request=#<Faraday::RequestOptions proxy=#<Faraday::ProxyOptions uri=#<URI::HTTP:0x0056075dce69d0 URL:proxy_url>>> #request_headers={"User-Agent"=>"Faraday v0.9.2"} #ssl=#<Faraday::SSLOptions (empty)> #response=#<Faraday::Response:0x0056075d353ad0 ...>>>
What am I doing wrong here?

The hardest issue with the conversion to Faraday is that you need to use a SOCKS5 proxy. Faraday does not support SOCKS proxies (there is an open pull-request for this).
The only way around this is to monkey-patch Faraday to use the socksify gem which adds support for SOCKS proxies to Net::HTTP (the default Faraday network adapter). The procedure is nicely described in this gist and I mostly copy-paste a slightly altered version of it here.
Basically you need to follow these steps:
Install the faraday and socksify gems
Monkey-patch Faraday to support SOCKS. Put this code into a Rails initializer. Note that the patch only applies if you don't need to authenticate to the SOCKS proxy (as your curl command suggests). If you need proxy authentication, see the gist for a patch version supporting that. The patch is as follows:
class Faraday::Adapter::NetHttp
def net_http_connection(env)
if proxy = env[:request][:proxy]
if proxy[:socks]
Net::HTTP::SOCKSProxy(proxy[:uri].host, proxy[:uri].port)
else
Net::HTTP::Proxy(proxy[:uri].host, proxy[:uri].port, proxy[:uri].user, proxy[:uri].password)
end
else
Net::HTTP
end.new(env[:url].host, env[:url].port)
end
end
Create the request. I noticed you are probably trying to make a HTTPS request so I took this into account, as well as the timeouts you have in the curl parameters:
PROXY_OPTS = {
uri: URI.parse('https://proxy_url:1080'),
socks: true
}
SSL_OPTS = { verify: false }
connection = Faraday.new(url: "https://example.com",
ssl: SSL_OPTS,
request: { proxy: PROXY_OPTS }) do |faraday|
faraday.options.timeout = 10 # open/read timeout in seconds
faraday.options.open_timeout = 10 # connection open timeout in seconds
faraday.response :logger # log debug info
faraday.adapter :net_http # use the Net:HTTP adapter
faraday.headers['Accept'] = 'application/json' # your custom headers
end
response = connection.get
response.body
Finally, please note that ignoring peer verification (verify: false in the SSL options) is insecure! You should instead properly configure Faraday to use a certificate store to verify peer certificates against. This is fully documented here.

This is how I use faraday for a post request to get an access token from Microsoft Exchange API for example.
url = "https://login.microsoftonline.com/#{ENV['TENANT']}/oauth2/token"
conn = Faraday.new url: url do |faraday|
faraday.request :url_encoded # form-encode POST params
faraday.response :logger # log requests to STDOUT
faraday.adapter Faraday.default_adapter # make requests with Net::HTTP
end
response = conn.post do |req|
req.headers['Content-Type'] = 'application/x-www-form-urlencoded'
req.body = {
client_id: URI::encode(ENV['CLIENT_ID']),
client_secret: URI::encode(ENV['CLIENT_SECRET']),
resource: URI::encode('https://graph.microsoft.com'),
grant_type: URI::encode('client_credentials'),
}
Rails.logger.info "Body #{req.body.inspect}"
end
if response.status.to_i == 200
response_body = ActiveSupport::JSON.decode(response.body)
return response_body['access_token']
else
return false
end
Hope it helps.

Related

How to run cURL commands in Rails

I'm using Ruby on Rails 5 and I need to execute the following command in my application:
curl -F 'client_id=126581840734567' -F 'client_secret=678ebe1b3b8081231aab27dff738313' -F 'grant_type=authorization_code' -F 'redirect_uri=https://uri.com/' -F 'code=AQBi4L2Ohy3Q_N3V48OygFm0zb3gEsL985x5TIyDTNDJaLs93BwXiT1tyGYWoCg1HlBDU7ZRjUfLL5HVlzw4G-7YkVEjp6Id2WuqOz0Ylt-k2ADwDC5upH3CGVtHgf2udQhLlfDnQz5NPsnmxjg4bW3PJpW5FaQs8fn1ztgYp-ssfAf6IRt2-sI45ZC8cqqr5K_12y0Nq_Joh0H-tTfVyNLKatIxHPCqRDb3tfqgmxim1Q' https://api.instagram.com/oauth/access_token
so that it returns something like:
{"access_token": "IGQVJYS0k8V6ZACRC10WjYxQWtyMVRZAN8VXamh0RVBZAYi34RkFlOUxXZnTJsbjlEfnFJNmprQThmQ4hTckpFUmJEaXZAnQlNYa25aWURnX3hpO12NV1VMWDNMWmdIT3FicnJfZAVowM3VldlVWZAEViN1ZAidHlyU2VDMUNuMm2V", "user_id": 17231445640157812}
Is there a way to make Rails execute those types of commands? I was trying the following:
uri = URI.parse('https://api.instagram.com/oauth/access_token')
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri)
request.set_form_data({
"client_id" => "126581840734567",
"client_secret" => "678ebe1b3b8081231aab27dff738313",
"grant_type" => "authorization_code",
"redirect_uri" => "http://nace.network/",
"code" => params[:code]
})
res = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request)
end
but I get the following error:
end of file reached
in this line:
res = Net::HTTP.start(uri.hostname, uri.port) do |http|
http.request(request)
end
You're using HTTPS, so you need to add this to your code:
Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
res = http.request(request)
end
But if you don't need persistent connections, you could also use this:
res = Net::HTTP.post_form(uri,
"client_id" => "126581840734567",
"client_secret" => "678ebe1b3b8081231aab27dff738313",
"grant_type" => "authorization_code",
"redirect_uri" => "http://nace.network/",
"code" => params[:code]
)
Also, you could consider using a library like Faraday, which is a lot easier to deal with.
Edit
This is from TinMan's comment below, sound points.
Using cURL from inside Ruby or Rails is extremely valuable. There is an incredible amount of functionality inside cURL that isn't implemented in Rails or Ruby; Even Ruby's HTTP clients have a hard time replicating it, so cURL is very acceptable depending on the needs of the application. And, depending on the application, because cURL is in compiled C, it could easily outrun pure Ruby clients.
Curl is a means of issuing HTTP (or HTTPs) requests from the command line.
You don't want to use CURL in Rails. You want to issue HTTP requests from within Rails. Using curl is okay, it's one way to issue HTTP requests from with Rails.
We can refine that down further to, you want to issue HTTP requests from Ruby. Narrowing/distilling down to the most basic version of the problem is always good to do.
We knew all this already probably - still worth writing down for us all to benefit from!
Use HTTP in Ruby
We want to use a HTTP Client. There are many but, for this I'm going to use Faraday (a gem) 'cause I like it.
You've made a good start with Ruby's built in NET:HTTP but I prefer Faraday's DSL. It results in more readable and extendable code.
So, here is a class! I barely tested this so, use as a starting point. Make sure you write some unit tests for it.
# This is a Plain Old Ruby Object (PORO)
# It will work in Rails but, isn't Rails specific.
require 'faraday' # This require is needed as it's a PORO.
class InstagramOAuth
attr_reader :code
# The code parameter will likely change frequently, so we provide it
# at run time.
def initialize(code)
#code = code
end
def get_token
connection.get('/oauth/access_token') do |request|
request.params[:code] = code
end
end
private
def connection
#connection ||= Faraday.new(
url: instagram_api_url,
params: params,
ssl: { :ca_path => https_certificate_location }
)
end
def instagram_api_url
#url ||= 'https://api.instagram.com'
end
# You need to find out where these are for your self.
def https_certificate_location
'/usr/lib/ssl/certs'
end
def params
# These params likely won't change to often so we set a write time
# in the class like this.
{
client_id: '126581840734567',
client_secret: '678ebe1b3b8081231aab27dff738313',
grant_type: 'authorization_code',
redirect_uri: 'https://uri.com/'
}
end
end
# How do we use it? Like so
# Your big old authorisation code from your question
code = 'AQBi4L2Ohy3Q_N3V48OygFm0zb3gEsL985x5TIyDTNDJaLs93BwXiT1tyGYWoCg1HlBDU'\
'7ZRjUfLL5HVlzw4G-7YkVEjp6Id2WuqOz0Ylt-k2ADwDC5upH3CGVtHgf2udQhLlfDnQz'\
'5NPsnmxjg4bW3PJpW5FaQs8fn1ztgYp-ssfAf6IRt2-sI45ZC8cqqr5K_12y0Nq_Joh0H'\
'-tTfVyNLKatIxHPCqRDb3tfqgmxim1Q'
# This will return a Faraday::Response object but, what is in it?
response = InstagramOAuth.new(code).get_token
# Now we've got a Hash
response_hash = response.to_hash
puts 'Request made'
puts "Request full URL: #{response_hash[:url]}"
puts "HTTP status code: #{response_hash[:status]}"
puts "HTTP response body: #{response_hash[:body]}"
When I ran the snippet above I got the following. The class works, you just need to tweak the request params until you get what you want. Hopefully the class demonstrates how to send HTTP requests in Ruby/Rails.
Request made
Request full URL: https://api.instagram.com/oauth/access_token?client_id=126581840734567&client_secret=678ebe1b3b8081231aab27dff738313&code=AQBi4L2Ohy3Q_N3V48OygFm0zb3gEsL985x5TIyDTNDJaLs93BwXiT1tyGYWoCg1HlBDU7ZRjUfLL5HVlzw4G-7YkVEjp6Id2WuqOz0Ylt-k2ADwDC5upH3CGVtHgf2udQhLlfDnQz5NPsnmxjg4bW3PJpW5FaQs8fn1ztgYp-ssfAf6IRt2-sI45ZC8cqqr5K_12y0Nq_Joh0H-tTfVyNLKatIxHPCqRDb3tfqgmxim1Q&grant_type=authorization_code&redirect_uri=https%3A%2F%2Furi.com%2F
HTTP status code: 405
HTTP response body:
Additional Reading
. https://lostisland.github.io/faraday/usage/
. https://github.com/lostisland/faraday/wiki/Setting-up-SSL-certificates

How to get the response from Faraday get request with token authentication?

I'm using Faraday with token authentication and unable to configure it to get the response.
So far i'm writing it like this
conn = Faraday.new(url: url) do |faraday|
faraday.token_auth(ENV.fetch('API_TOKEN'))
faraday.adapter Faraday.default_adapter
end
And when i'm using the conn object to get the response it is responding in a different manner like this
conn.response
ArgumentError (wrong number of arguments (given 0, expected 1+))
I'm not sure what i am missing here.
Here mentioned how to get response. So i configured it like this for the url:
url = https://some-api.com/products/1231/other_part_of_url
base_url = https://some-api.com
path = products/1231/other_part_of_url
conn = Faraday.new(url: base_url) do |faraday|
faraday.headers['Authorization'] = ENV.fetch('API_TOKEN')
faraday.adapter Faraday.default_adapter
end
response = conn.get(path)
response.body
Also i have moved the token to headers because of when using faraday.token_auth(ENV.fetch('API_TOKEN'))
it is adding additional string before the token like token token - 2342342 which is invalid for the api i'm getting the data.

Bad Request trying to call service with Digest Auth from ruby

I'm trying to call a service with Digest Auth from a rails application and it always returns a 400 bad request error.
I've used net-http-digest_auth gem to create the headers but I think I've missed something.
def get_digest(url)
uri = URI.parse(url)
http = Net::HTTP.new uri.host, uri.port
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
req = Net::HTTP::Get.new(uri.request_uri)
# Fist call with the 401 and auth headers
digest_response = http.request(req)
digest_auth_request = Net::HTTP::DigestAuth.new
uri.user = digest_auth[:user]
uri.password = digest_auth[:password]
auth = digest_auth_request.auth_header uri, digest_response['www-authenticate'], 'GET', true
req.add_field 'Authorization', auth
response = http.request(req)
# Response is always #<Net::HTTPBadRequest 400 Bad Request readbody=true>
if response.code.to_i == 200
response_body = response.body
else
error
end
response_body
end
The request's headers look like this:
Digest username=\"myuser#mydomain.com\", realm=\"Digest\", algorithm=MD5-sess, qop=\"auth\", uri=\"/path/WS/my%20user/path/path/path/path/service.svc\", nonce=\"+Upgraded+v1e3f88bce1c32bd15avn421e440ca6622ebadd4522f7ed201fab1421c39d8fd15b771b972c9eb59894f8879307b9e6a5544476bc05cc7885a\", nc=00000000, cnonce=\"d42e6ea8a37aadsasdbea1231232456709\", response=\"7fbfc75cc3aasdasd342230ebf57ac37df\""
I can't figure out what's happening, is there any other gem to make this easier?
Finally found the problem by comparing browser header vs ruby header.
I wasn't calculating "nc" (calls counter) correctly. After adding +1 it started to return a 401 error (now I have a different problem ;)).

curl to Ruby request for connecting to wheniwork

Back story
I've spent a few days on the Ruby gem for wheniwork, and it's been a nightmare. To make a long story short, the gem had a dependency of an older version of activesupport and I basically forked their repo and made a compatible version for my ruby project. I was able to create a new gem and attempted to use it, but I was getting the following json message:
{"error"=>"User login required for this resource.", "code"=>1000}
The username, password, and api key were set. What I basically need to do is access our account on wheniwork, and retrieve data.
This is an example using the curl command
curl https://api.wheniwork.com/2/login \
--data '{"username":"user#example.com","password":"*******"}' \
-H "W-Key: iworksoharditsnotfunny"
Essentially, I would get some kind of return object from wheniwork that would contain the token. I'd use this token for future requests to the wheniwork site.
To (Ruby)
A new attempt to translate this to Ruby
require 'net/http'
data = {'credentials' => {'username' => 'XXXXX#XXXXX.com','password' => 'XXXXX', 'W-Key' => '111111111XXXXX'}}
uri = URI.parse("https://api.wheniwork.com/2/login/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(uri.path, initheader = {'Content-Type' =>'application/json'})
request.add_field('Content-Type', 'application/json')
request.body = data.to_json
response = http.request(request)
Result:
#<Net::HTTPUnauthorized 401 Unauthorized readbody=true>
If you have any idea or clue (or basically anything that would help me here), I would appreciate it beyond words. Thanks!
Got it!! The major issue(s) were that the api key and (username+password) combo are two separate pieces.
I was so close the entire time, but here it is:
data = {'username' => 'XXXX#XXXXXXX.com','password' => 'XXXXX'}
uri = URI.parse("https://api.wheniwork.com/2/login/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(uri.path)
request.add_field('W-Key', 'XXXXXXXXXXXXX')
request.body = data.to_json
response = http.request(request)
puts JSON.parse(response.body)

Running a HTTP request with rails

It has been a while since I have used Rails. I currently have a curl request as follows
curl -X GET -H 'Authorization: Element TOKEN, User TOKEN' 'https://api.cloud-elements.com/elements/api-v2/hubs/marketing/ping'
All I am looking to do is to be able to run this request from inside of a rails controller, but my lack of understanding when it comes to HTTP requests is preventing me from figuring it out to how best handle this. Thanks in advance.
Use this method for HTTP requests:
def api_request(type , url, body=nil, header =nil )
require "net/http"
uri = URI.parse(url)
case type
when :post
request = Net::HTTP::Post.new(uri)
request.body = body
when :get
request = Net::HTTP::Get.new(uri)
when :put
request = Net::HTTP::Put.new(uri)
request.body = body
when :delete
request = Net::HTTP::Delete.new(uri)
end
request.initialize_http_header(header)
#request.content_type = 'application/json'
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') {|http| http.request request}
end
Your example will be:
api_request(:get, "https://api.cloud-elements.com/elements/api-v2/hubs/marketing/ping",nil, {"Authorization" => "Element TOKEN, User TOKEN" })
It would be something like the following. Note that the connection will be blocking, so it can tie up your server depending on how quickly the remote host returns the HTTP response and how many of these requests you are making.
require 'net/http'
# Let Ruby form a canonical URI from our URL
ping_uri = URI('https://api.cloud-elements.com/elements/api-v2/hubs/marketing/ping')
# Pass the basic configuration to Net::HTTP
# Note, this is not asynchronous. Ruby will wait until the HTTP connection
# has closed before moving forward
Net::HTTP.start(ping_uri.host, ping_uri.port, :use_ssl => true) do |http|
# Build the request using the URI as a Net::HTTP::Get object
request = Net::HTTP::Get.new(ping_uri)
# Add the Authorization header
request['Authorization'] = "Element #{ELEMENT_TOKEN}, User #{user.token}"
# Actually send the request
response = http.request(request)
# Ruby will automatically close the connection once we exit the block
end
Once the block exits, you can use the response object as necessary. The response object is always a subclass (or subclass of a subclass) of Net::HTTPResponse and you can use response.is_a? Net::HTTPSuccess to check for a 2xx response. The actual body of the response will be in response.body as a String.

Resources