Convert curl (with --data-urlencode) to ruby - ruby-on-rails

I am trying to convert the following curl command to ruby using net/http but I haven't figured out how to pass in the --data-urlencode script#files/jql/events.js part of the command.
curl https://mixpanel.com/api/2.0/jql -u <apikey>: --data-urlencode script#files/jql/events.js
Using net/http I had the following...
uri = URI.parse("https://mixpanel.com/api/2.0/jql")
request = Net::HTTP::Get.new(uri)
request.basic_auth("<apikey>", "")
response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") do |http|
http.request(request)
end
Is there anyway to do this? If not within net/http then maybe using another gem?

Mixpanel has it's official ruby gem
I didn't actually work with it, but I assume it have all needed methods.
But if you don't like to use it, you may use Faraday an awesome HTTP client library for Ruby.
I made a simple example with it. Please have a look:
class MixpanelClient
def initialize(url = "https://mixpanel.com/api/2.0/jql", api_key = "ce08d087255d5ceec741819a57174ce5")
#url = url
#api_key = api_key
end
def query_data
File.read("#{Rails.root}/lib/qry.js")
end
def query_params
'{"from_date": "2016-01-01", "to_date": "2016-01-07"}'
end
def get_events
resp = Faraday.new(url: #url, ssl: { verify: false }) do |faraday|
faraday.request :url_encoded
faraday.response :logger
faraday.adapter Faraday.default_adapter
faraday.basic_auth(#api_key, "")
end.get do |req|
req.params['script'] = query_data
req.params['params'] = query_params
end
raise MixpanelError.new("Mixpanel error") unless resp.status == 200
JSON.parse(resp.body)
end
end
class MixpanelError < StandardError; end
Here is the result:
[1] pry(main)> m = MixpanelClient.new
=> #<MixpanelClient:0x007fc1442d53b8 #api_key="ce08d087255d5ceec741819a57174ce5", #url="https://mixpanel.com/api/2.0/jql">
[2] pry(main)> m.get_events
I, [2016-06-09T09:05:51.741825 #36920] INFO -- : get https://mixpanel.com/api/2.0/jql?params=%7B%22from_date%22%3A+%222016-01-01%22%2C+%22to_date%22%3A+%222016-01-07%22%7D&script=function+main%28%29%7B+return+Events%28params%29.groupBy%28%5B%22name%22%5D%2C+mixpanel.reducer.count%28%29%29+%7D
D, [2016-06-09T09:05:51.741912 #36920] DEBUG -- request: Authorization: "Basic Y2UwOGQwODcyNTVkNWNlZWM3NDE4MTlhNTcxNzRjZTU6"
User-Agent: "Faraday v0.9.2"
I, [2016-06-09T09:05:52.773172 #36920] INFO -- Status: 200
D, [2016-06-09T09:05:52.773245 #36920] DEBUG -- response: server: "nginx/1.9.12"
date: "Thu, 09 Jun 2016 03:05:52 GMT"
content-type: "application/json"
transfer-encoding: "chunked"
connection: "close"
vary: "Accept-Encoding"
cache-control: "no-cache, no-store"
access-control-allow-methods: "GET, POST, OPTIONS"
access-control-allow-headers: "X-PINGOTHER,Content-Type,MaxDataServiceVersion,DataServiceVersion,Authorization,X-Requested-With,If-Modified-Since"
=> [{"key"=>["Change Plan"], "value"=>186}, {"key"=>["View Blog"], "value"=>278}, {"key"=>["View Landing Page"], "value"=>1088}, {"key"=>["login"], "value"=>1241}, {"key"=>["purchase"], "value"=>359}, {"key"=>["signup"], "value"=>116}]
A set ssl: {verufy: false} because Faraday need addtitional workaround to work with ssl certificates: https://github.com/lostisland/faraday/wiki/Setting-up-SSL-certificates

Related

Ruby Rest client POST, how to send headers

Ruby rest client is not able to send headers, my java service is not able to read headers when I trigger post request from ruby as below. My Service layers throws error Header companyID is missing. When run the same http request in Postman it works.
response = RestClient::Request.new({
method: :post,
url: 'https://example.com/submitForm',
headers:{content_type: 'application/x-www-form-urlencoded',
companyID:'Company1',
Authorization:'Basic HELLO1234'},
payload: { data: str_res}
}).execute do |response, request, result|
case response.code
when 400
[ :error, JSON.parse(response.to_str) ]
when 200
[ :success, JSON.parse(response.to_str) ]
puts request.headers
else
fail "Invalid response #{response.to_str} received."
end
end
Here is postman code that works. Need similar in Ruby Rest, pls advise.
POST /submitForm HTTP/1.1
Host: example.com
companyID: Company1
Authorization: Basic HELLO1234
Content-Type: application/x-www-form-urlencoded
Cache-Control: no-cache
Postman-Token: 528cafa1-2b5d-13a1-f227-bfe0171a9437
data=My Own data
Below worked. Looks like headers should be in same line.
payloadString = "data=My Own data"
response = RestClient::Request.new({
method: :post,
url: 'https://example.com/submitForm',
payload: payloadString,
headers: {content_type: 'application/x-www-form-urlencoded', companyID:'Company1', Authorization:'Basic HELLO1234'}
}).execute do |response, request, result|
case response.code
when 400
[ :error, JSON.parse(response.to_str) ]
when 200
[ :success, JSON.parse(response.to_str) ]
else
fail "Invalid response #{response.to_str} received."
end
end
Try using the RestClient post method:
result = RestClient.post(
'https://example.com/submitForm',
payload,
{
content_type: 'application/x-www-form-urlencoded',
companyID: 'Company1',
Authorization: 'Basic HELLO1234'
}
)
Payload in this instance is a string, so you'll need to figure out the appropriate structure for application/x-www-form-urlencoded. For example:
payload.to_json => '{"data": "str_res"}'

Get HTTPS GET request with headers in Ruby

I have the following curl command that works fine:
curl "https://api.multisafepay.com/v1/json/gateways" -X GET -H "api_key: e983177756a109e87aa5edbe05e0xxxxxxxxxxxx"
When I try to do the same from within Rails (I tried with HTTP and with Faraday I always get back a response that contains:
"error_info\": \"Invalid API key\"
This is the command I do for Faraday:
conn = Faraday.new 'https://api.multisafepay.com/v1/json/'
conn.headers = {'api_key' => 'e983177756a109e87aa5edbe05e0xxxxxxxxxxxx'}
conn.get 'gateways'
What is the difference with the Curl command that is giving a correct result set?
Edit: debugging into Faraday shows me that in this method:
def run_request(method, url, body, headers)
headers are nil? I'm continuing my search why this is.
I don't think you're giving Faraday the right URL. Try something like this
conn = Faraday.new do |f|
f.headers['api_token'] = <yourkey>
f.request :url_encoded
end
resp = conn.get('https://api.multisafepay.com/v1/json/gateways')
require 'rest-client'
require 'json'
response = RestClient.get("https://api.multisafepay.com/v1/json/gateways",
{'api_key' => 'e983177756a109e87aa5edbe05e0xxxxxxxxxxxx'}
)
puts JSON.parse(response, { symbolize_names: true })
#just in case try this one for header
#{:content_type => :json, :accept => :json, :api_key => 'e983177756a109e87aa5edbe05e0xxxxxxxxxxxx'}

converting a curl call to a faraday request

I have the following curl request -
`curl --socks5 #{proxy} --connect-timeout 10 -H "Accept: application/json" #{url}`
I want to write a faraday request instead of a curl call. I am doing this -
faraday = Faraday.new("#{url}", ssl:{verify: false} do |f|
f.proxy "#{proxy}"
end
faraday.get
I am getting a reponse but the response has no body or headers. Following is the response -
#<Faraday::Response:0x0056075d353ad0 #on_complete_callbacks=[], #env=#<Faraday::Env #method=:get #url=#<URI::HTTP:0x0056075d358328 URL:main_url> #request=#<Faraday::RequestOptions proxy=#<Faraday::ProxyOptions uri=#<URI::HTTP:0x0056075dce69d0 URL:proxy_url>>> #request_headers={"User-Agent"=>"Faraday v0.9.2"} #ssl=#<Faraday::SSLOptions (empty)> #response=#<Faraday::Response:0x0056075d353ad0 ...>>>
What am I doing wrong here?
The hardest issue with the conversion to Faraday is that you need to use a SOCKS5 proxy. Faraday does not support SOCKS proxies (there is an open pull-request for this).
The only way around this is to monkey-patch Faraday to use the socksify gem which adds support for SOCKS proxies to Net::HTTP (the default Faraday network adapter). The procedure is nicely described in this gist and I mostly copy-paste a slightly altered version of it here.
Basically you need to follow these steps:
Install the faraday and socksify gems
Monkey-patch Faraday to support SOCKS. Put this code into a Rails initializer. Note that the patch only applies if you don't need to authenticate to the SOCKS proxy (as your curl command suggests). If you need proxy authentication, see the gist for a patch version supporting that. The patch is as follows:
class Faraday::Adapter::NetHttp
def net_http_connection(env)
if proxy = env[:request][:proxy]
if proxy[:socks]
Net::HTTP::SOCKSProxy(proxy[:uri].host, proxy[:uri].port)
else
Net::HTTP::Proxy(proxy[:uri].host, proxy[:uri].port, proxy[:uri].user, proxy[:uri].password)
end
else
Net::HTTP
end.new(env[:url].host, env[:url].port)
end
end
Create the request. I noticed you are probably trying to make a HTTPS request so I took this into account, as well as the timeouts you have in the curl parameters:
PROXY_OPTS = {
uri: URI.parse('https://proxy_url:1080'),
socks: true
}
SSL_OPTS = { verify: false }
connection = Faraday.new(url: "https://example.com",
ssl: SSL_OPTS,
request: { proxy: PROXY_OPTS }) do |faraday|
faraday.options.timeout = 10 # open/read timeout in seconds
faraday.options.open_timeout = 10 # connection open timeout in seconds
faraday.response :logger # log debug info
faraday.adapter :net_http # use the Net:HTTP adapter
faraday.headers['Accept'] = 'application/json' # your custom headers
end
response = connection.get
response.body
Finally, please note that ignoring peer verification (verify: false in the SSL options) is insecure! You should instead properly configure Faraday to use a certificate store to verify peer certificates against. This is fully documented here.
This is how I use faraday for a post request to get an access token from Microsoft Exchange API for example.
url = "https://login.microsoftonline.com/#{ENV['TENANT']}/oauth2/token"
conn = Faraday.new url: url do |faraday|
faraday.request :url_encoded # form-encode POST params
faraday.response :logger # log requests to STDOUT
faraday.adapter Faraday.default_adapter # make requests with Net::HTTP
end
response = conn.post do |req|
req.headers['Content-Type'] = 'application/x-www-form-urlencoded'
req.body = {
client_id: URI::encode(ENV['CLIENT_ID']),
client_secret: URI::encode(ENV['CLIENT_SECRET']),
resource: URI::encode('https://graph.microsoft.com'),
grant_type: URI::encode('client_credentials'),
}
Rails.logger.info "Body #{req.body.inspect}"
end
if response.status.to_i == 200
response_body = ActiveSupport::JSON.decode(response.body)
return response_body['access_token']
else
return false
end
Hope it helps.

Submitting a POST request from a rails controller

I have followed the answer of this question. And this is what i have.
def index
$response = get_insta_feed
end
def get_insta_feed
require "net/http"
require 'net/https'
require "uri"
$tag = "test"
uri = URI.parse("https://api.instagram.com/v1/tags/"+$tag+"/media/recent")
uri.query = URI.encode_www_form(parameters)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(uri.request_uri)
http.request(request).to_json
end
private
def parameters
{
"access_token" => "my access token here"
}
end
on my view page i just want to display the full json response first before parsing the data that i want to display so this:
<div>
<%=$response%>
</div>
and this is what is displayed in my div:
{"server":["nginx"],"date":["Fri, 07 Nov 2014 06:13:34 GMT"],"content-type":["text/html; charset=utf-8"],"allow":["GET"],"content-language":["en"],"expires":["Sat, 01 Jan 2000 00:00:00 GMT"],"vary":["Cookie, Accept-Language"],"pragma":["no-cache"],"cache-control":["private, no-cache, no-store, must-revalidate"],"set-cookie":["csrftoken=e70981e518d478dd7362049f9ce89cc9; expires=Fri, 06-Nov-2015 06:13:34 GMT; Max-Age=31449600; Path=/","ccode=PH; Path=/"],"connection":["close"]}
What am I doing wrong?
Please use instagram gem for the connection tasks to Instagram. For example you can use it as follows:
client = Instagram.client(:access_token => session[:access_token])
for media_item in client.tag_recent_media(tag_name)
# Use the folowing fields...
# media_item.images.thumbnail.url
# media_item.id
# media_item.likes[:count]
end
For more information of the gem, please refer to its github page.

check https status code ruby

Is there a way to check for an HTTPS status code in ruby? I know that there are ways to do this in HTTP using require 'net/http', but I'm looking for HTTPS. Maybe there is a different library that I need to use?
You can do this in net/http:
require "net/https"
require "uri"
uri = URI.parse("https://www.secure.com/")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri)
res = http.request(request)
res.code #=> "200"
Refs:
Net::HTTP cheat sheet
How to Cure Net::HTTP’s Risky Default HTTPS Behavior
You can use any wrapper around Net::HTTP(S) to get much easier behavior.
I use Faraday here ( https://github.com/lostisland/faraday ) but HTTParty has almost the same functionality ( https://github.com/jnunemaker/httparty )
require 'faraday'
res = Faraday.get("https://www.example.com/")
res.status # => 200
res = Faraday.get("http://www.example.com/")
res.status # => 200
(as a bonus you get options for parsing responses, raising state exceptions, logging requests....
connection = Faraday.new("https://www.example.com/") do |conn|
# url-encode the body if given as a hash
conn.request :url_encoded
# add an authorization header
conn.request :oauth2, 'TOKEN'
# use JSON to convert the response into a hash
conn.response :json, :content_type => /\bjson$/
# ...
conn.adapter Faraday.default_adapter
end
connection.get("/")
# GET https://www.example.com/some/path?query=string
connection.get("/some/path", :query => "string")
# POST, PUT, DELETE, PATCH....
connection.post("/some/other/path", :these => "fields", :will => "be converted to a request string in the body"}
# add any number of headers. in this example "Accept-Language: en-US"
connection.get("/some/path", nil, :accept_language => "en-US")
require 'uri'
require 'net/http'
res = Net::HTTP.get_response(URI('http://www.example.com/index.html'))
puts res.code # -> '200'
Slightly more readable way:
response.kind_of?(Net::HTTPOK)

Resources