Too many open files - socket(2) with net/http - ruby-on-rails

I have a problem with http requests with net/http...
I writing a ruby script that interacts with the dailymotion api.
This script will upload a video "test.flv".
Basically it consists of four requests.
It works perfectly until step "#Create the video object".
The following error is raised on the last
"response = http.request(req)" command ->
Errno::EMFILE: Too many open files - socket(2)
Here is the code, thx for any advice...
require 'net/http'
require 'curb'
require 'json'
# Authenticate the user
url = URI.parse( 'https://api.dailymotion.com/oauth/token' )
req = Net::HTTP::Post.new(url.path)
req.set_form_data({ 'grant_type' => 'password',
'client_id' => 'my_client_id',
'client_secret' => 'my_client_secret',
'username' => 'myusername',
'password' => 'mypassword'
})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request(req)
access_token = JSON.parse( response.body )['access_token']
access_url = 'https://api.dailymotion.com/file/upload?access_token=' + access_token
# Get an upload URL
url = URI.parse( access_url )
req = Net::HTTP::Get.new( url.request_uri )
http = Net::HTTP.new( url.host, url.port )
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request( req )
upload_url = JSON.parse( response.body )['upload_url']
progress_url = JSON.parse( response.body )['progress_url']
# Post the video
fields_hash = {}
post_data = fields_hash.map { |k, v| Curl::PostField.content(k, v.to_s) }
post_data << Curl::PostField.file('file', 'C:/test.flv')
c = Curl::Easy.new(upload_url)
c.multipart_form_post = true
c.http_post(post_data)
file_url = JSON.parse( c.body_str )['url']
# Create the video object
url = URI.parse( 'https://api.dailymotion.com/me/videos' )
req = Net::HTTP::Post.new(url.path)
req.set_form_data({ 'url' => file_url,
'access_token' => access_token
})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request(req)
puts response.body

Try calling finish
http.finish
after the response = http.request( req ).

Related

FTX.com REST API POST Authentication FAILS with Ruby on Rails and net/https

Hoping for some help as this one has me baffled...
I created a user account and API credentials at FTX.com.
They have an interesting Auth setup which is detailed here: https://docs.ftx.com/?python#authentication
They only provide code examples for python, javascript and c#, but I need to implement the integration on a RoR app.
Here's a link which also provides an example for both GET and POST calls: https://blog.ftx.com/blog/api-authentication/
I'm using:
ruby '3.0.1'
gem 'rails', '~> 6.1.4', '>= 6.1.4.1'
also,
require 'uri'
require 'net/https'
require 'net/http'
require 'json'
I got the authentication working for GET calls as follows:
def get_market
get_market_url = 'https://ftx.com/api/markets/BTC-PERP/orderbook?depth=20'
api_get_call(get_market_url)
end
def api_get_call(url)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}GET/api/markets"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
rsp = http.get(uri, headers)
JSON.parse(rsp.body)
end
This works great and I get the correct response:
=>
{"success"=>true,
"result"=>
{"bids"=>
[[64326.0, 2.0309],
...
[64303.0, 3.1067]],
"asks"=>
[[64327.0, 4.647],
...
[64352.0, 0.01]]}}
However, I can't seem to authenticate correctly for POST calls (even though as far as I can tell I am following the instructions correctly). I use the following:
def create_subaccount
create_subaccount_url = 'https://ftx.com/api/subaccounts'
call_body =
{
"nickname": "sub2",
}.to_json
api_post_call(create_subaccount_url, call_body)
end
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri, headers)
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
Also tried passing headers via request[] directly:
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri)
request['FTX-KEY'] = ENV['FTX_API_KEY']
request['FTX-SIGN'] = signature
request['FTX-TS'] = ts.to_s
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
This is the error response:
=> {"success"=>false, "error"=>"Not logged in: Invalid signature"}
My feeling is the issue is somewhere in adding the body to signature_payload before generating the signature via HMAC here..?:
signature_payload = "#{ts}POST/api/subaccounts#{body}"
Thinking this because, if I leave out #{body} here, like so:
signature_payload = "#{ts}POST/api/subaccounts"
the response is:
=> {"success"=>false, "error"=>"Missing parameter nickname"}
I have tried several iterations of setting up the POST call method using various different net/https examples but have had no luck...
I have also contacted FTX support but have had no response.
Would truly appreciate if anyone has some insight on what I am doing wrong here?
try this headers
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
Here's a working example of a class to retrieve FTX subaccounts. Modify for your own purposes. I use HTTParty.
class Balancer
require 'uri'
require "openssl"
include HTTParty
def get_ftx_subaccounts
method = 'GET'
path = '/subaccounts'
url = "#{ENV['FTX_BASE_URL']}#{path}"
return HTTParty.get(url, headers: headers(method, path, ''))
end
def headers(*args)
{
'FTX-KEY' => ENV['FTX_API_KEY'],
'FTX-SIGN' => signature(*args),
'FTX-TS' => ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
end
def signature(*args)
OpenSSL::HMAC.hexdigest(digest, ENV['FTX_API_SECRET'], signature_payload(*args))
end
def signature_payload(method, path, query)
payload = [ts, method.to_s.upcase, "/api", path].compact
if method==:post
payload << query.to_json
elsif method==:get
payload << ("?" + URI.encode_www_form(query))
end unless query.empty?
payload.join.encode("UTF-8")
end
def ts
#ts ||= (Time.now.to_f * 1000).to_i
end
def digest
#digest ||= OpenSSL::Digest.new('sha256')
end
end

Add a single contact via sendgrid api in Ruby on Rails

I have a Rails 5 app and have a form where I want the user to enter their email address and be added as a contact. THIS QUESTION go me pretty close. The response error I am getting is: {"errors":[{"field":null,"message":"access forbidden"}]}which looks like an authentication issue. Here is my code...
def email_signup
email_address = params[:email_address]
url = URI("https://api.sendgrid.com/v3/contactdb/recipients")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
request["authorization"] = 'Bearer <<my_api_key>>'
request["content-type"] = 'application/json'
request.body = "[{\"email\" : email_address}]"
response = http.request(request)
redirect_to jobs_url, notice: response.read_body
end
What am I missing?
OK...I had two issues that needed to be addressed. First, I had the << in the string portion of my API key, the other was that the request body wasn't formatted quite right. Here is what worked for me:
def email_signup
email_address = params[:email_address]
url = URI("https://api.sendgrid.com/v3/contactdb/recipients")
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new(url)
request["authorization"] = 'Bearer SG.Q9sdfsd9s098sdf89sf809sdf809sd'
request["content-type"] = 'application/json'
request.body = [{"email": email_address}].to_json
response = http.request(request)
redirect_to jobs_url, notice: response.read_body
end

Microsoft Emotion API for Video Operation Result Not Found - Rails

I'm working with the Microsoft Emotion API for processing emotions in video in a Rails app. I was able to make the call to the API to submit an operation, but now I have to query another API to get the status of the operation and once it's done it will provide the emotions data.
My issue is that when I query the results API, the response is that my operation is not found. As in, it doesn't exist.
I first sent the below request through my controller, which worked great:
#static controller
uri = URI('https://api.projectoxford.ai/emotion/v1.0/recognizeinvideo')
uri.query = URI.encode_www_form({})
request = Net::HTTP::Post.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = ENV['MEA_SubscriptionKey1']
request['Content-Type'] = 'application/octet-stream'
request.body = File.read("./public/mark_zuck.mov")
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
# Get operation location and id of operation
operation_location = response["operation-location"]
oid = operation_location.split("/")[6]
The response of this first call is:
"operation-location => https://api.projectoxford.ai/emotion/v1.0/operations/e7ef2ee1-ce75-41e0-bb64-e33ce71b1668"
The protocol is for one to grab the end of the "operation-location" url, which is the operation id, and send it back to the results API url like below:
# parse operation ID from url and add it to results API url
url = 'https://api.projectoxford.ai/emotion/v1.0/operations/' + oid
uri = URI(url)
uri.query = URI.encode_www_form({})
request = Net::HTTP::Get.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = ENV['MEA_SubscriptionKey1']
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
The result I get is:
"{\"error\":{\"code\":\"Unspecified\",\"message\":\"Operation not found.\"}}"
I get the same result when I query the Microsoft online API console with the operation id of an operation created through my app.
Does anyone have any ideas or experience with this? I would greatly appreciate it.
You do not need parse the "oid" out of "operation-location" header, as it is already the URL you should GET the status.
The following code works for me. Use it to see if you still see the issue.
require 'net/http'
require 'uri'
uri = URI('https://api.projectoxford.ai/emotion/v1.0/recognizeinvideo')
uri.query = URI.encode_www_form({})
request = Net::HTTP::Post.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = '<your key>'
request['Content-Type'] = 'application/octet-stream'
videoFile = File.open("c:\\1mb.mp4", "rb")
request.body = videoFile.read
videoFile.close
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
puts response.message
puts response.read_body
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
# Get operation location url for subsequent calls
operation_location = response["operation-location"]
operation_url = operation_location
uri = URI(operation_url)
uri.query = URI.encode_www_form({})
loop do
request = Net::HTTP::Get.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = '<your key>'
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
puts response.read_body
response_msg = response.read_body
break if response_msg.include?("Succeeded") or response_msg.include?("Failed")
sleep 20
end
puts response.message
puts response.read_body

Net::HTTP::Post.new request returns empty body in Ruby 2

In Ruby 2.0.0p195, Rails 4.0.0, Net::HTTP::Post.new request returns empty body of response.
#toSend = {
"zuppler_store_id" => 'X3r82l89',
"user_id" => '1'
}.to_json
uri = URI("http://smoothpay.com/zuppler/gen_token_post.php")
http = Net::HTTP.new(uri.host,uri.port)
req = Net::HTTP::Post.new uri
req.content_type = "application/json"
req.body = #toSend # or "[ #{#toSend} ]" ?
res = Net::HTTP.start(uri.host, uri.port) {|http| http.request(req)}
puts "Response #{res.code} - #{res.message}: #{res.body}"
This code returns "Response 200 - OK:"
But it should return like this: {"result":"success","token":"843e5be88fb8cee7d324244929177b4e"}
You can check it by typing this url:
http://smoothpay.com/zuppler/gen_token_test.php
Why is res.body empty?
Seems like that service doesn't like the POST request to be application/json.
This works:
uri = URI("http://smoothpay.com/zuppler/gen_token_post.php")
http = Net::HTTP.new(uri.host,uri.port)
req = Net::HTTP::Post.new uri
req.body = "zuppler_store_id=X3r82l89&user_id=1"
res = Net::HTTP.start(uri.host, uri.port) {|http| http.request(req)}
res.body # => "{\"result\":\"success\",\"token\":\"9502e49d454ab7b7dd2699a26f742cda\"}"
In other words, give the service application/x-www-form-urlencoded. Peculiarly, it will hand you back text/html which you'll have to JSON.parse. Weird service.

EOFError (end of file reached) in Ruby on Rails with http.request

I am trying to get json form url :
uri = URI.parse("http://84.38.185.251:9262/send")
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
response.code # => 301
response.body # => The body (HTML, XML, blob, whatever)
response["cache-control"] # => public, max-age=2592000
puts response.body
but i get an error :`EOFError (end of file reached):
app/controllers/sensors_controller.rb:35:in sensinfo'
sensors_controller.rb:35:
response = http.request(request)
What am i did wrong?
this error mostly get for using https
If it is https then
Please try this one
uri = URI.parse("https://84.38.185.251:9262/send")
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
http.use_ssl = true
response = http.request(request)
Note aditional
http.use_ssl = true
If it is not https
http.use_ssl = false
or you can add the condition
http.use_ssl = true if domain =~ /^https/
you can get more on this https://web.archive.org/web/20140226183826/http://expressica.com/2012/02/10/eoferror-end-of-file-reached-issue-when-post-a-form-with-nethttp/
I think it is a some sort of bug; typhoeus seems to work:
require 'typhoeus'
response = Typhoeus.get("http://84.38.185.251:9262/send")
p response.body
#=> {"ids":"-1","data":{"temp":"nan","h":"-1"},"status":"255","voltage":"-1"}

Resources