Microsoft Emotion API for Video Operation Result Not Found - Rails - ruby-on-rails

I'm working with the Microsoft Emotion API for processing emotions in video in a Rails app. I was able to make the call to the API to submit an operation, but now I have to query another API to get the status of the operation and once it's done it will provide the emotions data.
My issue is that when I query the results API, the response is that my operation is not found. As in, it doesn't exist.
I first sent the below request through my controller, which worked great:
#static controller
uri = URI('https://api.projectoxford.ai/emotion/v1.0/recognizeinvideo')
uri.query = URI.encode_www_form({})
request = Net::HTTP::Post.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = ENV['MEA_SubscriptionKey1']
request['Content-Type'] = 'application/octet-stream'
request.body = File.read("./public/mark_zuck.mov")
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
# Get operation location and id of operation
operation_location = response["operation-location"]
oid = operation_location.split("/")[6]
The response of this first call is:
"operation-location => https://api.projectoxford.ai/emotion/v1.0/operations/e7ef2ee1-ce75-41e0-bb64-e33ce71b1668"
The protocol is for one to grab the end of the "operation-location" url, which is the operation id, and send it back to the results API url like below:
# parse operation ID from url and add it to results API url
url = 'https://api.projectoxford.ai/emotion/v1.0/operations/' + oid
uri = URI(url)
uri.query = URI.encode_www_form({})
request = Net::HTTP::Get.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = ENV['MEA_SubscriptionKey1']
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
The result I get is:
"{\"error\":{\"code\":\"Unspecified\",\"message\":\"Operation not found.\"}}"
I get the same result when I query the Microsoft online API console with the operation id of an operation created through my app.
Does anyone have any ideas or experience with this? I would greatly appreciate it.

You do not need parse the "oid" out of "operation-location" header, as it is already the URL you should GET the status.
The following code works for me. Use it to see if you still see the issue.
require 'net/http'
require 'uri'
uri = URI('https://api.projectoxford.ai/emotion/v1.0/recognizeinvideo')
uri.query = URI.encode_www_form({})
request = Net::HTTP::Post.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = '<your key>'
request['Content-Type'] = 'application/octet-stream'
videoFile = File.open("c:\\1mb.mp4", "rb")
request.body = videoFile.read
videoFile.close
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
puts response.message
puts response.read_body
# Get response headers
response.each_header do |key, value|
p "#{key} => #{value}"
end
# Get operation location url for subsequent calls
operation_location = response["operation-location"]
operation_url = operation_location
uri = URI(operation_url)
uri.query = URI.encode_www_form({})
loop do
request = Net::HTTP::Get.new(uri.request_uri)
request['Ocp-Apim-Subscription-Key'] = '<your key>'
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
puts response.read_body
response_msg = response.read_body
break if response_msg.include?("Succeeded") or response_msg.include?("Failed")
sleep 20
end
puts response.message
puts response.read_body

Related

FTX.com REST API POST Authentication FAILS with Ruby on Rails and net/https

Hoping for some help as this one has me baffled...
I created a user account and API credentials at FTX.com.
They have an interesting Auth setup which is detailed here: https://docs.ftx.com/?python#authentication
They only provide code examples for python, javascript and c#, but I need to implement the integration on a RoR app.
Here's a link which also provides an example for both GET and POST calls: https://blog.ftx.com/blog/api-authentication/
I'm using:
ruby '3.0.1'
gem 'rails', '~> 6.1.4', '>= 6.1.4.1'
also,
require 'uri'
require 'net/https'
require 'net/http'
require 'json'
I got the authentication working for GET calls as follows:
def get_market
get_market_url = 'https://ftx.com/api/markets/BTC-PERP/orderbook?depth=20'
api_get_call(get_market_url)
end
def api_get_call(url)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}GET/api/markets"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
rsp = http.get(uri, headers)
JSON.parse(rsp.body)
end
This works great and I get the correct response:
=>
{"success"=>true,
"result"=>
{"bids"=>
[[64326.0, 2.0309],
...
[64303.0, 3.1067]],
"asks"=>
[[64327.0, 4.647],
...
[64352.0, 0.01]]}}
However, I can't seem to authenticate correctly for POST calls (even though as far as I can tell I am following the instructions correctly). I use the following:
def create_subaccount
create_subaccount_url = 'https://ftx.com/api/subaccounts'
call_body =
{
"nickname": "sub2",
}.to_json
api_post_call(create_subaccount_url, call_body)
end
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s
}
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri, headers)
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
Also tried passing headers via request[] directly:
def api_post_call(url, body)
ts = (Time.now.to_f * 1000).to_i
signature_payload = "#{ts}POST/api/subaccounts#{body}"
key = ENV['FTX_API_SECRET']
data = signature_payload
digest = OpenSSL::Digest.new('sha256')
signature = OpenSSL::HMAC.hexdigest(digest, key, data)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.read_timeout = 1200
http.use_ssl = true
request = Net::HTTP::Post.new(uri)
request['FTX-KEY'] = ENV['FTX_API_KEY']
request['FTX-SIGN'] = signature
request['FTX-TS'] = ts.to_s
request.body = body
response = http.request(request)
JSON.parse(response.body)
end
This is the error response:
=> {"success"=>false, "error"=>"Not logged in: Invalid signature"}
My feeling is the issue is somewhere in adding the body to signature_payload before generating the signature via HMAC here..?:
signature_payload = "#{ts}POST/api/subaccounts#{body}"
Thinking this because, if I leave out #{body} here, like so:
signature_payload = "#{ts}POST/api/subaccounts"
the response is:
=> {"success"=>false, "error"=>"Missing parameter nickname"}
I have tried several iterations of setting up the POST call method using various different net/https examples but have had no luck...
I have also contacted FTX support but have had no response.
Would truly appreciate if anyone has some insight on what I am doing wrong here?
try this headers
headers = {
'FTX-KEY': ENV['FTX_API_KEY'],
'FTX-SIGN': signature,
'FTX-TS': ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
Here's a working example of a class to retrieve FTX subaccounts. Modify for your own purposes. I use HTTParty.
class Balancer
require 'uri'
require "openssl"
include HTTParty
def get_ftx_subaccounts
method = 'GET'
path = '/subaccounts'
url = "#{ENV['FTX_BASE_URL']}#{path}"
return HTTParty.get(url, headers: headers(method, path, ''))
end
def headers(*args)
{
'FTX-KEY' => ENV['FTX_API_KEY'],
'FTX-SIGN' => signature(*args),
'FTX-TS' => ts.to_s,
'Content-Type' => 'application/json',
'Accepts' => 'application/json',
}
end
def signature(*args)
OpenSSL::HMAC.hexdigest(digest, ENV['FTX_API_SECRET'], signature_payload(*args))
end
def signature_payload(method, path, query)
payload = [ts, method.to_s.upcase, "/api", path].compact
if method==:post
payload << query.to_json
elsif method==:get
payload << ("?" + URI.encode_www_form(query))
end unless query.empty?
payload.join.encode("UTF-8")
end
def ts
#ts ||= (Time.now.to_f * 1000).to_i
end
def digest
#digest ||= OpenSSL::Digest.new('sha256')
end
end

Parsing a API source, but the JSON souse is not updating (Cache?)

I have coded a rake file to monitor and fetch data from a website that has this data in JSON format. The following is the actual source of this data
https://www.thegazette.co.uk/company/07877158/filings/data.json
The rake file monitors the "total_count" in the above json and when it changes the rake will fetch and save any new information
The issue I have is after the first time it monitors that page, it simply doesn't update. As a real-world current example, the above json source was updated overnight with two new records, and consequently, the "total_count" increased from 40 to 42, but my rake is still telling me there is 40 (and subsequently doing nothing because it thinks nothing has changed)
I think it is a cache issue but have cleared my rails cache with no success. It is strange because I don't have this issue with other similar rakes I have created for other sites
My rake code is as follows
desc "Monitor"
task :S_01 => :environment do
require 'rubygems'
require 'open-uri'
require 'openssl'
def g_api(url)
uri = URI.parse(url)
request = Net::HTTP::Get.new(uri)
request.content_type = "application/json"
req_options = {
use_ssl: uri.scheme == "https",
}
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
data = JSON.parse(response.body)
end
company = CompanyBorrower.where(id: 43)
company.each do |f|
begin
#scrape source
tg_fh_url = "https://www.thegazette.co.uk/company/"+f.ch+"/filings/data.json"
gf_scrape = g_api(tg_fh_url)
ch_s = gf_scrape.fetch('total_count', nil) #scrape
puts ch_s
if not f.filing_count == ch_s # has teh cound change - if not, skip
f.update_attributes(cwdetail1: ch_s, filing_update: ch_fh3)
gf_scrape['items'].first(3).each_with_index do |f1, index|
#fetch & save data here
end
end
rescue
next
end
end
end
EDIT
Added the following to the code, but get an error
response["Cache-Control: no-cache"]
NoMethodError: undefined method `fetch' for nil:NilClass
def g_api(url)
uri = URI.parse(url)
request = Net::HTTP::Get.new(uri)
request.content_type = "application/json"
req_options = {
use_ssl: uri.scheme == "https",
}
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
data = JSON.parse(response.body)
response["Cache-Control: no-cache"]
end

use_ssl value changed, but session already started when trying to make https request

I’m using Rails 4.2.7 and this code for making a Net::Http get request
req = Net::HTTP::Get.new(url)
if !headers.nil?
headers.each do |k, v|
req[k] = v
end
end
res = Net::HTTP.new(uri.host, uri.port).start do |http|
http.use_ssl = (uri.scheme == "https")
http.request(req)
end
status = res.code
content_type = res['content-type']
content_encoding = res['content-encoding']
content = res.body
However, when I make one in which the scheme is “https”, I get the following error
Error during processing: use_ssl value changed, but session already started
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:758:in `use_ssl='
/Users/davea/Documents/workspace/myproject/app/helpers/webpage_helper.rb:118:in `block in get_content'
/Users/davea/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/net/http.rb:853:in `start'
How do I set https while still being able to make my GET request?
According to docs, use_ssl
must be set before starting session.
This is my usual flow:
uri = URI 'some endpoint with encoded params'
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
headers = headers.each_with_object({}) { |(k, v), hash| hash[k] = v }
http.get(uri.request_uri, initheader = headers)
See the docs on get.
Sidenote on your
if !headers.nil?
It would be more readable if you just check for presence:
if headers.present?
Or even shorter:
if headers # would return true unless it's nil or false

Net::HTTP::Post.new request returns empty body in Ruby 2

In Ruby 2.0.0p195, Rails 4.0.0, Net::HTTP::Post.new request returns empty body of response.
#toSend = {
"zuppler_store_id" => 'X3r82l89',
"user_id" => '1'
}.to_json
uri = URI("http://smoothpay.com/zuppler/gen_token_post.php")
http = Net::HTTP.new(uri.host,uri.port)
req = Net::HTTP::Post.new uri
req.content_type = "application/json"
req.body = #toSend # or "[ #{#toSend} ]" ?
res = Net::HTTP.start(uri.host, uri.port) {|http| http.request(req)}
puts "Response #{res.code} - #{res.message}: #{res.body}"
This code returns "Response 200 - OK:"
But it should return like this: {"result":"success","token":"843e5be88fb8cee7d324244929177b4e"}
You can check it by typing this url:
http://smoothpay.com/zuppler/gen_token_test.php
Why is res.body empty?
Seems like that service doesn't like the POST request to be application/json.
This works:
uri = URI("http://smoothpay.com/zuppler/gen_token_post.php")
http = Net::HTTP.new(uri.host,uri.port)
req = Net::HTTP::Post.new uri
req.body = "zuppler_store_id=X3r82l89&user_id=1"
res = Net::HTTP.start(uri.host, uri.port) {|http| http.request(req)}
res.body # => "{\"result\":\"success\",\"token\":\"9502e49d454ab7b7dd2699a26f742cda\"}"
In other words, give the service application/x-www-form-urlencoded. Peculiarly, it will hand you back text/html which you'll have to JSON.parse. Weird service.

Too many open files - socket(2) with net/http

I have a problem with http requests with net/http...
I writing a ruby script that interacts with the dailymotion api.
This script will upload a video "test.flv".
Basically it consists of four requests.
It works perfectly until step "#Create the video object".
The following error is raised on the last
"response = http.request(req)" command ->
Errno::EMFILE: Too many open files - socket(2)
Here is the code, thx for any advice...
require 'net/http'
require 'curb'
require 'json'
# Authenticate the user
url = URI.parse( 'https://api.dailymotion.com/oauth/token' )
req = Net::HTTP::Post.new(url.path)
req.set_form_data({ 'grant_type' => 'password',
'client_id' => 'my_client_id',
'client_secret' => 'my_client_secret',
'username' => 'myusername',
'password' => 'mypassword'
})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request(req)
access_token = JSON.parse( response.body )['access_token']
access_url = 'https://api.dailymotion.com/file/upload?access_token=' + access_token
# Get an upload URL
url = URI.parse( access_url )
req = Net::HTTP::Get.new( url.request_uri )
http = Net::HTTP.new( url.host, url.port )
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request( req )
upload_url = JSON.parse( response.body )['upload_url']
progress_url = JSON.parse( response.body )['progress_url']
# Post the video
fields_hash = {}
post_data = fields_hash.map { |k, v| Curl::PostField.content(k, v.to_s) }
post_data << Curl::PostField.file('file', 'C:/test.flv')
c = Curl::Easy.new(upload_url)
c.multipart_form_post = true
c.http_post(post_data)
file_url = JSON.parse( c.body_str )['url']
# Create the video object
url = URI.parse( 'https://api.dailymotion.com/me/videos' )
req = Net::HTTP::Post.new(url.path)
req.set_form_data({ 'url' => file_url,
'access_token' => access_token
})
http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request(req)
puts response.body
Try calling finish
http.finish
after the response = http.request( req ).

Resources