Microsoft Translator API answers 500 internal server error - ruby-on-rails

I'm trying to use Microsoft's Translator API in my Rails app. Unfortunately and mostly unexpected, the server answers always with an internal server error. I also tried it manually with Poster[1] and I get the same results.
In more detail, what am I doing? I'm creating an XML string which goes into the body of the request. I used the C# Example of the API documentation. Well, and then I'm just invoking the RESTservice.
My code looks like this:
xmlns1 = "http://schemas.datacontract.org/2004/07/Microsoft.MT.Web.Service.V2"
xmlns2 = "http://schemas.microsoft.com/2003/10/Serialization/Arrays"
xml_builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml|
xml.TranslateArrayRequest("xmlns:ms" => xmlns1, "xmlns:arr" => xmlns2) {
xml.AppId token #using temporary token instead of appId
xml.From source
xml.To target
xml.Options {
xml["ms"].ContentType {
xml.text "text/html"
}
}
xml.Texts {
translate.each do |key,val|
xml["arr"].string {
xml.text CGI::unescape(val)
}
end
}
}
end
headers = {
'Content-Type' => 'text/xml'
}
uri = URI.parse(##msTranslatorBase + "/TranslateArray" + "?appId=" + token)
req = Net::HTTP::Post.new(uri.path, headers)
req.body = xml_builder.to_xml
response = Net::HTTP.start(uri.host, uri.port) { |http| http.request(req) }
# [...]
The xml_builder produces something like the following XML. Differently to the example from the API page, I'm defining two namespaces instead of referencing them on the certain tags (mainly because I wanted to reduces the overhead) -- but this doesn't seem to be a problem, when I do it like the docu-example I also get an internal server error.
<?xml version="1.0" encoding="UTF-8"?>
<TranslateArrayRequest xmlns:ms="http://schemas.datacontract.org/2004/07/Microsoft.MT.Web.Service.V2" xmlns:arr="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<AppId>TX83NVx0MmIxxCzHjPwo2_HgYN7lmWIBqyjruYm7YzCpwnkZL5wtS5oucxqlEFKw9</AppId>
<From>de</From>
<To>en</To>
<Options>
<ms:ContentType>text/html</ms:ContentType>
</Options>
<Texts>
<arr:string>Bitte übersetze diesen Text.</arr:string>
<arr:string>Das hier muss auch noch übersetzt werden.</arr:string>
</Texts>
</TranslateArrayRequest>
Every time I request the service it answers with
#<Net::HTTPInternalServerError 500 The server encountered an error processing the request. Please see the server logs for more details.>
... except I do some unspecified things, like using GET instead of POST, then it answers with something like "method not allowed".
I thought it might be something wrong with the XML stuff, because I can request an AppIdToken and invoke the Translate method without problems. But to me, the XML looks just fine. The documentation states that there is a schema for the expected XML:
The request body is a xml string generated according to the schema specified at http:// api.microsofttranslator.com/v2/Http.svc/help
Unfortunately, I cannot find anything on that.
So now my question(s): Am I doing something wrong? Maybe someone experienced similar situations and can report on solutions or work-arounds?
[1] Poster FF plugin > addons.mozilla.org/en-US/firefox/addon/poster/

Well, after lot's of trial-and-error I think I made it. So in case someone has similar problems, here is how I fixed this:
Apparently, the API is kind of fussy with the incoming XML. But since there is no schema (or at least I couldn't find the one specified in the documentation) it's kind of hard to do it the right way: the ordering of the tags is crucial!
<TranslateArrayRequest>
<AppId/>
<From/>
<Options />
<Texts/>
<To/>
</TranslateArrayRequest>
When the XML has this ordering it works. Otherwise you'll only see the useless internal server error response. Furthermore, I read a couple of times that the API also breaks if the XML contains improper UTF-8. One can force untrusted UTF-8 (e.g. coming from a user form) this way:
ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
valid_string = ic.iconv(untrusted_string + ' ')[0..-2]

Related

Quickbooks v3 query url returning Error

First - my question:
When accessing the Quickbooks API, v3 (as has been forced on me as of this weekend by Intuit) I am trying to access Journal Entries (but the following problem persists across any other query) and I'm trying to use the prescribed query?query=SELECT * FROM JournalEntry (what?).
https://qb.sbfinance.intuit.com/v3/company/<id>/query?query=SELECT * FROM JournalEntry
I get as result:
{"Fault":{"Error":[{"Message":"message=Exception authenticating OAuth; errorCode=003200; statusCode=401","code":"3200"}],"type":"AUTHENTICATION"},"requestId":"6f5e5f14af7d4867ad0d8f639ade7d04","time":"2013-11-12T16:10:44.724Z"}
Which, yes, tells me that there was an error with authentication. However, when I access a URL that doesn't include this ridiculous query syntax, everything works fine:
https://qb.sbfinance.intuit.com/v3/company/<id>/journalentry/<id>
I had a similar error when accessing the v2 API, and that was bad formatting on my part, but I don't see what's wrong with my query.
And because my code for generating the authentication tokens etc is identical for both types of request, I doubt that the problem is with how I'm authenticating. Similarly "exception" tells me that there's something going wrong that the API isn't identifying. Probably a formatting of the URL that is going wrong.
I've tried replacing the query URL spaces with both a '+' and a '%20', which returns the same error.
I'm using python and rauth. The code works fine for v2 (but that was deprecated over the weekend without warning, and now is no longer documented).
As a bonus, and because apparently this is Intuit's primary mode of communication with their clients: I'm shocked that Intuit no longer has private support tickets available on their website, and that they rely on a community environment like SO to provide support. The least they could do is provide their own support. Especially if we're paying for use of the API. This is absolutely shocking.
On top of that, the API returns inconsistent responses (the same request will return an error or a valid result, depending on... no change at all). An error I have reported through their support tickets, and they have duly ignored.
Oh, and the documentation says to use
https://quickbooks.api.intuit.com/v3/v3/company/companyID/query?query=selectStmt
while the API Explorer uses:
https://qb.sbfinance.intuit.com/v3/company/<id>/query?query=SELECT * FROM JournalEntry
Anyone know which one I should actually use?
Edit
For the response that is failing, my request headers are:
{
'Content-Length': u'62',
'Accept-Encoding': 'gzip,
deflate,
compress',
'accept': 'application/json',
'User-Agent': 'python-requests/1.2.3CPython/2.7.5Darwin/13.0.0',
'Content-Type': 'application/x-www-form-urlencoded',
'authorization': 'OAuthrealm="<companyId>",
oauth_nonce="3ad98c5f71bc9f102cc31ac9815cb6d08994454e",
oauth_timestamp="1384280420",
oauth_consumer_key="<consumerKey>",
oauth_signature_method="HMAC-SHA1",
oauth_version="1.0",
oauth_token="<oauthToken>",
oauth_signature="<oauthSignature"'
}
My url is:
https://quickbooks.api.intuit.com/v3/company/<id>/query?query=SELECT+*+FROM+JournalEntry&
And my response headers are:
{'content-length': '227', 'server': 'Apache/2.2.22 (Unix)', 'connection': 'close', 'date': 'Tue, 12 Nov 2013 18:20:20 GMT', 'content-type': 'application/json;charset=ISO-8859-1', 'www-authenticate': 'OAuth oauth_problem="signature_invalid"'}
My signature hashing function is correct. It's the standard function used by Rauth, and works fine for more standard API calls (that don't have spaces or SQL select queries in them).
Pass the URL to your HTTP call without encoding:
URL = https://quickbooks.api.intuit.com/v3/company/123456789/query?query="Select * from Customer"
But to build the signature, separate the parameters from the URL, then encode separately, you should get:
"GET" + "&" +
URLEncode(https://quickbooks.api.intuit.com/v3/company/123456789/query) + "&" +
URLEncode(query=Select%20%2A%20from%20Customer), where Select%20%2A%20from%20Customer is the encoding of Select * from Customer
Note the SQL gets encoded a second time, when generating the signature.
Et voila ! I spent a week on this, I know what I'm talking about.
(notations are from VBA language, so replace as appropriate)
It turns out that the actual problem is that the Quickbooks documentation is wrong as of this writing (2013/11/14).
The documentation says that the query URL expects a GET request, which is not the case. This works when submitting SELECT statement as part of the body of a POST request.
See here for more details: https://intuitpartnerplatform.lc.intuit.com/questions/786661-python-script-to-integrate-with-quickbook
I had tried this API call using Java devkit.
JournalEntry je = GenerateQuery.createQueryEntity(JournalEntry.class);
String jeQuery = select($(je)).generate();
System.out.println("Query - " + jeQuery);
QueryResult JournalEntryRes = service.executeQuery(jeQuery);
Request URI : https://quickbooks.api.intuit.com/v3/company/688779980/query?query=SELECT+*+FROM+JournalEntry&
Response XML
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<IntuitResponse xmlns="http://schema.intuit.com/finance/v3" time="2013-11-12T09:50:39.836-08:00">
<QueryResponse startPosition="1" maxResults="1" totalCount="1">
<JournalEntry domain="QBO" sparse="false">
<Id>22734</Id>
<SyncToken>0</SyncToken>
<MetaData>
<CreateTime>2013-10-15T08:42:12-07:00</CreateTime>
<LastUpdatedTime>2013-10-15T08:42:12-07:00</LastUpdatedTime>
</MetaData>
<TxnDate>2013-10-15</TxnDate>
<Line>
<Id>0</Id>
<Amount>100.00</Amount>
<DetailType>JournalEntryLineDetail</DetailType>
<JournalEntryLineDetail>
<PostingType>Debit</PostingType>
<AccountRef name="Advertising">9</AccountRef>
</JournalEntryLineDetail>
</Line>
<Line>
<Id>1</Id>
<Amount>100.00</Amount>
<DetailType>JournalEntryLineDetail</DetailType>
<JournalEntryLineDetail>
<PostingType>Credit</PostingType>
<AccountRef name="Advertising">9</AccountRef>
</JournalEntryLineDetail>
</Line>
<Adjustment>false</Adjustment>
</JournalEntry>
</QueryResponse>
</IntuitResponse>
You can try this call from V3 QBO ApiExplorer as well.
Query - SELECT * FROM JournalEntry
Thanks
you need to encode the query, but not the whole url
https://quickbooks.api.intuit.com/v3/company/123456789/query?query=" & URLEncode("Select * from Customer")
see sample explained here :
https://developer.intuit.com/docs/0100_quickbooks_online/0300_references/0000_programming_guide/0050_data_queries

YouTube API: Delete video HTTP request from Ruby not working

I am trying to delete a video on YouTube from a Ruby on Rails application. I am following these instructions, from the YouTube API docs:
DELETE /feeds/api/users/default/uploads/VIDEO_ID HTTP/1.1
Host: gdata.youtube.com
Content-Type: application/atom+xml
Authorization: Bearer ACCESS_TOKEN
GData-Version: 2
X-GData-Key: key=DEVELOPER_KEY
I am not very familiar with Ruby's Net::HTTP class, but it seems that no matter what I try I cannot get the request to work properly. I have looked carefully at the many other StackOverflow questions regarding deleting videos from YouTube, but none that I could find address this particular problem. My code is below, where I've replaced the user name, video ID, access token, and developer key.
url = URI.parse("https://gdata.youtube.com/feeds/api/users/[USER_NAME]/uploads/[VIDEO_ID]")
post_args = { 'Host' => 'gdata.youtube.com', 'GData-Version' => '2', 'Content-Type' => 'application/atom+xml', 'Authorization' => "Bearer [ACCESS_TOKEN]", 'X-GData-Key' => 'key=[DEVELOPER_KEY]' }
req = Net::HTTP::Delete.new(url.path)
req.set_form_data(post_args)
httpreq = Net::HTTP.new(url.host, url.port)
httpreq.use_ssl = true
resp = httpreq.start {|http| http.request(req) }
Checking the response, I get an Error 400 (Bad Request) from YouTube. The response simply says "Your client has issued a malformed or illegal request. That's all we know".
Is there something wrong with the request I'm making? I've checked it against the template time and time again and I can't see anything wrong with it. I know that my access token and developer key are working because I can make other requests like video uploads just fine.
I printed the debug output from the HTTP request, and as far as I can tell it looks fine:
<- "DELETE /feeds/api/users/[USER_NAME]/uploads/[VIDEO_ID] HTTP/1.1\r\nAccept: */*\r\nUser-Agent: Ruby\r\nContent-Type: application/x-www-form-urlencoded\r\nHost: gdata.youtube.com\r\nContent-Length: 275\r\n\r\n"
<- "Host=gdata.youtube.com&GData-Version=2&Content-Type=application%2Fatom%2Bxml&Authorization=Bearer+[ACCESS_TOKEN]&X-GData-Key=key%3D[DEVELOPER_KEY]"
The only thing I could see as a possible problem was that in the first line of the request, the "Content-Type" is set to "application/x-www-form-urlencoded". Again, not being an expert on HTTP requests I'm not sure what the difference is between the Content-Type set in the first line and the Content-Type that I explicitly set as "application/atom+xml" which appears on the second line of the request. After some digging, though, I found out that the set_form_data method automatically sets the content type as "application/x-www-form-urlencoded", so I tried adding the following line to my code:
req.content_type = 'application/atom+xml'
right after the line
req.set_form_data(post_args)
When I do this, I do see a corresponding change in the request:
<- "DELETE /feeds/api/users/[USER_ID]/uploads/[VIDEO_ID] HTTP/1.1\r\nAccept: */*\r\nUser-Agent: Ruby\r\nContent-Type: application/atom+xml\r\nHost: gdata.youtube.com\r\nContent-Length: 275\r\n\r\n"
<- "Host=gdata.youtube.com&GData-Version=2&Content-Type=application%2Fatom%2Bxml&Authorization=Bearer+[ACCESS_TOKEN]&X-GData-Key=key%3D[DEVELOPER_KEY]"
However, I still get the exact same response from YouTube. Error 400, bad request. What the heck is going on here??
Of course, 10 minutes after asking my question, I find out the answer. I did not understand the distinction between the HTTP header fields and form arguments, which I don't feel so bad about since it's not explained anywhere either in the Ruby documentation on Net::HTTP or in the YouTube API. The reason I was confused was because for uploading a video, you can provide all the values like Authorization and Content-Type as form data, so the above approach from my question works fine. For deleting a video, you have to provide those values as part of the header, not form data. At least, that is now my understanding.
Anyway, in case anyone ever runs into this problem, this solved it for me:
req = Net::HTTP::Delete.new(url.path)
req['GData-Version'] = '2' # this syntax sets header fields & values
req['Authorization'] = "..."
req['X-GData-Key'] = "..."
req.content_type = 'application/atom+xml'
httpreq = Net::HTTP.new(url.host, url.port)
httpreq.use_ssl = true
resp = httpreq.start {|http| http.request(req) }
Another case where one explanatory sentence from the authors of the documentation would have saved two hours of wasted time. If I had a nickel...

Gzip decompress JSON POST body in Rails/Passenger/Nginx

We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.

How do I post XML to RESTFUL Web Service using Net::HTTP::Post?

I am having trouble getting this to work so any help would be appreciated! Basically, the request.body contains valid XML for the Web Service like so:
<somedata>
<name>Test Name 1</name>
<description>Some data for Unit testing</description>
</somedata>
...but the service returns empty XML. Note that the id field is returned suggesting that it does actually hit the database, but the name and description fields are nil:
<somedata>
<id type='integer'>1</id>
<name nil='true'></name>
<description nil='true'></description>
</somedata>
I have manually tested the RESTFUL service using Poster and it works fine.
Here is the code:
url = URI.parse('http://localhost:3000/someservice/')
request = Net::HTTP::Post.new(url.path)
request.body = "<?xml version='1.0' encoding='UTF-8'?><somedata><name>Test Name 1</name><description>Some data for Unit testing</description></somedata>"
response = Net::HTTP.start(url.host, url.port) {|http| http.request(request)}
#Note this test PASSES!
assert_equal '201 Created', response.get_fields('Status')[0]
Does anyone have any clues why the data in the XML post is not persisted?
Without knowing anything about the service, this is just a guess, but… is the service expecting a specific header that Poster is setting and you aren't?
If I were you, I'd use Wireshark, tcpflow or some other sniffer to look at the exact data Poster and your app are sending, and make sure they're identical. I've seen services that were sensitive to the weirdest things, like whitespace or user agent.

What's the best way to use SOAP with Ruby?

A client of mine has asked me to integrate a 3rd party API into their Rails app. The only problem is that the API uses SOAP. Ruby has basically dropped SOAP in favor of REST. They provide a Java adapter that apparently works with the Java-Ruby bridge, but we'd like to keep it all in Ruby, if possible. I looked into soap4r, but it seems to have a slightly bad reputation.
So what's the best way to integrate SOAP calls into a Rails app?
I built Savon to make interacting with SOAP webservices via Ruby as easy as possible.
I'd recommend you check it out.
We used the built in soap/wsdlDriver class, which is actually SOAP4R.
It's dog slow, but really simple. The SOAP4R that you get from gems/etc is just an updated version of the same thing.
Example code:
require 'soap/wsdlDriver'
client = SOAP::WSDLDriverFactory.new( 'http://example.com/service.wsdl' ).create_rpc_driver
result = client.doStuff();
That's about it
We switched from Handsoap to Savon.
Here is a series of blog posts comparing the two client libraries.
I also recommend Savon. I spent too many hours trying to deal with Soap4R, without results. Big lack of functionality, no doc.
Savon is the answer for me.
Try SOAP4R
SOAP4R
Getting Started with SOAP4R
And I just heard about this on the Rails Envy Podcast (ep 31):
WS-Deathstar SOAP walkthrough
Just got my stuff working within 3 hours using Savon.
The Getting Started documentation on Savon's homepage was really easy to follow - and actually matched what I was seeing (not always the case)
Kent Sibilev from Datanoise had also ported the Rails ActionWebService library to Rails 2.1 (and above).
This allows you to expose your own Ruby-based SOAP services.
He even has a scaffold/test mode which allows you to test your services using a browser.
I have used HTTP call like below to call a SOAP method,
require 'net/http'
class MyHelper
def initialize(server, port, username, password)
#server = server
#port = port
#username = username
#password = password
puts "Initialised My Helper using #{#server}:#{#port} username=#{#username}"
end
def post_job(job_name)
puts "Posting job #{job_name} to update order service"
job_xml ="<soapenv:Envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:ns=\"http://test.com/Test/CreateUpdateOrders/1.0\">
<soapenv:Header/>
<soapenv:Body>
<ns:CreateTestUpdateOrdersReq>
<ContractGroup>ITE2</ContractGroup>
<ProductID>topo</ProductID>
<PublicationReference>#{job_name}</PublicationReference>
</ns:CreateTestUpdateOrdersReq>
</soapenv:Body>
</soapenv:Envelope>"
#http = Net::HTTP.new(#server, #port)
puts "server: " + #server + "port : " + #port
request = Net::HTTP::Post.new(('/XISOAPAdapter/MessageServlet?/Test/CreateUpdateOrders/1.0'), initheader = {'Content-Type' => 'text/xml'})
request.basic_auth(#username, #password)
request.body = job_xml
response = #http.request(request)
puts "request was made to server " + #server
validate_response(response, "post_job_to_pega_updateorder job", '200')
end
private
def validate_response(response, operation, required_code)
if response.code != required_code
raise "#{operation} operation failed. Response was [#{response.inspect} #{response.to_hash.inspect} #{response.body}]"
end
end
end
/*
test = MyHelper.new("mysvr.test.test.com","8102","myusername","mypassword")
test.post_job("test_201601281419")
*/
Hope it helps. Cheers.
I have used SOAP in Ruby when i've had to make a fake SOAP server for my acceptance tests. I don't know if this was the best way to approach the problem, but it worked for me.
I have used Sinatra gem (I wrote about creating mocking endpoints with Sinatra here) for server and also Nokogiri for XML stuff (SOAP is working with XML).
So, for the beginning I have create two files (e.g. config.rb and responses.rb) in which I have put the predefined answers that SOAP server will return.
In config.rb I have put the WSDL file, but as a string.
##wsdl = '<wsdl:definitions name="StockQuote"
targetNamespace="http://example.com/stockquote.wsdl"
xmlns:tns="http://example.com/stockquote.wsdl"
xmlns:xsd1="http://example.com/stockquote.xsd"
xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
xmlns="http://schemas.xmlsoap.org/wsdl/">
.......
</wsdl:definitions>'
In responses.rb I have put samples for responses that SOAP server will return for different scenarios.
##login_failure = "<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
<s:Body>
<LoginResponse xmlns="http://tempuri.org/">
<LoginResult xmlns:a="http://schemas.datacontract.org/2004/07/WEBMethodsObjects" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
<a:Error>Invalid username and password</a:Error>
<a:ObjectInformation i:nil="true"/>
<a:Response>false</a:Response>
</LoginResult>
</LoginResponse>
</s:Body>
</s:Envelope>"
So now let me show you how I have actually created the server.
require 'sinatra'
require 'json'
require 'nokogiri'
require_relative 'config/config.rb'
require_relative 'config/responses.rb'
after do
# cors
headers({
"Access-Control-Allow-Origin" => "*",
"Access-Control-Allow-Methods" => "POST",
"Access-Control-Allow-Headers" => "content-type",
})
# json
content_type :json
end
#when accessing the /HaWebMethods route the server will return either the WSDL file, either and XSD (I don't know exactly how to explain this but it is a WSDL dependency)
get "/HAWebMethods/" do
case request.query_string
when 'xsd=xsd0'
status 200
body = ##xsd0
when 'wsdl'
status 200
body = ##wsdl
end
end
post '/HAWebMethods/soap' do
request_payload = request.body.read
request_payload = Nokogiri::XML request_payload
request_payload.remove_namespaces!
if request_payload.css('Body').text != ''
if request_payload.css('Login').text != ''
if request_payload.css('email').text == some username && request_payload.css('password').text == some password
status 200
body = ##login_success
else
status 200
body = ##login_failure
end
end
end
end
I hope you'll find this helpful!
I was having the same issue, switched to Savon and then just tested it on an open WSDL (I used http://www.webservicex.net/geoipservice.asmx?WSDL) and so far so good!
https://github.com/savonrb/savon

Resources