Rubyists,
something's wrong with my HTTP AUTH params that are coming into my Rails 3 app. The password has some whitespace at the end. I was debugging my client app and it looks like it is sending it correctly.
I am doing this in my app:
params[:auth_username], params[:auth_password] = user_name_and_password(request)
Then I am sending this into Warden.
I would like to see the raw data to see if the whitespace is there. How to do that?
Edit: I have debugged the wire between httpd and thin process and I am pretty sure the data are coming correctly. It must be something wrong in my Rails 3.0.10. I was able to decode the base64 string that is coming in the headers and it did not contain any whitespace.
This really looks like BASE64 decoder issue. Maybe a padding problem. My string is:
Qmxvb21iZXJnOnRjbG1lU1JT
which decodes to
Bloomberg:tclmeSRS
correctly using non-Ruby base64 decoders. Even in Ruby:
>> Base64.decode64 "Qmxvb21iZXJnOnRjbG1lU1JT"
=> "Bloomberg:tclmeSRS"
I don't get it. Searching for a bugreport in Rails or something like that.
Edit: So it turns out our Apache httpd proxy adds something to the header:
Authorization: Basic Qmxvb21iZXJnOnRjbG1lU1JT, Basic
This leads to the incorrect characters at the end of the password, because:
>> Base64.decode64('Basic Qmxvb21iZXJnOnRjbG1lU1JT, Basic'.split(' ', 2).last || '')
=> "Bloomberg:tclmeSRS\005\253\""
The question is now - is this correct? Is it a bug in httpd or rails?
Rails user_name_and_password method makes a call to decode_credentials that performs the following, then splits using ":" :
::Base64.decode64(request.authorization.split(' ', 2).last || '')
Applied to your data :
::Base64.decode64("Qmxvb21iZXJnOnRjbG1lU1JT".split(' ', 2).last || '').split(/:/, 2)
=> ["Bloomberg", "tclmeSRS"]
Everything seems to be ok, the problem sits elsewhere IMO. To dump the authorization data from your controller :
render :text => "Authorization: #{request.authorization}"
Related
I'm trying to follow this solution to add a params parser to my rails app, but all that happens is that I now get the headers but no parameters from the body of the JSON request at all. In other words, calling params from within the controller returns this:
{"controller"=>"residences", "action"=>"create",
"user_email"=>"wjdhamilton#wibble.com",
"user_token"=>"ayAJ8kDUKjCiy1r1Mxzp"}
but I expect this as well:
{"data"=>{"type"=>"residences",
"attributes"=>{"name-number"=>"The Byre",
"street"=>"Next Door",
"town"=>"Just Dulnain Bridge",
"postcode"=>"PH1 3SY",
"country-code"=>""},
"relationships"=>{"residence-histories"=>{"data"=>nil},
"occupants"=>{"data"=>nil}}}}
Here is my initializer, which as you can see is almost identical to the one in the other post:
Rails.application.config.middleware.swap(
::ActionDispatch::ParamsParser, ::ActionDispatch::ParamsParser,
::Mime::Type.lookup("application/vnd.api+json") => Proc.new { |raw_post|
# Borrowed from action_dispatch/middleware/params_parser.rb except for
# data.deep_transform_keys!(&:underscore) :
data = ::ActiveSupport::JSON.decode(raw_post)
data = {:_json => data} unless data.is_a?(::Hash)
data = ::ActionDispatch::Request::Utils.deep_munge(data)
# Transform dash-case param keys to snake_case:
data = data.deep_transform_keys(&:underscore)
data.with_indifferent_access
}
)
Can anyone tell me where I'm going wrong? I'm running Rails 4.2.7.1
Update 1: I decided to try and use the Rails 5 solution instead, the upgrade was overdue anyway, and now things have changed slightly. Given the following request:
"user_email=mogwai%40balnaan.com
&user_token=_1o3Kpzo4gTdPC2bivy
&format=json
&data[type]=messages&data[attributes][sent-on]=2014-01-15
&data[attributes][details]=Beautiful+Shetland+Pony
&data[attributes][message-type]=card
&data[relationships][occasion][data][type]=occasions
&data[relationships][occasion][data][id]=5743
&data[relationships][person][data][type]=people
&data[relationships][person][data][id]=66475"
the ParamsParser middleware only receives the following hash:
"{user":{"email":"mogwai#balnaan.com","password":"0h!Mr5M0g5"}}
Whereas I would expect it to receive the following:
{"user_email"=>"mogwai#balnaan.com", "user_token"=>"_1o3Kpzo4gTdPC2b-ivy", "format"=>"5743", "data"=>{"type"=>"messages", "attributes"=>{"sent-on"=>"2014-01-15", "details"=>"Beautiful Shetland Pony", "message-type"=>"card"}, "relationships"=>{"occasion"=>{"data"=> "type"=>"occasions", "id"=>"5743"}}, "person"=>{"data"=>{"type"=>"people", "id"=>"66475"}}}}, "controller"=>"messages", "action"=>"create"}
The problem was caused by the tests that I had written. I had not added the Content-Type to the requests in the tests, and had not explicitly converted the payload to JSON like so (in Rails 5):
post thing_path, params: my_data.to_json, headers: { "Content-Type" => "application/vnd.api+json }
The effects of this were twofold: Firstly, since params parsers are mapped to specific media types then withholding the media type meant that rails assumed its default media type (in this case application/json) so the parser was not used to process the body of the request. What confused me was that it still passed the headers to the parser. Once I fixed that problem, I was then faced with the body in the format of the request above. That is where the explicit conversion to JSON is required. I could have avoided all of this if I had just written accurate tests!
We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.
I have the following code:
http = Net::HTTP.new("www.something.com", 80)
http.set_debug_output($stdout)
http.post("/blah", "something", {'random-parameter' => 'value1="something",value2="somethingelse"'})
Then when I read the output from stdout, it looks like this:
<- "POST /blah HTTP/1.1\r\nAccept: */*\r\nContent-Type: application/x-www-form-urlencoded\r\nConnection: close\r\nrandom-parameter: value1=\"something\",value2=\"somethingelse\"\r\nContent-Length: 9\r\nHost: www.something.com\r\n\r\n"
<- "something"
with the quotes being escaped. The problem is that the slashes seem to be sent to the server, which doesn't like that. I'm getting an error that says
Unknown value for random-parameter in header: {value1=\\\"something\\\"value2=\\\"somethingelse\\\"}
So my question is, is there a way to tell Net::HTTP to not insert those slashes, or strip them out before sending the header?
Clarifications:
I'm using Ruby 1.8.7 with Rails 2.0.2.
I think it may be Rails that is escaping the characters, but I'm not sure how to make it stop.
Are you sure you're building up the header correctly? Net::HTTP does not quote quotes when sending the request. You can easily verify it by using for example netcat (nc):
Terminal 1:
> nc -v -l -p 2323
Terminal 2 (in irb):
> http = Net::HTTP.new("localhost", 2323)
> http.post("/blah", "something", {'random-parameter' => ... )
Result (in terminal 1):
listening on [any] 2323 ...
connect to [127.0.0.1] from localhost [127.0.0.1] 37598
POST /blah HTTP/1.1
Connection: close
Accept: */*
Random-Parameter: value1="something",value2="somethingelse"
Content-Type: application/x-www-form-urlencoded
Content-Length: 9
Host: localhost:2323
something
I think what you actually may want to do (not sure, but I'm guessing) is something more along the lines of the HTTP spec:
> http.post("/blah", "something", {
'random-parameter' => 'value1="something"; value2="somethingelse"' })
Rails is probably interpreting your first value1=... as the whole value..when you probably need to separate the values with ';', not ','.
Also note that you don't usually pass parameters through request headers. But maybe that's what you want to do in this case(?) Otherwise you should either pass the parameters in the URL itself like param1=foo¶m2=bar, or use x-www-form-urlencoded to pass the parameters.
See here for a cheat sheet:
http://www.rubyinside.com/nethttp-cheat-sheet-2940.html
Isn't the problem that your name/value pairs are the value for "random-parameter", hence they need to be escaped.
I'd expect that when you inspect the random-parameter value in the controller method that it would not have the slashes - can you debug the code or print the parameter to see whats being received?
I'm trying to use Microsoft's Translator API in my Rails app. Unfortunately and mostly unexpected, the server answers always with an internal server error. I also tried it manually with Poster[1] and I get the same results.
In more detail, what am I doing? I'm creating an XML string which goes into the body of the request. I used the C# Example of the API documentation. Well, and then I'm just invoking the RESTservice.
My code looks like this:
xmlns1 = "http://schemas.datacontract.org/2004/07/Microsoft.MT.Web.Service.V2"
xmlns2 = "http://schemas.microsoft.com/2003/10/Serialization/Arrays"
xml_builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml|
xml.TranslateArrayRequest("xmlns:ms" => xmlns1, "xmlns:arr" => xmlns2) {
xml.AppId token #using temporary token instead of appId
xml.From source
xml.To target
xml.Options {
xml["ms"].ContentType {
xml.text "text/html"
}
}
xml.Texts {
translate.each do |key,val|
xml["arr"].string {
xml.text CGI::unescape(val)
}
end
}
}
end
headers = {
'Content-Type' => 'text/xml'
}
uri = URI.parse(##msTranslatorBase + "/TranslateArray" + "?appId=" + token)
req = Net::HTTP::Post.new(uri.path, headers)
req.body = xml_builder.to_xml
response = Net::HTTP.start(uri.host, uri.port) { |http| http.request(req) }
# [...]
The xml_builder produces something like the following XML. Differently to the example from the API page, I'm defining two namespaces instead of referencing them on the certain tags (mainly because I wanted to reduces the overhead) -- but this doesn't seem to be a problem, when I do it like the docu-example I also get an internal server error.
<?xml version="1.0" encoding="UTF-8"?>
<TranslateArrayRequest xmlns:ms="http://schemas.datacontract.org/2004/07/Microsoft.MT.Web.Service.V2" xmlns:arr="http://schemas.microsoft.com/2003/10/Serialization/Arrays">
<AppId>TX83NVx0MmIxxCzHjPwo2_HgYN7lmWIBqyjruYm7YzCpwnkZL5wtS5oucxqlEFKw9</AppId>
<From>de</From>
<To>en</To>
<Options>
<ms:ContentType>text/html</ms:ContentType>
</Options>
<Texts>
<arr:string>Bitte übersetze diesen Text.</arr:string>
<arr:string>Das hier muss auch noch übersetzt werden.</arr:string>
</Texts>
</TranslateArrayRequest>
Every time I request the service it answers with
#<Net::HTTPInternalServerError 500 The server encountered an error processing the request. Please see the server logs for more details.>
... except I do some unspecified things, like using GET instead of POST, then it answers with something like "method not allowed".
I thought it might be something wrong with the XML stuff, because I can request an AppIdToken and invoke the Translate method without problems. But to me, the XML looks just fine. The documentation states that there is a schema for the expected XML:
The request body is a xml string generated according to the schema specified at http:// api.microsofttranslator.com/v2/Http.svc/help
Unfortunately, I cannot find anything on that.
So now my question(s): Am I doing something wrong? Maybe someone experienced similar situations and can report on solutions or work-arounds?
[1] Poster FF plugin > addons.mozilla.org/en-US/firefox/addon/poster/
Well, after lot's of trial-and-error I think I made it. So in case someone has similar problems, here is how I fixed this:
Apparently, the API is kind of fussy with the incoming XML. But since there is no schema (or at least I couldn't find the one specified in the documentation) it's kind of hard to do it the right way: the ordering of the tags is crucial!
<TranslateArrayRequest>
<AppId/>
<From/>
<Options />
<Texts/>
<To/>
</TranslateArrayRequest>
When the XML has this ordering it works. Otherwise you'll only see the useless internal server error response. Furthermore, I read a couple of times that the API also breaks if the XML contains improper UTF-8. One can force untrusted UTF-8 (e.g. coming from a user form) this way:
ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
valid_string = ic.iconv(untrusted_string + ' ')[0..-2]
I am using Ruby on Rails 3 and I would like to solve an issue with the following code where a web client application receive back some JSON data from a web service application that uses a Rack middleware in order to respond.
In the web client app model I have
response_parsed = JSON.parse(response.body)
if response_parsed["account"]
...
else
return response
end
In the above code the response.body come back from the web service app that uses a Rack middleware to respond to the web client:
accounts = Account.where(:id => ids)
[200, {'Content-Type' => 'application/json'}, accounts.to_json] # That is, response.body = accounts.to_json
Data transmission is ok, but I get the following error
TypeError
can't convert String into Integer
*Application Trace*
lib/accounts.rb:107:in `[]'
The line 107 corresponds to
if response_parsed["account"]
...
Where and what is the problem? How to solve that?
If I try to debug the respons.body I get
# Note: this is an array!
"[{\"account\":{\"firstname\":\"Semio\",\"lastname\":\"Iaven\"\"}}]"
If I'm saying something you already realize, forgive me.
It looks like your response is a one-element array with a hash in it as the first element. Because the response is an array, when you use the [] it is expecting a integer representing the index of the item in the array you'd like to access, and that is what the error message means--it expected that you'd tell it the integer value of the item you wanted, but instead you gave it a string.
If you instead do:
response_parsed[0]['account']
It seems like you'd get what you want.