I am creating a ruby on rails application using HTTParty. I create a request using this method
query = { :flyFrom => "london_gb",:dateFrom => "02%2F11%2F2015"}
response = HTTParty.get('https://api.blahblah.com/flights', :query => query)
However HTTParty seems to encode % signs as %25. So the request looks like this:
https://api.blahblah.com/flights?flyfrom=london&dateFrom==02%252F11%252F2015
As you can see the difference is:
02%2F11%2F2015
02%252F11%252F2015
My API expects the dates to be in a particular format, so is complaining about this. Does anyone know how you can get HTTParty to encode these characters correctly so that it sends just a % rather than %25?
Thanks.
Adjusted answer based on the additional comments.
"%25" is the correct URL encoding for "%". So HTTParty is encoding the percent.
However, "%2F" is the URL encoding for "/". So if you intend to send "02%2F11%2F2015" over the wire then you probably want to pass the unencoded version to HTTParty: "02/11/2015".
Related
I'm updating my rest-client gem from 1.8 to 2.0.
On 1.8 it sends an array of params on a get request as my-url?ids=1,2,3,4.
But on 2.0 it uses duplicated keys like my-url?ids=1&ids=2&ids=3. For reasons beyond the context of this question, our back end does NOT support the new multiple keys syntax (ok, it supports it, but we'll have to make a big refactor). So I'd like to know if there's a way to use the 2.0 client version and keep sending get array requests with only one key and separated by comma as before?
Based on the rest-client docs https://github.com/rest-client/rest-client#query-parameters it seems your only option would be to serialize the parameters yourself and add them to the URL as the query string.
If you don't like this behavior and want more control, just serialize params yourself (e.g. with URI.encode_www_form) and add the query string to the URL directly for GET parameters or pass the payload as a string for POST requests.
If you provide some sample code on how you're using the gem, we could help out a bit better with sample answers.
Ok yeah Leo Correa was right, so what I had to do is replace my old code
params = {
partner_key: #partner,
resources: ["front_end_config", "gui_settings"]
}
#response = JSON.parse( RestClient.get( "#{api_base_uri}/partner_config.json", params: params.merge({multipart:true}) ) )
with this new one, serializing and encoding by myself...
params = {
partner_key: #partner,
resources: '["front_end_config", "gui_settings"]'
}
params = URI.encode_www_form(params.merge({multipart:true}))
#response = JSON.parse( RestClient.get( "#{api_base_uri}/partner_config.json?#{params}" ) )
It's ugly as hell, but it worked for me. If there's some other idea on how to make it better, I'd appreciate.
So, Rails normally handles parsing of incoming Arrays sent via HTTP Post requests (forms), like this:
"Normally Rails ignores duplicate parameter names. If the parameter
name contains an empty set of square brackets [] then they will be
accumulated in an array." - Rails Guides
But when using Net::HTTP.Post to send a Post request to a third party service (API), then it seems this convention of handling arrays in HTTP Post requests is not followed.
This code:
data = {:categories => [one, two, three]}
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Post.new(uri.request_uri)
request.set_form_data(data)
response = http.request(request)
Then set_form_data will serialize the Array like this:
categories=one&categories=two&categories=three
And not like this (which I thought was the conventional Rails way):
categories[]=one&categories[]=two&categories[]=three
Why?
I can see that it has to do with the recent implementation of the URI.encode_www_form method that set_form_data uses. But what is the purpose deviating from the conventional Rails way?
And, more importantly, how do I easily modify this to send it in the latter way (without overloading a bunch of inherent Ruby/Rails methods)?
I found out that the solution was as easy as changing the table name:
data = {'categories[]' => [one, two, three]}
It even works if other elements of the data hash are :symbols.
I'd still be curious to find out why Rails makes this "hack" necessary when using the Net::HTTPHeader::set_form_data method, to get Rails' otherwise conventional way of handling arrays in the url parameters.
I want to make a http request (using httparty gem) to facebook graph to check if a token which comes form the client is valid or not, but I suppose that interpolation doesn't work with httparty
response = HTTParty.get("https://graph.facebook.com/me?access_token=#{params[:access_token]}")
render json: response
but this gives me the response
{"error":{"message":"Bad signature","type":"OAuthException","code":1}}
Am I missing something?
I suppose that interpolation doesn't work with httparty
Interpolation is a Ruby syntax feature; your string gets interpolated before it's ever passed to HTTParty.
You could have debugged this by inspecting the URL:
uri = "https://graph.facebook.com/me?access_token=#{params[:access_token]}"
Rails.logger.info(uri)
response = HTTParty.get(uri)
Regardless, when using query parameters with HTTParty, you shouldn't use interpolation anyway. You should pass them with the :query option, which will ensure that the parameters are encoded correctly:
response = HTTParty.get( "https://graph.facebook.com/me",
query: { access_token: params[:access_token] })
However, I'm not certain this will actually fix your problem. There's a good chance the problem is really somewhere else in your OAuth flow--there's a lot of room for error when working with OAuth.
I am sending this request to a service:
get_store_data = Typhoeus::Request.new("http://localhost:3000/api/v1/store?service=#{(proxy_ticket.service)}&ticket=#{proxy_ticket.ticket}")
proxy_ticket.service resolves to this string "http://localhost:3000/api/v1/store". When the request is sent, This string is escaped to this:
service=http%3A%2F%2Flocalhost%3A3000%2Fapi%2Fv1%2Fstore
The problem is that the service on the other end expects the service parameter as "http://localhost:3000/api/v1/store" how can i prevent this query string from being escaped?
The other side should be unescaping the params. If you would still like to know how to do it however here is the method uri.unescape which is used like so:
require 'uri'
enc_uri = URI.escape("http://example.com/?a=\11\15")
p enc_uri
# => "http://example.com/?a=%09%0D"
p URI.unescape(enc_uri)
# => "http://example.com/?a=\t\r"
If you ever want to quickly unescape a uri (and don't want to open a repl for some strange reason or other, like maybe it insulted your honour or something.) you can try this site
You can't. Its the 'other side' who should decode this param (and most likely they do).
For example rails do this encoding automatically. You can check it by altering some of your actions to raise params.pretty_inspect and invoke it with extra param your/action/route?service=http%3A%2F%2Flocalhost%3A3000%2Fapi%2Fv1%2Fstore. You will see that params include service: http://localhost:3000/api/v1/store.
If this is not working for you this mine that you need to get in touch with the other side so they implement this. There is no other way to pass urls within get urls.
Given: message = Mail.new(params[:message])
as seen here: http://docs.heroku.com/cloudmailin
It shows how to get the message.body as HTML, how to do you get the plain/text version?
Thanks
The code above:
message = Mail.new(params[:message])
will create a new instance of the mail gem from the full message. You can then use any of the methods on that message to get the content. You can therefore get the plain content using:
message.text_part
or the HTML with
message.html_part
These methods will just guess and find the first part in a multipart message of either text/plain or text/html content type. CloudMailin also provides these as convenience methods however via params[:plain] and params[:html]. It's worth remembering that the message is never guaranteed to have a plain or html part. It may be worth using something like the following to be sure:
plain_part = message.multipart? ? (message.text_part ? message.text_part.body.decoded : nil) : message.body.decoded
html_part = message.html_part ? message.html_part.body.decoded : nil
As a side note it's also important to extract the content encoding from the message when you use these methods and make sure that the output is encoded into the encoding method you desire (such as UTF-8).
What is Mail?
The message defined in the question appears to be an instance of the same Mail or Mail::Message class, which is also used in ActionMailer::Base, or in the mailman gem.
I'm not sure where this is integrated into rails, but Steve Smith has pointed out that this is defined in the mail gem.
Usage Section of the gem's readme on github.
Documentation of Mail::Message on rubydoc.info.
Extracting a Part From a Multipart Email
In the gem's readme, there is an example section on reading multipart emails.
Besides the methods html_part and text_part, which simply find the first part of the corresponding mime type, one can access and loop through the parts manually and filter by the criteria as needed.
message.parts.each do |part|
if part.content_type == 'text/plain'
# ...
elsif part.content_type == 'text/html'
# ...
end
end
The Mail::Part is documented here.
Encoding Issues
Depending on the source of the received mail, there might be encoding issues. For example, rails could identify the wrong encoding type. If, then, one tries to convert the body to UTF-8 in order to store it in the database (body_string.encode('UTF-8')), there might be encoding errors like
Encoding::UndefinedConversionError - "\xFC" from ASCII-8BIT to UTF-8
(like in this SO question).
In order to circumvent this, one can readout the charset from the message part and tell rails what charset it has been before encoding to UTF-8:
encoding = part_to_use.content_type_parameters['charset']
body = part_to_use.body.decoded.force_encoding(encoding).encode('UTF-8')
Here, the decoded method removes the header lines, as shown in the encoding section of the mail gem's readme.
EDIT: Hard Encoding Issues
If there are really hard encoding issues, the former approach does not solve, have a look at the excellent charlock_holmes gem.
After adding this gem to the Gemfile, there is a more reliable way to convert email encodings, using the detect_encoding method, which is added to Strings by this gem.
I found it helpful to define a body_in_utf8 method for mail messages. (Mail::Part also inherits from Mail::Message.):
module Mail
class Message
def body_in_utf8
require 'charlock_holmes/string'
body = self.body.decoded
if body.present?
encoding = body.detect_encoding[:encoding]
body = body.force_encoding(encoding).encode('UTF-8')
end
return body
end
end
end
Summary
# select the part to use, either like shown above, or as one-liner
part_to_use = message.html_part || message.text_part || message
# readout the encoding (charset) of the part
encoding = part_to_use.content_type_parameters['charset'] if part_to_use.content_type_parameters
# get the message body without the header information
body = part_to_use.body.decoded
# and convert it to UTF-8
body = body.force_encoding(encoding).encode('UTF-8') if encoding
EDIT: Or, after defining a body_in_utf8 method, as shown above, the same as one-liner:
(message.html_part || message.text_part || message).body_in_utf8
email = Mail.new(params[:message])
text_body = (email.text_part || email.html_part || email).body.decoded
I'm using this solution on RedmineCRM Helpdesk plugin
I believe if you call message.text_part.body.decoded you will get it converted to UTF-8 for you by the Mail gem, the documentation isn't 100% clear on this though.
Save HTML Body Format in Rails
USE <%= #email.body.html_safe%>
This will send text written in email text editor as it is to email.