I have post data that includes a '+' sign. Once it makes it to the server the raw post data is showing the '+' sign but once the post data makes it into the param hash the '+' sign has been converted to a blank. Any ideas on how to make it NOT do that?
If you replace your '+' signs with '%2B', this should resolve the issue.
However, also note that you probably need to check your ampersands, percent signs, and other characters as well. The server receiving your post data is probably expecting URLEncoded data.
In a nutshell, if you replace % signs with %25, then replace & with %26, replace ? with %3F, replace # with %23, and replace + signs with %2B; you will cover most of the issues you can encounter.
A more in-depth list of replacements can be found at these links.
HTML Url Encoding (w3schools)
Percent-Encoding (wikipedia)
Have a look at the CGI.escape method in the standard library that will do this for you:
irb(main):001:0> require 'cgi'
=> true
irb(main):002:0> CGI.escape 'foo+bar&baz?qux quux/corge'
=> "foo%2Bbar%26baz%3Fqux+quux%2Fcorge"
There's also a CGI.unescape method should you need to convert back.
Try replacing the + with %2B.
Not sure why that is happening. Normally + signs make it through to the params. Can you post your rails versions. Also try escaping the "+" sign with "+" or its CGI equivalent "%2B" to see if it makes a difference.
there is a Ruby call to handle all this for you so you don't need to figure out the characters yourself
require 'uri'
url = http://www.google.com?a=this is a test
URI.escape(url, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))
Related
I have a simple problem: users can post urls through specific input in a form in my website.
I would like to encode the posted url, because sometimes users send urls with strange and/or non ascii characters (like é à ç...). For instance: https://www.example.com/url-déjà-vu
So I tried to use URI.escape('https://www.example.com/url-déjà-vu') which does work, but then if you have the following url: URI.escape('https://somesite.com/page?stuff=stuff&%20') you get:
=> "https://somesite.com/page?stuff=stuff&%2520"
The % character is encoded and should not be as %20 is already an encoded character. Then I thought I could do this:
URI.escape(URI.decode('https://somesite.com/page?stuff=stuff&%20'))
=> "https://somesite.com/page?stuff=stuff&%20"
But there is a problem if you have a "/" encoded in your url, for instance:
URI.escape(URI.decode('http://example.com/a%2fb'))
=> "http://example.com/a/b"
The "/" should stay encoded.
So... putting it all together: I want to encode urls posted by users but leaving already encoded characters unchanged in ruby. Any idea how I may do that without getting an headache?
Thanks :)
I can't think of a way to do this that isn't a little bit of a kludge. So I propose a little bit of a kludge.
URI.escape appears to work the way you want in all cases except when characters are already encoded. With that in mind we can take the result of URI.encode and use String#gsub to "un-encode" only those characters.
The below regular expression looks for %25 (an encoded %) followed by two hex digits, turning e.g. %252f back into %2f:
require "uri"
DOUBLE_ESCAPED_EXPR = /%25([0-9a-f]{2})/i
def escape_uri(uri)
URI.encode(uri).gsub(DOUBLE_ESCAPED_EXPR, '%\1')
end
puts escape_uri("https://www.example.com/url-déjà-vu")
# => https://www.example.com/url-d%C3%A9j%C3%A0-vu
puts escape_uri("https://somesite.com/page?stuff=stuff&%20")
# => https://somesite.com/page?stuff=stuff&%20
puts escape_uri("http://example.com/a%2fb")
# => http://example.com/a%2fb
I don't promise that this is foolproof, but hopefully it helps.
I currently am getting an the following error when using httparty on heroku.
HTTParty works for the first 11 times but on the 12th time it shows this error.
I am trying to seed data into my database.
When I go to the URL via my browser, it works. I ran the same code in development and it works. I am unsure what to do. Please help!
You pass invalid URI -
"https://maps.googlemaps.com/maps/api/geocode/json?address=San Fransisco"
address has space in URI.
So, wherever you pass params, Do -
uri_path = https://maps.googlemaps.com/maps/api/geocode/json
params = {address: "San Fransisco",...............}
"#{uri_path}?#{params.to_param}"
Amit Suroliya was correct. I was using HTTParty as well and it was working through Curl and Development but would crash in production. Because the HTTParty parameter is the literal URL (as a string), it has to be a flawless URL/URI (meaning no spaces). My bad URI was as follows:
HTTParty.get("http://api.blahblahblah.com/v1/Boards/Search?&limit=79&query=#{query}&filter_date_from=1423353600")
Notice the interpolation query=#{query}"
So if query='Michelle Obama', notice space between Michelle and Obama. Because of interpolation, the HTTParty.get('string') it is incorrect.
Solution:
Replace all whitespaces within your string with +, or you could use %20.
I used query.gsub!(' ', '+')
For more info on whitespace in the URL check it out here: Spaces in URLs?
There are many discussion about URL escaping in Ruby, but unfortunately I didn't find an appropriate solution.
In general, URI.escape should do the job, but looks like it doesn't support all characters, for example it doesn't escape "[".
URI.parse(URI.escape("1111{3333"))
works well.
URI.parse(URI.escape("1111[3333"))
raises an exception.
I understand that "[" is not an eligible character in URL according to RFC, but when I enter it into the browser it takes it, and renders the page, so I need exactly the same behavior.
Do know any ready solution for escaping in Ruby?
I typically use
CGI.escape
to escape URI parameters.
require 'cgi'.
CGI.escape('1111[3333')
=> "1111%5B3333"
The character [ is a uri delimiter character and does not require escaping.
http://www.ietf.org/rfc/rfc2396.txt
section 2.4.3. Excluded US-ASCII Characters
I have built a form that submits values to Wufoo as a GET request in the URL. I cannot get it to work if any of the values (in a textarea) contain a line-break or a forward slash. Is there a way to encode these in a URL?
This is being done in Rails.
I thought Rails would do that for you. But if you need to do it manually, you can use CGI::escape, e.g.
> require 'cgi'
...
> CGI.escape("hello%there\nworld")
=> "hello%25there%0Aworld"
EDIT:
Actually, CGI does not seem to escape a dot. URI can be used instead, it takes an extra parameter that lets you list extra characters you want escaped:
URI.escape("hello.there%world", ".")
http://en.wikipedia.org/wiki/Percent-encoding
is there any danger if the rails html_escape function would stop escaping '&'? I tested a few cases and it doesn't seem to create any problems. Can you give me a contrary an example? Thanks.
If you put an unescaped "&" into an HTML attribute, it would make your page invalid. For example:
Link
The page is now invalid as the & indicates an entity. This is true for any usage of an & on a page (for example, view source and hopefully you'll notice that Stack Overflow escapes the & signs in this post!)
The following would make the above example valid:
Link
Additional Note
& characters do need to be escaped in URLs if you want to validate your markup against the W3C validator. Example:
Line 9, Column 38: & did not start a character reference.
(& probably should have been escaped as &.)
Example
change an url with adding some argument