Rails incompatible character encodings: UTF-8 and ASCII-8BIT - ruby-on-rails

In my Rails app,I send a post request:
require 'net/http'
url="http://192.168.0.84:809/Services/SDService.asmx/UserRegister"
Net::HTTP.post_form(URI(url),{:memtyp=>'CU',:memid=>'100867',:dob=>'1989-01-01'}).body
But I got the error:
incompatible character encodings: UTF-8 and ASCII-8BIT
I found that the response data include UTF-8 character just like 中文,and then I got this error.
so what should I do?

You can send data in json format if you want to do so you can do this by following:
require 'rest_client'
require "net/http"
require "uri"
require 'json'
RestClient.post 'localhost:3001/users',{:memtyp=>'CU',:memid=>'100867',:dob=>'1989-01-01'}.to_json , :content_type => :json, :accept => :json
Plese change the localhost url to actual url you want to hit.

Related

How to decode utf8 characters with ruby Base64

Just as the title describes, how can I do the following?
require 'base64'
text = 'éééé'
encode = Base64.encode64(text)
Base64.decode64(encode)
Result: éééé instead of \xC3\xA9\xC3\xA9
When you decode64 you get back a string with BINARY (a.k.a. ASCII-8BIT) encoding:
Base64.decode64(encode).encoding
# => #<Encoding:ASCII-8BIT>
The trick is to force-apply a particular encoding:
Base64.decode64(encode).force_encoding('UTF-8')
# => "éééé"
This assumes that your string is valid UTF-8, which it might not be, so use with caution.
Just use Base64's encode and decode method:
require 'base64'
=> true
Base64.encode64('aksdfjd')
=> "YWtzZGZqZA==\n"
Base64.decode64 "YWtzZGZqZA==\n"
=> "aksdfjd"

rails incompatible character encodings: UTF-8 and ASCII-8BIT in json

I use RestClient to retrieve a json string from a webservice via GET.
This works fine but as soon as there are Umlauts (ü) and other chars (e.g. ß) in the string, I get this error in my view
#output = RestClient.get 'https://myurl.com/api/v1/orders/53e0ae7f6630361c46060000', {:authorization => 'Token xxxxxx', :content_type => :json, :accept => :json}
<%= #output %>
=>
Encoding::CompatibilityError
incompatible character encodings: UTF-8 and ASCII-8BIT
any idea how to solve this?
Solved after adding this line
#output = #output.force_encoding('utf-8').encode

Encoding::UndefinedConversionError: "\xE4" from ASCII-8BIT to UTF-8

I tried to fetch this CSV-File with Net::HTTP.
File.open(file, "w:UTF-8") do |f|
content = Net::HTTP.get_response(URI.parse(url)).body
f.write(content)
end
After reading my local csv file again, i got some weird output.
Nationalit\xE4t;Alter 0-5
I tried to encode it to UTF-8, but got the error Encoding::UndefinedConversionError: "\xE4" from ASCII-8BIT to UTF-8
The rchardet gem tolds me the content is ISO-8859-2. But convert to UTF-8 will not work.
After open it in a normal Texteditor, i see it normal encoded.
You can go with force_encoding:
require 'net/http'
url = "http://data.linz.gv.at/katalog/population/abstammung/2012/auslg_2012.csv"
File.open('output', "w:UTF-8") do |f|
content = Net::HTTP.get_response(URI.parse(url)).body
f.write(content.force_encoding("UTF-8"))
end
But this will make you lose some acentuation in your .cvs file
If you are deadly sure that you always will use this URL as input, and the file will always keep this encoding, you can do
# encoding: utf-8
require 'net/http'
url = "http://data.linz.gv.at/katalog/population/abstammung/2012/auslg_2012.csv"
File.open('output', "w:UTF-8") do |f|
content = Net::HTTP.get_response(URI.parse(url)).body
f.write(content.encode("UTF-8", "ISO-8859-15"))
end
But this will only work to this file.

"\xC2" to UTF-8 in conversion from ASCII-8BIT to UTF-8

I have a rails project that runs fine with MRI 1.9.3. When I try to run with Rubinius I get this error in app/views/layouts/application.html.haml:
"\xC2" to UTF-8 in conversion from ASCII-8BIT to UTF-8
It turns out the page had an invalid character (an interpunct '·'), which I found out with the following code (credits to this gist and this question):
lines = IO.readlines("app/views/layouts/application.html.haml").map do |line|
line.force_encoding('ASCII-8BIT').encode('UTF-8', :invalid => :replace, :undef => :replace, :replace => '?')
end
File.open("app/views/layouts/application.html.haml", "w") do |file|
file.puts(lines)
end
After running this code, I could find the problematic characters with a simple git diff and moved the code to a helper file with # encoding: utf-8 at the top.
I'm not sure why this doesn't fail with MRI but it should since I'm not specifying the encoding of the haml file.

Ruby/Rails - Bad URI

Not sure why I'm getting the following error when the URI works just fine in the browser:
http://oracleofbacon.org/cgi-bin/xml?a=Kevin Bacon&b=Tom Cruise&u=1&p=google-apps
This is my code:
def kb(to)
uri = "http://oracleofbacon.org/cgi-bin/xml?a=Kevin Bacon&b=#{to.strip}&u=1&p=google-apps"
doc = Nokogiri::XML(open(uri)) # throws error on this line
return parse(doc)
end
I get the following error:
in `split': bad URI(is not URI?): http://oracleofbacon.org/cgi-bin/xml?a=Kevin Bacon&b=Tom Cruise&u=1&p=google-apps (URI::InvalidURIError)`
I execute the method in the following way:
kb("Tom Cruise")
It's because a browser is pathologically friendly, like a puppy, and will go to great lengths to render a page or resolve a URL. An application won't do that because you have to tell it how to be friendly.
Your URL is not valid because it has embedded spaces. Replace the spaces with %20:
irb -f
irb(main):001:0> require 'open-uri'
=> true
irb(main):002:0> open('http://oracleofbacon.org/cgi-bin/xml?a=Kevin%20Bacon&b=Tom%20Cruise&u=1&p=google-apps').read
=> "<?xml version=\"1.0\" standalone=\"no\"?>\n<link><actor>Tom Cruise</actor><movie>A Few Good Men (1992)</movie><actor>Kevin Bacon</actor></link>"
Escaping the characters needing to be escaped is easy:
irb -f
irb(main):001:0> require 'uri'
=> true
irb(main):002:0> URI.escape('http://oracleofbacon.org/cgi-bin/xml?a=Kevin Bacon&b=Tom Cruise&u=1&p=google-apps')
=> "http://oracleofbacon.org/cgi-bin/xml?a=Kevin%20Bacon&b=Tom%20Cruise&u=1&p=google-apps"

Resources