I am working on a rails app that submits a french translation via ajax and for some reason I keep getting the following error in the log:
Encoding::CompatibilityError incompatible character encodings: UTF-8 and ASCII-8BIT
Does anyone know how to fix this?
FIX:This works on the WEBrick sever
Place # encode: UTF-8 at the top of each file you want to work with different chars
I can't get this to work on a rails server with Thin... anyone else run into this?
https://rails.lighthouseapp.com/projects/8994/tickets/4336-ruby19-submitted-string-form-parameters-with-non-ascii-characters-cause-encoding-errors
the above link fixed my problem.
Specifically myString.force_encoding('UTF-8') on the string before sending it for translation.
Placed the sample code in the Application_controller.rb file and all is well
I know this is old, but I had the same problem and the solution was in the link #dennismonsewicz gave. In detail, the code was:
was:
before_filter :force_utf8_params
def force_utf8_params
traverse = lambda do |object, block|
if object.kind_of?(Hash)
object.each_value { |o| traverse.call(o, block) }
elsif object.kind_of?(Array)
object.each { |o| traverse.call(o, block) }
else
block.call(object)
end
object
end
force_encoding = lambda do |o|
o.force_encoding(Encoding::UTF_8) if o.respond_to?(:force_encoding)
end
traverse.call(params, force_encoding)
end
I fixed this issue by converting an utf8 file to ascii.
See the answer here: ruby 1.9 + sinatra incompatible character encodings: ASCII-8BIT and UTF-8
Related
My rails app uploads CSV files to S3, then subsequently pulls them down into a tempfile to send each row's data to a Sidekiq worker. I'm using Carrierwave and fog to handle the uploading.
This all worked beautifully until recently switching to Heroku, and now, when trying to create my tempfile I get the following error:
Error type Encoding::UndefinedConversionError
Error message "\xA2" from ASCII-8BIT to UTF-8
I've tried setting the encoding when creating the tempfile as well as working with the CSV file and continue to get the same error. I cannot reproduce this error on my local machine, which has made this entire process that much more fun :)
Currently, my Sidekiq worker calls the following method:
def upload_csv(filename, file_path)
file = Tempfile.new(filename, Rails.root.join('tmp'), encoding: "ISO8859-1:utf-8").tap do |f|
open(file_path).rewind
f.write(open(file_path).read)
f.close
end
CSV.foreach(file, headers: true, encoding: "ISO8859-1:utf-8")do |row|
#do stuff to rows
end
end
I understand the very basics of encoding, but I'm super stuck on this. Any insight would be appreciated.
Thanks!
Not sure if this will help anyone else, but I found a solution that works for me:
def upload_csv(filename, file_path)
file = Tempfile.new(filename, Rails.root.join('tmp')).tap do |f|
open(file_path).rewind
f.write(open(file_path).read.force_encoding('utf-8'))
f.close
end
CSV.foreach(file, headers: true)do |row|
#do stuff to rows
end
end
Even though I could confirm that the file was UTF-8 encoded before it was uploaded, open(#file_path).read.encoding returning an ASCII-8BIT encoding. It was getting confused on how to write the file and convert it from ASCII-8BIT to UTF-8.
I have the next method call:
Formatting.git_log_to_html(`git log --no-merges master --pretty=full #{interval}`)
The value of interval is something like release-20130325-01..release-20130327-04.
The git_log_to_html ruby method is the next (I am only pasting the line what raises the error):
module Formatting
def self.git_log_to_html(git_log)
...
git_log.gsub(/^commit /, "COMMIT_STARTcommit").split("COMMIT_STARTcommit").each do |commit|
...
end
end
This used to work, but actually I checked that gsub is raising an "invalid byte sequence in UTF-8" error.
Could you help to understand why and how can I fix it? :/
Here is the output of git_log:
https://dl.dropbox.com/u/42306424/output.txt
For some reason, this command:
git log --no-merges master --pretty=full #{interval}
is giving you a result that is not encoded in UTF-8, it may be that your computer is working with a different charset, try the following:
module Formatting
def self.git_log_to_html(git_log)
...
git_log.force_encoding("utf8").gsub(/^commit /, "COMMIT_STARTcommit").split("COMMIT_STARTcommit").each do |commit|
...
end
end
I'm not sure if that will work, but you can try.
If that doesn't work, you can check ruby iconv to detect the charset and encode it on utf-8: http://www.ruby-doc.org/stdlib-2.0/libdoc/iconv/rdoc/
Based on the file you added on the comment, I did:
require 'open-uri'
content = open('https://dl.dropbox.com/u/42306424/output.txt').read
content.gsub(/^commit /, "COMMIT_STARTcommit").split("COMMIT_STARTcommit")
and worked nice without any kind of troubles
btw, you can try:
require 'iconv'
module Formatting
def self.git_log_to_html(git_log)
...
git_log = Iconv.conv 'UTF-8', 'iso8859-1', git_log
git_log.gsub(/^commit /, "COMMIT_STARTcommit").split("COMMIT_STARTcommit").each do |commit|
...
end
end
but you should really detect the charset of the string before attempting a conversion to utf-8.
I need some help with the following. I got a string here which contains special characters e.g. ë, é etc. I can display them correctly in my view but once I call the truncate method, it throws the following error:
incompatible character encodings: ASCII-8BIT and UTF-8
The weird thing is that, when I inspect the encoding of the truncated string, it does give me UTF-8, which is what I need (and UTF-8 is used for my database).
my_string_with_special_characters.truncate(35).encoding.inspect
=> UTF-8
But is is when I call:
<%= my_string_with_special_characters.truncate(35) %>
=> incompatible character encodings: ASCII-8BIT and UTF-8
I have also tried the magic_encoding gem which prepends the magic comment
"encoding : utf-8" in all of my controller files, but I still got the incompatible character encoding error.
If anyone knows how to solve this, let me know. Much appreciated.
Alex
Try to use this string in the beginning of you file (for *.rb files)
# -*- encoding: utf-8 -*-
I am using Ruby 1.9.2, Rails 3.0.4/3.0.5 and Phusion Passenger 3.0.3/3.0.4. My templates are written in HAML and I am using the MySQL2 gem. I have a controller action that when passed a parameter that has a special character, like an umlaut, gives me the following error:
ActionView::Template::Error (incompatible character encodings: UTF-8 and ASCII-8BIT)
The error points to the first line of my HAML template, which has the following code on it:
<!DOCTYPE html>
My understanding is that this is caused because I have a UTF-8 string that is being concatenated with an ASCII-8BIT string, but I can't for the life of me figure out what that ASCII-8BIT string is. I have checked that the params in the action are encoded using UTF-8 and I have added an encoding: UTF-8 declaration to the top of the HAML template and the ruby files and I still get this error. My application.rb file has a config.encoding = "UTF-8" declaration in it as well and the following all result in UTF-8:
ENV['LANG']
__ENCODING__
Encoding.default_internal
Encoding.default_external
Here's the kicker: I cannot reproduce this result locally on my Mac-OSX using standalone passenger or mongrel in either development or production. I can only reproduce it on a production server running nginx+passenger on linux. I have verified in the production server's console that the latter mentioned commands all result in UTF-8 as well.
Have you experienced this same error and how did you solve it?
After doing some debugging I found out the issue occurs when using the ActionDispatch::Request object which happens to have strings that are all coded in ASCII-8BIT, regardless of whether my app is coded in UTF-8 or not. I do not know why this only happens when using a production server on Linux, but I'm going to assume it's some quirk in Ruby or Rails since I was unable to reproduce this error locally. The error occurred specifically because of a line like this:
#current_path = request.env['PATH_INFO']
When this instance variable was printed in the HAML template it caused an error because the string was encoded in ASCII-8BIT instead of UTF-8. To solve this I did the following:
#current_path = request.env['PATH_INFO'].dup.force_encoding(Encoding::UTF_8)
Which forced #current_path to use a duplicated string that was forced into the proper UTF-8 encoding. This error can also occur with other request related data like request.headers.
Mysql could be the source of troublesome ascii. Try putting the following in initializer to at least eliminate this possibility:
require 'mysql'
class Mysql::Result
def encode(value, encoding = "utf-8")
String === value ? value.force_encoding(encoding) : value
end
def each_utf8(&block)
each_orig do |row|
yield row.map {|col| encode(col) }
end
end
alias each_orig each
alias each each_utf8
def each_hash_utf8(&block)
each_hash_orig do |row|
row.each {|k, v| row[k] = encode(v) }
yield(row)
end
end
alias each_hash_orig each_hash
alias each_hash each_hash_utf8
end
edit
This may not be applicable to mysql2 gem. Works for mysql however.
I tried the ruby hacks for utf8 (from : http://gist.github.com/273741) ... and I'm still getting the following error:
ActionView::TemplateError (incompatible character encodings: ASCII-8BIT and UTF-8)
What is bizarre for me is that the same content if retrieved with a post action (searching the app with an html from) it is displaying well ... however, with get (using an html link) it telling that their is character incompatibility !
Do you have any idea where it comes from ? is there a rails/ruby patchs for this issue ?
Thanks,
I think you problem com from template encoding in UTF-8 not in ASCII, like attempts.
In Rails 3, there are a new configuration for that :
# Configure the default encoding used in templates for Ruby 1.9.
config.encoding = "utf-8"