Ruby on Rails locales utf-8 issues - ruby-on-rails

I allready have problems with utf-8 encoding in my ror app ...
some are fixed now. But some are still left.
I have now an utf-8 force in my layout
But still have problems with German special chars (ä, ö, ü). In my /config/locales/de.yml I have lots of them. In the File they look nice :) tested with rubymine and nano.
But when I start the app it crashes. The yml is encoded in utf-8 ..
I've also tried this:
f\xC3\xBCr --> should be für
always got this:
incompatible character encodings: UTF-8 and ASCII-8BIT
Does anyone have some hints for me?

It seems to me that the encoding of the app is set to UTF-8.
Are you sure that RubyMine saves your file with UTF-8?
You can add
# encoding: UTF-8 to the top of your files to assure it is set. (Not sure if this works in .yml)
Edit:
If you have pasted any text into the file it may still contain wrong encoding.
Move the de.yml out of the project.
Create a new file de.yml
de:
first_translation: Ich möchten ein bisschen Müsli
If this works, then you need to rewrite everything from the old file, no copying!

Related

How to solve byte order mark, BOM, in localization files

My Rails app's localization works fine locally, but when deployed to Heroku some word don't get translated and renders the "missing translation" span tag.
This question seems to have the same problem and have solved it by removing the offending BOM. But what does this imply, how do I identify and remove offending BOM in my yaml-file?
Or/and what should I look for to check if BOM exists in the file?
I'm editing in Sublime.
If BOM is your problem, in Sublime you can go to File --> Save With Encoding --> UTF8.
UTF8 options is implied to be without BOM. You have other option as well UTF8 with BOM.

What Windows 'hosts' encoding is?

What the Windows 'hosts' file encoding is? Is it UTF-8? Or ASCII + system codepage? How IDN (international domain names with umlauts etc.) entries should be added and can they be added at all?
It should be ANSI or UTF-8 without BOM. I just dealt with a server that had the hosts file encoding set to UCS-2 Little Endian, and that led to the file being ignored.
There is a wealth of information here:
https://serverfault.com/questions/452268/hosts-file-ignored-how-to-troubleshoot
The simple answer is
ANSI or UTF-8 WITH BOM.
(UTF-8 without BOM is NOT valid).
Details:
As far as I have tried, the encoding of the hosts file on Windows should be
ANSI or UTF-8 with BOM.
I know this question is many years old, but a colleague made the mistake of looking at this post and the ServerFault post, so I decided to add an answer.
1. Simple case only ASCII
Works.
Without any multi-byte characters, This is equivalent to ANSI, also equivalent to UTF-8 without BOM.
2. ANSI (with Japanese ANSI multi-byte characters)
Works.
note: There are Japanese characters but this is valid ANSI encoding in windows.
In Japanese editions of Windows, this code page cp932 is referred to as "ANSI",
https://en.wikipedia.org/wiki/Code_page_932_(Microsoft_Windows)
3. UTF-8 with BOM
Works.
note: BOM 付き means with BOM.
4. UTF-8 without BOM
DOES NOT work.
5. Additional test cases
If you use emoji instead of Japanese, the result will be the same.
Use emoji and save as UTF8 without BOM does not work.
(However, other lines not include emoji may be worked correctly.)
Use emoji and save as UTF8 with BOM can resolve host correctly.
note: If you use Notepad to check it yourself, be sure to put double quotes in the file name when you save it, or Notepad will be create hosts.txt.
Appended:
(Asked in comment)
The hosts file supports inline comments.

UTF-8 issue in Ruby on Rails with × character

<a class="close" href="#">×</a>
I get an error regarding the use of ×.
It's used in error messages on twitter's bootstrap framework, I get an invalid byte sequence in UTF-8 error when I try to use it. Is there any work-around? Apart from using a normal x or X.
I have:
# Configure the default encoding used in templates for Ruby 1.9.
config.encoding = "utf-8"
In my application.rb
This seems almost too simple, but why aren't you using ×?
You need to set the encoding at the top of the file where that character is used. You can do this with:
# coding: utf-8
class MyClass
end
I haven't tried it in an erb file, but I don't see why that would be any different. I think you can use the word "encoding" too instead of just "coding" if that feels better. All that is required is at minimum "coding".
What editor are you using?
I suspect that you are saving the source file using an encoding other than UTF-8 (such as Latin-1 or ANSI on Windows), which is then causing ruby to fail to interpret the file correctly.
I've tried adding the times symbol to one of my views (using HAML) and it worked correctly. I'm using VIM as my editor and saving in UTF-8 without any BOM.
#encoding: utf-8
class ClassiClass
end
everything works fine!

Rails and Utf-8 encoding

I know this is the most common problem in the history of programming but I reallu believe I have tried it all now but I still get: incompatible character encodings: UTF-8 and ASCII-8BIT in my pages#index.html file.
I am running Ruby 1.9.2 and Rails 3.1.1.
This is what I have verified / tested / set:
In database.yml, set encoding: utf8
development:
adapter: mysql
encoding: utf8
reconnect: false
database: kopa_development
pool: 5
username: root
password:
host: localhost
Checked with Notepad++ that the textfile(s) in term is encoded to utf-8
Checked with mysql query browser that the database is set to utf-8, so are the columns
I have tried with # coding: utf-8 in the controller related to the view I am trying to look at.
I have put SET NAMES utf8 in mysql
I have tried Encoding.default_internal = Encoding::UTF_8 in environment.rb as well as Encoding.default_external = Encoding::UTF_8
I have config.encoding = "utf-8" in application.rb
I have Googled
I have read http://www.joelonsoftware.com/articles/Unicode.html
I have tried both Notepad++ and RubyMine
Adding .force_encoding works but is very inconvenient to do
A funny thing is that if I set the index.html.erb to ANSI it will complain that application.html.erb does not have the correct encoding (rather than index.html.erb). I don't know what this means but it makes me think that there is something fishy with the editor.
I remember when using e text editor that there were two things I could set (with reference to encoding) the encoding and something else. That "else" I don't remember what it was though and I don't have e anymore.
Regarding the error-message, do I understand it correctly that the first mentioned means the database encoding and the second the Rails(?) encoding? I have seen the opposite error message as well (ASCII first and UTF-8 after).
Edit: I Googled some more this morning and found there is something wrong with the mysql-gem. The articles were a bit old and I tried the solutions there (such as trying mysql2 & ruby-mysql) with no better luck.
I am all out of ideas. How about you?
I couldn't fix this problem. I "solved" it by purchasing a Mac and continue to develop my Rails apps on it instead...
Encoding - me: 1-0
I was able to overcome that error but unfortunately I got the same error in yield statement (application.html.erb). After an half hour I gave up.
I saved some erb files from utf-8 with BOM to utf-8 without BOM. Mixing encoding is not good and I think it's the reason for your errors.
I also found files with different line ending: lf or crlf.
If you are developing in Windows, I recommend you the installation of virtual box to setup a linux virtual machine. You can access file system through samba, it works pretty good for me.
Also, configure your text editor for using utf-8 without BOM and lf end of line.

ruby on rails x charset

i'm having problem to deal with charset in ruby on rails app, specificially in my templates. Code that comes from my database, works fine, but codes like ç ~ that are located in my views are not working. I added the following codes to my code
I added a function like that, but that still not working i have ç ~ codes in my application.rhtml that are not working.
before_filter :configure_charsets
# Configuring charset to UTF-8 def configure_charsets
headers["Content-Type"] = "text/html; charset=UTF-8"
end
I added as well meta http-equiv html to utf-8 and a .htaccess parameter AddDefaultCharset UTF-8
That's still not working, any other tip?
Put this piece of code in your config (environment.rb)
Rails::Initializer.run do |config|
config.action_controller.default_charset = "iso-8859-1"
end
This will do it.
Also, remove the default charset line if any in layouts/application.html
Is the text editor you're using to put the special characters into the file (either source or views) treating those characters as UTF-8? For example, if you're using TextMate, you can deliberately save a file as UTF-8. If for some reason you used a different encoding earlier (a default, perhaps), those UTF-8 characters might be getting transcoded at the code editing stage, so even if the rendering process is using UTF-8 throughout, it'll still not work.
Further, if you're using something from a shell, like vi, or whatever, is your terminal set up to accept UTF-8 as default? If you had it set to ISO-8859-1 or whatever, you'd get the same issue.
Is your application.rhtml file written in the correct character set? Make sure it's UTF-8, and not ISO-8859-1.
So if the contents of your file are UTF-8, and the output is being interpreted as UTF-8, something in between is changing the data. Can give give us the the hex interpretation of the input bytes (anything non-ASCII will be at least two bytes in UTF-8) for one of your special characters, and the hex interpretation of the output byte or bytes? Perhaps we can figure out what the change is, and work back from there.

Resources