Ok, I've got an App built using Rails 4.2 and MongoDB (MongoId V4) for storage. The information (text) stored in some of the tables/collections is a mix of English and Danish. The App is not localized in any way/shape/form yet as it is an API only.
I have a Sidekiq worker that updates the documents in my database. The problem I'm having is that every time a document is updated in Sidekiq, I get the following error:
Encoding::UndefinedConversionError: "\xE9" from ASCII-8BIT to UTF-8 output in Sidekiq.
How can I fix that?
I think you can use enforce encoding to utf8 using string.force_encoding(Encoding::UTF_8) before you update that specific record
Related
Rails 4.0.3. I have a form with a date field. In other words, it only works when I just use date_select. that works perfectly on my dev machine. When I deploy it to production, the form refuses to submit and throws a NoMethodError upon reaching the date, complaining about the lack of the utc method. I can't trace it because the production server's log never tells me more than the line where it fails, but that line is the Model.new(params[:form]) line and the failure is during the type conversion magic.
The field is currently a text field with the value supplied by jQuery UI datepicker, but this problem exists also when it is a plain Rails formhelper date_field. As soon as it gets to the date, it fails... but it works perfectly in dev and I'm deploying to production using Git, so I know we are on the same page.
Locally, I'm working in jRuby and using Neo4j.rb 2.3 as my DB. I have mimicked the dev environment by running the production app server, Torquebox, and it still local, fails remote. I'm running the same version of jRuby in both locations and I have all of my gems locked to specific versions to rule out some weird version-specific bug. Any thoughts are very appreciated!
Try this code and see if it works:-
date_string = "#{review_params['date(1i)']}-#{review_params['date(2i)']}-#{review_params['date(3i)']}"
#concert = Concert.find_or_create_by!(artist: review_params[:artist], venue: review_params[:venue], date: date_string)
I realized the solution AS SOON AS I was clicking the submit button but wanted to document this in case someone else ever has this issue. This is actually a bug in the neo4j-wrapper gem... that had a pull request merged two months ago... that I wrote. And I forgot about it. My local gemset still had my modified version of the gem cached, I guess, but the server was just using the latest release, which was done before my modification went in.
This bug is in type_converters.rb. Neo4j saves all times and dates to the DB as integers so it needs to convert whatever is submitted. The original code checked the type of the object to be converted. If it was a date, it converted it to a time and then ran utc; else, it just ran utc. This is obviously no good for strings, so my fix ran Time.parse(s) first.
So, if anyone else is having this issue and is using Neo4j.rb 2.3, add this line to your gemfile
gem 'neo4j-wrapper', git: 'https://github.com/andreasronge/neo4j-wrapper', ref: '00e5665ead'
I deployed my app to heroku recently. When i try to send out an invite to anyone i get the following error.
ActionView::Template:Error (invalid type sequence in UTF-8)
In my template I added <%# encoding: utf-8 %>.
I am stuck at the moment since it was working well in development enviroment
Did you try running magic_encoding from the command line? You might have missed a Controller that needs the #encoding utf-8. Can you show us some code? Are you using any characters (Japanese, etc.) in your Controller?
I have a Rails application that accepts file uploads of CSV files. When developing the feature locally on my Mac, I received an "invalid byte sequence in UTF-8" error when trying to parse the uploaded file (using Ruby's standard library CSV).
So after doing some research and reading some answers to similar questions on StackOverflow, I tried using a gem to sniff out the character encoding (namely CharDet), and then when opening the file via the CSV library, I would specify the encoding. And this solved all my problems, and life was good.
content = File.read(fullpath)
self.file_encoding = CharDet.detect(content)['encoding']
CSV.table(fullpath, :encoding => file_encoding, :header_converters => :downcase).headers
But then I deployed this code to the production Linux environment, and again with the "invalid byte sequence in UTF-8" errors. What a mystery (to me anyway)! After quite some time trying to resolve the error, I tried removing the code that specified the encoding upon opening the file. And miraculously it fixed the problem on production, but now local Mac development is broken.
Keep in mind, that in both cases I'm uploading the same file using the same browser. Does anyone have any insight on what is going on here?
By the way, versions of ruby are close, but not the same. The Mac is ruby 1.9.3-p0, and the Linux server is 1.9.2-p180. The app is Rails 3.2.6.
A few thoughts:
Have you confirmed the encoding of the file that you're uploading?
Have you tested with 1.9.2-p180 on your Mac, as Frederick Cheung suggested?
Have you tried outputting the results of CharDet.detect on each platform to see what the encoding of the received file (as opposed to the uploaded file) is? I wonder if some configuration is different between Apache on Linux and WEBrick on your Mac?
Are you using the same version of CharDet on both platforms? What libraries does it use (e.g. iconv), and are they the same version on both platforms?
I'm not aware of any differences in behavior with regard to encoding between 1.9.2 and 1.9.3, but I haven't specifically researched it either. It could also be a difference in the configuration of the MRI build.
I just started work on a new Rails app, using the bright and shiny new version of Rails, 3.2.1. Previously, I had only used up to version 3.0.9. Before I describe my error, let it be known that I am using Ruby version ruby 1.9.2p290 (2011-07-09) [i386-mingw32] on Windows 7 32-bit. I have not changed my version of Ruby recently. I am using Notepad++ v5.9.3 and haven't (on purpose) changed any default settings.
When I ran my new app for the first time, I got an odd message:
ActionView::WrongEncodingError in Index#index
Your template was not saved as valid UTF-8. Please either specify UTF-8 as the encoding for your template in your text editor, or mark the template with its encoding by inserting the following as the first line of the template:
# encoding: <name of correct encoding>.
I do not understand why I am getting this error all of a sudden. Is it part of changes made to Rails 3.2.1? It is easily fixed by going into Notepad++ and using the Encoding menu option "Convert to UTF-8" but, like I said, I've never had to do this before.
The other odd thing is that even the files that Rails generates are generated with ANSI encoding when I use a generator. Overall, I'm confused and I want to make sure that I'm using good programming practices.
Is it part of changes made to Rails 3.2.1? It is easily fixed by going into Notepad++ and using the Encoding menu option "Convert to UTF-8" but, like I said, I've never had to do this before.
Yes. Rails 3.0+ (I think) requires all templates to be saved in UTF-8 encoding. You need to save the file as UTF-8. If that still doesn't work, set the encoding explicitly by adding on the first line of your .rb files the following:
# encoding: utf-8
Add this to the first line of your .erb templates:
<%# encoding: utf-8 %>
See this related question, and this similar problem. Sounds to me like your editor's encoding settings changed since you originally created the files.
The other odd thing is that even the files that Rails generates are generated with ANSI encoding when I use a generator. Overall, I'm confused and I want to make sure that I'm using good programming practices.
That is quite odd, and I'm not sure I have a good suggestion for that one, other then trying to add Encoding.default_external = "UTF-8" to your config.ru and config/environment.rb files.
I had tried the encoding: utf-8 method without luck, but I resolved the issue when I changed the encoding using Notepad++. Thanks!
I'm having huge problems with my populate.rake file - after importing my existing file from my Rails 2.3.5 installation, Rails 3 wants nothing to do with it. I'm getting tens (if not hundreds) of errors in my console, many of them for simple statements, and some even in the middle of a string of plaintext.
Has the syntax changed for Rails 3 populate or, perhaps do I need to install a gem/plugin for it?
Thanks very much.
db-populate is a Rails plugin which uses a populate.rake file, it's not core Rails functionality so yes, you'd need a gem/plugin for it (http://github.com/ffmike/db-populate).
However, since you've upgraded to Rails 3, it might be a nice time to switch to using the built-in Rails seed functionality - that way it's easy to run rake db:setup which will create the db from database.yml, load the schema, and run seeds.rb or rake db:reset which does the same thing after dropping the db. Keeps you from having to maintain third-party code as well. Just food for thought. :)
I found the problem after carefully reconstructing the populate.rake file from a copy. Some of the sample text I introduced into the file had quotes and these were interfering with the quotes belonging to the string itself.
Once Rails interprets the string as having ended, then it logically starts to return errors within the successive strings, which it is reading as interpretable code.
Once I removed all the quotes from within my strings during that reconstruction, the file started working. Thank you Ryan and JenJenut for your replies!