I'm having troubles with unicode characters in my Rails app. The problem is that when I store the object with some string containing scandinavian characters, those characters then are not rendered correctly when using to_json method. Basically all those characters are rendered like this:
\"description\":\"\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\"
Here is example:
Item.create(:title => "Test title", :description => "With scandic characters öäöäö")
=> #<Item id: 198, title: "Test title", priority: nil, description: "With scandic characters öäöäö", bought_when: nil, warranty_until: nil, created_at: "2012-12-04 15:22:19", updated_at: "2012-12-04 15:22:19", user_id: nil, custom_id: nil, uuid: nil, position: 1, public: nil, secure_details: "", bookmark_id: nil, giveaway: nil, trash: false>
At this point the created object looks ok.
Item.last
returns
=> #<Item id: 198, title: "Test title", priority: nil, description: "With scandic characters \xC3\xB6\xC3\xA4\xC3\xB6\xC3\xA4\xC3\xB6", bought_when: nil, warranty_until: nil, created_at: "2012-12-04 15:22:19", updated_at: "2012-12-04 15:22:19", user_id: nil, custom_id: nil, uuid: nil, position: 1, public: nil, secure_details: "", bookmark_id: nil, giveaway: nil, trash: false>
This "With scandic characters \xC3\xB6\xC3\xA4\xC3\xB6\xC3\xA4\xC3\xB6" - does it still look ok? Or do I miss some proper encoding already at the item creation stage?
Anyway, now when call Item.last.to_json, I get this:
"description\":\"With scandic characters \\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\\ufffd\"
And at this point the string is definitely broken.
I'm lacking some knowledge about the proper characters encoding in Rails, so any hints would be appreciated. The item, however, looks fine when rendered in the browser - this happens only when doing a JSON response.
Ok, the issue was that MySQL db was not in unicode. This is fixed now by migrating to PosgreSQL, ensuring it is in unicode and using Taps gem to migrate the data. All according to this Railscast: http://railscasts.com/episodes/342-migrating-to-postgresql
Related
Kind of an odd bug I am running into. Working on upgrading a Rails app from 3.2.22.1 tp 4.2.11.3. Everything is working pretty well. Test suite is passing on green. However when I go into the rails console and do user = User.new or list the attributes using User.new.attributes I am only getting a few of the fields returned.
Specifically
{"_id"=>BSON::ObjectId('XXxxXXxxxxXXXxXX'),
"email"=>"",
"encrypted_password"=>"",
"sign_in_count"=>0,
"time_zone"=>"Central Time (US & Canada)",
"admin"=>false}
This list should be much larger, for example, it is excluding all Devise fields like last_sign_in_at or any of them. Here is what it looks like, the same command, run on our production server which is the previous version of rails
_id: xxxXXXxxxXXXXxXXXXxx,
invited_by_type: nil,
invited_by_field: nil,
invited_by_id: nil,
email: "",
encrypted_password: "",
reset_password_token: nil,
reset_password_sent_at: nil,
remember_created_at: nil,
sign_in_count: 0,
current_sign_in_at: nil,
last_sign_in_at: nil,
current_sign_in_ip: nil,
last_sign_in_ip: nil,
confirmation_token: nil,
confirmed_at: nil,
confirmation_sent_at: nil,
unconfirmed_email: nil,
invitation_token: nil,
invitation_created_at: nil,
invitation_sent_at: nil,
invitation_accepted_at: nil,
invitation_limit: nil,
name: nil,
time_zone: "Central Time (US & Canada)",
admin: false
I am worried that there is something failing silently and I am not detecting it. I can save the unlisted values in the console, but it does not show them to me, which is not how it worked previously.
Anyone understand why this is happening?
specific versions are:
mongid -> 5.0.1
devise -> 4.7.2
Rails -> 4.2.11.3
Latest version of Mongoid does not include attributes that were never written in attributes.
Unless you have references to documentation that claims that Mongoid would return attributes that were never written, you are experiencing expected behavior.
This is a weird question... The context is that somehow, my program keeps creating weird arbitrary blank users with no information. I'm trying to debug it because it's certainly annoying but I don't want it to have some potentially other problems.
I'd like to debug it by figuring out when these random blank users were created and then looking at the logs with that time stamp. Unfortunately the user looks like this:
#<User id: 175, email: "", created_at: nil, updated_at: nil, encrypted_password: "", reset_password_token: nil, reset_password_sent_at: nil, remember_created_at: nil, sign_in_count: 0, current_sign_in_at: nil, last_sign_in_at: nil, current_sign_in_ip: nil, last_sign_in_ip: nil, confirmation_token: nil, confirmed_at: nil, confirmation_sent_at: nil, unconfirmed_email: nil, failed_attempts: 0, unlock_token: nil, locked_at: nil, stripe_customer_id: nil, stripe_card_id: nil, invite_code: nil, invited_by: nil, profile_pic: nil, first_name: nil, last_name: nil, full_name: nil>
I didn't realize it was possible to have this happen (creation without created_at), so am curious to see what could lead to this, and maybe that will help the debugging.
FYI I have tried to just look at my log over a long time period but the weird thing is I definitely do NOT see the actual create action being called so many times. I even have a puts statement right after the create just to catch this and it has only been logged twice (the appropriate amount). FWIW I'm using Devise.
Thanks
For some reason rails full_messages is missing labels and simply contains the error without any clue as to which field it relates to:
[3] pry(#<Spree::CheckoutController>)> #order.errors
=> #<ActiveModel::Errors:0x007fce1caa8218
#base=
#<Spree::Order id: 87, number: "R442456123", item_total: #<BigDecimal:7fce1c57a4a8,'0.6495E2',18(36)>, total: #<BigDecimal:7fce1c5cd0e0,'0.6495E2',18(27)>, state: "address", adjustment_total: #<BigDecimal:7fce1c5cd270,'0.0',9(27)>, user_id: 57, completed_at: nil, bill_address_id: nil, ship_address_id: nil, payment_total: #<BigDecimal:7fce1cb184a0,'0.0',9(27)>, shipping_method_id: nil, shipment_state: nil, payment_state: nil, email: "nick#1-night.co.nz", special_instructions: nil, created_at: "2014-10-10 09:47:21", updated_at: "2014-10-10 09:47:24", currency: "USD", last_ip_address: "127.0.0.1", created_by_id: 57, channel: "spree", tax_total: #<BigDecimal:7fce1cb217d0,'0.0',9(27)>>,
#messages=
{:"bill_address.phone"=>["can't be blank"],
:"ship_address.phone"=>["can't be blank"]}>
[4] pry(#<Spree::CheckoutController>)> #order.errors.full_messages
=> ["can't be blank", "can't be blank"]
I have added the attributes in en.yml - How do I get full messages to contain the model name ie: ["Billing address phone can't be blank", "Shipping address phone can't be blank"]
Using the following:
gem 'rails', '4.0.2'
gem 'spree', '2.1.4'
For me this was an issue of having (for a reason long forgotten) the following in my en.yml.
errors:
format: "%{message}"
After removing it labels show correctly.
I think the way you added the attributes in en.yml file is wrong. You should do it this way:
en:
activerecord:
attributes:
spree/order/bill_address:
phone: Billing address phone
spree/order/ship_address:
phone: Shipping address phone
This is my EmailContact model:
class EmailContact < ActiveRecord::Base
validates :email, :presence => true, :email => true
end
I am using the ruby gem valid_email.
I run the following in my rails console, in the same environment as my rake task I will show later:
>> email_contact = EmailContact.new(:email => 'a253545#gmail.com')
>> email_contact.valid?
true
So, as you can see, in the rails console I am building an EmailContact and it is valid.
Then I run this in my rake task:
list_entity = {:branch=>"Nashua Branch-YMCA of Greater Nashua", :branch_id=>"485", :call_type=>nil, :client_id=>"2264", :client_name=>"YMCA of Greater Nashua", :date_of_birth=>nil, :email=>"a253545#gmail.com", :first_name=>"Sridhar", :last_name=>"Tipirneni", :list_entity_id=>"277795", :mem_id=>"4085008", :mem_unit_id=>"2138728", :member_id=>"0213262-01", :membership_type=>"Dual 2 Adult Family", :membership_type_id=>"5203", :most_recent_join_date=>nil, :old_membership_type=>nil, :phone_number=>"(970)456-1010", :primary_language=>"English", :termination_date=>nil, :termination_reason=>nil, :unit_id=>"0213262", :unit_type=>nil, :visits=>nil, :"#i:type"=>"c:NpsListEntityDto"}
email_contact = EmailContact.new(list_entity.except(:"#i:type"))
puts email_contact.valid?
This returns false. The only validation, at all, is the email. Why does this email validate successfully in my console but fail in my rake task?
FYI, when I remove :email => true from my EmailContact model and only validate the presence of an :email, they both work fine. So the issue is definitely within the :email => true piece of my validation, but I don't understand why it passes in one place and fails in another.
EDIT
In my console, my model looks like this when using the full list_entity:
#<EmailContact id: nil, branch: "Nashua Branch-YMCA of Greater Nashua", branch_id: 485, call_type: nil, client_id: 2264, client_name: "YMCA of Greater Nashua", date_of_birth: nil, email: "a253545#gmail.com", first_name: "Sridhar", last_name: "Tipirneni", list_entity_id: 277795, mem_id: "4085008", mem_unit_id: "2138728", member_id: "0213262-01", membership_type: "Dual 2 Adult Family", membership_type_id: 5203, most_recent_join_date: nil, old_membership_type: nil, phone_number: "(970)456-1010", primary_language: "English", termination_date: nil, termination_reason: nil, unit_id: "0213262", visits: nil, loaded_at: nil, failed_at: nil, unit_type: nil, created_at: nil, updated_at: nil, list_id: nil>
In my rake task, when I run email_contact.inspect, this is returned:
#<EmailContact id: nil, branch: "Nashua Branch-YMCA of Greater Nashua", branch_id: 485, call_type: nil, client_id: 2264, client_name: "YMCA of Greater Nashua", date_of_birth: nil, email: "a253545#gmail.com", first_name: "Sridhar", last_name: "Tipirneni", list_entity_id: 277795, mem_id: "4085008", mem_unit_id: "2138728", member_id: "0213262-01", membership_type: "Dual 2 Adult Family", membership_type_id: 5203, most_recent_join_date: nil, old_membership_type: nil, phone_number: "(970)456-1010", primary_language: "English", termination_date: nil, termination_reason: nil, unit_id: "0213262", visits: nil, loaded_at: nil, failed_at: nil, unit_type: nil, created_at: nil, updated_at: nil, list_id: nil>
As you can see, they are both the exact same - The console model is valid, the rake model is invalid.
EDIT 2
I am using the valid_email gem, mentioned above. Here is the filepath:
/Users/luigi/.rvm/gems/ruby-2.0.0-p247#hub/gems/valid_email-0.0.4/lib/valid_email/email_validator.rb
All of my other gems are stored here as well it seems like.
It may also be worth mentioning that I get this warning before the validation fails:
[deprecated] I18n.enforce_available_locales will default to true in
the future. If you really want to skip validation of your locale you
can set I18n.enforce_available_locales = false to avoid this message.
20 hours later, I found the issue.
Using savon, all of the strings returned in my hash were being converted to a datatype of Nori::StringWithAttributes. The encoding was the same (UTF-8), but the class was different.
Running email_contact.email = email_contact.email.to_s prior to checking if the model is valid solves the issue.
I really can't explain this behavior, notice how after I make a request the class' object id has changed, and therefore my is_a? evaluation returns false.
any ideas? I'm not even sure how to debug this. Also, this isn't related to making a request from the command line. The same behavior is exhibited on the web server as well, it's just easier to explain from the command line.
staging$ RAILS_ENV=staging script/console
Loading staging environment (Rails 2.3.2)
>> c = CartItem.new
=> #<CartItem id: nil, order_id: nil, order_source: nil, date: nil, user_id: nil, created_at: nil, updated_at: nil, paid: nil, payment_id: nil, values: nil, cart_description: nil, type: nil, price: nil, email: nil, error: nil>
>> c.class.object_id
=> 70151495336400
>> CartItem.object_id
=> 70151495336400
>> c.is_a? CartItem
=> true
>> app = ActionController::Integration::Session.new
=> #<ActionController::Integration::Session:0x7f9ad5c55db0 .... >
>> app.get("site/favorite")
=> 200
>> c.class.object_id
=> 70151495336400
>> CartItem.object_id
=> 70151496019760
>> c.is_a? CartItem
=> false
>> c.class
=> CartItem(id: integer, order_id: string, order_source: string, date: date, user_id: integer, created_at: datetime, updated_at: datetime, paid: boolean, payment_id: integer, values: text, cart_description: string, type: string, price: integer, email: string, error: string)
Every new context (request/response cycle) will regenerate the object IDs. You may want to use responds_to?, instead of is_a?.
Since the ActionController::Integration module is used for integration testing, getting a url reloads your classes, therefore redefining the CartItem identifier. You now basically have two CartItem classes, one hanging around on your stack without an identifier pointing to it anymore (the "old one") and one referenced by the CartItem identifier.