When i am trying to upgrade Rails to 2.3.16 from 2.3.15, the ActiveSupport::JSON.decode(response.body) fails to handle NaN.
Am getting an error like: invalid character at "NaN,...
Does anyone know how to fix it?
ActiveSupport::JSON.decode uses MultiJson under the hood which tends to ignore any passed in options (At least in Rails 3, not sure about 2.3). This means you can't pass in the usually accepted option of allow_nan.
However, if you are using Ruby 1.9+ you can use the built-in JSON parser:
require 'json'
json_result = JSON.parse(response.body, allow_nan: true)
Related
I would like to decode a json string with a \u0022 character in it.
I am successful with:
>> ActiveSupport::JSON.decode("{\"json\":{\"difficulty\":1}}")
=> {"json"=>{"difficulty"=>1}}
But fail with:
>> ActiveSupport::JSON.decode("{\"json\":{\"difficulty\":\"test\\u0022test\"}}")
StandardError: Invalid JSON string
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/backends/yaml.rb:14:in `decode'
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/decoding.rb:14:in `__send__'
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/decoding.rb:14:in `decode'
from (irb):11
I would love to replace the \u0022 character with another one, but I can't do it because it's inside the parameters parsing process of rails when the app receives a request; except if I override the json decode chore method, which I would prefer to avoid.
FYI : I'm on Ruby 1.8.7-p358 & Rails 2.3.15 & I can't change that.
I think this has to do with Syck, the old YAML library used in (very) old versions of Ruby. What does this have to do with YAML? In ActiveSupport 2.3.15, JSON decoding uses YAML.load to parse JSON, because JSON happens to be a subset of YAML.
A quick aside: In ActiveSupport 2.3.18, ActiveSupport::JSON::Backends::Yaml no longer uses the YAML backend decode YAML, and ActiveSupport::JSON::Backends::Yaml#decode looks like this:
def decode(json)
raise "The Yaml backend has been deprecated due to security risks, you should set ActiveSupport::JSON.backend = 'OkJson'"
end
There are very serious consequences to using outdated versions of Ruby and Rails. Upgrade to versions that are still supported by security releases, or you will be sorry. Seriously.
Anyway, let's take a closer look at your data. As I suspect you know, the UTF-8 character U+0022 is a double-quote. All of the escaping makes things confusing, so let's see what string we actually have:
str = "{\"json\":{\"difficulty\":\"test\\u0022test\"}}"
puts str
# => {"json":{"difficulty":"test\u0022test"}}
FWIW it's more readable to use one of Ruby's alternate string syntaxes here, e.g.:
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
puts str
# => {"json":{"difficulty":"test\u0022test"}}
Either way, that's a literal \ followed by u0022. So far so good. Now let's try to parse it with Syck (I'm using Ruby 2.2, but it doesn't make a difference in this case):
require "syck"
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
Syck.load(str)
# => .../syck.rb:145:in `load': syntax error on line 0, col 38: `# Transforms the subclass name found i}}' (ArgumentError)
I don't know why Syck's error messages are so weird, but anyway we can see that Syck falls over trying to parse this JSON. Now let's try it with a modern YAML parser:
require "psych"
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
Psych.load(str)
# => {"json":{"difficulty":"test\u0022test"}}
Works great!
So you have a few options.
You could upgrade to a version of Ruby that's still maintained, doesn't have security holes, and has a YAML parser that isn't broken.
You could upgrade to a version of Rails that's still maintained, doesn't have security holes, and doesn't use a broken YAML parser to parse JSON.
You could install Psych, which is available as a gem for older versions of Ruby.
anyone seen this array error before?
I have a helper method that returns an array. In development mode on my laptop it returns the array in an expected format:
var fire =
[[1349083353000, 8.860000000000582], [1349085153000, 19.779999999999745],
[1349086953000, 20.289999999999964], [1349088753000, 29.850000000000364],
[1349090553000, 3.7999999999992724]];
BUT same code in production returns a strange array format:
var fire = 135175422800015.5135175602800020.0135175782800018.99135175962800012.33135176142800019.13135176322800029.55135176502800020.13135176682800077.34
I have tried checking the output in rails console on either machine and the production output the same weird array format. I have created a new array from within rails console on production and it works as expected to output the correct format of array.
Anyone seen this bit of weirdness?
Rails version:3.2.8
Ruby Version:1.9.3p-125
You're probably developing on Ruby 1.9 and deploying on Ruby 1.8. The default behaviors for treating arrays are different.
In Ruby 1.8 array.to_s is equivalent to array.join('').
In Ruby 1.9 array.to_s is equivalent to array.inspect.
If you want the proper behavior on both, and you're using JavaScript, you might want to render it as JSON using array.to_json instead.
I have the following structure :
a = { 'x' => [1,2,3] }
In Rails 3.0 with ruby 1.8, the conversion to JSON works as expected :
a.to_json
=> "{\"x\":[1,2,3]}"
In Rails 3.1 with ruby 1.9.3p125, the array is transformed into a hash :
a.to_json
=> "{\"x\":{\"1\":null,\"2\":null,\"3\":null}}"
I can't find any documentation explaining the difference in behaviour between the two versions, nor any way to preserve the expected output in Rails 3.1. Any clues ?
The problem is actually related to one of the gems installed on the Gemfile, which returns an incorrect result for to_json calls.
Thanks for the help.
I'm trying to roundtrip encode/decode plain strings in json, but I'm getting an error.
In rails 2.3. w/ ruby 1.8.6, it used to work.
>> puts ActiveSupport::JSON.decode("abc".to_json)
abc
=> nil
In rails 3.1beta1 w/ ruby 1.9.2, it raises an error.
ruby-1.9.2-p180 :001 > puts ActiveSupport::JSON.decode("abc".to_json)
MultiJson::DecodeError: 706: unexpected token at '"abc"'
from /home/stevenh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/json/common.rb:147:in `parse'
from /home/stevenh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/json/common.rb:147:in `parse'
from /home/stevenh/.rvm/gems/ruby-1.9.2-p180/gems/multi_json-1.0.1/lib/multi_json/engines/json_gem.rb:13:in `decode'
[...]
This is pretty much the same question discussed at nil.to_json cannot be parsed back to nil?
But nil used to work in 2.3/1.8.7 as well.
puts ActiveSupport::JSON.decode(nil.to_json)
nil
Is this the new normal?
This change occurred with the switch from ActiveSupport's JSON backend to MultiJson that was included in Rails 3.1.0.rc1. Per the MultiJson project team, the current behavior is correct and the previous implementation was faulty due to RFC4627's specification of the JSON grammar:
2. JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array
As neither "abc" nor "/"abc/"" are serialized objects or arrays, an error when attempting to decode them is appropriate.
The diagrams from the JSON website reinforce this specification.
That being said, this would seem to imply a bug in the to_json implementation that results in:
ruby-1.9.2-p180 :001 > "abc".to_json
=> "\"abc\""
Yes, what is happening in Rails3 is the new normal. The changes you illustrate seem like a reflection of a maturing framework.
Methods named "encode" & "decode" should be expected to be perfectly compliant with the JSON spec, and inverses of one another.
String#to_json, on the other hand is a behavior-ish type of method that functions as a convenience for building more complex JSON objects presumably used internally (within ActiveSupport) when Array#to_json or Hash#to_json encounter a String value as one of their consituent elements.
If you need to restore that behavior follow these steps, i.e.
# in your Gemfile
gem 'yajl-ruby'
# in your application.rb
require 'yajl/json_gem'
After those steps:
Loading development environment (Rails 3.2.8)
[1] pry(main)> puts ActiveSupport::JSON.decode("abc".to_json)
abc
=> nil
[2] pry(main)> puts ActiveSupport::JSON.decode(nil.to_json)
=> nil
I'm trying to port over some of my old rails apps to Ruby 1.9 and I keep getting warnings about how "Ruby 1.9 doesn't support Unicode normalization yet." I've tracked it down to this function, but I'm getting about 20 warning messages per request:
rails-2.3.5/activesupport/lib/active_support/inflector.rb
def transliterate(string)
warn "Ruby 1.9 doesn't support Unicode normalization yet"
string.dup
end
Any ideas how I should start tracking these down and resolving it?
If you are aware of the consequences, i.e. accented characters will not be transliterated in Ruby 1.9.1 + Rails 2.3.x, place this in config/initializers to silence the warning:
# http://stackoverflow.com/questions/2135247/ruby-1-9-doesnt-support-unicode-normalization-yet
module ActiveSupport
module Inflector
# Calling String#parameterize prints a warning under Ruby 1.9,
# even if the data in the string doesn't need transliterating.
if Rails.version =~ /^2\.3/
undef_method :transliterate
def transliterate(string)
string.dup
end
end
end
end
Rails 3 does indeed solve this issue, so a more future-proof solution would be to migrate towards that.
The StringEx Gem seems to work pretty well. It has no dependency on Iconv either.
It adds some methods to the String class, like "to_ascii" which does beautiful transliteration out of the box:
require 'stringex'
"äöüÄÖÜßë".to_ascii #=> "aouAOUsse"
Also, the Babosa Gem does a great job transliterating UTF-8 strings, even with language support:
"Jürgen Müller".to_slug.transliterate.to_s #=> "Jurgen Muller"
"Jürgen Müller".to_slug.transliterate(:german).to_s #=> "Juergen Mueller"
Enjoy.
That method definition is wrapped in an if-statement for Ruby 1.9. Right above it, you will find the regular definition, which shows a bit more of what this is doing. It's a method used to convert accented characters into their regular variants. E.g.: á => a, or ë => e
But this method is only used in parameterize, which is in turn defined right above transliterate. This is all still in ActiveSupport. I can't find anything that is directly calling parameterize.
So perhaps you're using parameterize or transliterate yourself, somewhere in your Rails application?
Common usage (according to the parameterize documentation) is for creating friendly permalinks from arbitrary strings, much like SO does, for example:
http://stackoverflow.com/questions/2135247/ruby-1-9-doesnt-support-unicode-normalization-yet
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Replace the body of the method with
raise "transliterate called"
and observe a backtrace which will show you where the stuff is coming from at the first call. Your app will of course collapse as well but that will likely give you the culprit from the first try.
I appreciate that this is a dirty way to solve the problem, but having read the error message I'm aware of the issue. So I want to get rid of the warnings. I dropped this code in environment.rb:
module ActiveSupport
module Inflector
# Calling String#parameterize prints a warning under Ruby 1.9,
# even if the data in the string doesn't need transliterating.
# Maybe Rails 3 will have fixed it...?
if RAILS_GEM_VERSION =~ /^2\.3/
undef_method :transliterate
def transliterate(string)
string.dup
end
end
end
end
If you'd rather not monkey patch the Inflector module, you can also do this...
Both of the following worked for me to silence this annoying "Ruby 1.9 doesn't support Unicode normalization yet" warning:
silence_stream(STDERR) {
whatever_code_caused_transliterate_to_be_called
}
or
silence_warnings {
whatever_code_caused_transliterate_to_be_called
}
This does have the disadvantage that it requires cluttering up your calling code, but it is a technique you can use generally whenever you don't want to see warnings or other output.
activesupport provides silence_stream and silence_warnings in activesupport-2.3.11/lib/active_support/core_ext/kernel/reporting.rb
String#unicode_normalize, String#unicode_normalize!, String#unicode_normalized? will be introduced in Ruby 2.2. Sample code and implementation can be seen in test case, lib/unicode_normalize.rb and lib/unicode_normalize/normalize.rb.
// U+00E1: LATIN SMALL LETTER A WITH ACUTE
// U+U+0301: COMBINING ACUTE ACCENT
puts "\u00E1" == "a\u0301".unicode_normalize(:nfc)
puts true == "a".unicode_normalized?(:nfc)