I'm trying to roundtrip encode/decode plain strings in json, but I'm getting an error.
In rails 2.3. w/ ruby 1.8.6, it used to work.
>> puts ActiveSupport::JSON.decode("abc".to_json)
abc
=> nil
In rails 3.1beta1 w/ ruby 1.9.2, it raises an error.
ruby-1.9.2-p180 :001 > puts ActiveSupport::JSON.decode("abc".to_json)
MultiJson::DecodeError: 706: unexpected token at '"abc"'
from /home/stevenh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/json/common.rb:147:in `parse'
from /home/stevenh/.rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/json/common.rb:147:in `parse'
from /home/stevenh/.rvm/gems/ruby-1.9.2-p180/gems/multi_json-1.0.1/lib/multi_json/engines/json_gem.rb:13:in `decode'
[...]
This is pretty much the same question discussed at nil.to_json cannot be parsed back to nil?
But nil used to work in 2.3/1.8.7 as well.
puts ActiveSupport::JSON.decode(nil.to_json)
nil
Is this the new normal?
This change occurred with the switch from ActiveSupport's JSON backend to MultiJson that was included in Rails 3.1.0.rc1. Per the MultiJson project team, the current behavior is correct and the previous implementation was faulty due to RFC4627's specification of the JSON grammar:
2. JSON Grammar
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array
As neither "abc" nor "/"abc/"" are serialized objects or arrays, an error when attempting to decode them is appropriate.
The diagrams from the JSON website reinforce this specification.
That being said, this would seem to imply a bug in the to_json implementation that results in:
ruby-1.9.2-p180 :001 > "abc".to_json
=> "\"abc\""
Yes, what is happening in Rails3 is the new normal. The changes you illustrate seem like a reflection of a maturing framework.
Methods named "encode" & "decode" should be expected to be perfectly compliant with the JSON spec, and inverses of one another.
String#to_json, on the other hand is a behavior-ish type of method that functions as a convenience for building more complex JSON objects presumably used internally (within ActiveSupport) when Array#to_json or Hash#to_json encounter a String value as one of their consituent elements.
If you need to restore that behavior follow these steps, i.e.
# in your Gemfile
gem 'yajl-ruby'
# in your application.rb
require 'yajl/json_gem'
After those steps:
Loading development environment (Rails 3.2.8)
[1] pry(main)> puts ActiveSupport::JSON.decode("abc".to_json)
abc
=> nil
[2] pry(main)> puts ActiveSupport::JSON.decode(nil.to_json)
=> nil
Related
I've recently returned to a Rails side project that's been sitting on the shelf for some time and am in the process of upgrading the various components, including Ruby and Rails. The app runs on Heroku, recently upgraded from cedar-14 to the heroku-16 stack. I've upgraded from Rails 4.1.0 to 4.2.10 (need to improve test coverage before pushing to Rails 5), and from Ruby 2.1.0 to 2.3.4-p301, and things are mostly working well. Unfortunately I'm seeing some regex behavior that I can't figure out. I'm referencing ruby-doc.org for the expected behavior.
The content is unimportant, but as an example, say I'm trying to parse a URL and capture the protocol, which I've always done with the following code:
m = /(https?)/.match("https://www.example.com")
=> #<MatchData "https" 1:"https">
m[1]
=> "https"
This still works as expected when I run the code above in the Console, both locally on my dev machine and on Heroku. But in the application itself, both locally and on Heroku, the first statement now returns the string "https" instead of the MatchData object, leading to the following:
m = /(https?)/.match("https://www.example.com")
=> "https"
m[1]
=> "t"
In my searching and review of the documentation, I can't find any scenarios where a string is returned instead of a MatchData object.
Of course, I could just use the string, but that gives me pause for obvious reasons since I can't reconcile it with the docs. Has anyone seen this before? Maybe a bigger question is - can you suggest configs or other factors that might cause me to see different results in Rails C (expected behavior) vs. Rails S (unexpected behavior) on such straightforward code (doesn't touch database, etc.)?
Thanks so much for your help.
Check to see if the match method has been redefined. For example, in a clean unmodified irb console, I would expect:
regex = /(https?)/
url_str = "https://www.example.com"
p regex.match(url_str)
p url_str.match(regex)
# => #<MatchData "https" 1:"https">
# should return nil, meaning it's defined in C and not in ruby code
p regex.method(:match).source_location
# reproduces your situation
class Regexp
alias_method :old_match, :match
def match(str)
old_match(str).to_s
end
end
p regex.match(url_str)
p url_str.match(regex)
# => "https"
# this should reveal if it was changed in a way similar to above
p regex.method(:match).source_location
I would like to decode a json string with a \u0022 character in it.
I am successful with:
>> ActiveSupport::JSON.decode("{\"json\":{\"difficulty\":1}}")
=> {"json"=>{"difficulty"=>1}}
But fail with:
>> ActiveSupport::JSON.decode("{\"json\":{\"difficulty\":\"test\\u0022test\"}}")
StandardError: Invalid JSON string
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/backends/yaml.rb:14:in `decode'
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/decoding.rb:14:in `__send__'
from /home/.../.rbenv/versions/1.8.7-p358/lib/ruby/gems/1.8/gems/activesupport-2.3.15/lib/active_support/json/decoding.rb:14:in `decode'
from (irb):11
I would love to replace the \u0022 character with another one, but I can't do it because it's inside the parameters parsing process of rails when the app receives a request; except if I override the json decode chore method, which I would prefer to avoid.
FYI : I'm on Ruby 1.8.7-p358 & Rails 2.3.15 & I can't change that.
I think this has to do with Syck, the old YAML library used in (very) old versions of Ruby. What does this have to do with YAML? In ActiveSupport 2.3.15, JSON decoding uses YAML.load to parse JSON, because JSON happens to be a subset of YAML.
A quick aside: In ActiveSupport 2.3.18, ActiveSupport::JSON::Backends::Yaml no longer uses the YAML backend decode YAML, and ActiveSupport::JSON::Backends::Yaml#decode looks like this:
def decode(json)
raise "The Yaml backend has been deprecated due to security risks, you should set ActiveSupport::JSON.backend = 'OkJson'"
end
There are very serious consequences to using outdated versions of Ruby and Rails. Upgrade to versions that are still supported by security releases, or you will be sorry. Seriously.
Anyway, let's take a closer look at your data. As I suspect you know, the UTF-8 character U+0022 is a double-quote. All of the escaping makes things confusing, so let's see what string we actually have:
str = "{\"json\":{\"difficulty\":\"test\\u0022test\"}}"
puts str
# => {"json":{"difficulty":"test\u0022test"}}
FWIW it's more readable to use one of Ruby's alternate string syntaxes here, e.g.:
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
puts str
# => {"json":{"difficulty":"test\u0022test"}}
Either way, that's a literal \ followed by u0022. So far so good. Now let's try to parse it with Syck (I'm using Ruby 2.2, but it doesn't make a difference in this case):
require "syck"
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
Syck.load(str)
# => .../syck.rb:145:in `load': syntax error on line 0, col 38: `# Transforms the subclass name found i}}' (ArgumentError)
I don't know why Syck's error messages are so weird, but anyway we can see that Syck falls over trying to parse this JSON. Now let's try it with a modern YAML parser:
require "psych"
str = %Q[{"json":{"difficulty":"test\\u0022test"}}]
Psych.load(str)
# => {"json":{"difficulty":"test\u0022test"}}
Works great!
So you have a few options.
You could upgrade to a version of Ruby that's still maintained, doesn't have security holes, and has a YAML parser that isn't broken.
You could upgrade to a version of Rails that's still maintained, doesn't have security holes, and doesn't use a broken YAML parser to parse JSON.
You could install Psych, which is available as a gem for older versions of Ruby.
It appears that in Rails 3.2.21, saving a serialized hash fails to save a value that comes from one specifc NumberHelper, helper.number_with_delimiter
In a Rails 3.2 app, in model Foo I have:
serialize :stuff, Hash
In the console:
f = Foo.create
f.stuff = { a: "aaaa", b: 1111, c: helper.number_with_delimiter(123456) }
=> {:a=>"aaaa", :b=>1111, :c=>"123,456"} # so far so good
f.save!
f.stuff
=> {:a=>"aaaa", :b=>1111, :c=>123456} # c should be a STRING
It DOES work correctly with helper.number_to_currency().
And it works if I set c: String.new(helper.number_with_delimiter(123456)).
This is a Rails bug, or am I doing something wrong?
Yes, this is a Rails (ActiveSupport) bug that was eventually fixed in Rails 4.2.1. From the 4.2.1 release notes:
Fixed a roundtrip problem with AS::SafeBuffer where primitive-like strings will be dumped as primitives
When you use helper.number_with_delimiter, the resulting object looks and behaves like a String, but in reality it is an ActiveSupport::SafeBuffer.
helper.number_with_delimiter(123456).class # => ActiveSupport::SafeBuffer < String
When you use:
serialize :stuff, Hash
That means behind the scenes Rails is using YAML format to save the data to the database. There was a bug in SafeBuffer that caused SafeBuffers like "123" to be mistakenly converted to integers (i.e. 123) instead of remaining strings when saving and loading to/from YAML.
Again, this is now fixed as of Rails 4.2.1. You can see the fix here:
https://github.com/rails/rails/commit/debe7aedda3665702d1f99a3ffb4a123a6c44e9c
When i am trying to upgrade Rails to 2.3.16 from 2.3.15, the ActiveSupport::JSON.decode(response.body) fails to handle NaN.
Am getting an error like: invalid character at "NaN,...
Does anyone know how to fix it?
ActiveSupport::JSON.decode uses MultiJson under the hood which tends to ignore any passed in options (At least in Rails 3, not sure about 2.3). This means you can't pass in the usually accepted option of allow_nan.
However, if you are using Ruby 1.9+ you can use the built-in JSON parser:
require 'json'
json_result = JSON.parse(response.body, allow_nan: true)
anyone seen this array error before?
I have a helper method that returns an array. In development mode on my laptop it returns the array in an expected format:
var fire =
[[1349083353000, 8.860000000000582], [1349085153000, 19.779999999999745],
[1349086953000, 20.289999999999964], [1349088753000, 29.850000000000364],
[1349090553000, 3.7999999999992724]];
BUT same code in production returns a strange array format:
var fire = 135175422800015.5135175602800020.0135175782800018.99135175962800012.33135176142800019.13135176322800029.55135176502800020.13135176682800077.34
I have tried checking the output in rails console on either machine and the production output the same weird array format. I have created a new array from within rails console on production and it works as expected to output the correct format of array.
Anyone seen this bit of weirdness?
Rails version:3.2.8
Ruby Version:1.9.3p-125
You're probably developing on Ruby 1.9 and deploying on Ruby 1.8. The default behaviors for treating arrays are different.
In Ruby 1.8 array.to_s is equivalent to array.join('').
In Ruby 1.9 array.to_s is equivalent to array.inspect.
If you want the proper behavior on both, and you're using JavaScript, you might want to render it as JSON using array.to_json instead.