YAML dump inconsistent string encoding - ruby-on-rails

Ruby version: 2.2.5
Rails version: '~> 4.0.13'
Why I and my friend have received different result when trying to convert a string to YAML using YAML.dump method?
[1] pry(main)> YAML.dump("foo")
=> "--- foo\n"
[2] pry(main)> "foo".to_yaml
=> "--- foo\n"
While he has:
[1] pry(main)> YAML.dump("foo")
=> "--- foo\n...\n"
[2] pry(main)> "foo".to_yaml
=> "--- foo\n...\n"
With triple dots after new line (...)
UPDATED
I have confirmed that my rails is using Psych as YAML parser engine, in rails console:
2.2.5 :002 > YAML
=> Psych
2.2.5 :004 > Psych.dump("foo")
=> "--- foo\n"
2.2.5 :005 > YAML.dump("foo")
=> "--- foo\n"
2.2.5 :006 >
But still the result is somehow different. For additional informations, I don't have any syck gem installed and not required it in any files in my rails project.

It appears "You" might be using syck as a YAML processor while "He" is using psych. E.g.
require 'syck'
require 'psych'
Syck.dump("foo")
#=> "--- foo\n"
Psych.dump("foo")
#=> "--- foo\n...\n"
Both are valid YAML parser/emitters per se although Psych has been preferred since 1.9 and Syck really only exists as a gem for backwards compatibility and was completely removed from ruby standard lib as of 2.0.0
As for why "You" are somehow using Syck instead I cannot say without far more information than was provided in this post.

I met this issue this week and spent a great deal of time investigating the root cause. I have found that the difference of libyaml in the system will affect the dump.
For libyaml version 0.1.7, it will always include the ending characters like that, that's why you have the extra "...\n".
For libyaml version 0.2.1 and above, it supports implicit ending so those characters can be ignored

Related

Unicode issues in rails console, but not in pry or irb

I've got content â\u0080\u0099 in some database columns I'm trying to find and replace with ActiveRecord, but Rails runtime and console appear to have trouble with that string, misinterpreting the first character. I see this in the Rails console, but not in pry or "plain" irb.
When I paste that string into the Rails console, it's converted to \U+FFC3\U+FFA2\, then ignored in the resulting string (see below).
I'd like to understand:
what makes the Rails console different from irb/pry concerning encoding and how I may make it behave similarly regarding encoding
how Rails was able to insert that data in the database in the first place (I'm having issues creating a unit test for this scenario for what I think are the same reasons as the console)
I'm on OS X Mojave using rbenv. I've tried readline and environment variable solutions with no change.
bin/rails console
Running via Spring preloader in process 99194
Loading development environment (Rails 5.2.2)
Cannot read termcap database;
using dumb terminal settings.
irb(main):001:0> RUBY_VERSION
=> "2.5.3"
irb(main):002:0> Encoding::default_internal
=> #<Encoding:UTF-8>
irb(main):004:0> s = '\U+FFC3\U+FFA2\u0080\u0099'
=> "\\u0080\\u0099"
irb(main):005:0> s = "\U+FFC3\U+FFA2\u0080\u0099"
=> "\u0080\u0099"
irb
irb(main):018:0> RUBY_VERSION
=> "2.5.3"
irb(main):019:0> s = 'â\u0080\u0099'
=> "â\\u0080\\u0099"
pry
$ gem list | grep pry
pry (0.12.2)
$ pry
[1] pry(main)> RUBY_VERSION
=> "2.5.3"
[2] pry(main)> s = "â\u0080\u0099"
=> "â\u0080\u0099"
Etc.
I noticed that Encoding::default_internal resolves to #<Encoding:UTF-8> in the Rails console, but to nil in irb and pry. Setting it to nil in the Rails console also did not solve my issue.
Ultimately these strings display like St. Patrickâs Day in the browser

Psych can't parse simple key: value pair, Ruby, Yaml

Running in to some parsing problems with ruby 1.9.2-p290 and rails 3.1.3.
My YAML file looks like this:
api_key: 12345
The other YAML files parse fine, like the database.yml and locale files. It's just this one.
Any ideas as to why?
there must be an error somwhere else, cause the line that you are showing is parsable by psych and syck engines:
YAML::ENGINE.yamler = 'psych'
YAML.load("api_key: 12345") # => {"api_key"=>12345}
YAML::ENGINE.yamler = 'syck'
YAML.load("api_key: 12345") # => {"api_key"=>12345}

Ruby file is in search path but can't be found by 'require'

I have a Rails app that's trying to use the gnuplot gem, but Rails won't bother to load it. It's in my Gemfile and I installed it with bundle install. Bundle knows about it:
vp117025:src tim$ bundle show gnuplot
/Users/tim/.rvm/gems/ruby-1.9.2-p290/gems/gnuplot-2.3.6
But it doesn't appear in the Rails console (and isn't accessible from my app), even after an explicit require, which shouldn't be necessary.
vp117025:src tim$ rails console
Loading development environment (Rails 3.1.0)
ruby-1.9.2-p290 :001 > require 'gnuplot'
=> false
ruby-1.9.2-p290 :002 > require 'gnuplot.rb'
=> false
Now, check to see whether the name of the module is in the constants table:
ruby-1.9.2-p290 :003 > Module.constants.include? :Gnuplot
=> false
It isn't! The module must not really be accessible. This is happening even though the require search path contains the directory that holds gnuplot.rb.
ruby-1.9.2-p290 :004 > $:.include? "/Users/tim/.rvm/gems/ruby-1.9.2-p290/gems/gnuplot-2.3.6/lib"
=> true
If I include the file explicitly by its full path, it works!
ruby-1.9.2-p290 :005 > require "/Users/tim/.rvm/gems/ruby-1.9.2-p290/gems/gnuplot-2.3.6/lib/gnuplot.rb"
=> true
ruby-1.9.2-p290 :006 > Module.constants.include? :Gnuplot
=> true
The module is now visible to the interpreter. It works just fine from outside Rails.
vp117025:src tim$ irb -rubygems -r gnuplot
ruby-1.9.2-p290 :001 > Module.constants.include? :Gnuplot
=> true
Why can't the Rails environment load gnuplot.rb, even though its parent directory is in the search path?
It is loading. The return value of a require statement will be false if the library was already loaded. When you start the Rails console, it requires all the gems in your bundle, so requiring it again will give you false as you see here. Require treats every path passed to it fairly literally, so you can require a file by both its relative and absolute paths, and it will load both times. That's why you get a true result when using the full path – Ruby is treating it as a different file.

Why would yaml be unavailable in production?

On my OS X development system:
$ ruby --version
ruby 1.8.6 (2007-03-13 patchlevel 0) [universal-darwin8.0]
$ script/console
Loading development environment (Rails 2.3.4)
>> require 'yaml'
=> []
On CentOS 5.3 production system:
$ script/console production
Loading production environment (Rails 2.3.4)
/opt/ruby-enterprise-1.8.7-2009.10/lib/ruby/gems/1.8/gems/activerecord-2.3.4/lib/active_record/base.rb:1959:in `method_missing':NoMethodError: undefined method `define_index' for #<Class:0x2e6f7b0>
>> require 'yaml'
=> false
Anything I can do about that NoMethodError?
Why would yaml be unavailable. Isn't it part of the core Ruby libraries?
Returning false means that the loading was successful or was already done. If it couldn't be loaded, it'd raise an exception instead.
It is part of the Ruby core libraries, but I'm not sure what gets bundled up with Ruby Enterprise Edition. Check that yaml.rb is inside your $LOAD_PATH. For example, try this in irb:
$LOAD_PATH.collect { |path| File.join(path, 'yaml.rb') }.find { |path| File.exist?(path) }
On OS X it will produce something like:
=> "/opt/local/lib/ruby/1.8/yaml.rb"

Getting Rails to play with Hpricot

I'm trying to get Hpricot working with Rails on my dev machine.
I've installed Hpricot [0.8.1] using the standard 'gem install hpricot' and confirmed it works fine with my standard Ruby installation [1.8.7]; however when I try the same with my Rails [2.1.0] installation, I get an error -
TypeError: superclass mismatch for class BogusETag from /usr/lib/ruby/1.8/hpricot/tag.rb:130
Seems like there's some kind of conflict, but googling the error hasn't turned up any useful information.
Any ideas ? Thanks in advance.
If you're free to choose your HTML parsing library, switch it.
Why, the creator of Hpricot, recently posted that you should better use Nokogiri instead of HPricot, nowadays.
This error occurs because there is no compiled library for the platform. To solve this for your current hpricot version, go to your rails directory dir and do the following (this assumes you are using an unpacked gem - this problem wouldn't arrise otherwise, unless your OS has been upgraded since installing the gem):
cd vendor/gems/hpricot-0.6/ext/hpricot_scan/
ruby extconf.rb
make
Then copy the compiled library to the correct platform dir for your system. Each version of OS X has a slightly different platform name, so mine (Snow Leopard) appears as:
ruby-1.8.6-p383 > RUBY_PLATFORM
=> "i686-darwin10.0.0"
This means I needed to copy the file "hpricot_scan.bundle" (OS X uses bundles for shared libraries, on Linux that would be "hpricot_scan.so") to the following directory, creating it if necessary:
vendor/gems/hpricot-0.6/lib/i686-darwin10.0.0/
Substitute the appropriate platform name (puts RUBY_PLATFORM from irb) for your machine.
Try using Hpricot in a irb session, because i don't think this should generally happen
irb(main):001:0> require 'rubygems'
=> true
irb(main):002:0> require 'hpricot'
=> true
irb(main):003:0> doc=Hpricot.parse("<html><head><title>test</title></head><body> Wooo Hooo </body></html>")
=> # {elem {elem "test" } } {elem " Wooo Hooo " } }>
irb(main):004:0> doc.search('title')
=> # "test" }]>
irb(main):005:0> doc.search('title').text
=> "test"
That means your Hpricot is working fine. And the problem is in way you are using it in Rails. ( It would be easier if you could paste your code ).
I had few issues with Hpricot(0.8.1) earlier, had to switch back to 0.6.164 version. you can try that if you want.
HTH
Execute Below command and its works
gem install hpricot --platform=mswin32

Resources