Rails: 3.0.3
Ruby: 1.9.2
Trying to deserialize a very simple object using YAML.load or Marshal.load produces a corrupted object because the class which belongs to is not required on the deserializing process.
Example:
# app/models/my_model.rb
class MyModel
attr_accessor :id
end
# test/unit/serializing_test.rb
require 'test_helper'
class SerializingTest < Test::Unit::TestCase
def test_yaml_serialize_structure
my_model = MyModel.new
my_model.id = 'my model'
File.open( "#{Rails.root}/tmp/object.yml" , 'w' ) do |f|
YAML::dump(my_model, f)
end
end
def test_yaml_deserialize_structure
object = YAML.load_file "#{Rails.root}/tmp/object.yml"
assert( object.instance_of? MyModel )
assert_equal( 'my model', object.id )
end
end
With this code we can run this shell console session without any error:
$ ruby -Itest test/unit/serializing_test.rb -n test_yaml_serialize_structure
$ ruby -Itest test/unit/serializing_test.rb -n test_yaml_deserialize_structure
But if I run the deserialization calls from a Rails console the object is not deserialized properly because the class is never required:
$ rails c
ruby-1.9.2-p0 > object = YAML.load_file "#{Rails.root}/tmp/object.yml"
=> #<Syck::Object:0x0000010322ea30 #class="MyModel", #ivars={"id"=>"my model"}>
I know the only problem is that the class is not required because if I require it by hand everything works:
ruby-1.9.2-p0 > require "#{Rails.root}/app/models/my_model"
=> ["MyModel"]
ruby-1.9.2-p0 > object = YAML.load_file "#{Rails.root}/tmp/object.yml"
=> #<MyModel:0x0000010320c8e0 #id="my model">
I have presented only the YAML examples but with Marshal is pretty the same.
Also say that although I'm reproducing the problem in a Rails console originally this problem was turning me crazy in a normal request to my application.
So the question is: How can I deserialize objects in Rails without have to require all my classes by hand?
Thanks
f.
Well, after read #tadman and a bunch of answers I have received in the spanish ror mailing list [1] I have collected a few hot tips when you have to deal with Ruby deserializing and class loading in Rails:
Super fast solution
Use config.cache_classes = true in your development.rb but you will lost the class auto-refreshing.
Better solution
Require all the classes that are gonna be deserialized but not with require but with require_dependency[2] so in development environment the class auto-refreshing will remain working.
Elegant solution
Monkey-patch the YAML and the Marshal gem to tell them to call require_dependency when they find a non-defined class to deserialize.
And #Xavi has sent me a proposition of monkey-patch Marshal (he says he wrote it on the air and it is not tested so use it in your own risk) [3]
[1] http://lists.simplelogica.net/pipermail/ror-es/2011-January/024787.html
[2] http://apidock.com/rails/ActiveSupport/Dependencies/Loadable/require_dependency
[3] http://lists.simplelogica.net/pipermail/ror-es/2011-January/024796.html
I described this "issue" on GitHub: https://github.com/rails/rails/issues/1585
To automatically require classes on YAML loading in the manner #fguillen suggests is elegant, I wrote this short monkey-patch.
It simply attempts to require_dependency any class the Psych ToRuby class resolves to classes.
Works for me in a serialised Active Record that stores a custom class instance, YMMV.
module Psych::Visitors
ToRuby.class_eval do
alias :resolve_class_without_autoload :resolve_class
def resolve_class klassname
begin
require_dependency klassname.underscore
rescue NameError, LoadError
end
resolve_class_without_autoload klassname
end
end
end
I had to adapt #ben-patterson's answer a bit to make it work (using Rails 5.0.2):
module Psych::Visitors
ToRuby.class_eval do
def resolve_class(klassname)
begin
class_loader.load klassname
rescue ArgumentError
require_dependency klassname.underscore
klassname.constantize
end
end
end
end
As far as I know, both YAML and Marshal do not make use of the Rails autoloader. You must go ahead and pre-load any classes that might need to be deserialized.
It's a bit if a fuss, especially in the development environment where almost nothing is loaded before it is needed.
In rails version 7 the monkey patch has to catch Psych::DisallowedClass error instead of ArgumentError or NameError, LoadError as proposed by
#ben-patterson and #panzi
module Psych::Visitors
ToRuby.class_eval do
def resolve_class(klassname)
begin
class_loader.load klassname
rescue Psych::DisallowedClass => e
require_dependency klassname.underscore
klassname.constantize
end
end
end
end
Related
Why isn't the Ruby interpreter throwing a NameError in this instance here?
class OrangeTurtle
self.table_name = 'turtles'
end
Filename: orange_turtles.rb
This answer might sound like a cop out, but it doesn't throw an error because Ruby doesn't care even the slightest what your filenames are called.
e.g. in file asdfasdf.no_rb_ending_here we can have
#!/usr/bin/env ruby
module Something
class Test
def test
puts 'test'
end
end
end
class SomethingElse
def otherThings
puts 'haha'
end
end
Then to make things even weirder, I can have a separate file that modifies (monkey patches) the classes defined in that file.
in more_stuff.rb
#!/usr/bin/env ruby
require_relative 'asdfasdf.no_rb_ending_here'
module Something
class Test
def test2
test()
puts '2'
end
end
end
class SomethingElse
def moreThings
otherThings()
puts 'MOAR'
end
end
Something::Test.new.test2()
# test
# 2
SomethingElse.new.moreThings()
# haha
# MOAR
Ruby is pretty cool - you don't get errors for things that don't NEED to cause an error.
The name error, or uninitialized constant error only appears in Rails. The reason for that is, that active record (which is also a general design pattern) is mapping the tables in the database with the models (or with objects in general).
Active Record can only make that connection via the naming conventions for files and the classes.
As mentioned in the other answer, pure ruby doesn't need to comply with these conventions. However, it is a general rule to name the files like the classes they contain to have better organised code.
I have the following standard Rails ActiveRecord Foo defined:
# app/models/foo.rb
class Foo < ApplicationRecord
end
And I'm trying to call Foo.find(..) from within a hierarchy that contains a module also named Foo..
# lib/commands/bar.rb
module Commands
module Bar
module Create
class Command
def initialize(params)
...
Foo.find(params[:foo_id]
...
end
end
end
end
end
# lib/commands/foo.rb
module Commands
module Foo
module Create
class Command
...
end
end
end
end
Ruby/Rails is finding Commands::Foo instead of my Foo Model and throwing undefined method 'find' for Commands::Foo:Module.. how can I point at the correct ActiveModel implementation?
The obvious answer is to rename Commands::Foo.. to Commands::Foos.. but I'm curious to know if there's another way :o)
If you want to avoid the clash then you should rename the modules. The existing structure is unwieldy and will present similar problems to all future maintainers.
The best solution that I find in your code is to ensure you call the appropriate module and method via its full path:
2.3.3 :007 > ::Commands::Foo::Create::Command.new
"Commands::Foo::Command reached"
=> #<Commands::Foo::Create::Command:0x007ffa1b05e2f0>
2.3.3 :008 > ::Commands::Bar::Create::Command.new
"Commands::Bar::Command reached"
=> #<Commands::Bar::Create::Command:0x007ffa1b04f110>
You shouldn't try to override or modify internal Rails calls, because then you've modified the framework to fit code, which leads to unpredictable side effects.
You can try to call::Foo in Commands::Foo, it should go with your Foo model
This issue will surface for many who depend on Ruby's JSON serialization outside of a Rails projects. When they try to use their code in a Rails project, it will not work as expected.
The following code run from Ruby (no Rails), prints A.
When run from rails console, it prints Hash.
That means my json serialization works in my command line lib/app, but not when it's imported into a Rails project.
What is the reason/workaround for this?
require 'json'
class A
def to_json(*a)
{:json_class => self.class.name}.to_json(*a)
end
def self.json_create(o)
A.new
end
end
class B
attr_accessor :value
def initialize(value)
#value = value
end
def to_json(*a)
{:json_class => self.class.name, :value => value}.to_json(*a)
end
def self.json_create(o)
B.new(o['value'])
end
end
b = JSON.parse(B.new(A.new).to_json)
puts b.value.class
Ruby is 1.9.3, Rails is 3.2.10
The problem is that Rails uses ActiveSupport::JSON.
For serializing, it uses as_json, not to_json. So the line
{:json_class => self.class.name, :value => value}.to_json(*a)
does not include a JSON version of value in the hash because Class A does not have a as_json method. To get your code to work the same in both Ruby and Rails, you need to explicitly call your A::to_json and A::json_create methods, like this:
def to_json(*a)
{:json_class => self.class.name, :value => JSON.dump(value)}.to_json(*a)
end
def self.json_create(o)
B.new(A.json_create(o['value']))
end
Then call, b = JSON.parse(JSON.dump(B.new(A.new)))
This wlll fix the example, but I think you may want to read this explanation of to_json vs as_json and revise your code appropriately.
According to others, the answer is yes.
http://www.rubyhood.com/2011/06/rails-spoiled-standard-json-library.html
In short, make as_json do what to_json does. That got me what I wanted/expected (and what I've been getting from pure Ruby - Rails).
For those still wandering why the strange behavior is occurring in rails the explanation can be found in:
https://github.com/flori/json/compare/v1.6.7...v1.6.8
and
https://github.com/intridea/multi_json/compare/v1.5.0...v1.5.1
Since in these version upgrades JSON.parse works different. JSON.load might still be helpful. The fastest fix would be:
gem 'json', '1.6.7'
gem 'multi_json', '1.5.0'
but leave some security issues open. Explicitly supplying create_additions: true to JSON parse when needed is recommended.
I'm struggling to use Delayed_job (collective idea v2.0 in a Rails 2.3.8 app).
I'm calling the job from an application_controller method:
...
Delayed::Job.enqueue(S3MoverJob.new(docs))
Where docs is a Hash with ids and names of files.
At my Lib directory I have the class S3MoverJob:
class S3MoverJob < Struct.new(:docs)
def perform
#setup connection to Amazon
...
#connect
...
#loop to move files not transfered already
docs.each do |id,file_name|
# Move file
begin
doc = Document.find(id)
... perform actions
rescue Exception => exc
raise "failed!"
end
end
end
end
The problem is that it's raising: S3MoverJob failed with NoMethodError: You have a nil object when you didn't expect it!
I looked into the handler, in the DB, and it was delivering to the perform method the Yaml file with the list of ids and file names, like this:
docs:
3456: name_of_file_01.png
4567: name_of_file_02.txt
What am I missing? Thanks for helping me.
I think you should puts more information for debug.
Such as: add statements like 'Rails.logger.info "some stuff"' under the perform method to see where the exception been throw
My fault. I didn't know it was an idenpendent process and needed some requirements:
require 'yaml'
require 'uri'
class Document < ActiveRecord::Base
end
After that, everything works fine.
Thanks anyway.
Setting
In my gem there is a Component base class:
module Component
class Base
def self.inherited(component_class)
Component.class_eval do
define_method(component_class.to_s.underscore) do |*args, &block|
component_class.new(*args, &block)
end
end
end
end
end
For every class inheriting from this base class (e.g. FancyComponent < Component::Base), a helper should be defined in the Component module (e.g. fancy_component).
This does work for any components delivered with my gem, i.e. Component.instance_methods.include? :fancy_component #=> true
Now Rails comes into play
I want users of my gem to be able to add components. These are stored in app/components.
This folder is included in all of the following:
MyApp::Application.config.load_paths
MyApp::Application.config.autoload_paths
MyApp::Application.config.eager_load_paths
A new component UserComponent < Component::Baseis stored in app/components/user_component.rb.
The Problem
If I launch the rails console, the situation is as follows:
Loading development environment (Rails 3.0.4, ruby-1.9.2-p0)
Component.instance_methods.include? :fancy_component #=> true
Component.instance_methods.include? :user_component #=> false
UserComponent #=> UserComponent
Component.instance_methods.include? :user_component #=> true
So the helper method is not available until the component class is somehow accessed.
So how to force eager loading of that class so that inherited is executed?
Your idea is nice but I'd greatly advise you against implementing something like this, `cos this would usually be done by pre-loading the models before Rails takes notice of them and this could lead to hard to figure loading issues in your app (like classes that should have been re-loaded but were not).
Here's a basic example, of the ways for you to implement this feature would be, at your "root" gem file (if your gem is named "my_component", the "lib/my_component.rb" file) do something like this:
require 'component/base/component'
# require here all other classes necessary for your gem
rb_files = File.join( Rails.root, 'app', 'components', '**', '*.rb' )
Dir.glob( rb_files ).each do |file|
require file.gsub('.rb')
end
With this you'd load all files under "app/components" but then Rails would not reload these objects, as they were not required by Rails, but by your own gem. If you don't mind not being able to change these files, this might be ok, but then you could have issues in the development environment (or any environment that has "cache_classes" set to false).
This is a bug in rails (but wontfix), see also this ticket here:
https://github.com/rails/rails/issues/3364