I'm having a hard time believing what I'm seeing, but it sure looks like DJ is failing to deserialize an object properly. I look at the DJ record in mongo and I see in the YAML that the object has its text field set, but when the code runs, the text field is not set. Here is some minimal repro code:
class Board
include Mongoid::Document
field :text, type: String
def process_text_field
if not self.text
raise "Text field is blank"
end
# Text field gets processed
end
end
# in a controller
def start_doing_something_slow
board = Board.find(params[:id])
board.text = "Text field is set"
board.save!
raise "Text disappeared!" unless board.text
board.delay.process_text_field
render json: {:result=>'ok'}
end
I invoke the controller method with the browser, and check the DJ record directly in mongo. I see in the YAML that the Board object has the text field correctly set. But when it executes in DJ, it raises the Text field is blank exception.
Somehow it's not deserializing the object properly.
Well this took me about a week to figure out, so I'm posting it here to help others who fall into this trap. Turns out this is a known bug in delayed_job_mongoid. And it's had a simple fix listed right there in the bug report for 10 months.
The problem arises if you use the identity map in mongoid, which acts as an in-process caching layer to the database. For normal web requests, the cache gets cleared between each request, so your controller methods don't use stale versions of the objects. But delayed_job_mongoid doesn't clear the cache between jobs without this patch (which I just put together): https://github.com/collectiveidea/delayed_job_mongoid/pull/38
The result is your delayed jobs are sometimes using old versions of the objects, depending on what ran before them, which creates truly bizarre, mysterious failures that are extremely difficult to track down until you understand what's happening.
Related
I have an existing application (Rails 6) with a set of tests (minitest). I've just converted my tests to use factory_bot instead of fixtures but I'm having a strange problem with records created and confirmed in the test and controller being unavailable in a PORO that does the actual work. This problem occurs inconsistently and never seems to happen when I run an individual test, only when tests are run in bulk (e.g. a whole file or directory). The more tests are run, the more likely the failure, it seems.
(NB I've never seen this code fail in actual use - it only happens during tests.)
Summary
Previously, when using fixtures, every test ran successfully both individually and when run all together with rails t. Now, with factory_bot, a few of my tests often (but not always) fail, all related to the use of the same object that is defined as a PORO.
Drilling down, I have found that there's an issue with records sometimes mysteriously going missing or being unavailable within the PORO during the test, even though they're confirmed as present in the test and in the controller that calls the PORO!
Details
In my application, I have a RichText object that receives some text and processes it, highlighting words in the text that match those stored in a Dictionary table. In my tests, I create several test Dictionary records, and expect the RichText object to perform appropriately when passed test data. And it does, when the individual test files are run (and always did when I used fixtures).
However, now, the records are created and available in the test and in the controller it calls, but then are not available within the RichText object created by the controller. With no Dictionary records available in the RichText object, the test naturally fails because no words are highlighted in the text. And, again, this only seems to happen when I run the tests as a group rather than running just a single test file (e.g. rails t test/objects/rich_text.rb passes, but rails t test/objects will fail within the same rich_text.rb test file).
It doesn't seem to matter whether I create the records with factory_bot#create or directly with Dictionary#create, which suggests it's nothing to do with factory_bot - but then why has this just started happening?
I do have parallelisation enabled in minitest but disabling it makes no difference - the tests still fail the same way.
Code
Example test code that runs and passes, up to the last assertion here, which sometimes fails as described above:
test 'can create new content' do
create(:dictionary, word_root: 'word_1')
create(:dictionary, word_root: 'word_2')
create(:dictionary, word_root: 'word_3')
assert_equal 3, Dictionary.all.count
...
# This next line is the one that calls the relevant controller code below
post '/api/v0/content', headers: #auth_headers, params: #new_content_params
...
# This assertion passes, as it did above, even though the error's already happened after the post above
assert_equal 3, Dictionary.all.count
# This assertion checks the response from the above post and fails under certain circumstances, as described above
assert_equal #new_content_output, response.body
...
end
I've added checks to the controller as below and, again, everything's fine through this code, which is called by the post line in the test above (i.e. the database records are present and correct just before the RichText object is called):
def create
...
byebug unless Dictionary.all.count == 3
rich_text = RichText::Basic.new(#organisation, new_version[:content])
...
end
However, the RichText object's initialize method immediately fails the same check for these records - but only if the test is being run in bulk rather than individually:
class RichText::Basic
def initialize(organisation, text)
byebug unless Dictionary.all.count == 3
...
end
end
Rails 6.1.4, ruby 2.7.1
Having tried various things (like disabling transactions in the affected tests), I found that the root cause was a constant defined in the RichText class (a line I didn't include in the question!). It looks like there was a race condition or similar that meant that the RichText class sometimes ran before the database was populated, leaving it with an empty constant.
Replacing the constant with a direct database call resolved the problem. It does mean slightly more database calls but, on the flip side, does mean it's slightly easier to update the Dictionary table. (This happens rarely - on the order of once a month - which is why I'd put it into a constant.)
From:
class RichTest::Basic
WORDS = Dictionary.standard
def some_method
WORDS.each do...
end
end
to
class RichTest::Basic
def some_method
Dictionary.standard.each do...
end
end
I'm just trying to use ReactRB with reactive-record.
So the deal is in render part I think. When I'm setting param :user, type: User in React Component class, I can't see any data in my table. Of course Model User in public folder, as this requirement in ReactRB.
Well, in console I see that server is fetching nothing, but right data returned.
What I'm missing? Thanks for the help!
The key for answer is in this screenshot
The details are that the data comes back from the server as a json blob
reactive-record decodes it, but counts on the fact that if you try to json parse a simple string, it raises an error.
opal 0.10 no longer raises standard error, so the whole thing just hangs up.
Just thinking about this... there is a known problem in Opal https://github.com/opal/opal/issues/1545 and this causes a problem in reactive-record. Please make sure that you are not using opal 0.10
One thing to keep in mind is that reactive-record lazy loads records, and attributes. So unless someplace in your render, you access a particular record/attribute that attribute will not show up on the client.
Its hard to tell more without a bit more of your code posted, but here is some help:
Lets say your component looks like this:
class Foo < React::Component::Base
param :user, type: User
def render
"user.name = #{user.name}"
end
end
and someplace either in a controller or in a layout you do this:
render_component '::Foo', {user: User.first}
You might try something very simple like this, just to get familiar with how things work.
What happens should be this: You will render your view and a placeholder for the first User will be sent to the component, during rendering the component looks for that user's name attribute, which it does not have, so that is queued up to fetch from the server. Rendering will complete, and eventually the data will come down from the server, the local model's data will be updated, and components displaying that data will be rerendered.
During prerendering all the above happens internal to the server, and when the component has been rendered the final html is delivered along with all the model data that was used in rendering the component. So on first load if all is working you will not see any fetches from the server.
So if you try out the above small example, and then go into your javascript console you can say things like this:
Opal.User.$first()
and you will see the underlying model data structure returned (I am translating from JS into ruby above... ruby methods all start with $)
You can then do this:
Opal.User.$first().$name()
And you can even do this (assuming there are at least 2 user models):
Opal.User.$find(2).$name()
You should have something like "DummyValue" returned, but then there will be a server fetch cycle in the console, then if you repeat the above command you will get back the actual value!!!
This may not be the best forum for more details, if you need to drop by https://gitter.im/reactrb/chat for more help
I'm having an issue with a date format. I have a time picker that has the date in a funky format (well, it's a nice format, actually, but not to the computer). I'm trying to have Chronic parse the date so that it can be saved properly.
At first, I was doing, in the create action of my controller:
params[:event][:start] = Chronic.parse(params[:event][:start])
but if and when validation fails, it sends the parsed value back to the view, and my datetimepicker is all botched, then.
So, I thought... callback? In my model, I added:
private
def date_change
self.start = Chronic.parse(self.start)
end
I tried before_save, before_validation, after_validation... but nothing seems to get that date formatted correctly.
As it stands, I keep getting ArgumentError in EventsController#create - Argument out of range. I assume that's because the database is expecting a properly formatted datetime object.
Any idea on how I can accomplish my goal, here, of not changing the params, but still being able to save a properly formatted object?
I'm guessing that the problem is occurring the the start= mutator method that ActiveRecord supplies. If you're doing things like this in your controller:
#event.update_attributes(params[:events])
#event = Event.create(params[:event])
#...
then create and update_attributes should call start= internally. That should allow you to put the Chronic stuff in your own start=:
def start=(t)
super(Chronic.parse(t))
end
You might need to adjust that for non-String ts, I'm not sure what Chronic.parse(Time.now), for example, would do. You could also call write_attribute(:start, Chronic.parse(t)) or self[:start] = Chronic.parse(t) if you didn't want to punt to super.
Note that before_validation and similar handlers will be called too late to bypass whatever default string-to-timestamp conversion ActiveRecord is doing but a mutator override should happen at the right time.
Alternatively, you could parse the time in the controller with something like this:
event = params[:events].dup
events[:start] = Chronic.parse(events[:start])
#event = Event.create(event)
Assumption is the mother of all mess ups :)
are you sure the callback is hit? Because if it would, and the error occurred (like it did), wouldn't it still send back the incorrect data (because parsed) back to the view? In case of doubt: log something to make sure it is hit.
are you sure which field causes the Argument out of range error.
Most cases bugs are so hard to find/fix because we assume we know the error, but we are looking at the error in the wrong way.
Easy ways to test which attribute causes the error:
open rails console, build an object with the parameters, save it, and ask the errors. Something like
e = Event.new(params[:event]) # copy params verbatim from your logfile
e.save
e.errors
and that will display which field causes the error.
Alternatively: use pry and add a line binding.pry just after the save, so you inspect the errors (more info)
Answer (assuming your assumption was correct)
I see two options to do what you want:
use the after_validation callback, if you are sure the data will always be correct, and correctly parsed by Chronic. This way if validation is passed, then convert the field and normally nothing can go wrong anymore, and the value is never sent to the browser again.
Note: if some other attribute is causing the error, this callback is never hit, of course. Because it does not pass the validation.
use a virtual attribute, e.g. start_str, which is a visual representation of your start, and
before_save convert it to start. It does not really matter that much here, because if validation fails, you just show start_str and not the "real" start field.
In my model I have:
class Log < ActiveRecord::Base
serialize :data
...
def self.recover(table_name, row_id)
d = Log.where(table_name: table_name, row_id: row_id).where("log_type != #{symbol_to_constant(:delete)}").last
row = d.data
raise "Nothing to recover" if d.nil?
raise "No data to recover" if d.data.nil?
c = const_get(table_name)
ret = c.create(row.attributes)
end
And in my controller I calling it as:
def index
Log.recover params[:t], params[:r]
redirect_to request.referer
end
The problem is, if I access this page for the first time, I am getting error specified below, but after refresh, is everything OK. Where can be problem?
undefined method `attributes' for #<String:0x00000004326fc8>
In data column are saved instances of models. For the first time column isn't properly unserialized, it's just yaml text. But after refresh everything is fine. That's confusing, what is wrong? Bug in rails?
It's not every time, sometimes in first access everything is okey.
Deyamlizing an object of class Foo will do funny things if there is no class Foo. This can quite easily happen in development becauses classes are only loaded when needed and unloaded when rails thinks they might have changed.
Depending on whether the class is loaded or not the YAML load will have different results (YAML doesn't know about rail's automatic loading stuff)
One solution worth considering is to store the attributes hash rather than the activerecord object. You'll probably avoid problems in the long run and it will be more space efficient in the long wrong - there's a bunch of state in an activerecord object that you probably don't care about in this case.
If that's not an option, your best bet is probably to make sure that the classes that the serialized column might contain are loaded - still a few calls to require_dependency 'foo' at the top of the file.
I have a Rails 3.0.9 app that, once it is deployed, suffers from a bunch of ActiveModel::MissingAttributeErrors that crop up causing 500s. The errors occur fairly randomly, sometimes a page will load, other times it won't, but the attributes are all existing attributes in the database and should be found.
The strange part is that after a while, the errors go away. Suddenly, they stop causing an issue.
I have searched about for a solution to this, but this error mostly occurs either when someone has done Model.all(:select => 'column_x,column_y') and are calling for column_z or when they are using cache_money. I am doing neither of these things.
Can anyone help?
You probably have a query that doesn't return all the columns (i.e. uses :select) and then cache_money; or some other ActiveRecord plugin uses an after_initialize callback, which executes whenever a new ActiveRecord object is created (i.e. when fetched from the database).
In that initialize callback, something tries to access or use an attribute that wasn't included in the :select. You'd expect this to return nil for that attribute, but an ActiveRecord::MissingAttributeError is thrown instead.
You can rescue ActiveRecord::MissingAttributeError like the article suggests, or patch the plugin(s) to use has_attribute?(:attribute_name) before they try to access or modify the attribute.
If you have been having this issue only directly after updating your database without any deploys or server restarts following, then what worked for me may work for you:
Run heroku restart and it should be fixed. Before the dyno restarts old data sometimes remains cached on the server, so starting it up again will scrub all of that data and prevent it from causing errors of that sort. Hope this helps.
I found an interesting take on this that resulted in the same error. In an attempt to reuse code we subclasses a presenters class with a presenters class that performed grouping to use in a graph view.
To simplify, it was something like:
class PostPresenter
def query
Post.where(...stuff....).includes(:wombat)
end
end
The the aggregator did something like the following to build a table of posts per day:
class AggregatePostPresenter < PostPresenter
def group_query
query.select('count(*) as cnt, date(created_at)').group('date(created_at)')
end
end
A call to "group_query" results in an ActiveModel::MissingAttributeError since, I think, the attempt to "includes" Wombat fails because "wombat_id" wasn't in the attributes included in the "select".
This is probably not your answer, however, since it happens regardless of whether or not cache is enabled.
I encountered this issue. Make sure your select: includes all fields referenced in your view, including any relationship IDs and any attributes called within your methods.
The missing attribute can be difficult to identify whenever your views and relationships are complex. The easiest way to debug this is to remove the select portion of your where clause and see if the query/scope/method runs correctly. If so, then add all of the attributes to the select and remove unneeded attributes one-at-a-time until you find the offending attribute.
A similar problem was annoying me when I was trying to make Ajax (actually angularjs) calls to populate an edit-in-place select fields.
I just wanted an id and name attributes to_json and kept getting the MissingAttributeError.
Realised I gotcha'd myself by having an as_json method in the model which is used for the main index and show calls on the model. Basically it was the as_json that was not seeing the attributes it expected.
#foo=Foo.select("id,name")
respond_to do |format|
format.json { render :json => #foo.to_json }
end
gave the error but
respond_to do |format|
format.json { render :json => { :foo=>#foo.as_json(:only=>[:id,:name]) } }
end
seems to be working. I was close to sussing it myself but I found a great explanation at.
http://jonathanjulian.com/2010/04/rails-to_json-or-as_json/
I fixed this by adding .to_json to the end of my controller render.
you need to add line
rescue ActiveRecord::MissingAttributeError
in your after_initialize() method of the model