I am doing a complex series of database interactions within nested transactions using ActiveRecord (Rails), involving various model.update(...), model.where(...).first_or_create(..) etc
Right before the transaction ends I'd like to report on what's actually changed and about to be written. Presumably ActiveRecord holds this information but I've not been able to work out where.
My code would be something like (semi-pseudocode)
def run options
begin
ActiveRecord::Base.transaction do |tranny|
options[:files].each do |file|
raw_data = open(file)
complex_data_stuff raw_data, options
end
report
rescue => e
"it all went horribly wrong, here's why: #{e.message}"
end
end
def report tranny
changes = {}
tranny.whatschanged?.each do |ch|
changes[ch.model.class.name] = {} unless changes[ch.model.class.name]
if changes[ch.model.class.name][ch.kind_of_update]
changes[ch.model.class.name][ch.kind_of_update] += 1
else
changes[ch.model.class.name][ch.kind_of_update] = 1
end
end
changes
end
How would I achieve something like this?
http://api.rubyonrails.org/classes/ActiveModel/Dirty.html
This is the latest version of "dirty models", which keeps track of the differences between the current object and the saved version. You access the changes via a "changes" method like in your attempt.
I added some extra stuff in one of my projects to store what was changed in the last update: this is stored in an instance variable, so is only accessable in the specific object in memory (ie you can't see it if you reload it from the database).
module ActiveRecord
class Base
attr_accessor :changed_in_last_save
before_save :set_changed_in_last_save_hash
def set_changed_in_last_save_hash
self.changed_in_last_save_hash = self.changes
end
def changed_in_last_save
self.changed_in_last_save_hash || {}
end
end
end
You definitely need ActiveModel::Dirty, you probably don't need my thing, just mentioned it as it's similar :)
Related
I'm experimenting & learning how to work with PostgreSQL, namely its Notify/Listen feature, in the context of making Server-Sent Events according to this tutorial.
The tutorial publishes NOTIFY to the user channel (via its id) whenever a user is saved and an attribute, authy_status is changed. The LISTEN method then yields the new authy_status Code:
class Order < ActiveRecord::Base
after_commit :notify_creation
def notify_creation
if created?
ActiveRecord::Base.connection_pool.with_connection do |connection|
execute_query(connection, ["NOTIFY user_?, ?", id, authy_status])
end
end
end
def on_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_?", id])
connection.raw_connection.wait_for_notify do |event, pid, status|
yield status
end
ensure
execute_query(connection, ["UNLISTEN user_?", id])
end
end
end
end
I would like to do something different, but haven't been able to find information on how to do this. I would like to NOTIFY when a user is created in the first place (i.e., inserted into the database), and then in the LISTEN, I'd like to yield up the newly created user itself (or rather its id).
How would I modify the code to achieve this? I'm really new to writing SQL so for example, I'm not very sure about how to change ["NOTIFY user_?, ?", id, authy_status] to a statement that looks not at a specific user, but the entire USER table, listening for new records (something like... ["NOTIFY USER on INSERT", id] ?? )
CLARIFICATIONS
Sorry about not being clear. The after_save was a copy error, have corrected to after_commit above. That's not the issue though. The issue is that the listener listens to changes in a SPECIFIC existing user, and the notifier notifies on changes to a SPECIFIC user.
I instead want to listen for any NEW user creation, and therefore notify of that. How does the Notify and Listen code need to change to meet this requirement?
I suppose, unlike my guess at the code, the notify code may not need to change, since notifying on an id when it's created seems to make sense still (but again, I don't know, feel free to correct me). However, how do you listen to the entire table, not a particular record, because again I don't have an existing record to listen to?
For broader context, this is the how the listener is used in the SSE in the controller from the original tutorial:
def one_touch_status_live
response.headers['Content-Type'] = 'text/event-stream'
#user = User.find(session[:pre_2fa_auth_user_id])
sse = SSE.new(response.stream, event: "authy_status")
begin
#user.on_creation do |status|
if status == "approved"
session[:user_id] = #user.id
session[:pre_2fa_auth_user_id] = nil
end
sse.write({status: status})
end
rescue ClientDisconnected
ensure
sse.close
end
end
But again, in my case, this doesn't work, I don't have a specific #user I'm listening to, I want the SSE to fire when any user has been created... Perhaps it's this controller code that also needs to be modified? But this is where I'm very unclear. If I have something like...
User.on_creation do |u|
A class method makes sense, but again how do I get the listen code to listen to the entire table?
Please use after_commit instead of after_save. This way, the user record is surely committed in the database
There are two additional callbacks that are triggered by the completion of a database transaction: after_commit and after_rollback. These callbacks are very similar to the after_save callback except that they don't execute until after database changes have either been committed or rolled back.
https://guides.rubyonrails.org/active_record_callbacks.html#transaction-callbacks
Actually it's not relevant to your question, you can use either.
Here's how I would approach your use case: You want to get notified when an user is created:
#app/models/user.rb
class User < ActiveRecord::Base
after_commit :notify_creation
def notify_creation
if id_previously_changed?
ActiveRecord::Base.connection_pool.with_connection do |connection|
self.class.execute_query(connection, ["NOTIFY user_created, '?'", id])
end
end
end
def self.on_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
connection.raw_connection.wait_for_notify do |event, pid, id|
yield self.find id
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
def self.clean_sql(query)
sanitize_sql(query)
end
def self.execute_query(connection, query)
sql = self.clean_sql(query)
connection.execute(sql)
end
end
So that if you use
User.on_creation do |user|
#do something with the user
#check user.authy_status or whatever attribute you want.
end
One thing I am not sure why you want to do this, because it could have a race condition situation where 2 users being created and the unwanted one finished first.
Need help understanding this code, as what to my knowledge I know "<<" append to a collection but here it saves the record correctly, how come it does without calling .save method?
#user.rb
has_many :saved_properties, through: :property_saves, source: :property
#users_controller.rb
def update
if #user.saved_properties << Property.find(params[:saved_property_id])
render plain: "Property saved"
end
In the has_many documentation it says:
Adds one or more objects to the collection by setting their foreign
keys to the collection's primary key. Note that this operation
instantly fires update SQL without waiting for the save or update
call on the parent object, unless the parent object is a new record.
Maybe looking at the source code will help you. This is my trail of searches based on the << method in activerecord:
def <<(*records)
proxy_association.concat(records) && self
end
rails/collection_proxy.rb at 5053d5251fb8c03e666f1f8b765464ec33e3066e · rails/rails · GitHub
def concat(*records)
records = records.flatten
if owner.new_record?
load_target
concat_records(records)
else
transaction { concat_records(records) }
end
end
rails/collection_association.rb at 5053d5251fb8c03e666f1f8b765464ec33e3066e · rails/rails · GitHub
def concat_records(records, should_raise = false)
result = true
records.each do |record|
raise_on_type_mismatch!(record)
add_to_target(record) do |rec|
result &&= insert_record(rec, true, should_raise) unless owner.new_record?
end
end
result && records
end
rails/collection_association.rb at 5053d5251fb8c03e666f1f8b765464ec33e3066e · rails/rails · GitHub
def insert_record(record, validate = true, raise = false)
set_owner_attributes(record)
set_inverse_instance(record)
if raise
record.save!(validate: validate)
else
record.save(validate: validate)
end
end
https://github.com/rails/rails/blob/5053d5251fb8c03e666f1f8b765464ec33e3066e/activerecord/lib/active_record/associations/has_many_association.rb#L32
def insert_record(record, validate = true, raise = false)
ensure_not_nested
if record.new_record? || record.has_changes_to_save?
if raise
record.save!(validate: validate)
else
return unless record.save(validate: validate)
end
end
save_through_record(record)
record
end
https://github.com/rails/rails/blob/5053d5251fb8c03e666f1f8b765464ec33e3066e/activerecord/lib/active_record/associations/has_many_through_association.rb#L38
As you can see, in the end, it does call the save method.
Disclaimer: I'm not that familiar with Rails souce code, but you have interesting question.
In a has_many relationship the link information is saved in the target record. This means that << would have to modify that record in order to add it to the set.
Perhaps intending convenience, ActiveRecord automatically saves these for you when making an assignment if the assignment was successful. The exception is for new records, the record they're being associated with doesn't have any identifier so that has to be delayed. They are saved when the record they're associated with is finally created.
This can be a little confusing, perhaps unexpected, but it's actually the thing you'd want to happen 99% of the time. If you don't want that to happen you should manipulate the linkage manually:
property = Property.find(params[:saved_property_id])
property.user = #user
property.save!
That's basically equivalent but a lot more verbose.
I have a class I've extended from ActiveRecord::Base...
class Profile < ActiveRecord::Base
and I collect the records from it like so...
records = #profile.all
which works fine, but it doesn't seem that I can successfully Update the attributes. I don't want to save them back to the database, just modify them before I export them as JSON. My question is, why can't I update these? I'm doing the following (converting date formats before exporting):
records.collect! { |record|
unless record.term_start_date.nil?
record.term_start_date = Date.parse(record.term_start_date.to_s).strftime('%Y,%m,%d')
end
unless record.term_end_date.nil?
record.term_end_date = Date.parse(record.term_end_date.to_s).strftime('%Y,%m,%d')
end
record
}
At first I had just been doing this in a do each loop, but tried collect! to see if it would fix things, but no difference. What am I missing?
P.S. - I tried this in irb on one record and got the same results.
I suggest a different way to solve the problem, that keeps the logic encapsulated in the class itself.
Override the as_json instance method in your Profile class.
def as_json(options={})
attrs = super(options)
unless attrs['term_start_date'].nil?
attrs['term_start_date'] = Date.parse(attrs['term_start_date'].to_s).strftime('%Y,%m,%d')
end
unless attrs['term_end_date'].nil?
attrs['term_end_date'] = Date.parse(attrs['term_end_date'].to_s).strftime('%Y,%m,%d')
end
attrs
end
Now when you render the records to json, they'll automatically use this logic to generate the intermediate hash. You also don't run the risk of accidentally saving the formatted dates to the database.
You can also set up your own custom option name in the case that you don't want the formatting logic.
This blog post explains in more detail.
Try to add record.save! before record.
Actually, by using collect!, you just modifying records array, but to save modified record to database you should use save or save! (which raises exception if saving failed) on every record.
This is probably one of the things that all new users find out about Rails sooner or later. I just realized that rails is updating all fields with the serialize keyword, without checking if anything really changed inside. In a way that is the sensible thing to do for the generic framework.
But is there a way to override this behavior? If I can keep track of whether the values in a serialized fields have changed or not, is there a way to prevent it from being pushed in the update statement? I tried using "update_attributes" and limiting the hash to the fields of interest, but rails still updates all the serialized fields.
Suggestions?
Here is a similar solution for Rails 3.1.3.
From: https://sites.google.com/site/wangsnotes/ruby/ror/z00---topics/fail-to-partial-update-with-serialized-data
Put the following code in config/initializers/
ActiveRecord::Base.class_eval do
class_attribute :no_serialize_update
self.no_serialize_update = false
end
ActiveRecord::AttributeMethods::Dirty.class_eval do
def update(*)
if partial_updates?
if self.no_serialize_update
super(changed)
else
super(changed | (attributes.keys & self.class.serialized_attributes.keys))
end
else
super
end
end
end
Yes, that was bugging me too. This is what I did for Rails 2.3.14 (or lower):
# config/initializers/nopupdateserialize.rb
module ActiveRecord
class Base
class_attribute :no_serialize_update
self.no_serialize_update = false
end
end
module ActiveRecord2
module Dirty
def self.included(receiver)
receiver.alias_method_chain :update, :dirty2
end
private
def update_with_dirty2
if partial_updates?
if self.no_serialize_update
update_without_dirty(changed)
else
update_without_dirty(changed | (attributes.keys & self.class.serialized_attributes.keys))
end
else
update_without_dirty
end
end
end
end
ActiveRecord::Base.send :include, ActiveRecord2::Dirty
Then in your controller use:
model_item.no_serialize_update = true
model_item.update_attributes(params[:model_item])
model_item.increment!(:hits)
model_item.update_attribute(:nonserializedfield => "update me")
etc.
Or define it in your model if you do not expect any changes to the serialized field once created (but update_attribute(:serialized_field => "update me" still works!)
class Model < ActiveRecord::Base
serialize :serialized_field
def no_serialize_update
true
end
end
I ran into this problem today and ended up hacking my own serializer together with a getter and setter. First I renamed the field to #{column}_raw and then used the following code in the model (for the media attribute in my case).
require 'json'
...
def media=(media)
self.media_raw = JSON.dump(media)
end
def media
JSON.parse(media_raw) if media_raw.present?
end
Now partial updates work great for me, and the field is only updated when the data is actually changed.
The problem with Joris' answer is that it hooks into the alias_method_chain chain, disabling all the chains done after (like update_with_callbacks which accounts for the problems of triggers not being called). I'll try to make a diagram to make it easier to understand.
You may start with a chain like this
update -> update_with_foo -> update_with_bar -> update_with_baz
Notice that update_without_foo points to update_with_bar and update_without_bar to update_with_baz
Since you can't directly modify update_with_bar per the inner workings of alias_method_chain you might try to hook into the chain by adding a new link (bar2) and calling update_without_bar, so:
alias_method_chain :update, :bar2
Unfortunately, this will get you the following chain:
update -> update_with_bar2 -> update_with_baz
So update_with_foo is gone!
So, knowing that alias_method_chain won't let you redefine _with methods my solution so far has been to redefine update_without_dirty and do the attribute selection there.
Not quite a solution but a good workaround in many cases for me was simply to move the serialized column(s) to an associated model - often this actually was a good fit semantically anyway.
There is also discussions in https://github.com/rails/rails/issues/8328.
Basically what I want to do is to log an action on MyModel in the table of MyModelLog. Here's some pseudo code:
class MyModel < ActiveRecord::Base
validate :something
def something
# test
errors.add(:data, "bug!!")
end
end
I also have a model looking like this:
class MyModelLog < ActiveRecord::Base
def self.log_something
self.create(:log => "something happened")
end
end
In order to log I tried to :
Add MyModelLog.log_something in the something method of MyModel
Call MyModelLog.log_something on the after_validation callback of MyModel
In both cases the creation is rolled back when the validation fails because it's in the validation transaction. Of course I also want to log when validations fail. I don't really want to log in a file or somewhere else than the database because I need the relationships of log entries with other models and ability to do requests.
What are my options?
Nested transactions do seem to work in MySQL.
Here is what I tried on a freshly generated rails (with MySQL) project:
./script/generate model Event title:string --skip-timestamps --skip-fixture
./script/generate model EventLog error_message:text --skip-fixture
class Event < ActiveRecord::Base
validates_presence_of :title
after_validation_on_create :log_errors
def log_errors
EventLog.log_error(self) if errors.on(:title).present?
end
end
class EventLog < ActiveRecord::Base
def self.log_error(event)
connection.execute('BEGIN') # If I do transaction do then it doesn't work.
create :error_message => event.errors.on(:title)
connection.execute('COMMIT')
end
end
# And then in script/console:
>> Event.new.save
=> false
>> EventLog.all
=> [#<EventLog id: 1, error_message: "can't be blank", created_at: "2010-10-22 13:17:41", updated_at: "2010-10-22 13:17:41">]
>> Event.all
=> []
Maybe I have over simplified it, or missing some point.
Would this be a good fit for an Observer? I'm not sure, but I'm hoping that exists outside of the transaction... I have a similar need where I might want to delete a record on update...
I've solved a problem like this by taking advantage of Ruby's variable scoping. Basically I declared an error variable outside of a transaction block then catch, store log message, and raise the error again.
It looks something like this:
def something
error = nil
ActiveRecord::Base.transaction do
begin
# place codez here
rescue ActiveRecord::Rollback => e
error = e.message
raise ActiveRecord::Rollback
end
end
MyModelLog.log_something(error) unless error.nil?
end
By declaring the error variable outside of the transaction scope the contents of the variable persist even after the transaction has exited.
I am not sure if it applies to you, but i assume you are trying to save/create a model from your controller. In the controller it is easy to check the outcome of that action, and you most likely already do to provide the user with a useful flash; so you could easily log an appropriate message there.
I am also assuming you do not use any explicit transactions, so if you handle it in the controller, it is outside of the transaction (every save and destroy work in their own transaction).
What do you think?
MyModelLog.log_something should be done using a different connection.
You can make MyModelLog model always use a different connection by using establish_connection.
class MyModelLog < ActiveRecord::Base
establish_connection Rails.env # Use different connection
def self.log_something
self.create(:log => "something happened")
end
end
Not sure if this is the right way to do logging!!
You could use a nested transaction. This way the code in your callback executes in a different transaction than the failing validation. The Rails documentations for ActiveRecord::Transactions::ClassMethods discusses how this is done.