I'm experimenting & learning how to work with PostgreSQL, namely its Notify/Listen feature, in the context of making Server-Sent Events according to this tutorial.
The tutorial publishes NOTIFY to the user channel (via its id) whenever a user is saved and an attribute, authy_status is changed. The LISTEN method then yields the new authy_status Code:
class Order < ActiveRecord::Base
after_commit :notify_creation
def notify_creation
if created?
ActiveRecord::Base.connection_pool.with_connection do |connection|
execute_query(connection, ["NOTIFY user_?, ?", id, authy_status])
end
end
end
def on_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_?", id])
connection.raw_connection.wait_for_notify do |event, pid, status|
yield status
end
ensure
execute_query(connection, ["UNLISTEN user_?", id])
end
end
end
end
I would like to do something different, but haven't been able to find information on how to do this. I would like to NOTIFY when a user is created in the first place (i.e., inserted into the database), and then in the LISTEN, I'd like to yield up the newly created user itself (or rather its id).
How would I modify the code to achieve this? I'm really new to writing SQL so for example, I'm not very sure about how to change ["NOTIFY user_?, ?", id, authy_status] to a statement that looks not at a specific user, but the entire USER table, listening for new records (something like... ["NOTIFY USER on INSERT", id] ?? )
CLARIFICATIONS
Sorry about not being clear. The after_save was a copy error, have corrected to after_commit above. That's not the issue though. The issue is that the listener listens to changes in a SPECIFIC existing user, and the notifier notifies on changes to a SPECIFIC user.
I instead want to listen for any NEW user creation, and therefore notify of that. How does the Notify and Listen code need to change to meet this requirement?
I suppose, unlike my guess at the code, the notify code may not need to change, since notifying on an id when it's created seems to make sense still (but again, I don't know, feel free to correct me). However, how do you listen to the entire table, not a particular record, because again I don't have an existing record to listen to?
For broader context, this is the how the listener is used in the SSE in the controller from the original tutorial:
def one_touch_status_live
response.headers['Content-Type'] = 'text/event-stream'
#user = User.find(session[:pre_2fa_auth_user_id])
sse = SSE.new(response.stream, event: "authy_status")
begin
#user.on_creation do |status|
if status == "approved"
session[:user_id] = #user.id
session[:pre_2fa_auth_user_id] = nil
end
sse.write({status: status})
end
rescue ClientDisconnected
ensure
sse.close
end
end
But again, in my case, this doesn't work, I don't have a specific #user I'm listening to, I want the SSE to fire when any user has been created... Perhaps it's this controller code that also needs to be modified? But this is where I'm very unclear. If I have something like...
User.on_creation do |u|
A class method makes sense, but again how do I get the listen code to listen to the entire table?
Please use after_commit instead of after_save. This way, the user record is surely committed in the database
There are two additional callbacks that are triggered by the completion of a database transaction: after_commit and after_rollback. These callbacks are very similar to the after_save callback except that they don't execute until after database changes have either been committed or rolled back.
https://guides.rubyonrails.org/active_record_callbacks.html#transaction-callbacks
Actually it's not relevant to your question, you can use either.
Here's how I would approach your use case: You want to get notified when an user is created:
#app/models/user.rb
class User < ActiveRecord::Base
after_commit :notify_creation
def notify_creation
if id_previously_changed?
ActiveRecord::Base.connection_pool.with_connection do |connection|
self.class.execute_query(connection, ["NOTIFY user_created, '?'", id])
end
end
end
def self.on_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
connection.raw_connection.wait_for_notify do |event, pid, id|
yield self.find id
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
def self.clean_sql(query)
sanitize_sql(query)
end
def self.execute_query(connection, query)
sql = self.clean_sql(query)
connection.execute(sql)
end
end
So that if you use
User.on_creation do |user|
#do something with the user
#check user.authy_status or whatever attribute you want.
end
One thing I am not sure why you want to do this, because it could have a race condition situation where 2 users being created and the unwanted one finished first.
Related
I have an admin dashboard where I want an alert to be fired whenever a user is created (on a separate page). The code below works, however, there's a race condition. If 2 users are created very close together, it will only fire once.
class User < ApplicationRecord
after_commit :notify_creation, on: :create
def notify_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
self.class.execute_query(connection, ["NOTIFY user_created, '?'", id])
end
end
def self.listen_to_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
connection.raw_connection.wait_for_notify do |event, pid, id|
yield id
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
def self.clean_sql(query)
sanitize_sql(query)
end
private
def self.execute_query(connection, query)
sql = self.clean_sql(query)
connection.execute(sql)
end
end
class AdminsController < ApplicationController
include ActionController::Live
def update
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream, event: 'notice')
begin
User.listen_to_creation do |user_id|
sse.write({user_id: user_id})
end
rescue ClientDisconnected
ensure
sse.close
end
end
end
This is my first time doing this, so I followed this tutorial, which like most tutorials are focused on updates to a single record, rather than listening to an entire table for new creation.
This is happening because you send only one update at once and then the request ends. If you make a request at the AdminsController#update. You have one subscriber waiting for your notification. Look at this block
begin
execute_query(connection, ["LISTEN user_created"])
connection.raw_connection.wait_for_notify do |event, pid, id|
yield id
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
As soon as you get one notification, the block yields and then you close the channel. So if you are relying on frontend to make one more connection attempt once it gets the result, if a record gets created before you start listening to the channel again in the new connection, you won't get a notification as there was no listener attached to Postgres at that time.
This is sort of a common issue in any realtime notification system. You would ideally want a pipe to frontend(Websocket, SSE or even LongPolling) which is always open. If you get a new item you send it to the frontend using that pipe and you should ideally keep that pipe open as in case of Websockets and SSE. Right now you are kind of treating your SSE connection as a long poll.
So your code should look something like
# Snippet 2
def self.listen_to_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
loop do
connection.raw_connection.wait_for_notify do |event, pid, id|
yield id
end
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
But this will run into a problem where it will keep thread alive forever even if the connection is closed until some data comes to thread and it is encounters an error while writing back then. You can choose to either run it a fixed number of times with short lived notify intervals or you can add sort of a hearbeat to it. There are two simple ways of accomplishing a hearbeat. I will add them as quick hack codes.
# Snippet 3
def self.listen_to_creation(heartbeat_interval = 10)
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
last_hearbeat = Time.now
loop do
connection.raw_connection.wait_for_notify(heartbeat_interval) do |event, pid, id|
yield({id: id})
end
if Time.now - last_heartbeat >= heartbeat_interval
yield({heartbeat: true})
last_heartbeat = Time.now
end
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
In the above example you will at least be sending something in the pipe every heartbeat_interval seconds. So, if the pipe closes it should error out and close the pipe thus freeing up the thread.
This approach kind of adds controller related logic to model and if you want to hold postgres notify without a time interval, the other thing that you can do to do a heartbeat is, just launch a thread in the controller itself. Launch a thread in the controller method that sleeps for heartbeat_interval and writes sse.write({heartbeat: true}) after waking up. You can leave the model code the same as Snippet 2 in that case.
Also, I added the other things to watch with SSEs with Puma & Rails in an answer to your other question:
I'm using Rails 4 with Oracle 12c and I need to update the status of an User, and then use the new status in a validation for another model I also need to update:
class User
has_many :posts
def custom_update!(new_status)
relevant_posts = user.posts.active_or_something
ActiveRecord::Base.transaction do
update!(status: new_status)
relevant_posts.each { |post| post.update_stuff! }
end
end
end
class Post
belongs_to :user
validate :pesky_validation
def update_stuff!
# I can call this from other places, so I also need a transaction here
ActiveRecord::Base.transaction do
update!(some_stuff: 'Some Value')
end
end
def pesky_validation
if user.status == OLD_STATUS
errors.add(:base, 'Nope')
end
end
end
However, this is failing and I receive the validation error from pesky_validation, because the user inside Post doesn't have the updated status.
The problem is, when I first update the user, the already instantiated users inside the relevant_posts variable are not yet updated, and normally all I'd need to fix this was to call reload, however, maybe because I'm inside a transaction, this is not working, and pesky_validation is failing.
relevant_users.first.user.reload, for example, reloads the user to the same old status it had before the update, and I'm assuming it's because the transaction is not yet committed. How can I solve this and update all references to the new status?
we have already used sidekiq for inserting records into our table asynchronously and we very often check production sidekiq dashboard to monitor no. of processed, queued, retry, busy for inserting records.
And we have got a new requirement to delete records (say users tables : delete expired users) asynchronously. we also need to monitor sidekiq dashboard for processes, queued, retry very often.
For insert records we use :
In my User controller:
def create_user
CreateUserWorker.perform_async(#client_info, #input_params)
end
In my lib/workers/createuser_worker.rb
class CreateUserWorker
include Sidekiq::Worker
def perform(client_info, input_params)
begin
#client_info = client_info
#user = User.new(#client_info)
#user.create(input_params)
rescue
raise
end
end
end
If I do the same for delete users asynchronously using sidekiq, how can i differentiate inserted process with deleted process without any messup?
First, If you want to check error for creating in begin-rescue block, you should use create! method. not create method.
Create method do not raise error.
Check here
Destroy method is same to Create method.
Use destroy method with ! (destroy!)
Of course, You should add new worker for destroy user.
because perform method should exists only 1.
If you do not want to add new worker, try pattern below!
UserWorker
def perform(~, flag)
#flag meaning is create or destroy
is_success = false # result of creating or destroying
# create or destroy
# ..
# ..
LogModel.create({}) # user info with is_success and flag
end
ebd
P.S
I think create() next new() is some awkward(?).
I recommend
#user = User.create(client_info)
or
#user = User.new(client_info)
#user.save! (bang meaning is same to above)
And no need begin-rescue block. Just use Create, Destroy method with bang.
def perform(client_info, input_params)
User.create!(client_info) # if failed raise Error
end
++Added for comments
I think if you have many user deleted or destroyed, pass user_ids (or user_infos) array to Worker perform method and in perform method, loop creating or destroying (if there is failed record created or destroyed, create log file or log model entry about a failed record).
If all user_id must be created or destroyed at once, use transaction block.
def perform(params)
begin
ActiveRecord::Base.transaction do
# loop create or destroy
end
rescue
end
end
if not, just loop
def perform(params)
#loop
if Create or destroy method (without bang)
#success
else
#failed
end
end
XWorker.perform_async() method maybe is called from admin page(?).
I am doing a complex series of database interactions within nested transactions using ActiveRecord (Rails), involving various model.update(...), model.where(...).first_or_create(..) etc
Right before the transaction ends I'd like to report on what's actually changed and about to be written. Presumably ActiveRecord holds this information but I've not been able to work out where.
My code would be something like (semi-pseudocode)
def run options
begin
ActiveRecord::Base.transaction do |tranny|
options[:files].each do |file|
raw_data = open(file)
complex_data_stuff raw_data, options
end
report
rescue => e
"it all went horribly wrong, here's why: #{e.message}"
end
end
def report tranny
changes = {}
tranny.whatschanged?.each do |ch|
changes[ch.model.class.name] = {} unless changes[ch.model.class.name]
if changes[ch.model.class.name][ch.kind_of_update]
changes[ch.model.class.name][ch.kind_of_update] += 1
else
changes[ch.model.class.name][ch.kind_of_update] = 1
end
end
changes
end
How would I achieve something like this?
http://api.rubyonrails.org/classes/ActiveModel/Dirty.html
This is the latest version of "dirty models", which keeps track of the differences between the current object and the saved version. You access the changes via a "changes" method like in your attempt.
I added some extra stuff in one of my projects to store what was changed in the last update: this is stored in an instance variable, so is only accessable in the specific object in memory (ie you can't see it if you reload it from the database).
module ActiveRecord
class Base
attr_accessor :changed_in_last_save
before_save :set_changed_in_last_save_hash
def set_changed_in_last_save_hash
self.changed_in_last_save_hash = self.changes
end
def changed_in_last_save
self.changed_in_last_save_hash || {}
end
end
end
You definitely need ActiveModel::Dirty, you probably don't need my thing, just mentioned it as it's similar :)
Let us say that we have the model Champ, with the following attributes, all with default values of nil: winner, lose, coach, awesome, should_watch.
Let's assume that two separate operations are performed: (1) a new record is created and (2) c.the_winner is called on a instance of Champ.
Based on my mock code, and the observer on the model, what values are saved to the DB for these two scenarios? What I am trying to understand is the principles of how callbacks work within the context of Base.save operation, and if and when the Base.save operation has to be called more than once to commit the changes.
class Champ
def the_winner
self.winner = 'me'
self.save
end
def the_loser
self.loser = 'you'
end
def the_coach
self.coach = 'Lt Wiggles'
end
def awesome_game(awesome_or_not=false)
self.awesome = awesome_or_not
end
def should_watch_it(should=false)
self.should_watch = should
end
end
class ChampObserver
def after_update(c)
c.the_loser
end
def after_create(c)
c.the_coach
end
def before_create(c)
c.awesome_game(true)
c.should_watch_it(true) if c.awesome_game
end
end
With your example, if you called champ.winner on a new and unmodified instance of Champ, the instance of Champ would be committed to the DB and would look like this in the database:
winner: 'me'
awesome: true
should_watch: true
loser: nil
coach: nil
The after_create callback would be called if it is a new record, and if not, the after_update callback would (this is why loser would be nil if the instance was new). However, because they just call a setter method on the instance, they will only update the instance and will not commit more changes to the DB.
You could use update_attribute in your observer or model methods to commit the change, but unless you actually need to have the record in the database and then update it, it's wasteful. In this example, if you wanted those callbacks to actually set loser and coach in the database, it'd be more efficient to use before_save and before_create.
The Rails guides site has a good overview of callbacks here, if you haven't read it already.