Using the Neo4jrb gem, is it possible to use one transaction across multiple blocks - trailblazer

Using the neo4jrb/neo4j gem (8.x), I know it is possible to run multiple queries inside a single transaction like so
person = Person.find_by(id: person_id)
Neo4j::ActiveBase.run_transaction do |tx|
person.update(name: 'New name')
person.update(number: 1)
end
But it possible to open a transaction, and then use that same transaction across multiple blocks. Something like:
person = Person.find_by(id: person_id)
transaction = Neo4j::ActiveBase.new_transaction
transaction.run do |tx|
person.update(name: 'New name')
end
transaction.run do |tx|
person.update(number: 1)
end
transaction.close
The reason why this functionality is desirable is because I'm using Neo4jrb inside of a Trailblazer-Operation. The trailblazer operation is broken up into discrete steps which are themselves written as separate methods. I want several of the steps wrapped in a transaction, but, without monkey patching the operation, I don't have the ability to execute some steps inside one transaction block.
Thanks!

So it turns out that my problem has multiple solutions.
While it wasn't immediately apparent to me that this was the case, after a new transaction is created in Neo4jrb, any subsequent queries in that same session are automatically associated with the transaction as long as it remains open.
person = Person.find_by(id: person_id)
transaction = Neo4j::ActiveBase.new_transaction
one = person.update(name: 'New name')
two = person.update(number: 1)
transaction.close
In the above example, both one and two are committed as part of transaction. So this solves my problem.
Another solution to my problem, is that trailblazer-operation has a specific method for wrapping steps in a db transaction block, as documented on Trailblazer's website.
step Wrap ->(*, &block) { Sequel.transaction do block.call end } {
step Model( Song, :new )
step Contract::Build( constant: MyContract )
step Contract::Validate( )
step Contract::Persist( method: :sync )
}
In the above example, all of the step methods inside step Wrap are called within the scope of the Sequel.transaction block.

Related

Ruby on Rails - rollback on transactions doesnt work for multiple saves

I have this simple piece of code.
ActiveRecord::Base.transaction do
User.new(username: "michael").save!
User.new(username: "lebron").save!
User.new(username: "michael").save! # not unique error
end
Fun fact is that error in the 3rd line doesn't rollback all saves, but just the last one.
What am I not understanding about TX in Ruby so it behaves like that?
I know from documentation that
Both #save and #destroy come wrapped in a transaction
So I assume they are wrapped into a different one then the parent transaction.
Thuss making rollback from the 3rd statement not effective on the two prior ones.
How that should be addressed, so I have all statements rolled back during the course of the transaction?
Actually, ActiveRecord::Base.transaction will verify a single transaction so you can convert this into a single transaction so that it will work properly. So push user details into an array and create it.
Example:
user_details = [{username: "michael"}, {username: "lebron"}, {username: "michael"}] User.create(user_details)

How to prevent parallel Sidekiq jobs from executing code in Rails

I have around 10 workers that performs a job that includes the following:
user = User.find_or_initialize_by(email: 'some-email#address.com')
if user.new_record?
# ... some code here that does something taking around 5 seconds or so
elsif user.persisted?
# ... some code here that does something taking around 5 seconds or so
end
user.save
The problem is that at certain times, two or more workers run this code at the exact time, and thus I later found out that two or more Users have the same email, in which I should always end up only unique emails.
It is not possible for my situation to create DB Unique Indexes for email as unique emails are conditional -- some Users should have unique email, some do not.
It is noteworthy to mention that my User model has uniqueness validations, but it still doesn't help me because, between .find_or_initialize_by and .save, there is a code that is dependent if the user object is already created or not.
I tried Pessimistic and Optimistic locking, but it didn't help me, or maybe I just didn't implement it properly... should you have some suggestions regarding this.
The solution I can only think of is to lock the other threads (Sidekiq jobs) whenever these lines of codes get executed, but I am not too sure how to implement this nor do I know if this is even a suggestable approach.
I would appreciate any help.
EDIT
In my specific case, it is gonna be hard to put email parameter in the job, as this job is a little more complex than what was just said above. The job is actually an export script in which a section of the job is the code above. I don't think it's also possible to separate the functionality above into another separate worker... as the whole job flow should be serial and that no parts should be processed parallely / asynchronously. This job is just one of the jobs that are managed by another job, in which ultimately is managed by the master job.
Pessimistic locking is what you want but only works on a record that exists - you can't use it with new_record? because there's nothing to lock in the DB yet.
I managed to solve my problem with the following:
I found out that I can actually add a where clause in Rails DB Uniqueness Partial Index, and thus I can now set up uniqueness conditions for different types of Users on the database-level in which other concurrent jobs will now raise an ActiveRecord::RecordNotUnique error if already created.
The only problem now then is the code in between .find_or_initialize_by and .save, since those are time-dependent on the User objects in which always only one concurrent job should always get a .new_record? == true, and other concurrent jobs should then trigger the .persisted? == true as one job would always be first to create it, but... all of these doesn't work yet because it is only at the line .save where the db uniqueness index validation gets called. Therefore, I managed to solve this problem by putting .save before those conditions, and at the same time I added a rescue block for .save which then adds another job to the queue of itself should it trigger the ActiveRecord::RecordNotUnique error, to make sure that async jobs won't get conflicts. The code now looks like below.
user = User.find_or_initialize_by(email: 'some-email#address.com')
begin
user.save
is_new_record = user.new_record?
is_persisted = user.persisted?
rescue ActiveRecord::RecordNotUnique => exception
MyJob.perform_later(params_hash)
end
if is_new_record
# do something if not yet created
elsif is_persisted
# do something if already created
end
I would suggest a different architecture to bypass the problem.
How about a producer-worker model, where one master Sidekiq process gets a list of email addresses, and then spawns a worker Sidekiq process for each email? Sidekiq makes this easy with a dedicated queue for master and workers to communicate.
Doing so, the email address becomes an input parameter of workers, so we know by construction that workers will not stump on each other data.

Thread in rails for creating object

In my controller, I have :
#a = Myobject.new params
#b = Myobject.new params2
In the models folder, I have a class (no a model) :
class Myobject
attr_accessor: :var1, :var2, :var3
def initialize params
# Some processes with SQL queries and classes variables.
#var1 = result_of_a_sql_query
#var2 = a_hash
#var3 = params[:varpost]
end
end
But it's slow.
I would like to launch this 2 object creations in parallel. (with thread or other)
I've tried :
t1 = Thread.new {#a = Myobject.new params}
#b = Myobject.new params2
t1.join
But I've got an error : Circular dependency detected while autoloading constant Myobject
How can I launch this 2 commands in parallel ?
Thank you.
If you are working with complex object creation, but you need to get a prompt response back to your user, then it may be best to actually queue your object creation jobs where they can be processed on the system's own time rather than going through the whole thing during your action.
You might look at the various background job gems available to you. On the current application I am working with, updates are dropped into a Resque queue asynchronously and then pulled out and actions performed on the database as possible. This approach can work very well and is fairly standard in cloud environments.
If you need the data available on the front, that would not work as well but you could create simple proxy objects to return most of the data in a view, or simply perform async updates and maintain a front-end version of the data model too.

Run rails code after an update to the database has commited, without after_commit

I'm trying to battle some race cases with my background task manager. Essentially, I have a Thing object (already exists) and assign it some properties, and then save it. After it is saved with the new properties, I queue it in Resque, passing in the ID.
thing = Thing.find(1)
puts thing.foo # outputs "old value"
thing.foo = "new value"
thing.save
ThingProcessor.queue_job(thing.id)
The background job will load the object from the database using Thing.find(thing_id).
The problem is that we've found Resque is so fast at picking up the job and loading the Thing object from the ID, that it loads a stale object. So within the job, calling thing.foo will still return "old value" like 1/100 times (not real data, but it does not happen often).
We know this is a race case, because rails will return from thing.save before the data has actually been commit to the database (postgresql in this case).
Is there a way in Rails to only execute code AFTER a database action has commit? Essentially I want to make sure that by the time Resque loads the object, it is getting the freshest object. I know this can be achieved using an after_commit hook on the Thing model, but I don't want it there. I only need this to happen in this one specific context, not every time the model has commit changed to the DB.
You can put in a transaction as well. Just like the example below:
transaction do
thing = Thing.find(1)
puts thing.foo # outputs "old value"
thing.foo = "new value"
thing.save
end
ThingProcessor.queue_job(thing.id)
Update: there is a gem which calls After Transaction, with this you may solve your problem. Here is the link:
http://xtargets.com/2012/03/08/understanding-and-solving-race-conditions-with-ruby-rails-and-background-workers/
What about wrapping a try around the transaction so that the job is queued up only upon success of the transaction?
I had a similar issue, where by I needed to ensure that a transaction had commited before running a series of action. I ended up using this Gem:
https://github.com/Envek/after_commit_everywhere
It meant that I could do the following:
def finalize!
Order.transaction do
payment.charge!
# ...
# Ensure that we only send out items and perform after actions when the order has defintely be completed
after_commit { OrderAfterFinalizerJob.perform_later(self) }
end
end
One gem to allow that is https://github.com/Ragnarson/after_commit_queue
It is a little different than the other answer's after_commit_everywhere gem. The after_commit_everywhere call seems decoupled from current model being saved or not.
So it might be what you expect or not expect, depending on your use case.

ActiveRecord - working within one connection

For example, suppose there is the code in Rails 3.2.3
def test_action
a = User.find_by_id(params[:user_id])
# some calculations.....
b = Reporst.find_by_name(params[:report_name])
# some calculations.....
c = Places.find_by_name(params[:place_name])
end
This code does 3 requests to database and opens 3 different connections. Most likely it's going to be a quite long action.
Is there any way to open only one connection and do 3 requests within it? Or I want to control which connection to use by myself.
You would want to bracket the calls with transaction:
Transactions are protective blocks where SQL statements are only
permanent if they can all succeed as one atomic action. The classic
example is a transfer between two accounts where you can only have a
deposit if the withdrawal succeeded and vice versa. Transactions
enforce the integrity of the database and guard the data against
program errors or database break-downs. So basically you should use
transaction blocks whenever you have a number of statements that must
be executed together or not at all.
def test_action
User.transaction do
a = User.find_by_id(params[:user_id])
# some calculations.....
b = Reporst.find_by_name(params[:report_name])
# some calculations.....
c = Places.find_by_name(params[:place_name])
end
end
Even though they invoke different models the actions are encapsulated into one call to the DB. It is all or nothing though. If one fails in the middle then the entire capsule fails.
Though the transaction class method is called on some Active Record
class, the objects within the transaction block need not all be
instances of that class. This is because transactions are per-database
connection, not per-model.
You can take a look at ActiveRecord::ConnectionAdapters::ConnectionPool documentation
Also AR doesn't open a connection for each model/query it reuses the existent connection.
[7] pry(main)> [Advertiser.connection,Agent.connection,ActiveRecord::Base.connection].map(&:object_id)
=> [70224441876100, 70224441876100, 70224441876100]

Resources