I have posts and organisations in my database. Posts belongs_to organisation and organisation has_many posts.
I have an existing post_id column in my post table which I by now increment manually when I create a new post.
How can I add auto increment to that column scoped to the organisation_id?
Currently I use mysql as my database, but I plan to switch to PostgreSQL, so the solution should work for both if possible :)
Thanks a lot!
#richard-huxton has the correct answer and is thread safe.
Use a transaction block and use SELECT FOR UPDATE inside that transaction block. Here is my rails implementation. Use 'transaction' on a ruby class to start a transaction block. Use 'lock' on the row you want to lock, essentially blocking all other concurrent access to that row, which is what you want for ensuring unique sequence number.
class OrderFactory
def self.create_with_seq(order_attributes)
order_attributes.symbolize_keys!
raise "merchant_id required" unless order_attributes.has_key?(:merchant_id)
merchant_id = order_attributes[:merchant_id]
SequentialNumber.transaction do
seq = SequentialNumber.lock.where(merchant_id: merchant_id, type: 'SequentialNumberOrder').first
seq.number += 1
seq.save!
order_attributes[:sb_order_seq] = seq.number
Order.create(order_attributes)
end
end
end
We run sidekiq for background jobs, so I tested this method by creating 1000 background jobs to create orders using 8 workers with 8 threads each. Without the lock or the transaction block, duplicate sequence number occur as expected. With the lock and the transaction block, all sequence numbers appear to be unique.
OK - I'll be blunt. I can't see the value in this. If you really want it though, this is what you'll have to do.
Firstly, create a table org_max_post (org_id, post_id). Populate it when you add a new organisation (I'd use a database trigger).
Then, when adding a new post you will need to:
BEGIN a transaction
SELECT FOR UPDATE that organisation's row to lock it
Increment the post_id by one, update the row.
Use that value to create your post.
COMMIT the transaction to complete your updates and release locks.
You want all of this to happen within a single transaction of course, and with a lock on the relevant row in org_max_post. You want to make sure that a new post_id gets allocated to one and only one post and also that if the post fails to commit that you don't waste post_id's.
If you want to get clever and reduce the SQL in your application code you can do one of:
Wrap the hole lot above in a custom insert_post() function.
Insert via a view that lacks the post_id and provides it via a rule/trigger.
Add a trigger that overwrites whatever is provided in the post_id column with a correctly updated value.
Deleting a post obviously doesn't affect your org_max_post table, so won't break your numbering.
Prevent any updates to the posts at the database level with a trigger. Check for any changes in the OLD vs NEW post_id and throw an exception if there is one.
Then delete your existing redundant id column in your posts table and use (org_id,post_id) as your primary key. If you're going to this trouble you might as well use it as your pkey.
Oh - and post_num or post_index is probably better than post_id since it's not an identifier.
I've no idea how much of this will play nicely with rails I'm afraid - the last time I looked at it, the database handling was ridiculously primitive.
Its good to know how to implement it. I would prefer to use a gem myself.
https://github.com/austinylin/sequential (based on sequenced)
https://github.com/djreimer/sequenced
https://github.com/felipediesel/auto_increment
First, I must say this is not a good practice, but I will only focus on a solution for your problem:
You can always get the organisation's posts count by doing on your PostsController:
def create
post = Post.new(...)
...
post.post_id = Organization.find(organization_id).posts.count + 1
post.save
...
end
You should not alter the database yourself. Let ActiveRecord take care of it.
Related
i have Account Model,Asset, Capital and Revenue this table are all inherited in my Account model. i have 3 kind of attributes in my Account model. name, code and type. when i create an account where will be to insert will happen one in my account and the other one is in my type for example
Account.create(name: "test123", code:"test123", type:"Asset")
sql will run Two Insert one for Account model and one for Asset Table
and my sunspot work well it will reindex my database and i can search my params
but when i update my model Account my sql run one insert and one update
my question is how can i reindex my model when i update. with a particular data. i can do Sunspot.reindex but this is will load all data in my sql. that will cause me to slow
sql will run Two Insert one for Account model and one for Asset Table
FYI you use STI when you want to share same database table between multiple models because they are similar in attributes and behavior. Like AdminUser model is likely to have almost same attributes/columns as PublisherUser or ReaderUser. Therefore you might wish to have a common table called users or model User and share this table among the above mentioned models.
Point is: ActiveRecord will run a single SQL query not two, like:
INSERT INTO "accounts" ("name", "code", "type") VALUES ('test123', 'test123', 'Asset')
my question is how can i reindex my model when i update. with a particular data. i can do Sunspot.reindex but this is will load all data in my sql. that will cause me to slow
Actually sunspot_rails is designed to auto-reindex whenever you make changes to your model/record. It listens to the save callbacks.
But you need to make sure that you are not using methods like update_column(s). See the list of silent create/update methods which do not trigger callbacks and validations at all.
In addition, you need to understand the concept of batch size in terms of Solr. For performance reasons, all of your new indexes are not immediately committed. Committed means, writing indexes to database like in RDBMS commits.
By default the batch_size for commits is 50. Meaning after 50 index method executions only the indexes will be committed and you will be able to search the records. To change it, use following
# in config/initializers/sunspot_config.rb
Sunspot.config.indexing.default_batch_size = 1 # or any number
or
# in models; its not considered good though
after_commit do
Sunspot.commit
end
For manual re-indexing, you can use like #Kathryn suggested.
But, I don't think you need to intervene in the auto-operation. I think you were not seeing immediate results so you were worrying.
According to the documentation, objects will be indexed automatically if you are on Rails. But it also mentions you can reindex a class manually:
Account.reindex
Sunspot.commit
It also suggests using Sunspot.index on individual objects.
i put this to my model
after_update do
Sunspot.index Account.where(id: self.id)
end
I'm diving into Rails 4 and I'm trying to understand how to safely access model data while it's being accessed by multiple DB connections. I have some match-making logic that finds the oldest user in a queue, removes that user from the queue, and returns the user...
# UserQueue.rb
class UserQueue < ActiveRecord::Base
has_many :users
def match_user(user)
match = nil
if self.users.count > 0
oldest = self.users.oldest_in_queue
if oldest.id != user.id
self.users.delete(oldest)
match = oldest
end
end
end
end
If two different threads executed this match_user method around the same time, is it possible for them to both find the same oldest user and try to delete it from the queue, and return it to the caller? If so, how do I prevent that?
I looked into transactions, but they don't seem to be a solution since there's only one model being modified in this case (the queue).
Thanks in advance for your wisdom!
ActiveRecord has support for row locking.
This is taken from the Rails guide, locking records for update:
11.1 Optimistic Locking
Optimistic locking allows multiple users to access the same record for edits, and assumes a minimum of conflicts with the data. It does this by checking whether another process has made changes to a record since it was opened. An ActiveRecord::StaleObjectError exception is thrown if that has occurred and the update is ignored.
Optimistic locking column
In order to use optimistic locking, the table needs to have a column called lock_version of type integer. Each time the record is updated, Active Record increments the lock_version column. If an update request is made with a lower value in the lock_version field than is currently in the lock_version column in the database, the update request will fail with an ActiveRecord::StaleObjectError. Example:
c1 = Client.find(1)
c2 = Client.find(1)
c1.first_name = "Michael"
c1.save
c2.name = "should fail"
c2.save # Raises an ActiveRecord::StaleObjectError
You're then responsible for dealing with the conflict by rescuing the exception and either rolling back, merging, or otherwise apply the business logic needed to resolve the conflict.
This behavior can be turned off by setting ActiveRecord::Base.lock_optimistically = false.
To override the name of the lock_version column, ActiveRecord::Base provides a class attribute called locking_column:
class Client < ActiveRecord::Base
self.locking_column = :lock_client_column
end
I suggest reading this section in the Rails guide.
Yes, that absolutely could happen. How you prevent it depends on the rest of your app/framework/database/etc.
Transactions won't help because two clients could start the request at the same time and both would see the same UserQueue record as oldest.
You want a mutex. But a mutex in the code isn't ideal if there are other ways to modify the data (ie. directly via SQL, etc.) It can also get messy as the first time you forget to use the mutex, you've opened up the race condition. Then again, maybe this is enough. Just remember your mutex needs to work across threads and processes, etc.
You might see if your database has a mutex or other row level lock you can use to mark the oldest queue record, then extract it.
Or find another way to grab the oldest queue that avoids the race condition entirely. Something like:
SQL to update the oldest queue whose id is not user.id and set "marked_for_work" to some unique ID.
Fetch the queue row whose marked_for_work value is our unique ID.
You could run multiple threads with the above without worry as the SQL update is (well, it should be!) atomic.
In our database, we have a table for comments and blogs.
There's a field comments.comment_blog_index that increments for each comment in the blog.
...so if we have 3 comments for a particular blog, the value for comment_blog_index for each comment is: 1, 2, 3 (respectively)
The code that sets comment_blog_index looks like this:
#comment = Comment.new
#comment.comment_blog_index = #blog.comments.count + 1
The problem happens when two users trigger this code simultaneously. It will calculate the same value for both users, and the comment_blog_index is duplicated.
I've seen code for Item.increment_counter( :total_bids, item.id ), but that requires you to already have a record in the database in a table that stores a summation. In our case, the record is being created inside the `commments`` table.
How do we prevent this?
As you've already seen your current method is not safe. You could add validations and callbacks and all manner of things to try and make it safe but I would suggest that kind of work should happen at the database level.
Either implement a propert auto-incrementing field or offload that work to a gem.
It will help
Generate an auto increment field in rails
It shows how auto increment can be implemented in rails
long time reader first time poster.
I recently started using ruby on rails so I am still very new to the environment (even though I have completed a few guides) so be gentle please.
What I want to do is create a sort of archive table of another table that the user can access at any time(via a different link on the website).
So for example, if I have the "users" table, I want to be able to archive old users but still give the option for someone to go and view them.
Basically, it will sort of have to delete the user from the initial table, and save his/her info in to the archived_users table.
Thank you for your time.
I figured my comment was more of an answer, so posting it here and adding more info
In this situation you're better off adding some sort if "active" flag to the users table, which you can flip on or off as needed. That way you don't need to worry about dealing with yet another model class, and you can reuse all the same view and controller structures. In your views, you can then simply "hide" any inactive users (and maybe only show inactive folks if the logged in user is an admin...etc).
You also have the freedom to include other meta data such as "deactivated on" (time stamp) for example.
Long story short, if you're concerned about performance, with proper indexing (and partitioning if necessary), you shouldn't really need to create a separate archive table.
The only reason I can think of to do this is if you're dealing with billions upon billions of records, and/or growing by an insane amount (which is probably not your case).
The best way to do this is probably to add a column called deleted on the original Users table. You can then filter out the old users in normal circumstances (preferably using a default scope) but allow them to be seen/queried when needed.
Papertrail might work for you.
It creates a "versions" table and logs create/update/destroy events for any class which includes has_paper_trail. For example:
class User < ActiveRecord::Base
has_paper_trail
end
deleted_users = Papertrail::Version.where(item_type: User, event: "destroy")
deleted_users.last.reify.name # assuming the users table has a 'name' column
I've set up a trigger-based partitioning scheme on one of our pg 8.3 databases according to the pg docs here:. Basically, I have a parent table, along with several child tables. An insert trigger on the parent redirects any inserts on the parent into the appropriate child table -- this works well.
The ActiveRecord pg adapter, however, seems to rely on the postgres INSERT ... RETURNING "id" extension to get the id of the returned row after the initial insert. But the trigger seems to break the RETURNING clause -- no id is returned, although the row is created correctly.
While I suppose this behavior makes sense -- after all, nothing is being inserted in the main table, I really need to find some kind of work-around, as other child records will be inserted that require the row id of the just-inserted row.
I suppose I could add some kind of unique id to row prior to insert and then re-read it using this key after insert, but this seems pretty kludgy. Does anyone have a better work-around?
Since Rails v.2.2.1, you can turn off 'returning id' behavior just by overriding #supports_insert_with_returning method in PostgreSQLAdapter.
class ActiveRecord::ConnectionAdapters::PostgreSQLAdapter
def supports_insert_with_returning?
false
end
end
Currently it looks like my best option is to just change the table prefix in a before_create event so that the insert happens on the underlying partition table directly, bypassing the insert trigger altogether. This is not a perfect solution, however, but seems to be the most performant and the simplest.
The only other solution I can come up with is to add a guid column to each table, and re-read the row from the parition table by guid immediately after insert to get the id.
Any other suggestions are welcome. Thanx -- m