I'm using gem bulk_insert for the first time: https://github.com/jamis/bulk_insert
I successfully used the gem to bulk copy records from one table to another. Later on, I needed to report the number of new records. I don't see anyway to get a row count back from bulk_insert, so I turned to return_primary_keys and result_sets to get a count, as shown in the Readme.
I added inserted = to line 3 and added the last line below:
columns = %i[first_name, last_name, email, referal]
inserted = User.bulk_insert(*columns, ignore: true, return_primary_keys: true) do |bulk|
bulk.set_size = BATCH_SIZE
registrants.select(:fname, :lname, :email).find_in_batches(batch_size: BATCH_SIZE) do |batch|
batch.each do |reg|
bulk.add [reg.fname, reg.lname, reg.email, 'self-registered']
end
end
end
puts "added #{inserted.result_sets.count} self-registered users"
Now, I get NoMethodError: undefined method 'result_sets' for nil:NilClass on the puts line.
I've read the Readme several times and searched for the problem with no results. Also checked that I've got the latest version - 1.7.0
What am I missing? How can I get to result_sets? Or better yet, can I get a record count without retrieving the entire list of new primary keys?
According to this issue on the repo, you need to create a bulk_insert worker first. So, in your case I think it would look something like this:
columns = %i[first_name, last_name, email, referal]
insert_worker = User.bulk_insert(*columns, ignore: true, return_primary_keys: true)
insert_worker.set_size = BATCH_SIZE
registrants.select(:fname, :lname, :email).find_in_batches(batch_size: BATCH_SIZE) do |batch|
batch.each do |reg|
insert_worker.add [reg.fname, reg.lname, reg.email, 'self-registered']
end
end
puts "added #{insert_worker.result_sets.count} self-registered users"
I found the answer in https://github.com/jamis/bulk_insert/issues/35, with credit to butnaruandrei. I had to create the bulk_insert object first, then call .add() later...as opposed to passing a block to bulk_insert with .add() within.
IMPORTANT: I also had to add inserter.save! after all add's. Without this, the last half-batch never gets saved...where it did get saved when passing a block in. This was not obvious at first, as everything seemed to be working...but the last few records were not saved without it.
This code works correctly:
columns = %i[first_name, last_name, email, referal]
inserter = User.bulk_insert(*columns, ignore: true, return_primary_keys: true)
inserter.set_size = BATCH_SIZE
registrants.select(:fname, :lname, :email).find_in_batches(batch_size: BATCH_SIZE) do |batch|
batch.each do |reg|
inserter.add [reg.fname, reg.lname, reg.email, 'self-registered']
end
end
inserter.save!
count = inserter.result_sets.map(&:rows).flatten.count
puts "added #{count} self-registered users"
Related
We're trying to optimize our search requests by making a single bulk search, so we're trying to use Searchkick.multi_search. However, it only returns Searchkick::Query with the results populated in a results attribute, as a regular Array.
Then, now if I try results.with_highlights... it fails with
undefined method `with_highlights' for #<Array:0x000055a82a7440f0>
Or if I try on the search_query.with_highlights it fails with
undefined method `with_highlights' for #<Searchkick::Query:0x00007f47c5d0cde8>
How can I get the highlights when using multi_search?
Updated Answer for Searchkick 4.6.1+
Talked to Ankane from Searchkick here https://github.com/ankane/searchkick/pull/1518. He ended up releasing a new version with fixes to this then the original answer here is only valid up to Searchkick version 4.6.0.
For 4.6.1+ just do:
groups = Group.search(query, execute: false)
users = User.search(query, execute: false)
Searchkick.multi_search([groups, users])
highlighted_groups_results = groups.with_highlights(...
...
Original answer for 4.6.0-
Got it!
After diving into the Searchkick codebase and checking the Searchkick::Query implementation, discovered that the execute method is what I need.
def execute
#execute ||= begin
begin
response = execute_search
if retry_misspellings?(response)
prepare
response = execute_search
end
rescue => e # TODO rescue type
handle_error(e)
end
handle_response(response)
end
end
https://github.com/ankane/searchkick/blob/230ec8eb996ae93af4dc7686e02555d995ba1870/lib/searchkick/query.rb#L101
handle_response(response) is exactly what we need for making with_highlights work.
Then my final implementation ended up being something like the following:
groups = Group.search(query, execute: false)
users = User.search(query, execute: false)
Searchkick.multi_search([groups, users])
# execute here won't do any additional requests as it's already cached in an instance variable #execute
highlighted_groups_results = groups.execute.with_highlights(...
...
I have a concept called Transactions. Each transaction can consist of a specific number_of_stocks- say 10.
What I'm now looking at doing, is creating a new record (Share) for each of that number_of_stocks.
Example: If #transaction.number_of_stocks = 10 then we should create 10 new records of Share.
The code below works if I just create one at a time, but I haven't found a way to create a variable number of new records depending on the #transaction.number_of_stocks variable.
In addition, and to make things trickier - each Share has an attribute share_number. What I would like to do is that for each new Share record created, the share_number should increment, but it should also check if there are any existing share_numbers belonging to the cap-table already, and "start there".
Just for context, I'll probably wrap this functionality into a TransactionsHelper somewhere so I can use it elsewhere.
#number_of_share_numbers_to_create = #transaction.number_of_stocks # Example: 10
Share.create( #TODO: Now we need to create 10 records here
owner_id: params[:buying_shareholder_id],
captable_id: #transaction.captable.id,
company_id: #transaction.company.id,
share_number: #TODO: Increment based on the latest share_number linked to this captable and then for each new record in this transaction
)
The model
class Share < ApplicationRecord
belongs_to :captable
belongs_to :company
end
You'll want to look at the times method on Integer. Something, perhaps, like:
#transaction.number_of_stocks.times do |i|
Share.create( #TODO: Now we need to create 10 records here
owner_id: params[:buying_shareholder_id],
captable_id: #transaction.captable.id,
company_id: #transaction.company.id,
share_number: #TODO: Increment based on the latest share_number linked to this captable and then for each new record in this transaction
)
end
If number_of_stocks is really a decimal and not an integer, as you state in your comment above, then you'll need to fiddle in order to make times work and to manage fractional shares. But, that's a different issue.
As an aside, will you always be transacting only stocks? What about other asset forms (e.g., bonds), derivatives (e.g., options), or other transactable assets/instruments? If at some point you will be transacting other assets/instruments, you might want to think about that now. Just a thought.
I solved the share_number piece with the following
A quick method to get the latest share_number
def get_latest_share_number_from_captable(captable)
#captable = Captable.find(captable.id)
#shares = #captable.shares
#share_numbers = []
#shares.each do |s|
#share_numbers.push(s.share_number)
end
#latest_share = #share_numbers.max
return #latest_share
end
Then I use that method and the .times solution suggested in order to create new records.
# Add share numbering for this transaction
#number_of_share_numbers_to_create = #transaction.number_of_stocks.to_i
# Get last sharenumber in captable
#latest_share = get_latest_share_number_from_captable(#transaction.captable)
#number_of_share_numbers_to_create.times do |i|
Share.create(
owner_id: params[:buying_shareholder_id],
captable_id: #transaction.captable.id,
company_id: #transaction.company.id,
share_number: #latest_share += 1 #Increment for each new record
)
end
I've generated User model with attributes: first_name, last_name, city. Then, I've created 200 instances of the class using seed.rb and gem 'faker'. After all, I have added one more attribute to this model - age:string, so now in every instance of the class the age = 'nil'. Now, I want to populate every single user with randomly generated number from range 10..99.
Here is my code from seed.rb:
users = User.all
users.each do |element|
element.age = rand(10..99).to_s
end
When I type rake db:seed it seems to be successfull but when I check the database, each age:string is still 'nil'. Do you have any idea what might have gone wrong?
You are assigning the value, but you don't save it. You should change your code to
users = User.all
users.each do |element|
element.age = rand(10..99).to_s
element.save!
end
or just
users = User.all
users.each do |user|
user.update_attribute(:age, rand(10..99))
end
If you don't need to validate the record, the following is even faster
users = User.all
users.each do |user|
user.update_column(:age, rand(10..99))
end
#Simone Carletti's answer is good... maybe you'd benefit from find_each:
User.find_each do |user|
user.update_attribute :age, rand(10..99)
end
Also, you can combine Faker with Fabrication for more cool seeds.
Faker::Name.name #=> "Christophe Bartell"
Faker::Internet.email #=> "kirsten.greenholt#corkeryfisher.info"
You can use it to generate random data from db/seed.rbrunning rake db:seed.
n00b question. I'm trying to loop through every User record in my database. The pseudo code might look a little something like this:
def send_notifications
render :nothing => true
# Randomly select Message record from DB
#message = Message.offset(rand(Message.count)).first
random_message = #message.content
#user = User.all.entries.each do
#user = User.find(:id)
number_to_text = ""
#user.number = number_to_text #number is a User's phone number
puts #user.number
end
end
Can someone fill me in on the best approach for doing this? A little help with the syntax would be great too :)
Here is the correct syntax to iterate over all User :
User.all.each do |user|
#the code here is called once for each user
# user is accessible by 'user' variable
# WARNING: User.all performs poorly with large datasets
end
To improve performance and decrease load, use User.find_each (see doc) instead of User.all. Note that using find_each loses the ability to sort.
Also a possible one-liner for same purpose:
User.all.map { |u| u.number = ""; puts u.number }
The ActiveRecord find_or_create_by dynamic finder method allows me to specify a block. The documentation isn't clear on this, but it seems that the block only runs in the create case, and not in the find case. In other words, if the record is found, the block doesn't run. I tested it with this console code:
User.find_or_create_by_name("An Existing Name") do |u|
puts "I'M IN THE BLOCK"
end
(nothing was printed). Is there any way to have the block run in both cases?
As far as I understand block will be executed if nothing found. Usecase of it looks like this:
User.find_or_create_by_name("Pedro") do |u|
u.money = 0
u.country = "Mexico"
puts "User is created"
end
If user is not found the it will initialized new User with name "Pedro" and all this stuff inside block and will return new created user. If user exists it will just return this user without executing the block.
Also you can use "block style" other methods like:
User.create do |u|
u.name = "Pedro"
u.money = 1000
end
It will do the same as User.create( :name => "Pedro", :money => 1000 ) but looks little nicer
and
User.find(19) do |u|
..
end
etc
It doesn't seem to me that this question is actually answered so I will. This is the simplest way, I think, you can achieve that:
User.find_or_create_by_name("An Existing Name or Non Existing Name").tap do |u|
puts "I'M IN THE BLOCK REGARDLESS OF THE NAME'S EXISTENCE"
end
Cheers!