My latest migration runs very slowly (600 seconds) even though it is not doing much
I have a model that contains tags in a string format separated with commas like so :
Model.tags = "TAG1, TAG2, TAG3"
I want to create a new Tag model that has a has_and_belongs_to_many relation with my model
Here is the migration
def self.up
rename_column :pizzas, :tags, :tags_migration
create_table :tags do |t|
t.string :name
t.integer :count
t.timestamps
end
create_join_table :tags, :pizzas do |t|
t.index [:tag_id, :pizza_id]
end
Pizza.all.each do |pizza|
pizza_tags = pizza.tags_migration
unless pizza_tags.empty?
pizza_tags_array = pizza_tags.split(', ')
pizza_tags_array.each do |tag|
t = Tag.find_by(name: tag)
if t.nil?
t = Tag.new
t.name = tag
t.count = 1
else
t.count = t.count + 1
end
t.pizzas << pizza
t.save
pizza.tags << t
pizza.save
end
end
puts "pizza n" + pizza.id.to_s
end
end
I don't think this code is supposed to take this long ( i have around 2000 entries )
It looks like you may be holding quite a bit in memory (due to Pizza.all). A simple performance benefit would be to change Pizza.all.each to Pizza.find_each
A good point by #codenamev about find_each
You have n+1 query in Tag.find_by(name: tag)
And also here
-
t.pizzas << pizza
t.save
pizza.tags << t
pizza.save
You can run the whole thing in a transaction to commit all the changes at once (but it may cause locks)
I'm not sure about the implementation details but the code above could be cut in half as t.pizzas << pizza will assign both models. has_many ... :through association should handle this
Also, consider moving the update part outside of migration as it will lock the database for a while
Related
I am trying to populate a new table from an existing database but my method does not seem to be working properly. Below is my code.
class CreateEmployees < ActiveRecord::Migration
def change
create_table :employees do |t|
t.string :first_name, null: false
t.string :last_name, null: false
t.string :email, null: false
t.timestamps
end
Sale.find_each do |sale|
unless Employee.exists?(sale.employee)
puts "Employee #{sale.employee} created!"
else
puts "Employee #{sale.employee} already existed!"
end
employee_info = sale.employee.split
Employee.find_or_create_by(first_name: employee_info[0], last_name: employee_info[1], email:employee_info[2])
end
end
end
What I have is a main database called sales that with a field that contains employee. In that field you will find a string entry as so: "Mary Higgins higgins#korning.com".
Basically the sales database contains four distinct employees but the employees are listed many times. What I'm trying to do is to create four unique rows. I thought the code above would work but something seems to be off with my logic. When I run the above code it, goes through the n amount of rows and creates the Employee object so, essentially the unless statement never results to true for some reason. Could the problem lie in the .find_each method. Would a .each suffice? I don't know if any more information would need to be provided with my database but if its needed I'll supply more details.
sale.employee is a string eg "Mary Higgins higgins#korning.com"
exists? excepts a hash with the conditions like Employee.exists?(:email => "higgins#korning.com"). If you pass a string like you did, first, it converts the string to an integer then tries to find the record with that id which in your case will be 0 and because of that it always returns false.
I would change the find_each loop like this:
Sale.find_each do |sale|
employee_info = sale.employee.split
employee = Employee.find_or_create_by(first_name: employee_info[0], last_name: employee_info[1], email:employee_info[2])
if employee.new_record?
puts "Employee #{sale.employee} created!"
else
puts "Employee #{sale.employee} already existed!"
end
end
I'm worried about the efficiency of this line in the controller of my Rails project
posts_list = Post.where(:title => params[:title])
If the number of "Posts" in the database grows, will the line become slow to execute? Is there any possible optimization?
It just fires this query,
select * from posts where title = params[:title]
You can index the title column in your migration file
add_index(:posts, :title)
Add index on title field can be a first approach :
class AddIndexTitleToPost < ActiveRecord::Migration
def change
add_index :posts, :title, uniq: false
end
end
And you can use find_each on your iteration for prevent your database growing
Post.where(title: params[:title]).find_each(batch_size: 10) do |post|
...
end
That all for applicative enhancement
I'm trying to write a migration and it looks something like this:
class AddStatusToWorks < ActiveRecord::Migration
def self.up
change_table :works do |t|
t.string :status
end
end
def self.down
change_table :works do |t|
t.remove :status
end
end
end
Thing is, I want to set different default values for "status" based on a boolean value that's already in the table, "complete." If complete = true, status = "complete." If not, status = "work in progress." (The reason I want a string instead of keeping complete as the boolean is because I want there to be able to be more than two possibilites for status.) Any idea how to do that? Do I just stick an if statement in there like this
change_table :works do |t|
t.string :status
if (:complete == true)
:value => "complete"
else
:value => "wip"
end
Er, so that doesn't look quite right. I googled a bit and found that you can set :default values, but that's not quite what I'm going for. Any ideas/help would be lovely. Thanks!
You don't need a default at all, you just need to add the new column and give it values. Something like this should work:
def self.up
change_table :works do |t|
t.string :status
end
Works.reset_column_information
Works.where(:complete => true).update_all(:status => 'complete')
Works.where(:complete => [false, nil]).update_all(:status => 'wip')
end
See the Migrations Guide for information on reset_column_information.
You could also do it straight in the database but you have to be careful about different boolean representations (PostgreSQL wants 't' and 'f', MySQL wants 1 and 0, SQLite wants 1 and 0 but Rails mistakenly uses 't' and 'f', ...):
t = connection.quote(true)
connection.execute(%Q{
update works
set status = case complete
when #{t} then 'complete'
else 'wip'
end
})
Remember that default values are created immediately when the record is created, and thus don't yet have values for fields w/o defaults. You'll likely want to move this sort of logic to your model. Perhaps in an ActiveRecord callback, i.e. before_validation or before_save.
I'm using Ruby on Rails and I'm storing business hours like this:
CREATE TABLE "business_hours" (
"id" integer NOT NULL PRIMARY KEY,
"business_id" integer NOT NULL FOREIGN KEY REFERENCES "businesses",
"day" integer NOT NULL,
"open_time" time,
"close_time" time)
(which came from the thread at:
Storing Business Hours in a Database )
Now I want to pull the hours out for each day of the week and display them, and I'm trying to find the best (or at least a good) way.
Should I just have a helper method that loops through getting the days (from 0..6) for a given business_id and assign it to a variable for the associated day? I feel like there must be a better way -- with an array, or something, but it's hurting my head thinking about it, because I also have a form of 'select's where any of the hours for a given business can be updated at once.
Thanks for any guidance!
Use the enum column plugin to declare the day field as a enum field.
class BusinessHours < ActiveRecord::Migration
def self.up
create_table :business_hours do |t|
t.integer :business_id, :null => false
t.enum :day, :limit =>[:sun, :mon, :tue, :wed, :thu, :fri, :sat], :nill => false
t.time :open_time, :null => false
t.time :close_time, :null => false
end
end
def self.down
drop_table :business_hours
end
end
Now when you do find on the BusinessHour model you will get the day as a string.
b = BusinessHour.find_by_business_id(2).first
p b.day.to_s.camelize #prints Sun/Mon/Tue etc.
You can use the enum_select and enum_radio form helpers to create list box/radio button group for the enum group:
Since the number of days in a week really is fixed, you can join the table 6 times (plus the original) and do a query for a single row. I'd probably just do a single query and loop through the rows though.
Have you considered serializing the business hours? Using serialization you are essentially storing objects in the database.
class BusinessHour < ActiveRecord::Base
serialize :hours
...
end
BusinessHour.create :business => #business, :hours =>
{:mon => [mon_start_time, mon_end_time], :wed => [wed_start_time, wed_end_time],
...}
Personally I would go with the bitwise approach described in linked question. All you really need to do to make it work is write new accessor methods.
It would be easier to find the business and use the associations to retrieve the business_hours rows.
Try this in your view
<% #business.business_hours.each do |hrs| %>
<%= hrs.day_name %>: Open-<%= hrs.open_time %> Close-<%= hrs.close_time %>
<%- end -%>
In your business_hour.rb model file, create a default scope to make sure the days are always listed in order. You can also create the day_name method to make it easier to display the day.
default_scope :order => 'day ASC'
def day_name
case self.day
when 0 then "Sun"
when 1 then "Mon"
...
end
end
For the purposes of the discussion I cooked up a test with two tables:
:stones and :bowls (both created with just timestamps - trivial)
create_table :bowls_stones, :id => false do |t|
t.integer :bowl_id, :null => false
t.integer :stone_id, :null => false
end
The models are pretty self-explanatory, and basic, but here they are:
class Stone < ActiveRecord::Base
has_and_belongs_to_many :bowls
end
class Bowl < ActiveRecord::Base
has_and_belongs_to_many :stones
end
Now, the issue is: I want there to be many of the same stone in each bowl. And I want to be able to remove only one, leaving the other identical stones behind. This seems pretty basic, and I'm really hoping that I can both find a solution and not feel like too much of an idiot when I do.
Here's a test run:
#stone = Stone.new
#stone.save
#bowl = Bowl.new
#bowl.save
#test1 - .delete
5.times do
#bowl.stones << #stone
end
#bowl.stones.count
=> 5
#bowl.stones.delete(#stone)
#bowl.stones.count
=> 0
#removed them all!
#test2 - .delete_at
5.times do
#bowl.stones << #stone
end
#bowl.stones.count
=> 5
index = #bowl.stones.index(#stone)
#bowl.stones.delete_at(index)
#bowl.stones.count
=> 5
#not surprising, I guess... delete_at isn't part of habtm. Fails silently, though.
#bowl.stones.clear
#this is ridiculous, but... let's wipe it all out
5.times do
#bowl.stones << #stone
end
#bowl.stones.count
=> 5
ids = #bowl.stone_ids
index = ids.index(#stone.id)
ids.delete_at(index)
#bowl.stones.clear
ids.each do |id|
#bowl.stones << Stone.find(id)
end
#bowl.stones.count
=> 4
#Is this really the only way?
So... is blowing away the whole thing and reconstructing it from keys really the only way?
You should really be using a has_many :through relationship here. Otherwise, yes, the only way to accomplish your goal is to create a method to count the current number of a particular stone, delete them all, then add N - 1 stones back.
class Bowl << ActiveRecord::Base
has_and_belongs_to_many :stones
def remove_stone(stone, count = 1)
current_stones = self.stones.find(:all, :conditions => {:stone_id => stone.id})
self.stones.delete(stone)
(current_stones.size - count).times { self.stones << stone }
end
end
Remember that LIMIT clauses are not supported in DELETE statements so there really is no way to accomplish what you want in SQL without some sort of other identifier in your table.
(MySQL actually does support DELETE ... LIMIT 1 but AFAIK ActiveRecord won't do that for you. You'd need to execute raw SQL.)
Does the relationship have to be habtm?
You could have something like this ...
class Stone < ActiveRecord::Base
has_many :stone_placements
end
class StonePlacement < ActiveRecord::Base
belongs_to :bowl
belongs_to :stone
end
class Bowl < ActiveRecord::Base
has_many :stone_placements
has_many :stones, :through => :stone_placements
def contents
self.stone_placements.collect{|p| [p.stone] * p.count }.flatten
end
def contents= contents
contents.sort!{|a, b| a.id <=> b.id}
contents.uniq.each{|stone|
count = (contents.rindex(stone) - contents.index(stone)) + 1
if self.stones.include?(stone)
placement = self.stone_placements.find(:first, :conditions => ["stone_id = ?", stone])
if contents.include?(stone)
placement.count = count
placement.save!
else
placement.destroy!
end
else
self.stone_placements << StonePlacement.create(:stone => stone, :bowl => self, :count => count)
end
}
end
end
... assuming you have a count field on StonePlacement to increment and decrement.
How about
bowl.stones.slice!(0)