I have a table called google_records in which every row is a snapshot of a google adwords account for that day.
Here is the schema -
create_table "google_records", :force => true do |t|
t.string "user_id"
t.string "date"
t.text "stats"
t.text "account_name"
t.datetime "created_at", :null => false
t.datetime "updated_at", :null => false
t.decimal "total_cost", :precision => 12, :scale => 2, :default => 0.0, :null => false
t.integer "total_conversions", :default => 0, :null => false
end
add_index "google_records", ["date"], :name => "index_google_records_on_date"
stats contains a hash of the day's stats by campaign and an entry would look something like this:
:Carpet Cleaning:
:conversions: 3
:cost: 391.47
:Upholstery Cleaning:
:conversions: 0
:cost: 69.96
:Air Duct Cleaning:
:conversions: 0
:cost: 8.68
I just added those total_cost and total_conversion columns and I'd like to update those with the totals for each respective value each day.
I can get the total I want to work in my console like so(the cost total doesn't match up with the sample I gave but its only because I shortened the sample to fit better - the total is correct) -
user = User.find(6)
GoogleRecord.where(user_id: user).where(date: "20140328").map {|m| m.stats}.map{ |s| s.map{ |key, value| value[:cost] || 0 } }.map {|m| m.inject(:+)}.compact.reduce(&:+)
=> 660.26
I'd like to update all records in the table this way for conversions and cost and save them but when I try something like:
GoogleRecord.all.each do |g|
g.map {|m| m.stats}.map{ |s| s.map{ |key, value| value[:cost] || 0 } }.map {|m| m.inject(:+)}.compact.reduce(&:+)
end
I keep getting NoMethodError: undefined method map for GoogleRecord:0x5f09868
It seems like since I'm just grabbing one record in the example that is working, I should be able to apply code to each record.
Inside your example code, what the where method return is something like array, so map can work. But inside the GoogleRecord.all.each block, the g is reference to the db record. Remove the first map call, and see whether it will work.
GoogleRecord.all.each do |g|
g.stats.map{ |s| s.map{ |key, value| value[:cost] || 0 } }.map {|m| m.inject(:+)}.compact.reduce(&:+)
end
Related
Hi i'm try to check search engine's performance of my ROR application.
I have 4 search input forms : title, content, created_on (date) and updated_on (date)
I want to check performace of search depending on the presence or absence of an index. (in my case, index presence on created_on and absence on updated_on)
My controller of Post
def index
search_start_time = Time.now
#posts = Post.search(params[:title], params[:content], params[:created_on], params[:updated_on])
# this line for check performance of search
puts Time.now - search_start_time
end
My schema
create_table 'posts', force: :cascade do |t|
t.string 'title', null: false
t.string 'content', null: false
t.date 'created_on', null: false, index: true
t.date 'updated_on', null: false
end
In my post.rb, i maked search method like this
def self.search(title, content, started_on, finished_on)
where([
"title LIKE ? AND content LIKE ? AND CAST(started_on AS text) LIKE ? AND CAST(finished_on AS text) LIKE ?",
"%#{title}%", "%#{content}%", "%#{started_on}%", "%#{finished_on}%"
])
end
With my code, i performance but there were not big difference with search performance of "indexed" and "not indexed" columns.
Is there a problem with my code? Or does the index not affect the search results?
The number of records is 10 million, and an indexed column always comes out similar to the speed of an unindexed column.
I tried to change my search method like this ->
def self.search(title = '', content = '', started_on = '', finished_on = '')
But there was not difference.
I'm looking to find records where #messages.visiblity (array) matches the current_user.id with this simple call:
#messages = Message.where(visibility: [current_user.id])
However it's not returning anything! So frustrating. Here's a sample of a message that should be in that #messages collection:
2.3.3 :007 > Message.last
Message Load (0.4ms) SELECT "messages".* FROM "messages" ORDER BY "messages"."id" DESC LIMIT $1 [["LIMIT", 1]]
=> #<Message id: 35, subject: "hello!", content: "can you see this", user_id: 40, created_at: "2020-08-06 03:22:59", updated_at: "2020-08-06 03:22:59", visibility: ["55", "49"], active: true>
Here the current user.id is 49.
It wont add that message to the #message instance until all user.ids are met, not just the one id that is 49. The record will be added to #messages if I do this:
#messages = Message.where(visibility: ["55", current_user.id]) #DON'T WANT THIS!
But that's ridiculous. Everywhere I'm looking says the first call should work, what am I doing wrong?
Here is my Message schema:
create_table "messages", force: :cascade do |t|
t.string "subject"
t.string "content"
t.bigint "user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.text "visibility", default: [], array: true
and here is the form that collects the data:
<%= form_with(model: #message) do |form| %>
<%= form.check_box(:visibility, {:multiple => true}, u.id, nil) %> - <%= u.email %>
<% end %>
this should work:
Message.where('visibility #> ARRAY[?]::string[]', ["49"])
or this:
Message.where("'49' = ANY(visibility)")
Following is the way how we can do it in databases irrespective of if database supports Array or not.
There is a pattern how serialized array is stored in database.
["55", "49"] will be stored as "55"\n "49"\n (or 55\n 49\n)
So you can search using LIKE operator
#messages = Message.where("visibility LIKE ?", "% #{current_user.id}\n%")
OR
#messages = Message.where("visibility LIKE ?", '% "#{current_user.id}"\n%')
Note: There is space between % and # in % #{current_user.id}
You can use this also
Message.where("visibility && array[?]",current_user.id.to_s)
I am casting current_user.id to STRING, because visibility is a STRING ARRAY and it cannot be compared to integer value of ID
You can also specify it as below:
Message.where("visibility && array[?]",[current_user.id.to_s])
Or with Multiple value array as below:
Message.where("visibility && array[?]",['55','49'])
Multiple values are compared as AND operation.
I create Rails APP and I would like to fetch data batch starting from specific point. I use AR and my table structure looks following:
create_table(:types) do |t|
t.string :name, null: false
t.string :type, null: false
t.string :type_id, null: false
t.text :metadata
t.timestamps
end
To get data I use type_id which is in following format (GUID):
"b2d506fd-409d-4ec7-b02f-c6d2295c7edd"
I would like to fetch specific count of data, ascending or descending ,starting from specific type_id. To be more specific I want do do something like this:
Model.get_batch(type_id: type, count: 20).desc
Can I do it simply in ActiveRecord?
You can use ActiveRecord::Batches to find records in batches
example
Model.where('your condition').find_in_batches(start: 2000, batch_size: 2000) do |group|
# do something with batch
end
check also ActiveRecord::Batches.find_in_batch
You can do like following
Model.find_by_type_id(type).offset(batch_offset).limit(amount_in_batch)
Or as on answer above
Model.where(type_id: your_value).find_in_batches(start: 2000, batch_size: 2000) do |group|
# do something with batch
end
I have just changed a column (called time) from t.string to t.datetime and then dropped and re-created the database and run the migration.
I have a script set to run every minute that scrapes information from a remote website and then adds the information to new records based on the time column that I adjusted to be a datetime rather than string.
# Add each row to a new call record
page = agent.page.search("table tbody tr").each do |row|
next if (!row.at('td'))
time, source, destination, duration = row.search('td').map{ |td| td.text.strip }
call = Call.find_or_create_by_time(time)
call.update_attributes({:time => time, :source => source, :destination => destination, :duration => duration})
end
Since changing the time column to integer the script doesn't seem to be importing any information at all. I wondered if there is an extra step that I need to do to make this work again?
My schema looks like this:
create_table "calls", :force => true do |t|
t.string "source"
t.string "duration"
t.datetime "time"
t.string "destination"
t.string "recording"
t.string "cost"
t.datetime "created_at"
t.datetime "updated_at"
end
In this part
time, source, destination, duration = row.search('td').map{ |td| td.text.strip }
call = Call.find_or_create_by_time(time)
you get time variable as a string, and trying to find_by it. I think smth like
call = Call.find_or_create_by_time(Time.parse(time))
should do the trick
HI
whats the best way to implement a simple search in rails to retrieve all models with the same date?
e.g retrieve all customer with birthday date on 23-10-1975, which will be of a date format.
i.e
create_table "customer", :force => true do |t|
t.string "f_name"
t.string "m_name"
t.string "l_name"
t.date "date_of_birth"
end
I presume you want to congratulate all users that have their birthday today?
Well, I'm not on a dev-machine now, so I can't test it, but I would go for something like this:
#bday_customers = Customer.find(:all, :conditions => { :birthday => Date.today } )
or even
#bday_customers = Customer.find(:all, :conditions => { :birthday => (Date.today)..(Date.today + 2.weeks) } )
This will cause your database to do the work as it its optimised for such searches.
Assuming date is a string of the form 23-10-1975 then something like this;
Customer.all(:conditions => { :date_of_birth => Date.strptime(date, '%d-%m-%Y') })