I'm loading data from my database and I'm doing a sum calculation with a group by.
ElectricityReading.sum(:electricity_value, :group => "electricity_timestamp", :having => ["electricity_timestamp = '2010-02-14 23:30:00'"])
My data sets are extremely large, 100k upwards so I was wondering if its possible to use the find_each to batch this to help with memory overhead.
I can write the batching manually use limit and offset I guess but I'd like to avoid that if the code already exists.
From http://railsforum.com/viewtopic.php?pid=88198#p88198
#categories = Categories.find(:all, :joins => :animals,
:select => "categories.*, SUM(animals.weight) as weight_sum",
:group => "categories.id")
# ATTENTION: weight_sum is now a temporary attribute of the categories returned!
# and the animals are NOT eager-loaded`
<% #categories.each do |c| %>
Category: <%= c.name %><br />
Sum of Weight in this category: <%= c.weight_sum %><br />
<% end %>
It isn't ActiveRecord.sum, but it should do the trick.
Related
Am I doing something wrong?
If I don't have the with options added location works and I get results shown as soon as I enter the browser page of my website.
When i hit the page all users are shown by 20 per page. If I add with_all gender => params[:gender], location still works and I type in a location and filter results by gender and results are successfully returned.
If I add ethnicity to the with_all hash then ethnicity works and results are turned but gender and location no longer work.
It's like it only allows 1 attribute for filtering.
I have rebuilt the index several times so I don't get what's going on.
I've got text search for location and 2 filters set 1. gender, 2. ethnicity
Here is my Profile model for the profiles table that stores all the attributes above:
define_index do
indexes location
has ethnicity, :type => :integer
has gender, :type => :integer
end
Here is my controller:
class BrowsersController < ApplicationController
def index
#default_image = "/assets/default_avatar.jpg"
#gender = params[:gender].to_i
#users = Profile.search params[:location],
:page => params[:page],
:per_page => 20,
:with_all =>
{
:gender => params[:gender],
:ethnicity => params[:ethnicity]
}
end
end
my view form:
<%= form_tag browsers_path, :method => 'get' do %>
<p>
Location: <%= text_field_tag :location, params[:location] %><br />
Gender: <%= select_tag :gender,
options_for_select([["Select", nil],
["Male", 1],
["Female", 2]], params[:gender]) %>
<br />
Ethnicity: <%= select_tag :ethnicity,
options_for_select([["Select", nil],['Black', 1 ],['White / Caucasian', 2 ],['European', 3 ],['Asian', 4 ],['Indian', 5 ],['Middle Eastern', 6 ],['Native American', 7 ],['Hispanic', 8 ],['Mixed Race', 9 ],['Other Ethnicity', 10 ]], params[:ethnicity]) %>
<br />
<%= submit_tag "Search", :name => nil %>
</p>
<% end %>
There's a lot to digest in your question, but here's a few things to note - perhaps they will help:
:with_all is for matching multiple values in a single multi-value attribute - for example, matching an article that has all three tag ids would use this: :with_all => {:tag_ids => [1, 2, 3]}.
:with, however, is perfectly fine for having filters on more than one attribute - which is what you seem to be after (although :with_all with single filter values behaves in just the same way).
Sphinx treats nils/NULLs as 0's - so, if you're filtering by a gender but not an ethnicity, then what your controller code is doing is searching for profiles with the given gender and an ethnicity of 0. Perhaps try something like this instead:
filters = {}
filters[:gender] = params[:gender].to_i if params[:gender].present?
filters[:ethnicity] = params[:ethnicity].to_i if params[:ethnicity].present?
#users = Profile.search params[:location],
:page => params[:page],
:per_page => 20,
:with => filters
Finally - the gender and ethnicity columns are integers, yes? If so, you don't need to specify :type => :integer in your index definition - that'll be done automatically.
I have a collection of Episodes which is connected to a Season and a Show.
I need to display them as such:
Show title
....Season number 1
........Episode name
........Episode name
....Season number 2
........Episode name
........Episode name
My controller:
def index
#show_ids = Following.find_all_by_user_id(current_user.id).collect(&:show_id)
#seen_ids = SeenEpisode.find_all_by_user_id(current_user.id).collect(&:episode_id)
if #seen_ids.any?
#episodes = Episode.find(:all, :conditions => ["show_id IN (?) AND episodes.id NOT IN (?)", #show_ids, #seen_ids], :joins => [:show, :season])
else
#episodes = Episode.find(:all, :conditions => ["show_id IN (?)", #show_ids], :joins => [:show, :season])
end
end
My view:
<ul>
<% #episodes.group_by(&:show).each do |show, episodes| %>
<li><h2><%= show.name %></h2></li>
<% episodes.group_by(&:season).each do |season, episodes| %>
<li><strong><%= season.number %></strong></li>
<% episodes.each do |episode| %>
<li><%= episode.name %></li>
<% end %>
<% end %>
<% end %>
</ul>
This works fine, although I know this is not a good method, and the performance is SHIT (like 10 seconds for about 150 records). How can I have a grouping like this with good performance?
Also, how can I refactor this?
if #seen_ids.any?
#episodes = Episode.find(:all, :conditions => ["show_id IN (?) AND episodes.id NOT IN (?)", #show_ids, #seen_ids], :joins => [:show, :season])
else
#episodes = Episode.find(:all, :conditions => ["show_id IN (?)", #show_ids], :joins => [:show, :season])
end
First, make sure your database has indexes on any foreign key columns you're querying against (I generally index anything_id as a matter of course:
add_index :episodes, :show_id
add_index :followings, :user_id
To clean up your finds, try something like this (from your updated post):
#episodes = Episode.scoped(
:conditions => ["show_id IN (?)", #show_ids],
:include => :show )
if #seen_ids.present?
#episodes = #episodes.scoped(
:conditions => "seen_episodes.show_id IS NULL",
:joins => :seen_episodes )
end
The above assumes you're using Rails 2 (since you were using the .find(:all) syntax...) but you can clean that up further by using .where, etc. instead of .scoped if you're on Rails 3.
using the "NOT IN" clause is generally slow. Instead left join on the SeenEpisode table and add a condition where SeenEpisode is NULL
Episode.find(:all, :joins => "LEFT JOIN SeenEpisode ON SeenEpisode.show_id = Episode.show_id", :conditions => "SeenEpisode.show_id IS NULL")
Note that I omitted some of the clauses for clarity. What this does is keep all records from Episode and add in columns from SeenEpisode that match. The condition then takes out those matching records.
I noticed that my db got queried Select * from shows Where Id = 100 for each record in the loop (show.name). I'm guessing the join did not work because of ambiguous named columns (episodes.name and shows.name)
This is what I ended up with.
query:
#episodes = Episode.find(:all, :select => "episodes.*, shows.name AS show_name", :conditions => ["show_id IN (?) AND seen_episodes.episode_id IS NULL", #show_ids], :joins => "INNER JOIN shows ON shows.id = episodes.show_id LEFT JOIN seen_episodes ON seen_episodes.episode_id = episodes.id")
view:
<ul>
<% #episodes.group_by(&:show_name).each do |show_name, episodes| %>
<li><h2><%= show_name %></h2></li>
<% episodes.group_by(&:season_number).each do |season_number, episodes| %>
<li><strong><%= season_number %></strong></li>
<% episodes.each do |episode| %>
<li><%= episode.name %></li>
<% end %>
<% end %>
<% end %>
</ul>
Also, I already had the season_number in a "cache column" on each episode.
I think this is OK. The query is not very pretty, but at least I like the result:
Completed in 109ms (View: 31, DB: 15)
I'm creating an application that'll display a random picture based upon a defined letter in a word.
Images are attached to a Pictures model (containing another "letter" field) using Paperclip, and will be iterated through in an each block.
How would I go about passing the letter back from the each block to the model for random selection.
This is what I've come up with so far, but it's throwing the following error.
undefined method `%' for {:letter=>"e"}:Hash
Model:
def self.random(letter)
if (c = count) != 0
find(:first, :conditions => [:letter => letter], :offset =>rand(c))
end
end
View:
<% #letters.each do |a| %>
<%= Picture.random(a).image(:thumb) %>
<% end %>
Thanks
One problem is your conditions has a syntax error. The hash notation is wrong:
:conditions => [:letter => letter]
should be
:conditions => {:letter => letter}
Also, it seems to me that your random scope will always exclude the first Picture if you don't allow an offset of 0. Besides that, do you really want to return nil if the random number was 0?
Picture.random(a).image(:thumb) would throw "undefined method 'image' for nil:NilClass" exception every time c==0. Can probably just use:
def self.random(letter)
find(:first, :conditions => {:letter => letter}, :offset =>rand(count))
end
EDIT: You'll either need to guarantee that your db has images for all letters, or tell the user no image exists for a given letter.
<% #letters.each do |a| %>
<% if pic = Picture.random(a).image(:thumb) %>
<%= pic.image(:thumb) %>
<% else %>
No image available for <%= a %>
<% end %>
<% end %>
Or the like...
EDIT: Actually I don't think your offset strategy will work. One other approach would be to return the set of images available for the given letter and randomly select from that collection, something like:
def self.random(letter)
pics = find(:all, :conditions => {:letter => letter})
pics[rand(pics.size)] if !pics.blank?
end
I have the following line in a view:
<%= f.select(:province_id, options_from_collection_for_select(Province.find(:all, :conditions => { :country_id => #property.country_id }, :order => "provinces.name ASC"), :id, :name) %>
In the province model I have the following:
def name
I18n.t(super)
end
Problem is that the :name field is translated (through the province model) and that the ordering is done by activerecord on the english name. The non-english result set can be wrongly sorted this way. We have a province in Belgium called 'Oost-Vlaanderen'. In english that is 'East-Flanders". Not good for sorting:)
I need something like this, but it does not work:
<%= f.select(:province_id, options_from_collection_for_select(Province.find(:all, :conditions => { :country_id => #property.country_id }, :order => "provinces.I18n.t(name) ASC"), :id, :name) %>
What would be the best approach to solve this?
As you may have noticed, my coding knowledge is very limited, sorry for that.
You need to sort the entries after they have been loaded from the database. This should probably do the trick:
Provinces.find(:all, :conditions => {:country_id => #property.country_id}).sort_by{|p| p.name}
I am fairly new to RoR and hacking on a small application. In order to display existing values of some model column in a form, I currently do:
<% form_for([#testbox, #testitem]) do |f| %>
<%= f.label :sortkey %><br />
<%= f.collection_select :sortkey, Testitem.groups , :sortkey, :sortkey, {:include_blank => true} %>
<% end %>
In the model Testitem I have:
def Testitem.groups
return find_by_sql("SELECT DISTINCT sortkey from testitems; ")
end
I am sure there is a more elegant solution to this? I have tried find(:all).sortkey.unique but that throws undefined method 'sortkey' for #<Array:0x59d1b60>
This accomplishes the same thing but I'm not sure it's any more elegant:
Testitem.find(:all, :select => "DISTINCT sortkey")
It's probably much faster to do it Patrick Robertson's way. You can also do it with the group option:
Testitem.find(:all, :group => "sortkey")
using the :select option will put the DISTICT sortkey into the SQL query, while using :group adds GROUP BY sortkey. I don't know which one is faster.
You could also do it by just returning the strings of sortkey as well:
Testitem.find(:all).map(&:sortkey).uniq
This is probably slower because it will retrieve all the items and do the filtering on the Ruby side instead of the SQL side.