I was implementing my first HABTM relationship and have run into an issue with my query.
I am looking to validate my approach and to see if I have found a bug in the AREL (or some other part of Rails) code.
I have the following models
class Item < ActiveRecord::Base
belongs_to :user
belongs_to :category
has_and_belongs_to_many :regions
end
class Region < ActiveRecord::Base
has_ancestry
has_and_belongs_to_many :items
end
I have the associated items_regions table:
class CreateItemsRegionsTable < ActiveRecord::Migration
def self.up
create_table :items_regions, :id => false do |t|
t.references :item, :null => false
t.references :region, :null => false
end
add_index(:items_regions, [:item_id, :region_id], :unique => true)
end
def self.down
drop_table :items_regions
end
end
My goal is to create a scope/query is follows:
Find all items in a region (and its subregions)
The ancestory gem provides a method to retrieve descendant categories for Region as an array. In this case,
ruby-1.9.2-p180 :167 > a = Region.find(4)
=> #<Region id: 4, name: "All", created_at: "2011-04-12 01:14:00", updated_at: "2011-04-12 01:14:00", ancestry: nil, cached_slug: "all">
ruby-1.9.2-p180 :168 > region_list = a.subtree_ids
=> [1, 2, 3, 4]
If there is only one element in the array, the following works
items = Item.joins(:regions).where(["region_id = ?", [1]])
The sql generated is
"SELECT `items`.* FROM `items` INNER JOIN `items_regions` ON `items_regions`.`item_id` = `items`.`id` INNER JOIN `regions` ON `regions`.`id` = `items_regions`.`region_id` WHERE (region_id = 1)"
However, if there are multiple items in the array and I try to use IN
Item.joins(:regions).where(["region_id IN ?", [1,2,3,4]])
ActiveRecord::StatementInvalid: Mysql::Error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '1,2,3,4)' at line 1: SELECT `items`.* FROM `items` INNER JOIN `items_regions` ON `items_regions`.`item_id` = `items`.`id` INNER JOIN `regions` ON `regions`.`id` = `items_regions`.`region_id` WHERE (region_id IN 1,2,3,4)
The sql generated has an error at the end
"SELECT `items`.* FROM `items` INNER JOIN `items_regions` ON `items_regions`.`item_id` = `items`.`id` INNER JOIN `regions` ON `regions`.`id` = `items_regions`.`region_id` WHERE (region_id IN 1,2,3,4)"
the last part of the generated code should be
(region_id IN ("1,2,3,4"))
If I edit the sql manually and run it, I get what I expect.
So, two questions:
Is my approach for the single value case correct?
Is the sql generation a bug or have I configured things incorrectly?
Thanks
Alan
.where('regions.id' => array)
Should work in all cases, whether or not you specify one value or multiple.
The reason your original query doesn't work is that you actually need to specify valid SQL. So alternatively you can do
.where('region_id IN (?)', [1,2,3,4])
The other responders are correct regarding use of the conditions hash, but the specific issue you're running into after that has to do with field specificity:
Mysql::Error: Unknown column 'items.region_id' in 'where clause'
You're trying to draw a condition based on "region_id", but since you didn't explicitly give a table it uses "items" by default. It sounds like your column is actually on the "item_regions" table. Try this:
where("item_regions.region_id IN (?)", [1,2,3,4])
Or alternatively:
where(:item_regions => {:region_id => [1,2,3,4]})
Did you try the
.where('region_id IN (?)', [1,2,3,4])
form? You need the () to be valid.
I think the cleanest and most idiomatic way to do this in Arel is the nested hash syntax which avoids string literals (and any direct reference to the HABTM join table):
Item.joins(:regions).where(regions: { id: [1,2,3,4] })
Related
I am trying to make a scope for my EventsLog model which looks something along the lines of EventsLog.with_values({"value_name" => "value", "other_value_name" => "other_value"}).
The results of which would be the EventsLog records that have an associated EventsLogValue for each of the key-value pairs in the hash.
Here is what I have to work with.
Two tables whose definitions look like this:
--table for tracking events
CREATE TABLE events_log(
id INT PRIMARY KEY IDENTITY(1,1),
event_name VARCHAR(25), --name of the event
created_at DATETIME
);
--table for tracking the values corresponding to the event
CREATE TABLE events_log_values(
id INT PRIMARY KEY IDENTITY(1,1),
event_id INT,
value VARCHAR(255),
value_name VARCHAR(25),
);
From these two tables two models which look like:
class EventsLog < BaseAPIDatabase
self.table_name = "events_log"
self.primary_key = "id"
has_many :events_log_values, :foreign_key => "event_id", :primary_key => "id", :class_name => "EventsLogValue", :autosave => true
scope :since, ->(since){ where("created_at > ?", since)}
scope :named, ->(event_name){ where(:event_name => event_name) }
def values
events_log_values.inject({}) do |hsh, v|
hsh.merge({v.value_name => v.value})
end
end
end
class EventsLogValue < BaseAPIDatabase
self.table_name = "events_log_values"
self.primary_key = "id"
end
My approach so far has been to try and create a function that returns an active record relation which has applied one key-value pair at a time and then later to add a scope (or probably just a class method returning a relation) which chains them for me (something along the lines of scope :with_values, ->(values){values.inject(self){|slf, (k, v)| slf.with_value(k, v)} }).
Originally I tried to implement with_value as a fairly standard scope, scope :with_value, ->(val_name, val){ eager_load(:events_log_values).where(:events_log_values => {:value_name => val_name, :value => val}) }, which works fine by itself but when chained results in a single join with multiple conditions on the joined values.
Deciding that this would be solved by joining the values table with an alias for each condition; my new approach has been to define a has_many association in my with_value function then to eager_load that association and to add a where condition based on each new association:
def self.with_value(val_name, val)
has_many val_name.to_sym, ->(){ where(:value_name => val_name) }, :foreign_key => "event_id", :primary_key => "id", :class_name => "EventsLogValue"
res = eager_load(:events_log_values)
res.eager_load(val_name.to_sym).where("#{val_name.pluralize}_events_log" => {:value => val})
end
This actually works pretty well but has a few problems. The first one being that I have a difficult time knowing what the name in the where condition is going to be for the association. The second (and bigger problem) being that my values function now only has whatever value_names have not had an association made for them.
Here is some sql which was generated by the multiple has_manys and may help to illustrate what I am trying to do:
EventsLog.with_values("hello" => "world", "foo" => "bar").to_sql
SELECT ...
FROM [events_log]
LEFT OUTER JOIN [events_log_values] ON [events_log_values].[event_id] = [events_log].[id]
LEFT OUTER JOIN [events_log_values] [hellos_events_log] ON [hellos_events_log].[event_id] = [events_log].[id] AND [hellos_events_log].[value_name] = 'hello'
LEFT OUTER JOIN [events_log_values] [foos_events_log] ON [foos_events_log].[event_id] = [events_log].[id] AND [foos_events_log].[value_name] = 'foo'
WHERE [hellos_events_log].[value] = 'world' AND [foos_events_log].[value] = 'bar'
How can I go about getting a record which has several associated records meeting several separate conditions?
This is the answer I have been able to come up with since asking my question. It uses arel to generate sql joins with aliases for each of the values and also generates a where condition for each value.
It's not the cleanest thing but it seems to get the job done.
def self.with_values(values)
el = EventsLog.arel_table
arel_joins = el
arel_wheres = []
values.each do |k, v|
ev = EventsLogValue.arel_table.alias("#{k}_join")
arel_joins = arel_joins.join(ev).on(el[:id].eq(ev[:event_id]).and(ev[:value_name].eq(k)))
arel_wheres << ev[:value].eq(v)
end
arel_wheres.inject(EventsLog.joins(arel_joins.join_sources)){|rel, con| rel.where(con)}
end
p.s. I think I read somewhere that Model.arel_table is undocumented and should not be used? It may be prudent to use Arel::Table.new('table_name') instead.
IMPORTANT - READ EDIT BELOW FOR UPDATE ON THE ISSUE
I'm getting what I think is a bogus error when I try to add a new record to a join table with a unique composite key index on SQLite3. Note that for all (manual) tests I've done, the database has been completely rebuilt through db:drop followed by db:migrate.
The error:
ActiveRecord::RecordNotUnique
SQLite3::ConstraintException: columns adventurer_id, item_id are not unique:
INSERT INTO "adventurers_items" ("adventurer_id", "item_id") VALUES (1, 68)
The code that generates the error:
class Adventurer < ActiveRecord::Base
after_create :set_starting_skills
after_create :set_starting_items
has_and_belongs_to_many :items
has_and_belongs_to_many :skills
# automatically add starting skills on creation
def set_starting_skills
self.skills = self.profession.starting_skills
end
# automatically add starting items on creation
def set_starting_items
self.items = self.profession.items
end
The migration creating the join table adventurers_skills:
class AdventurersItems < ActiveRecord::Migration
def change
create_table :adventurers_items do |t|
t.integer :item_id, :null => false
t.integer :adventurer_id, :null => false
end
add_index :adventurers_items, :item_id
add_index :adventurers_items, :adventurer_id
add_index :adventurers_items, [:adventurer_id, :item_id], :unique => true
The table exists and is completely empty. Why is my application failing to insert this record due to the uniqueness constraint? I also have the same error with an equivalent table "adventurers_skills" -- am I doing something wrong architecturally?
EDIT
The system is trying to add the same item/skill twice. When I change the private method to this:
def set_starting_skills
skills = profession.starting_skills
end
It doesn't attempt to create anything in the join table. But reverting the first line to self.skills as below attempts to create the same skill TWICE
def set_starting_skills
self.skills = profession.starting_skills
end
returns
(0.4ms) INSERT INTO "adventurers_skills" ("adventurer_id", "skill_id") VALUES (4, 54)
(4.9ms) INSERT INTO "adventurers_skills" ("adventurer_id", "skill_id") VALUES (4, 54)
SQLite3::ConstraintException: columns adventurer_id, skill_id are not unique:
INSERT INTO "adventurers_skills" ("adventurer_id", "skill_id") VALUES (4, 54)
(3.2ms) rollback transaction
There is only one skill returned for profession.starting_skills:
1.9.3-p194 :022 > Profession.find(7).starting_skills.each {|x| puts x.id}
54
So the real question has become: why is Rails trying to add this HABTM record twice?
You need to put the callback declaration (after_create :set_starting_skills) after the relation declaration (has_and_belongs_to_many :skills).
i.e. The ordering of the lines in the model is significant else you get this bug.
This is madness, and there is a GitHub issue for it.
I am trying to get a list, and I will use books as an example.
class Book < ActiveRecord::Base
belongs_to :type
has_and_belongs_to_many :genres
end
class Genre < ActiveRecord::Base
has_and_belongs_to_many :books
end
So in this example I want to show a list of all Genres, but it the first column should be the type. So, if say a genre is "Space", the types could be "Non-fiction" and "Fiction", and it would show:
Type Genre
Fiction Space
Non-fiction Space
The Genre table has only "id", "name", and "description", the join table genres_books has "genre_id" and "book_id", and the Book table has "type_id" and "id". I am having trouble getting this to work however.
I know the sql code I would need which would be:
SELECT distinct genres.name, books.type_id FROM `genres` INNER JOIN genres_books ON genres.id = genres_books.genre_id INNER JOIN books ON genres_books.book_id = books.id order by genres.name
and I found I could do
#genre = Genre.all
#genre.each do |genre|
#type = genre.book.find(:all, :select => 'type_id', :group => 'type_id')
#type.each do |type|
and this would let me see the type along with each genre and print them out, but I couldn't really work with them all at once. I think what would be ideal is if at the Genre.all statement I could somehow group them there so I can keep the genre/type combinations together and work with them further down the road. I was trying to do something along the lines of:
#genres = Genre.find(:all, :include => :books, :select => 'DISTINCT genres.name, genres.description, books.product_id', :conditions => [Genre.book_id = :books.id, Book.genres.id = :genres.id] )
But at this point I am running around in circles and not getting anywhere. Do I need to be using has_many :through?
The following examples use your models, defined above. You should use scopes to push associations back into the model (alternately you can just define class methods on the model). This helps keep your record-fetching calls in check and helps you stick within the Law of Demeter.
Get a list of Books, eagerly loading each book's Type and Genres, without conditions:
def Book < ActiveRecord::Base
scope :with_types_and_genres, include(:type, :genres)
end
#books = Book.with_types_and_genres #=> [ * a bunch of book objects * ]
Once you have that, if I understand your goal, you can just do some in-Ruby grouping to corral your Books into the structure that you need to pass to your view.
#books_by_type = #books.group_by { |book| book.type }
# or the same line, more concisely
#books_by_type = #books.group_by &:type
#books_by_type.each_pair do |type, book|
puts "#{book.genre.name} by #{book.author} (#{type.name})"
end
I've hit a slight block with the new scope methods (Arel 0.4.0, Rails 3.0.0.rc)
Basically I have:
A topics model, which has_many :comments, and a comments model (with a topic_id column) which belongs_to :topics.
I'm trying to fetch a collection of "Hot Topics", i.e. the topics that were most recently commented on. Current code is as follows:
# models/comment.rb
scope :recent, order("comments.created_at DESC")
# models/topic.rb
scope :hot, joins(:comments) & Comment.recent & limit(5)
If I execute Topic.hot.to_sql, the following query is fired:
SELECT "topics".* FROM "topics" INNER JOIN "comments"
ON "comments"."topic_id" = "topics"."id"
ORDER BY comments.created_at DESC LIMIT 5
This works fine, but it potentially returns duplicate topics - If topic #3 was recently commented on several times, it would be returned several times.
My question
How would I go about returning a distinct set of topics, bearing in mind that I still need to access the comments.created_at field, to display how long ago the last post was? I would imagine something along the lines of distinct or group_by, but I'm not too sure how best to go about it.
Any advice / suggestions are much appreciated - I've added a 100 rep bounty in hopes of coming to an elegant solution soon.
Solution 1
This doesn't use Arel, but Rails 2.x syntax:
Topic.all(:select => "topics.*, C.id AS last_comment_id,
C.created_at AS last_comment_at",
:joins => "JOINS (
SELECT DISTINCT A.id, A.topic_id, B.created_at
FROM messages A,
(
SELECT topic_id, max(created_at) AS created_at
FROM comments
GROUP BY topic_id
ORDER BY created_at
LIMIT 5
) B
WHERE A.user_id = B.user_id AND
A.created_at = B.created_at
) AS C ON topics.id = C.topic_id
"
).each do |topic|
p "topic id: #{topic.id}"
p "last comment id: #{topic.last_comment_id}"
p "last comment at: #{topic.last_comment_at}"
end
Make sure you index the created_at and topic_id column in the comments table.
Solution 2
Add a last_comment_id column in your Topic model. Update the last_comment_id after creating a comment. This approach is much faster than using complex SQL to determine the last comment.
E.g:
class Topic < ActiveRecord::Base
has_many :comments
belongs_to :last_comment, :class_name => "Comment"
scope :hot, joins(:last_comment).order("comments.created_at DESC").limit(5)
end
class Comment
belongs_to :topic
after_create :update_topic
def update_topic
topic.last_comment = self
topic.save
# OR better still
# topic.update_attribute(:last_comment_id, id)
end
end
This is much efficient than running a complex SQL query to determine the hot topics.
This is not that elegant in most SQL implementations. One way is to first get the list of the five most recent comments grouped by topic_id. Then get the comments.created_at by sub selecting with the IN clause.
I'm very new to Arel but something like this could work
recent_unique_comments = Comment.group(c[:topic_id]) \
.order('comments.created_at DESC') \
.limit(5) \
.project(comments[:topic_id]
recent_topics = Topic.where(t[:topic_id].in(recent_unique_comments))
# Another experiment (there has to be another way...)
recent_comments = Comment.join(Topic) \
.on(Comment[:topic_id].eq(Topic[:topic_id])) \
.where(t[:topic_id].in(recent_unique_comments)) \
.order('comments.topic_id, comments.created_at DESC') \
.group_by(&:topic_id).to_a.map{|hsh| hsh[1][0]}
In order to accomplish this you need to have a scope with a GROUP BY to get the latest comment for each topic. You can then order this scope by created_at to get the most recent commented on topics.
The following works for me using sqlite
class Comment < ActiveRecord::Base
belongs_to :topic
scope :recent, order("comments.created_at DESC")
scope :latest_by_topic, group("comments.topic_id").order("comments.created_at DESC")
end
class Topic < ActiveRecord::Base
has_many :comments
scope :hot, joins(:comments) & Comment.latest_by_topic & limit(5)
end
I used the following seeds.rb to generate the test data
(1..10).each do |t|
topic = Topic.new
(1..10).each do |c|
topic.comments.build(:subject => "Comment #{c} for topic #{t}")
end
topic.save
end
And the following are the test results
ruby-1.9.2-p0 > Topic.hot.map(&:id)
=> [10, 9, 8, 7, 6]
ruby-1.9.2-p0 > Topic.first.comments.create(:subject => 'Topic 1 - New comment')
=> #<Comment id: 101, subject: "Topic 1 - New comment", topic_id: 1, content: nil, created_at: "2010-08-26 10:53:34", updated_at: "2010-08-26 10:53:34">
ruby-1.9.2-p0 > Topic.hot.map(&:id)
=> [1, 10, 9, 8, 7]
ruby-1.9.2-p0 >
The SQL generated for sqlite(reformatted) is extremely simple and I hope Arel would render different SQL for other engines as this would certainly fail in many DB engines as the columns within Topic are not in the "Group by list". If this did present a problem then you could probably overcome it by limiting the selected columns to just comments.topic_id
puts Topic.hot.to_sql
SELECT "topics".*
FROM "topics"
INNER JOIN "comments" ON "comments"."topic_id" = "topics"."id"
GROUP BY comments.topic_id
ORDER BY comments.created_at DESC LIMIT 5
Since the question was about Arel, I thought I'd add this in, since Rails 3.2.1 adds uniq to the QueryMethods:
If you add .uniq to the Arel it adds DISTINCT to the select statement.
e.g. Topic.hot.uniq
Also works in scope:
e.g. scope :hot, joins(:comments).order("comments.created_at DESC").limit(5).uniq
So I would assume that
scope :hot, joins(:comments) & Comment.recent & limit(5) & uniq
should also probably work.
See http://apidock.com/rails/ActiveRecord/QueryMethods/uniq
Have Addresses and Lists with many-to-many relationship, as shown below.
Sometimes need all the Lists an Address is not in.
Using the find_by_sql query shown, and it works great. But is there a way to do it without using direct SQL?
class List
has_many :address_list_memberships
has_many :addresses, :through => :address_list_memberships
end
class Address
has_many :address_list_memberships, :dependent => :destroy
has_many :lists, :through => :address_list_memberships
# Lists that this Address is not in
def Address.lists_not_in(address_id)
sql = %Q|
SELECT
l.*
FROM
lists l
WHERE
l.id
NOT IN
(
SELECT
l.id
FROM
addresses a, lists l, address_list_memberships alm
WHERE
a.id = alm.address_id AND l.id = alm.list_id
AND
a.id = #{address_id}
)
|
List.find_by_sql(sql)
end
end
I would do this as a scope in List
class List
named_scope :without_address, lambda { |address_id| { :joins => 'inner join address_list_memberships alm on alm.list_id = lists.id', :conditions => ['alm.address_id <> ?', address_id]}}
end
Now you can call List.without_address(4), and you can call scopes on top of that.
As Matchu points out, you can do it without writing out the join SQL:
class List
named_scope :without_address, lambda { |address_id| { :joins => :address_list_memberships, :conditions => ['address_list_memberships.address_id <> ?', address_id]}}
end
And make sure your join table has indices!
In a migration:
add_index "address_list_memberships", "address_id"
add_index "address_list_memberships", "list_id"
For other ways you can format the named_scope, see Sam Saffron's gist: http://gist.github.com/162489
WHERE (address_list_memberships.address_id <> 13896)
is going to be expensive on a database with 21849 Addresses and 1483 Lists.
Flip your logic:
def lists_not_in
List.all - self.lists
end
That way you are only subtracting one array from another instead of checking each record in the database to see if it's in the list.
You are not going to get the flexibility you get with direct SQL from ActiveRecord, in particular, it is not going to be possible for you to craft the not in clause in active record.
If you want to get a little bit more control you could try using Sequel http://sequel.rubyforge.org/ or just hand crafting.
Note, the solution you have is risky cause you are allowing for a sql injection. (a.id = #{address_id})