Specific search implementation - ruby-on-rails

So I'm trying to improve the search feature for my app
My model relationships/associations are like so (many>one, one=one):
Clients < Projects < Activities = Assignments = Users
Assignments < Tasks
Tasks table has only a foreign key to assignments.
Search params look something like this:
params[:search]==User: 'user_handle', Client: 'client_name', Project: 'project_name', Activity: 'activity_name'
So I need to porbably search Clients.where().tasks, Projects.where().tasks and so on.
Then I need to somehow concatenate those queries and get rid of all the duplicate results. How to do that in practice however, I have no clue.
I've been hitting my head against a brick wall with this and internet searches didn't really help... so any help is greatly apreciated. Its probably a simple solution too...
I am on rails 4.2.5 sqlite for dev pg for production

A few things I would change/recommend based on the code in your own answer:
Move the search queries into scopes on each model class
Prefer AREL over raw SQL when composing queries (here's a quick
guide)
Enhance rails to use some sort of or when querying Models
The changes I suggest will enable you to do something like this:
search = search_params
tasks = Tasks.all
tasks = tasks.or.user_handle_matches(handle) if (handle = search[:user].presence)
tasks = tasks.or.client_name_matches(name) if (name = search[:client].presence)
tasks = tasks.or.project_name_matches(name) if (name = search[:project].presence)
tasks = tasks.or.activity_name_matches(name) if (name = search[:activity].presence)
#tasks = tasks.uniq
First, convert each of your queries to a scope on your models. This enables you to reuse your scopes later:
class User
scope :handle_matches, ->(handle) {
where(arel_table[:handle].matches("%#{handle}%"))
}
end
class Client
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
class Project
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
class Activity
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
You can then use these scopes on your Task model to allow for better searching capabilities. For each of the scopes on Task we are doing an join (inner join) on a relationship and using the scope to limit the results of the join:
class Task
belongs_to :assignment
has_one :user, :through => :assignment
has_one :activity, :through => :assignment
has_one :project, :through => :activity
scope :user_handle_matches, ->(handle) {
joins(:user).merge( User.handle_matches(handle) )
}
scope :client_name_matches, ->(name) {
joins(:client).merge( Client.name_matches(name) )
}
scope :activity_name_matches, ->(name) {
joins(:activity).merge( Activity.name_matches(name) )
}
scope :project_name_matches, ->(name) {
joins(:project).merge( Project.name_matches(name) )
}
end
The final problem to solve is oring the results. Rails 4 and below don't really allow this out of the box but there are gems and code out there to allow this functionality.
I often include the code in this GitHub gist in an initializer to allow oring of scopes. The code allows you to do things like Person.where(name: 'John').or.where(name: 'Jane').
Many other options are discussed in this SO question.
If you don't want include random code and gems, another option is to pass an array of ids into the where clause. This generates a query similar to SELECT * FROM tasks WHERE id IN (1, 4, 5, ...):
tasks = []
tasks << Tasks.user_handle_matches(handle) if (handle = search[:user].presence)
tasks << tasks.or.client_name_matches(name) if (name = search[:client].presence)
tasks << tasks.or.project_name_matches(name) if (name = search[:project].presence)
tasks << tasks.or.activity_name_matches(name) if (name = search[:activity].presence)
# get the matching id's for each query defined above
# this is the catch, each call to `pluck` is another hit of the db
task_ids = tasks.collect {|query| query.pluck(:id) }
tasks_ids.uniq!
#tasks = Tasks.where(id: tasks_ids)

So I solved it, it is supper sloppy however.
first I wrote a method
def add_res(ar_obj)
ar_obj.each do |o|
res += o.tasks
end
return res
end
then I wrote my search logic like so
if !search_params[:user].empty?
query = add_res(User.where('handle LIKE ?', "%#{search_params[:user]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:client].empty?
query = add_res(Client.where('name LIKE ?', "%#{search_params[:client]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:project].empty?
query = add_res(Project.where('name LIKE ?', "%#{search_params[:project]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:activity].empty?
query = add_res(Activity.where('name LIKE ?', "%#{search_params[:activity]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if #tasks.nil?
#tasks=Task.all
end
#tasks=#tasks.uniq
If someone can provide a better answer I would be forever greatful

Related

Count number of associations with a status in Ruby on Rails

I have a model named Project and Project has many Tasks
Task can have 3 different status(integer).
I want to get a list of Projects with counts of associated Tasks in status = 1, 2 and 3.
The best i can get to is have a method on Project
def open_tasks
self.tasks.where(:status => 1).count
end
But this will make another SQL for each count and it is very bad performance when loading 100 projects.
Is there a way to get it out in one SQL statement?
I can think of a couple of ways to do this...
(It's not a single sql statement but two, still quite performant though)...
Task.where(status: 1).group(:project_id).count
will give you a hash where the keys are project ids and the values are the task counts. You can then combine this with the list of projects.
You can use the ActiveRecord counter_cache to save in the project records a value for the number of open tasks. ActiveRecord will automatically update this for you. I believe you will need to add an association to the project model like this:
# app/models/project.rb
# needs to include a column called open_task_count
class Project < ActiveRecord::Base
has_many :open_tasks, class_name: Task, -> { where status: 1 }
end
class Task < ActiveRecord::Base
belongs_to :project, counter_cache: true
end
Project.select(
'projects.*',
'(SELECT COUNT(tasks.*) FROM tasks WHERE tasks.project_id = projects.id AND tasks.status = 0) AS status_0_count',
'(SELECT COUNT(tasks.*) FROM tasks WHERE tasks.project_id = projects.id AND tasks.status = 1) AS status_1_count'
).left_joins(:tasks)
Although there are more elegant ways (like lateral joins and CTEs) subqueries work on most DBs. If statuses is an ActiveRecord::Enum you can construct the subqueries by looping over the enum mapping:
class Project < ApplicationRecord
has_many :tasks
def self.with_task_counts
# constucts an array of SQL strings
statuses = Task.statuses.map do |key, int|
sql = Task.select('COUNT(*)')
.where('tasks.project_id = projects.id')
.where(status: key)
.to_sql
"(#{sql}) AS #{key}_tasks_count"
end
select(
'projects.*',
*statuses # * turns the array into a list of args
).left_joins(:tasks)
end
end
In Rails 4 you can still do a LEFT OUTER JOIN by using a SQL string:
class Project
def self.left_joins_tasks(*args)
deprecator = ActiveSupport::Deprecation.new("5.0", "MyApp")
deprecator.deprecation_warning("left_joins_tasks is deprecated, use `.left_joins(:tasks)` instead")
joins('LEFT OUTER JOIN tasks ON tasks.project_id = projects.id')
end
end
Using .joins works as well but gives an INNER join so rows with no tasks are filtered out. You can also use .includes.
I ended up using the counter_culture gem.
https://github.com/magnusvk/counter_culture

In a Rails 3 many to many association, what is the most efficient way to query objects based on conditions on their associations?

I have a many-to-many model relation:
class Movie
has_many :movie_genres
has_many :genres, :through => :movie_genres
class Genre
has_many :movie_genres
has_many :movies, :through => :movie_genres
class MovieGenre
belongs_to :movie
belongs_to :genre
I want to query all movies with a certain genre but not associated with another genre. Example: All movies that are Action but not Drama.
What I have done is this:
action_movies = Genre.find_by_name('action').movies
drama_movies = Genre.find_by_name('drama').movies
action_not_drama_movies = action_movies - drama_movies
Is there a more efficient way of doing this? It should be noted that the query can become more complex like: All movies that are Action but not Drama or All movies that are Romance and Comedy
You can indeed improve efficiency by avoid having to instantiate the Movie instances for all action and drama movies by removing the drama movies from the set of action movies in via the sql statement.
The basic building block is a dynamic scope similar to what widjajayd proposed
class Movie
...
# allows to be called with a string for single genres or an array for multiple genres
# e.g. "action" will result in SQL like `WHERE genres.name = 'action'`
# ["romance", "comedy"] will result in SQL like `WHERE genres.name IN ('romance', 'comedy')`
def self.of_genre(genres_names)
joins(:genres).where(genres: { name: genres_names })
end
...
end
You can use that scope as a building block to get the movies you want
All movies that are action but not drama:
Movie
.of_genre('action')
.where("movies.id NOT IN (#{Movie.of_genre('drama').to_sql}")
This will result in an sql subquery. Using a join would be nicer but it should be good enough for most cases and is a better read that the join alternative.
If your app where a rails 5 application you could even type
Movie
.of_genre('action')
.where.not(id: Movie.of_genre('drama'))
All movies that are Action but not Drama or All movies that are Romance and Comedy
Because it is a rails 3 app you will have to type move most of the sql by hand and can not make a lot of use of the scope. The or method is only introduced in rails 5. So this will mean having to type:
Movie
.joins(:genres)
.where("(genres.name = 'action' AND movies.id NOT IN (#{Movie.of_genre('drama').to_sql}) OR genres.name IN ('romance', 'comedy')" )
Again, if it where a rails 5 application this would be much simpler
Movie
.of_genre('action')
.where.not(id: Movie.of_genre('drama'))
.or(Movie.of_genre(['romance', 'comedy']))
probably using scope is better, here is sample and explanation (but not tested), create scope in movie model as follow
Movie.rb
scope :action_movies, joins(movie_genre: :genre).select('movies.*','genres.*').where('genres.name = ?', 'action')
scope :drama_movies, joins(movie_genre: :genre).select('movies.*','genres.*').where('genres.name = ?', 'drama')
in your controller, you can call as follow
#action_movies = Movie.action_movies
#drama_movies = Movie.drama_movies
#action_not_drama_movies = #action_movies - #drama_movies
edit for dynamic scope
if you want dynamic then you can send parameter to scope below is scope using block.
scope :get_movies, lambda { |genre_request|
joins(movie_genre: :genre).select('movies.*','genres.*').where('genres.name = ?', genre_request)
}
genre_request = parameter variable for scope
in your controller
#action_movies = Movie.get_movies('action')
#drama_movies = Movie.get_movies('drama')
Don't see a way to do it with one query (not without using subqueries anyway). But here is one that I think makes it a little better:
scope :by_genres, lambda { |genres|
genres = [genres] unless genres.is_a? Array
joins(:genres).where(genres: { name: genre }).uniq
}
scope :except_ids, lambda { |ids|
where("movies.id NOT IN (?)", ids)
}
scope :intersect_ids, lambda { |ids|
where("movies.id IN (?)", ids)
}
## all movies that are action but not drama
action_ids = Movie.by_genres("action").ids
drama_movies = Movie.by_genres("drama").except_ids(action_ids)
## all movies that are both action and dramas
action_ids = Movie.by_genres("action").ids
drama_movies = Movie.by_genres("drama").intersect_ids(action_ids)
## all movies that are either action or drama
action_or_drama_movies = Movie.by_genres(["action", "drama"])
It's possible to do except and intersect with raw SQL in Rails. But I think that's in general not a good idea as it still requires more than one query and also might make the code dependent on the DB used.
My original answer is rather naive. I'll leave it here so others won't make the same mistake:
Use joins and you can get it with one query:
Movie.joins(:genres).where("genres.name = ?", "action").where("genres.name != ?", "drama")
As noted in the comments, this will get all the movies that are both action and drama too.

Rails query with condition in count

I'm having a little trouble with a query in Rails.
Actually my problem is:
I want to select all users which do not have any user_plans AND his role.name is equals to default... OR has user_plans and all user_plans.expire_date are lower than today
user has_many roles
user has_many user_plans
users = User.where(gym_id: current_user.id).order(:id)
#users = []
for u in users
if u.roles.pluck(:name).include?('default')
add = true
for up in u.user_plans
if up.end_date > DateTime.now.to_date
add = false
end
end
if add
#users << u
end
end
end
This code up here, is doing exactly what I need, but with multiple queries.
I don't know how to build this in just one query.
I was wondering if it is possible to do something like
where COUNT(user_plans.expire_date < TODAY) == 0
User.joins(:user_plans, :roles).where("roles.name = 'default' OR user_plans.expire_date < ?", Date.today)
Should work, not tested, but should give you some idea you can play with (calling .distinct at the end may be necessary)
There is also where OR in Rails 5:
User.joins(:user_plans, :roles).where(roles: { name: 'default' }).or(
User.joins(:user_plans).where('user_plans.expire_date < ?', Date.today)
)
FYI: Calling .joins on User will only fetch those users who have at least one user_plan (in other words: will not fetch those who have no plans)

Use Ruby's select method on a Rails relation and update it

I have an ActiveRecord relation of a user's previous "votes"...
#previous_votes = current_user.votes
I need to filter these down to votes only on the current "challenge", so Ruby's select method seemed like the best way to do that...
#previous_votes = current_user.votes.select { |v| v.entry.challenge_id == Entry.find(params[:entry_id]).challenge_id }
But I also need to update the attributes of these records, and the select method turns my relation into an array which can't be updated or saved!
#previous_votes.update_all :ignore => false
# ...
# undefined method `update_all' for #<Array:0x007fed7949a0c0>
How can I filter down my relation like the select method is doing, but not lose the ability to update/save it the items with ActiveRecord?
Poking around the Google it seems like named_scope's appear in all the answers for similar questions, but I can't figure out it they can specifically accomplish what I'm after.
The problem is that select is not an SQL method. It fetches all records and filters them on the Ruby side. Here is a simplified example:
votes = Vote.scoped
votes.select{ |v| v.active? }
# SQL: select * from votes
# Ruby: all.select{ |v| v.active? }
Since update_all is an SQL method you can't use it on a Ruby array. You can stick to performing all operations in Ruby or move some (all) of them into SQL.
votes = Vote.scoped
votes.select{ |v| v.active? }
# N SQL operations (N - number of votes)
votes.each{ |vote| vote.update_attribute :ignore, false }
# or in 1 SQL operation
Vote.where(id: votes.map(&:id)).update_all(ignore: false)
If you don't actually use fetched votes it would be faster to perform the whole select & update on SQL side:
Vote.where(active: true).update_all(ignore: false)
While the previous examples work fine with your select, this one requires you to rewrite it in terms of SQL. If you have set up all relationships in Rails models you can do it roughly like this:
entry = Entry.find(params[:entry_id])
current_user.votes.joins(:challenges).merge(entry.challenge.votes)
# requires following associations:
# Challenge.has_many :votes
# User.has_many :votes
# Vote.has_many :challenges
And Rails will construct the appropriate SQL for you. But you can always fall back to writing the SQL by hand if something doesn't work.
Use collection_select instead of select. collection_select is specifically built on top of select to return ActiveRecord objects and not an array of strings like you get with select.
#previous_votes = current_user.votes.collection_select { |v| v.entry.challenge_id == Entry.find(params[:entry_id]).challenge_id }
This should return #previous_votes as an array of objects
EDIT: Updating this post with another suggested way to return those AR objects in an array
#previous_votes = current_user.votes.collect {|v| records.detect { v.entry.challenge_id == Entry.find(params[:entry_id]).challenge_id}}
A nice approach this is to use scopes. In your case, you can set this up the scope as follows:
class Vote < ActiveRecord::Base
scope :for_challenge, lambda do |challenge_id|
joins(:entry).where("entry.challenge_id = ?", challenge_id)
end
end
Then your code for getting current votes will look like:
challenge_id = Entry.find(params[:entry_id]).challenge_id
#previous_votes = current_user.votes.for_challenge(challenge_id)
I believe you can do something like:
#entry = Entry.find(params[:entry_id])
#previous_votes = Vote.joins(:entry).where(entries: { id: #entry.id, challenge_id: #entry.challenge_id })

How can I find records by "count" of association using rails and mongoid?

With these models:
class Week
has_many :proofs
end
class Proof
belongs_to :week
end
I want to do something like:
Week.where(:proof.count.gt => 0)
To find only weeks that have multiple proofs.
There is one answer that seems to address this:
Can rails scopes filter on the number of associated classes for a given field
But in this example, there is no such attribute as proof_ids in Week since the ids are stored with the proofs. This does not work for example:
Week.where(:proof_ids.gt => 0)
How is this query possible? Conceptually simple but I can't figure out how to do this with mongo or mongoid.
Similarly, I'd like to order by the number of proofs for example like:
Week.desc(:proofs.size)
But this also does not work.
I do realize that a counter-cache is an option to both my specific questions but I'd also like to be able to do the query.
Thanks in advance for any help.
With rails (and without counter_cache), you could do:
class Week < ActiveRecord::Base
has_many :proofs
def self.by_proofs_size
sort_by { |week| week.proofs.size }
end
def self.with_at_least_n_proofs(n = 1)
select { |week| week.proofs.size >= n }
end
end
Even though each of those operations produces 2 queries, this is far from ideal.
The pair of queries is repeated (=> 4 queries for each operation) with scopes (bug?):
scope :with_at_least_n_proofs, -> (n = 1) { select { |w| w.proofs.size >= n } }
scope :by_proofs_size, -> { sort_by { |w| w.proofs.size } }
The ideal is probably to use counter_cache
scope :with_at_least_n_proofs, -> (n = 1) { where('proofs_count >= ?', n) }
scope :by_proofs_size, -> { order(proofs_count: :desc) }
I don't know if this is the best solution, as it maps it through a array, but this does the job: (the other solutions mentioned here gives me exceptions)
class Week < ActiveRecord::Base
scope :has_proofs, -> { any_in(:_id => includes(:proofs).select{ |w| w.proofs.size > 0 }.map{ |r| r.id }) }
end
Pardon me if I'm way off - but would you be able to use a simple counter_cache in the weeks table? Then you could do something like week.proofs_count.

Resources