How to use find_by_sql properly? - ruby-on-rails

I have the following model
class Backup < ActiveRecord::Base
belongs_to :component
belongs_to :backup_medium
def self.search(value)
join_tables = "backups, components, backup_media"
joins = "backups.backup_medium_id = backup_media.id and components.id = backups.component_id"
c = Backup.find_by_sql "select * from #{join_tables} where components.name like '%#{value}%' and #{joins}"
b = Backup.find_by_sql "select * from #{join_tables} where backup_media.name like '%#{value}%' and #{joins}"
c.count > 0 ? c : b
end
end
In pry, when I run Backup.all.class, I get
=> Backup::ActiveRecord_Relation
but when I run Backup.search('xxx').class, I get
=> Array
Since the search should return a subset of all, I think I need to return an Active Record_Relation. What am I missing?

From the documentation:
Executes a custom SQL query against your database and returns all the
results. The results will be returned as an array with columns
requested encapsulated as attributes of the model you call this method
from. If you call Product.find_by_sql then the results will be
returned in a Product object with the attributes you specified in the
SQL query.
So you will get an array of Backup instances.
Note that you probably should not do it this way. Using string interpolation in a query opens you up to SQL injection attacks and gains you nothing. Also, you can get quite a bit more flexibility using ActiveRecord scopes for this.
def self.my_includes
includes(:components, :backup_media)
end
def self.by_component_name(name)
media_includes.where("components.name like ?", "'%#{name}%'")
end
def self.by_media_name(name)
media_includes.where("backup_media.name like ?", "'%#{value}%'")
end
def self.search(name)
by_component(name).any? ? by_component_name : by_media_name
end
You can then call
Backup.search(name)
as well as
Backup.by_component_name(name)
or
Backup.by_media_name(name)

find_by_sql returns an array of objects, not a Relation. If you want to return relation for consistency try to rewrite your search to use ActiveRecord api:
def self.search(value)
query = Backup.includes(:component, :backup_medium)
by_component_name = query.where("components.name like ?", "'%#{value}%'")
by_media_name = query.where("backup_media.name like ?", "'%#{value}%'")
by_component_name.any? ? by_component_name : by_media_name
end
or, if you still want to use sql, you can try to fetch record ids and then make a second query:
def self.search(value)
# ...
c = Backup.find_by_sql "select id from #{join_tables} where components.name like '%#{value}%' and #{joins}"
b = Backup.find_by_sql "select id from #{join_tables} where backup_media.name like '%#{value}%' and #{joins}"
ids = c.count > 0 ? c : b
Backup.where(id: ids)
end

So I am unable to get the syntax right for the media_includes, but inspired by your solution I have succeeded by using joins.
I created a small demo project which just shows the code related to search. You can take a look at https://github.com/pamh09/rails-search-demo. If you want to collaborate on a solution, I think this would be more efficient than trying to paste all the code here. That said, I do have a working solution if you'd rather not bother. But I would like to see what the right syntax is.
Below is the model code. It's very possible that I just have some kind of syntactic mismatch since I am not very familiar with how rails does its database magic (obviously).
class Backup < ApplicationRecord
belongs_to :component
belongs_to :backup_medium
#---- code below does not work ---
# in pry
# pry(Backup):1> by_media('bak').any?
# (0.0ms) SELECT COUNT(*) FROM "backups" WHERE (backup_media = 'bak')
# ActiveRecord::StatementInvalid: SQLite3::SQLException: no such column: backup_media.name: SELECT COUNT(*) FROM "backups" WHERE (backup_media.name = 'bak')
def self.my_includes
includes(:component, :backup_medium)
end
def self.by_component(name)
my_includes.where("components.name = ?", name)
end
def self.by_media(name)
my_includes.where("backup_media.name = ?", name)
end
def self.search_by(name)
by_component(name).any? ? by_component_name : by_media_name
end
# ----- code below works ... call search('string') -----
# I was unable to get the like query to work without using #{name}
def self.by_component_like(name)
# Note: joins (singular).where (plural.column ...)
joins(:component).where("components.name like '%#{name}%'")
end
def self.by_media_like(name)
joins(:backup_medium).where("backup_media.name like '%#{name}%'")
end
def self.search(name)
by_component_like(name).any? ? by_component_like(name) : by_media_like(name)
end
end
And, as noted in the code. I could not figure you how to use the ? with LIKE as the query would come in as LIKE '%'xxx'%' instead of '%xxx%'.

Related

Specific search implementation

So I'm trying to improve the search feature for my app
My model relationships/associations are like so (many>one, one=one):
Clients < Projects < Activities = Assignments = Users
Assignments < Tasks
Tasks table has only a foreign key to assignments.
Search params look something like this:
params[:search]==User: 'user_handle', Client: 'client_name', Project: 'project_name', Activity: 'activity_name'
So I need to porbably search Clients.where().tasks, Projects.where().tasks and so on.
Then I need to somehow concatenate those queries and get rid of all the duplicate results. How to do that in practice however, I have no clue.
I've been hitting my head against a brick wall with this and internet searches didn't really help... so any help is greatly apreciated. Its probably a simple solution too...
I am on rails 4.2.5 sqlite for dev pg for production
A few things I would change/recommend based on the code in your own answer:
Move the search queries into scopes on each model class
Prefer AREL over raw SQL when composing queries (here's a quick
guide)
Enhance rails to use some sort of or when querying Models
The changes I suggest will enable you to do something like this:
search = search_params
tasks = Tasks.all
tasks = tasks.or.user_handle_matches(handle) if (handle = search[:user].presence)
tasks = tasks.or.client_name_matches(name) if (name = search[:client].presence)
tasks = tasks.or.project_name_matches(name) if (name = search[:project].presence)
tasks = tasks.or.activity_name_matches(name) if (name = search[:activity].presence)
#tasks = tasks.uniq
First, convert each of your queries to a scope on your models. This enables you to reuse your scopes later:
class User
scope :handle_matches, ->(handle) {
where(arel_table[:handle].matches("%#{handle}%"))
}
end
class Client
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
class Project
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
class Activity
scope :name_matches, ->(name) {
where(arel_table[:name].matches("%#{name}%"))
}
end
You can then use these scopes on your Task model to allow for better searching capabilities. For each of the scopes on Task we are doing an join (inner join) on a relationship and using the scope to limit the results of the join:
class Task
belongs_to :assignment
has_one :user, :through => :assignment
has_one :activity, :through => :assignment
has_one :project, :through => :activity
scope :user_handle_matches, ->(handle) {
joins(:user).merge( User.handle_matches(handle) )
}
scope :client_name_matches, ->(name) {
joins(:client).merge( Client.name_matches(name) )
}
scope :activity_name_matches, ->(name) {
joins(:activity).merge( Activity.name_matches(name) )
}
scope :project_name_matches, ->(name) {
joins(:project).merge( Project.name_matches(name) )
}
end
The final problem to solve is oring the results. Rails 4 and below don't really allow this out of the box but there are gems and code out there to allow this functionality.
I often include the code in this GitHub gist in an initializer to allow oring of scopes. The code allows you to do things like Person.where(name: 'John').or.where(name: 'Jane').
Many other options are discussed in this SO question.
If you don't want include random code and gems, another option is to pass an array of ids into the where clause. This generates a query similar to SELECT * FROM tasks WHERE id IN (1, 4, 5, ...):
tasks = []
tasks << Tasks.user_handle_matches(handle) if (handle = search[:user].presence)
tasks << tasks.or.client_name_matches(name) if (name = search[:client].presence)
tasks << tasks.or.project_name_matches(name) if (name = search[:project].presence)
tasks << tasks.or.activity_name_matches(name) if (name = search[:activity].presence)
# get the matching id's for each query defined above
# this is the catch, each call to `pluck` is another hit of the db
task_ids = tasks.collect {|query| query.pluck(:id) }
tasks_ids.uniq!
#tasks = Tasks.where(id: tasks_ids)
So I solved it, it is supper sloppy however.
first I wrote a method
def add_res(ar_obj)
ar_obj.each do |o|
res += o.tasks
end
return res
end
then I wrote my search logic like so
if !search_params[:user].empty?
query = add_res(User.where('handle LIKE ?', "%#{search_params[:user]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:client].empty?
query = add_res(Client.where('name LIKE ?', "%#{search_params[:client]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:project].empty?
query = add_res(Project.where('name LIKE ?', "%#{search_params[:project]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if !search_params[:activity].empty?
query = add_res(Activity.where('name LIKE ?', "%#{search_params[:activity]}%"))
#tasks.nil? ? #tasks=query : #tasks=#tasks&query
end
if #tasks.nil?
#tasks=Task.all
end
#tasks=#tasks.uniq
If someone can provide a better answer I would be forever greatful

active_model_serializer returning more than one result

I'm trying to return the another_id for a related record. I would just add a has_many and belongs_to relation for each project, but I need to have the user id in order to return the correct results. However, with the code I have below, it returns all of the possible another_ids for the current_user.
If I enter this into psql, it works fine:
WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = 595
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = 2;
another_id
-----------
52
(1 row)
But if I run the code from the serializer, it returns: [52, 51]:
class ProjectSerializer < ActiveModel::Serializer
attributes :id, :another_id
def another_id__sql
"(WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = projects.thing_id
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = #{options[:current_user].id})"
end
end
class API::V1::ProjectsController < API::APIController
def index
render json: Project.all
end
private
def default_serializer_options
{ current_user: #current_user }
end
end
From what I can gather, I'm not understanding how active_model_serializers serializes more than one record.
I'm using rails 4.2.3 and active_model_serializers 0.8.3. I'm afraid I can't change the schema. Also, it probably doesn't matter, but this is the API for an Ember app.
Thanks in advance. I'm a bit embarrassed that I'm having trouble with this.
Edit:
I should probably mention that this is what my project model looks like:
class Project < ActiveRecord::Base
belongs_to :thing
has_many :user_thing, through: :thing
attr_accessor :another_id
def set_another_id(user)
connection = ActiveRecord::Base.connection
result = connection.execute("(WITH RECURSIVE t(id, parent_id, path) AS (
SELECT thing.id, thing.parent_id, ARRAY[thing.id]
FROM thing, projects
WHERE thing.id = #{thing_id}
UNION
SELECT i.id, i.parent_id, i.parent_id || t.path
FROM thing i
INNER JOIN t ON t.parent_id = i.id
)
SELECT DISTINCT user_thing.another_id FROM user_thing
INNER JOIN t on t.id = user_thing.thing_id
WHERE t.id = user_thing.thing_id AND user_thing.user_id = #{user.id})")
#another_id = result[0]["another_id"].to_i
end
end
And this is the show action in the controller:
def show
#project = Project.find(params[:id])
#project.set_another_id(#current_user)
render json: #project
end
The show action does return the correct id.
Also, I know what I have is incorrect. The thing is that I can't just use the activerecord associations, because it depends on that session's current user.
Edit 2:
I thought I was able to get it to work if I just rendered it using: render json: Project.all.to_json, and got rid of the another_id__sql method in the serializer. That does work if it does have another_id. However, if that's nil, I get the error: "NoMethodError in API::V1::ProjectsController#index undefined method []' for nil:NilClass". It looks like this is a possible bug in 0.8, so I'll either have to ask another Stack Overflow question, or I'll have to see if I can upgrade theactive_model_serializers` gem. I was wrong! See my answer below.
All the DB logic belongs in your model, not in your serializer. The serializers simply state what is supposed to be exposed, but it should not be responsible for computing it.
So here, I'd advise to make this another_id a method on your model, which won't solve your issue (as it seems it is more of an SQL issue than anything else), but it will make it so that you don't have a problem with AMS anymore.
Serializers take a record and return a serialized representation suitable for JSON or XML encoding.
They are meant as an alternative to littering your controllers with this:
render json: #users, except: [:foo, :bar, :baz], include: [..........]
And the mental flatulence that is jbuilder.
SQL queries and scopes instead belong in your models.
You can set the serializer by using the each_serializer option. But in this case it will not do you much good the objects you serialize must at least implement the base methods for a serializable model.
So you need to re-write your query so that it returns a collection or array of records.
see:
http://apidock.com/rails/ActiveRecord/Base/find_by_sql/class
https://github.com/rails-api/active_model_serializers/blob/master/lib/active_model_serializers/model.rb
https://github.com/rails-api/active_model_serializers
Got it! It appears that I needed one more method in the serializer:
project_serializer.rb
def another_id
object.another_id
end
def another_id__sql
# sql from before
end
I'm not 100% sure why this works, but I had noticed that, if I left out the another_id__sql, I would get the error column.projects.another_id does not exist. So, I'm guessing that the another_id__sql is called when it's returning an array, but uses the another_id method when the object is a single project record.
I'd still love to hear better ways to do this!

Dynamic query with multiple OR conditions

On an ActiveRecord model, I'm trying to dynamically create a query that has multiple OR conditions. ie:
SELECT * from articles
WHERE (name LIKE '%a%' OR desc LIKE '%a%' OR
name LIKE '%b%' OR desc LIKE '%b%' OR
name LIKE '%c%' OR desc LIKE '%c%')
I think I'm on the right track using arel, but I can't work out how to start off the query.
class Article < ActiveRecord::Base
attr_accessor :title, :text
def self.search(terms)
terms = *terms
t = self.arel_table
query = terms.reduce(???) do |query, word|
search_term = "%#{word}%"
query.or(t[:title].matches(search_term).or(t[:text].matches(search_term)).expr).expr
end
where(query)
end
end
I originally got the idea from this answer, but the original query is a string obviously and not something I can chuck .or onto.
What do I need to replace ??? in the reduce method to make this work, or do I need to take a completely different path (as I suspect)?
This is what I did to get it working:
class Article < ActiveRecord::Base
attr_accessor :title, :text
def self.search(terms)
terms = *terms
t = self.arel_table
# generate array of conditions
query = terms.collect do |word|
search_term = "%#{word}%"
t[:title].matches(search_term).or(t[:text].matches(search_term)).expr
end
# combine conditions
query = query.reduce {|query, condition| query.or(condition).expr }
where(query)
end
end

How do I combine results from two queries on the same model?

I need to return exactly ten records for use in a view. I have a highly restrictive query I'd like to use, but I want a less restrictive query in place to fill in the results in case the first query doesn't yield ten results.
Just playing around for a few minutes, and this is what I came up with, but it doesn't work. I think it doesn't work because merge is meant for combining queries on different models, but I could be wrong.
class Article < ActiveRecord::Base
...
def self.listed_articles
Article.published.order('created_at DESC').limit(25).where('listed = ?', true)
end
def self.rescue_articles
Article.published.order('created_at DESC').where('listed != ?', true).limit(10)
end
def self.current
Article.rescue_articles.merge(Article.listed_articles).limit(10)
end
...
end
Looking in console, this forces the restrictions in listed_articles on the query in rescue_articles, showing something like:
Article Load (0.2ms) SELECT `articles`.* FROM `articles` WHERE (published = 1) AND (listed = 1) AND (listed != 1) ORDER BY created_at DESC LIMIT 4
Article Load (0.2ms) SELECT `articles`.* FROM `articles` WHERE (published = 1) AND (listed = 1) AND (listed != 1) ORDER BY created_at DESC LIMIT 6 OFFSET 4
I'm sure there's some ridiculously easy method I'm missing in the documentation, but I haven't found it yet.
EDIT:
What I want to do is return all the articles where listed is true out of the twenty-five most recent articles. If that doesn't get me ten articles, I'd like to add enough articles from the most recent articles where listed is not true to get my full ten articles.
EDIT #2:
In other words, the merge method seems to string the queries together to make one long query instead of merging the results. I need the top ten results of the two queries (prioritizing listed articles), not one long query.
with your initial code:
You can join two arrays using + then get first 10 results:
def self.current
(Article.listed_articles + Article.rescue_articles)[0..9]
end
I suppose a really dirty way of doing it would be:
def self.current
oldest_accepted = Article.published.order('created_at DESC').limit(25).last
Artcile.published.where(['created_at > ?', oldest_accepted.created_at]).order('listed DESC').limit(10)
end
If you want an ActiveRecord::Relation object instead of an Array, you can use:
ActiveRecordUnion gem.
Install gem: gem install active_record_union and use:
def self.current
Article.rescue_articles.union(Article.listed_articles).limit(10)
end
UnionScope module.
Create module UnionScope (lib/active_record/union_scope.rb).
module ActiveRecord::UnionScope
def self.included(base)
base.send :extend, ClassMethods
end
module ClassMethods
def union_scope(*scopes)
id_column = "#{table_name}.id"
if (sub_query = scopes.reject { |sc| sc.count == 0 }.map { |s| "(#{s.select(id_column).to_sql})" }.join(" UNION ")).present?
where "#{id_column} IN (#{sub_query})"
else
none
end
end
end
end
Then call it in your Article model.
class Article < ActiveRecord::Base
include ActiveRecord::UnionScope
...
def self.current
union_scope(Article.rescue_articles, Article.listed_articles).limit(10)
end
...
end
All you need to do is sum the queries:
result1 = Model.where(condition)
result2 = Model.where(another_condition)
# your final result
result = result1 + result2
I think you can do all of this in one query:
Article.published.order('listed ASC, created_at DESC').limit(10)
I may have the sort order wrong on the listed column, but in essence this should work. You'll get any listed items first, sorted by created_at DESC, then non-listed items.

Rails - find_by_sql - Querying with multiple values for one field

I'm having trouble joining the values for querying multiple values to one column. Here's what I got so far:
def self.showcars(cars)
to_query = []
if !cars.empty?
to_query.push cars
end
return self.find_by_sql(["SELECT * FROM cars WHERE car IN ( ? )"])
end
That makes the query into:
SELECT * FROM cars WHERE car IN (--- \n- \"honda\"\n- \"toyota\"\n')
It seems find_by_sql sql_injection protection adds the extra characters. How do I get this to work?
Do you really need find_by_sql? Since you're performing a SELECT *, and assuming your method resides on the Car model, a better way would be:
class Car < ActiveRecord::Base
def self.showcars(*cars)
where('car in :cars', :cars => cars)
# or
where(:car => cars)
end
end
Note the * right after the parameter name... Use it and you won't need to write code to make a single parameter into an array.
If you really need find_by_sql, try to write it this way:
def self.showcars(*cars)
find_by_sql(['SELECT * FROM cars where car in (?)', cars])
end
Try joining the to_query array into a comma separated string with all values in single quotes, and then passing this string as a parameter "?".
Problem resolve.
def self.average_time(time_init, time_end)
query = <<-SQL
SELECT COUNT(*) FROM crawler_twitters AS twitter WHERE CAST(twitter.publish AS TIME) BETWEEN '#{time_init}' AND '#{time_end}'
GROUP BY user) AS total_tweets_time;
SQL
self.find_by_sql(sanitize_sql(query))
end

Resources