I'm having trouble joining the values for querying multiple values to one column. Here's what I got so far:
def self.showcars(cars)
to_query = []
if !cars.empty?
to_query.push cars
end
return self.find_by_sql(["SELECT * FROM cars WHERE car IN ( ? )"])
end
That makes the query into:
SELECT * FROM cars WHERE car IN (--- \n- \"honda\"\n- \"toyota\"\n')
It seems find_by_sql sql_injection protection adds the extra characters. How do I get this to work?
Do you really need find_by_sql? Since you're performing a SELECT *, and assuming your method resides on the Car model, a better way would be:
class Car < ActiveRecord::Base
def self.showcars(*cars)
where('car in :cars', :cars => cars)
# or
where(:car => cars)
end
end
Note the * right after the parameter name... Use it and you won't need to write code to make a single parameter into an array.
If you really need find_by_sql, try to write it this way:
def self.showcars(*cars)
find_by_sql(['SELECT * FROM cars where car in (?)', cars])
end
Try joining the to_query array into a comma separated string with all values in single quotes, and then passing this string as a parameter "?".
Problem resolve.
def self.average_time(time_init, time_end)
query = <<-SQL
SELECT COUNT(*) FROM crawler_twitters AS twitter WHERE CAST(twitter.publish AS TIME) BETWEEN '#{time_init}' AND '#{time_end}'
GROUP BY user) AS total_tweets_time;
SQL
self.find_by_sql(sanitize_sql(query))
end
Related
I have a database query where I want to get an array of Users that are distinct for the set:
#range is a predefinded date range
#shift_list is a list of filtered shifts
def listing
Shift
.where(date: #range, shiftname: #shift_list)
.select(:user_id)
.distinct
.map { |id| User.find( id.user_id ) }
.sort
end
and I read somewhere that for readability, or isolating for testing, or code reuse, you could split this into seperate methods:
def listing
shiftlist
.select(:user_id)
.distinct
.map { |id| User.find( id.user_id ) }
.sort
end
def shift_list
Shift
.where(date: #range, shiftname: #shift_list)
end
So I rewrote this and some other code, and now the page takes 4 times as long to load.
My question is, does this type of method splitting cause the database to be hit twice? Or is it something that I did elsewhere?
And I'd love a suggestion to improve the efficiency of this code.
Further to the need to remove mapping from the code, this shift list is being created with the following code:
def _month_shift_list
Shift
.select(:shiftname)
.distinct
.where(date: #range)
.map {|x| x.shiftname }
end
My intention is to create an array of shiftnames as strings.
I am obviously missing some key understanding in database access, as this method is clearly creating part of the problem.
And I think I have found the solution to this with the following:
def month_shift_list
Shift.
.where(date: #range)
.pluck(:shiftname)
.uniq
end
Nope, the database will not be hit twice. The queries in both methods are lazy loaded. The issue you have with the slow page load times is because the map function now has to do multiple finds which translates to multiple SELECT from the DB. You can re-write your query to this:
def listing
User.
joins(:shift).
merge(Shift.where(date: #range, shiftname: #shift_list).
uniq.
sort
end
This has just one hit to the DB and will be much faster and should produce the same result as above.
The assumption here is that there is a has_one/has_many relationship on the User model for Shifts
class User < ActiveRecord::Base
has_one :shift
end
If you don't want to establish the has_one/has_many relationship on User, you can re-write it to:
def listing
User.
joins("INNER JOIN shifts on shifts.user_id = users.id").
merge(Shift.where(date: #range, shiftname: #shift_list).
uniq.
sort
end
ALTERNATIVE:
You can use 2 queries if you experience issues with using ActiveRecord#merge.
def listing
user_ids = Shift.where(date: #range, shiftname: #shift_list).uniq.pluck(:user_id).sort
User.find(user_ids)
end
If a query can't be efficiently expressed using ActiveRecord, how to safely use ActiveRecord::Base.connection.execute when interpolating passed params attributes?
connection.execute "... #{params[:search]} ..."
You can use the methods in ActiveRecord::Sanitization::ClassMethods.
You do have to be slightly careful as they are protected and therefore only readily available for ActiveRecord::Base subclasses.
Within a model class you could do something like:
class MyModel < ActiveRecord::Base
def bespoke_query(params)
query = sanitize_sql(['select * from somewhere where a = ?', params[:search]])
connection.execute(query)
end
end
You can send the method to try it out on the console too:
> MyModel.send(:sanitize_sql, ["Evening Officer ?", "'Dibble'"])
=> "Evening Officer '\\'Dibble\\''"
ActiveRecord has a sanitize method that allows you to clean the query first.
Perhaps it's something you can look into: http://apidock.com/rails/v4.1.8/ActiveRecord/Sanitization/ClassMethods/sanitize
I'd be very careful inserting parameters directly like that though.
What problem are you experiencing, that you cannot use ActiveRecord?
You can use functions from ActiveRecord::Base to sanitize your sql query. E.g. sanitize_sql_array. As mentioned in other answers they are protected, but that's possible to get around without having to deal with inheritance.
sanitize_sql_array accepts an array of strings where the first element is the query and the subsequent elements will replace ? characters in the query.
query = 'SELECT * FROM users WHERE id = ? OR first_name = ?'
id = 1
name = 'Alice'
sanitized_query = ActiveRecord::Base.send(:sanitize_sql_array, [query, id, name])
response = ActiveRecord::Base.connection.execute(sanitized_query)
I'm trying to figure out the count of certain types of articles. I have a very inefficient query:
Article.where(status: 'Finished').select{|x| x.tags & Article::EXPERT_TAGS}.size
In my quest to be a better programmer, I'm wondering how to make this a faster query. tags is an array of strings in Article, and Article::EXPERT_TAGS is another array of strings. I want to find the intersection of the arrays, and get the resulting record count.
EDIT: Article::EXPERT_TAGS and article.tags are defined as Mongo arrays. These arrays hold strings, and I believe they are serialized strings. For example: Article.first.tags = ["Guest Writer", "News Article", "Press Release"]. Unfortunately this is not set up properly as a separate table of Tags.
2nd EDIT: I'm using MongoDB, so actually it is using a MongoWrapper like MongoMapper or mongoid, not ActiveRecord. This is an error on my part, sorry! Because of this error, it screws up the analysis of this question. Thanks PinnyM for pointing out the error!
Since you are using MongoDB, you could also consider a MongoDB-specific solution (aggregation framework) for the array intersection, so that you could get the database to do all the work before fetching the final result.
See this SO thread How to check if an array field is a part of another array in MongoDB?
Assuming that the entire tags list is stored in a single database field and that you want to keep it that way, I don't see much scope of improvement, since you need to get all the data into Ruby for processing.
However, there is one problem with your database query
Article.where(status: 'Finished')
# This translates into the following query
SELECT * FROM articles WHERE status = 'Finished'
Essentially, you are fetching all the columns whereas you only need the tags column for your process. So, you can use pluck like this:
Article.where(status: 'Finished').pluck(:tags)
# This translates into the following query
SELECT tags FROM articles WHERE status = 'Finished'
I answered a question regarding general intersection like queries in ActiveRecord here.
Extracted below:
The following is a general approach I use for constructing intersection like queries in ActiveRecord:
class Service < ActiveRecord::Base
belongs_to :person
def self.with_types(*types)
where(service_type: types)
end
end
class City < ActiveRecord::Base
has_and_belongs_to_many :services
has_many :people, inverse_of: :city
end
class Person < ActiveRecord::Base
belongs_to :city, inverse_of: :people
def self.with_cities(cities)
where(city_id: cities)
end
# intersection like query
def self.with_all_service_types(*types)
types.map { |t|
joins(:services).merge(Service.with_types t).select(:id)
}.reduce(scoped) { |scope, subquery|
scope.where(id: subquery)
}
end
end
Person.with_all_service_types(1, 2)
Person.with_all_service_types(1, 2).with_cities(City.where(name: 'Gold Coast'))
It will generate SQL of the form:
SELECT "people".*
FROM "people"
WHERE "people"."id" in (SELECT "people"."id" FROM ...)
AND "people"."id" in (SELECT ...)
AND ...
You can create as many subqueries as required with the above approach based on any conditions/joins etc so long as each subquery returns the id of a matching person in its result set.
Each subquery result set will be AND'ed together thus restricting the matching set to the intersection of all of the subqueries.
I am struggling with the best way to meta program a dynamic method, where I'll be limiting results based on conditions... so for example:
class Timeslip < ActiveRecord::Base
def self.by_car_trans(car, trans)
joins(:car)
.where("cars.trans IN (?) and cars.year IN (?) and cars.model ILIKE ?", trans, 1993..2002, car)
.order('et1320')
end
end
Let's say instead of passing in my arguments, i pass in an array of conditions with key being the fieldname, and value being the field value. so for example, I'd do something like this:
i'd pass in [["field", "value", "operator"],["field", "value", "operator"]]
def self.using_conditions(conditions)
joins(:car)
conditions.each do |key, value|
where("cars.#{key} #{operator} ?", value)
end
end
However, that doesn't work, and it's not very flexible... I was hoping to be able to detect if the value is an array, and use IN () rather than =, and maybe be able to use ILIKE for case insensitive conditions as well...
Any advice is appreciated. My main goal here is to have a "lists" model, where a user can build their conditions dynamically, and then save that list for future use. This list would filter the timeslips model based on the associated cars table... Maybe there is an easier way to go about this?
First of all, you might find an interest in the Squeel gem.
Other than that, use arel_table for IN or LIKE predicates :
joins( :car ).where( Car.arel_table[key].in values )
joins( :car ).where( Car.arel_table[key].matches value )
you can detect the type of value to select an adequate predicate (not nice OO, but still):
column = Car.arel_table[key]
predicate = value.respond_to?( :to_str ) ? :in : :matches # or any logic you want
joins( :car ).where( column.send predicate, value )
you can chain as many as those as you want:
conditions.each do |(key, value, predicate)|
scope = scope.where( Car.arel_table[key].send predicate, value )
end
return scope
So, you want dynamic queries that end-users can specify at run-time (and can be stored & retrieved for later use)?
I think you're on the right track. The only detail is how you model and store your criteria. I don't see why the following won't work:
def self.using_conditions(conditions)
joins(:car)
crit = conditions.each_with_object({}) {|(field, op, value), m|
m["#{field} #{op} ?"] = value
}
where crit.keys.join(' AND '), *crit.values
end
CAVEAT The above code as is is insecure and prone to SQL injection.
Also, there's no easy way to specify AND vs OR conditions. Finally, the simple "#{field} #{op} ?", value for the most part only works for numeric fields and binary operators.
But this illustrates that the approach can work, just with a lot of room for improvement.
How would i do a query like this.
i have
#model = Model.near([latitude, longitude], 6.8)
Now i want to filter another model, which is associated with the one above.
(help me with getting the right way to do this)
model2 = Model2.where("model_id == :one_of_the_models_filtered_above", {:one_of_the_models_filtered_above => only_from_the_models_filtered_above})
the model.rb would be like this
has_many :model2s
the model2.rb
belongs_to :model
Right now it is like this (after #model = Model.near([latitude, longitude], 6.8)
model2s =[]
models.each do |model|
model.model2s.each do |model2|
model2.push(model2)
end
end
I want to accomplish the same thing, but with an active record query instead
i think i found something, why does this fail
Model2.where("model.distance_from([:latitude,:longitude]) < :dist", {:latitude => latitude, :longitude => longitude, :dist => 6.8})
this query throws this error
SQLite3::SQLException: near "(": syntax error: SELECT "tags".* FROM "tags" WHERE (model.distance_from([43.45101666666667,-80.49773333333333]) < 6.8)
, why
use includes. It will eager-load associated models (only two SQL queries instead of N+1).
#models = Model.near( [latitude, longitude], 6.8 ).includes( :model2s )
so when you will do #models.first.model2s, associated model2s will already be loaded (see RoR guides for more info).
If you want to get an array of all model2s belonging to your collection of models, you can do :
#models.collect( &:model2s )
# add .flatten at the end of the chain if you want a one level deep array
# add .uniq at the end of the chain if you don't want duplicates
collect (also called map) will gather in an array the result of any block passed to each of the caller's elements (this does exactly the same as your code, see Enumerable's doc for more info). The & before the symbol converts it into a Proc passed to each element of the collection, so this is the same as writing
#models.collect {|model| model.model2s }
one more thing : #mu is right, seems SQLite does not know about your distance_from stored procedure. As i suspect this is a GIS related question, you may ask about this particular issue on gis.stackexchange.com