I have a simple model setup in my Ruby on Rails app. (User {name, username, lat, lon}) and am writing a basic extension to the model. I would like the method to return users within a certain distance. It all works just fine in the page view, but as I am debugging I would like to work through some testing using the script/console.
My question: It seems to be printing to the screen the entire result set when I run it from the command line and script/console.
My model:
class User < ActiveRecord::Base
def distance_from(aLat, aLon)
Math.sqrt((69.1*(aLat - self.lat))**2 + (49*(aLon - self.lon))**2 )
end
def distance_from_user(aUser)
distance_from(aUser.lat, aUser.lon)
end
def users_within(distance)
close_users = []
users = User.find(:all)
users.each do |u|
close_users << u if u.distance_from_user(self) < distance
end
return close_users
end
end
and from the command line I am running
>> u = User.find_by_username("someuser")
>> print u.users_within(1)
So, I guess I would like to know why it's printing the whole result set, and if there is a way to suppress it so as to only print what I want?
you should handle it using a mysql query.
it's not good practise to pull all data from the database perofrm on them and show top 10 data out of them.
do something like following.
User.find_by_sql("select u.*,
(((69.1*(aLat - self.lat))**2 + (49*(aLon - self.lon))**2 ) < 100")) as distance
from users u
where u.username='someuser' and
distance > 100 order by distance DESC LIMIT 10)
Above query is just an example but it's always good practice to do something like above.
It appears that the console interface of the Ruby application evaluates each line as though it were entered from the console.
I would have expected it to some encapsulation of the scripts but it makes sense as it is being interpreted on the fly by the ruby interactive console line.
And to clarify as to the prefered sql base solution, doing find_by_sql is frowned upon in most refereces I have found. The SQL based solution that I reccomend is
def f
User.find(:all, :conditions => ["sqrt(pow(69.1*(lat - ?),2) + pow(49*(lon - ?),2)) < 1", lat.to_s,lon.to_s])
end
Related
I'm just wondering if there's a way to access the raw SQL that's executed for an update_all ActiveRecord request. As an example, take the simple example below:
Something.update_all( ["to_update = ?"], ["id = ?" my_id] )
In the rails console I can see the raw SQL statement so I'm guessing it's available for me to access in some way?
PS - I'm specifically interested in update_all and can't change it to anything else.
Thanks!
If you look at the way update_all is implemented you can't call to_sql on it like you can on relations since it executes directly and returns an integer (the number of rows executed).
There is no way to tap into the flow or get the desired result except by duplicating the entire method and changing the last line:
module ActiveRecord
# = Active Record \Relation
class Relation
def update_all_to_sql(updates)
raise ArgumentError, "Empty list of attributes to change" if updates.blank?
if eager_loading?
relation = apply_join_dependency
return relation.update_all(updates)
end
stmt = Arel::UpdateManager.new
stmt.set Arel.sql(#klass.sanitize_sql_for_assignment(updates))
stmt.table(table)
if has_join_values? || offset_value
#klass.connection.join_to_update(stmt, arel, arel_attribute(primary_key))
else
stmt.key = arel_attribute(primary_key)
stmt.take(arel.limit)
stmt.order(*arel.orders)
stmt.wheres = arel.constraints
end
#- #klass.connection.update stmt, "#{#klass} Update All"
stmt.to_sql
end
end
end
The reason you see the log statements is that they are logged by the connection when it executes the statements. While you can override the logging its not really possible to do it for calls from a single AR method.
If you have set RAILS_LOG_LEVEL=debug Rails shows you which SQL statement it executed.
# Start Rails console in debug mode
$ RAILS_LOG_LEVEL=debug rails c
# Run your query
[1] pry(main)> Something.update_all( ["to_update = ?"], ["id = ?" my_id] )
SQL (619.8ms) UPDATE "somethings" WHERE id = 123 SET to_update = my_id;
# ^it prints out the query it executed
Given this model:
class User < ActiveRecord::Base
has_many :things
end
Then we can do this::
#user = User.find(123)
#user.things.find_each{ |t| print t.name }
#user.thing_ids.each{ |id| print id }
There are a large number of #user.things and I want to iterate through only their ids in batches, like with find_each. Is there a handy way to do this?
The goal is to:
not load the entire thing_ids array into memory at once
still only load arrays of thing_ids, and not instantiate a Thing for each id
Rails 5 introduced in_batches method, which yields a relation and uses pluck(primary_key) internally. And we can make use of the where_values_hash method of the relation in order to retrieve already-plucked ids:
#user.things.in_batches { |batch_rel| p batch_rel.where_values_hash['id'] }
Note that in_batches has order and limit restrictions similar to find_each.
This approach is a bit hacky since it depends on the internal implementation of in_batches and will fail if in_batches stops plucking ids in the future. A non-hacky method would be batch_rel.pluck(:id), but this runs the same pluck query twice.
You can try something like below, the each slice will take 4 elements at a time and them you can loop around the 4
#user.thing_ids.each_slice(4) do |batch|
batch.each do |id|
puts id
end
end
It is, unfortunately, not a one-liner or helper that will allow you to do this, so instead:
limit = 1000
offset = 0
loop do
batch = #user.things.limit(limit).offset(offset).pluck(:id)
batch.each { |id| puts id }
break if batch.count < limit
offset += limit
end
UPDATE Final EDIT:
I have updated my answer after reviewing your updated question (not sure why you would downvote after I backed up my answer with source code to prove it...but I don't hold grudges :)
Here is my solution, tested and working, so you can accept this as the answer if it pleases you.
Below, I have extended ActiveRecord::Relation, overriding the find_in_batches method to accept one additional option, :relation. When set to true, it will return the activerecord relation to your block, so you can then use your desired method 'pluck' to get only the ids of the target query.
#put this file in your lib directory:
#active_record_extension.rb
module ARAExtension
extend ActiveSupport::Concern
def find_in_batches(options = {})
options.assert_valid_keys(:start, :batch_size, :relation)
relation = self
start = options[:start]
batch_size = options[:batch_size] || 1000
unless block_given?
return to_enum(:find_in_batches, options) do
total = start ? where(table[primary_key].gteq(start)).size : size
(total - 1).div(batch_size) + 1
end
end
if logger && (arel.orders.present? || arel.taken.present?)
logger.warn("Scoped order and limit are ignored, it's forced to be batch order and batch size")
end
relation = relation.reorder(batch_order).limit(batch_size)
records = start ? relation.where(table[primary_key].gteq(start)) : relation
records = records.to_a unless options[:relation]
while records.any?
records_size = records.size
primary_key_offset = records.last.id
raise "Primary key not included in the custom select clause" unless primary_key_offset
yield records
break if records_size < batch_size
records = relation.where(table[primary_key].gt(primary_key_offset))
records = records.to_a unless options[:relation]
end
end
end
ActiveRecord::Relation.send(:include, ARAExtension)
here is the initializer
#put this file in config/initializers directory:
#extensions.rb
require "active_record_extension"
Originally, this method forced a conversion of the relation to an array of activrecord objects and returned it to you. Now, I optionally allow you to return the query before the conversion to the array happens. Here is an example of how to use it:
#user.things.find_in_batches(:batch_size=>10, :relation=>true).each do |batch_query|
# do any kind of further querying/filtering/mapping that you want
# show that this is actually an activerecord relation, not an array of AR objects
puts batch_query.to_sql
# add more conditions to this query, this is just an example
batch_query = batch_query.where(:color=>"blue")
# pluck just the ids
puts batch_query.pluck(:id)
end
Ultimately, if you don't like any of the answers given on an SO post, you can roll-your-own solution. Consider only downvoting when an answer is either way off topic or not helpful in any way. We are all just trying to help. Downvoting an answer that has source code to prove it will only deter others from trying to help you.
Previous EDIT
In response to your comment (because my comment would not fit):
calling
thing_ids
internally uses
pluck
pluck internally uses
select_all
...which instantiates an activerecord Result
Previous 2nd EDIT:
This line of code within pluck returns an activerecord Result:
....
result = klass.connection.select_all(relation.arel, nil, bound_attributes)
...
I just stepped through the source code for you. Using select_all will save you some memory, but in the end, an activerecord Result was still created and mapped over even when you are using the pluck method.
I would use something like this:
User.things.find_each(batch_size: 1000).map(&:id)
This will give you an array of the ids.
This is an existing code written by someone else and am trying to enhance it. I am a java developer working on Ruby on Rails, so kindly be considerate.
I have entities like this
User
Delivery entity,
Delivery
belongs_to :user
named_scope :for_abcs, :conditions => {'deliveries.xyz_type' => ['Xyz1', 'Xyz2']},
many such named-scopes are defined.
Now to fetch the deliveries its written like this
#deliveries = current_user.deliveries.send("for_abcs").with(:xyz, :sender, :receiver)
...
...
...
# few other conditions added to #deliveries
finally
#deliveries.sort(...)
This sort is taking huge sql and giving performance issues. I want to use find_each, but find_each is only for Active Entity in Ruby on Rails, How can I achieve this (if possible) without much code change)
Earlier I used to do
Delevery.find_each
wherever it is
Delivery.find
Now I cant do as it is an array, what is the workaround or right procedure to do that in Ruby on Rails.
EDIT :
What I tried :
deliveries_temp = []
#deliveries.find_each(:batch_size=>999) do |delivery_temp|
deliveries_temp.push(delivery_temp)
end
This gave me error
undefined method `find_each' for []:Array
type(#deliveries) returned ActiveRecord::NamedScope::Scope , rails version 2.3.18
find_each should work on anything that returns a Relation (which includes scopes).
#deliveries = current_user.deliveries.for_abcs(:xyz, :sender, :receiver).find_each
Update
It sounds like you're using Rails 2.3. find_each is a class method in 2.3, so you'll need a way to extract the conditions from your scope and pass them to find_each. I found an article that looks promising, so give this a try:
Delivery.find_each(current_user.deliveries.for_abcs.scope(:find))
Also, I'm still not sure what that #with is doing. Maybe it's supposed to be #includes?
After lot of research for a week and learning about named_scopes by checking its source code. I understood what the problem was. The #deliveries is an object of class ActiveRecord::NamedScope::Scope . This class do not have find_each method. So I wrote a new named_scope for limit and offset in Delivery model file as follows :
named_scope :limit_and_offset, lambda { |lim,off| { :limit => lim, :offset=>off } }
After this , I called it in a loop passing offset and limit , for ex. first loop has offset=0, limit=999 , second loop has offset=999, limit=999 . I will add all the results into an emptry array. This loop continues till the result size is less than the limit value . This is working exactly the way I wanted , in batches.
set = 1
total_deliveries = []
set_limit=999
original_condition = #deliveries
loop do
offset = (set-1) * set_limit
temp_condition = original_condition.limit_and_offset(set_limit,offset)
temp_deliveries = temp_condition.find(:all)
total_deliveries+= temp_deliveries
set += 1
break if temp_deliveries.size < set_limit
end
#deliveries = total_deliveries.sort do |a, b|
I have a dashboard(esque) view in a Rails app which is showing some data in similar ways but broken out into many time periods.
I have some code in my controller like so:
#issues_this_month = Issue.where('issues.created_at BETWEEN ? AND ?', DateTime.now.in_time_zone.beginning_of_month, DateTime.now.in_time_zone.end_of_month)
and I also want to create a variables which shows issues this year and issues all time so I have this code:
#issues_this_year = Issue.where('issues.created_at BETWEEN ? AND ?', DateTime.now.in_time_zone.beginning_of_year, DateTime.now.in_time_zone.end_of_year)
I am curious if someone can think of a good way of doing one query, and from that inferring the date ranges all while avoiding the extra queries. Should I pass the results to a helper method and do the logic there?
in the model... you can define
def date
self.created_at.to_date
end
then in the controller
start = Date.today.beginning_of_year
end = Date.today.end_of_year
#issues_this_year = Issue.where(create_at: start..end).group_by(&:date)
now you have a hash of [month_1, {issues that exist in month_1}, month_2, {issues that exist in month_2}, etc]. play with it in the console to find the proper keys... #issues_this_year.keys
How about defining a method like
def self.in_timeframe(start_time=DateTime.now.in_time_zone.beginning_of_month,
end_time=DateTime.now.in_time_zone.end_of_month)
Issue.where('issues.created_at BETWEEN ? AND ?', start_time, end_time)
end
You can now invoke this as follows:
Issue.in_timeframe # For issues in the month
Issue.in_timeframe(x,y) # For issues within x-y timeframe
If you want the data in a single query, you could do stuff like:
def self.in_timeframes(time_frames)
data = {}
times_frames.each do |time_frame|
data[time_frame[:name]] = Issue.in_timeframe(time_frame[:srtart]. time_frame[:end])
end
data
end
You can invoke the above method using:
time_frames = [{:name=>"month"},
{:name=>"x-y", :start=>x, :end=>y}]
Issue.in_timeframes(time_frames)
I need to take some random documents using Rails and MongoId. Since I plan to have very large collections I decided to put a 'random' field in each document and to select documents using that field. I wrote the following method in the model:
def random(qty)
if count <= qty
all
else
collection = [ ]
while collection.size < qty
collection << where(:random_field.gt => rand).first
end
collection
end
end
This function actually works and the collection is filled with qty random elements. But as I try to use it like a scope like this:
User.students.random(5)
I get:
undefined method `random' for #<Array:0x0000000bf78748>
If instead I try to make the method like a lambda scope I get:
undefined method `to_criteria' for #<Array:0x0000000df824f8>
Given that I'm not interested in applying any other scopes after the random one, how can I use my method in a chain?
Thanks in advance.
I ended up extending the Mongoid::Criteria class with the following. Don't know if it's the best option. Actually I believe it's quite slow since it executes at least qty queries.
I don't know if not_in is available for normal ActiveRecord modules. However you can remove the not_in part if needed. It's just an optimization to reduce the number of queries.
On collections that have a double (or larger) number of documents than qty, you should have exactly qty queries.
module Mongoid
class Criteria
def random(qty)
if count <= qty
all
else
res = [ ]
ids = [ ]
while res.size < qty
el = where(:random_field.gt => rand).not_in(id: ids).first
unless el.nil?
res << el
ids << el._id
end
end
res
end
end
end
end
Hope you find this useful :)