Using Rails 3.2, Ruby 1.9, geocoder gem. The following query I have in my controller:
# 1500+ms to load
nearby_shops = (current_shop.nearbys(10, :order => 'overall_rating DESC')
.where(:shop_type => shop_type).includes(:photos, :active_property_list =>
:hotel_image_lists).limit(5))
# SQL
Shop Load (4045.6ms) SELECT shops.*, 3958.755864232 * 2 * ASIN(SQRT(POWER(
SIN((36.111927 - shops.lat) * PI() / 180 / 2), 2) + COS(36.111927 * PI()
/ 180) * COS(shops.lat * PI() / 180) * POWER(SIN((-115.171229 - shops.lng)
* PI() / 180 / 2), 2))) AS distance, CAST(DEGREES(ATAN2( RADIANS(shops.lng
- -115.171229), RADIANS(shops.lat - 36.111927))) + 360 AS decimal) % 360 AS
bearing FROM `shops` WHERE `shops`.`shop_type` = 'food' AND (shops.lat BETWEEN
35.96719521688915 AND 36.25665878311085 AND shops.lng BETWEEN
-115.3503819353204 AND -114.9920760646796 AND 3958.755864232 * 2 *
ASIN(SQRT(POWER(SIN((36.111927 - shops.lat) * PI() / 180 / 2), 2) +
COS(36.111927 * PI() / 180) * COS(shops.lat * PI() / 180) * POWER(SIN((
-115.171229 - shops.lng) * PI() / 180 / 2), 2))) <= 10 AND shops.id != 85155)
ORDER BY overall_rating DESC LIMIT 5
- shops.lat) * PI() / 180 / 2), 2) + COS(48.8582411618 * PI() / 180) *
COS(shops.lat * PI() / 180) * POWER(SIN((2.2945044899 - shops.lng) * PI() /
180 / 2), 2))) <= 100 AND shops.id != 517) ORDER BY distance ASC LIMIT 25
OFFSET 0
The problem lies in the nearbys which does calculation on the longitude and latitude. I have already added indexes to the longitude and latitude columns, but it doesn't improve anything.
How can I improve this?
P/S: I removed unrelated conditions which doesn't contribute to the speed of the query.
This might not be the most exact or elegant solution, but why not just fudge the math a little. You could write a select clause that did something like this:
.select(["id, name, 69.0975851*sqrt(POWER(shops.lat-?,2) + COS(?*PI()/180)*POWER(shops.lon-?,2)) AS DISTANCE", loc.lat, loc.lat, loc.lon])
You don't really need to use that hairy great circle formula if you're just dealing with small (< 500 mile) distances. You'll end up getting the same ~almost~ answers for a fraction of the computational cost.
Related
The following line of code throws an error and I don't understand why.
fav_arr.push(Address.near([#address.latitude, #address.longitude], 5).where(addressable_type: 'User').pluck(:addressable_id))
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column
"distance" does not exist
As soon as I add the .pluck(:addressable_id), it throws the error that the distance column does not exist.
This one works tho:
Address.near([#address.latitude, #address.longitude], 5).where(addressable_type: 'User').map{ |add| fav_arr << add.addressable_id }
This is the SQL query
Address.near([#address.latitude, #address.longitude], 5).where(addressable_type: 'User').to_sql :
: SELECT "addresses"."addressable_id" FROM "addresses" WHERE (addresses.latitude BETWEEN 45.42903640844458 AND 45.57376819155542 AND addresses.longitude BETWEEN -73.68277080563716 AND -73.47627419436283 AND (3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((45.5014023 - addresses.latitude) * PI() / 180 / 2), 2) + COS(45.5014023 * PI() / 180) * COS(addresses.latitude * PI() / 180) * POWER(SIN((-73.5795225 - addresses.longitude) * PI() / 180 / 2), 2)))) BETWEEN 0.0 AND 5) AND "addresses"."addressable_type" = $1 ORDER BY distance ASC
Thanks for the help
Please try this
Address.near([#address.latitude, #address.longitude], 5).where(addressable_type: 'User').reorder('').pluck(:addressable_id)
Extending #engineersmnky comment, near and pluck both modifies the select statement.
So, I have this concern and scope
module GeocoordinatesHelpers
extend ActiveSupport::Concern
included do
scope :within_range, -> (lat, lon, range) { where("(6371.0 * 2 * ASIN(SQRT(POWER(SIN((self.latitude - :lat) * PI() / 180 / 2), 2) + COS(self.latitude * PI() / 180) * COS(:lat * PI() / 180) * POWER(SIN((self.longitude - :lon) * PI() / 180 / 2), 2)))) <= :range", {lat: lat, lon: lon, range: range})}
end
end
that I need to use across several models
Is it possible to refer to the models column that the scope is being called against? Like, replace the self bits in the where with something that actually works
For reasons, I can't just use the geocoder gem
If you really need the full column name, try changing self to #{table_name}.
Using Rails 4.2 and Postgres I've created the following query to give me a uniq list of bird ids in the Sightings table ordered by the number of sightings for each bird. Also note I'm using the Kaminari gem for pagination.
Sighting.select("bird_id, COUNT(sightings.id) as sightings_count").group(:bird_id).order('sightings_count DESC').page(1)
This works great returning the ActiveRecordRelation intended. The problem arises when i try to combine it with the geocoder gems .near method
Sighting.near([-31.0, 151.0], 1000, units: :km).select("bird_id, COUNT(sightings.id) as sightings_count").group(:bird_id).order('sightings_count DESC').page(1)
This generates the query and error
SELECT sightings.*, 6371.0 * 2 * ASIN(SQRT(POWER(SIN((-31.0 - sightings.lat) * PI() / 180 / 2), 2) + COS(-31.0 * PI() / 180) * COS(sightings.lat * PI() / 180) * POWER(SIN((151.0 - sightings.lng) * PI() / 180 / 2), 2))) AS distance, MOD(CAST((ATAN2( ((sightings.lng - 151.0) / 57.2957795), ((sightings.lat - -31.0) / 57.2957795)) * 57.2957795) + 360 AS decimal), 360) AS bearing, bird_id, COUNT(sightings.id) as sightings_count FROM "sightings" WHERE (sightings.lat BETWEEN -39.993216059187304 AND -22.006783940812696 AND sightings.lng BETWEEN 140.50821379697885 AND 161.49178620302115 AND (6371.0 * 2 * ASIN(SQRT(POWER(SIN((-31.0 - sightings.lat) * PI() / 180 / 2), 2) + COS(-31.0 * PI() / 180) * COS(sightings.lat * PI() / 180) * POWER(SIN((151.0 - sightings.lng) * PI() / 180 / 2), 2)))) BETWEEN 0.0 AND 1000) GROUP BY bird_id ORDER BY distance ASC, sightings_count DESC LIMIT 25 OFFSET 0
PG::GroupingError: ERROR: column "sightings.id" must appear in the GROUP BY clause or be used in an aggregate function
Adding id to the group by means the bird counts aren't correct and as far as I understood COUNT within select was an aggregate function which does include sightings.id.
How can I successfully combine the two?
Note: I did try the following but this returns a Hash rather than AR Relation.
Sighting.near([#lat, #lng], #range, units: :km, order:nil).group(:bird_id).order('count_id DESC').page(#page).count(:id)
Thanks for any help!!
Creating a custom near scope was the easiest work around as I wasn't using the bearing or distance attributes that get added to each record and hence the whole select sql generated by near I didn't need. Instead of select(options[:select]) I replaced it with the select I wanted.
scope :birds_by_sighting, lambda{ |location, *args|
latitude, longitude = Geocoder::Calculations.extract_coordinates(location)
if Geocoder::Calculations.coordinates_present?(latitude, longitude)
options = near_scope_options(latitude, longitude, *args)
select("bird_id, COUNT(sightings.id) as sightings_count").group(:bird_id).where(options[:conditions]).order('sightings_count DESC')
else
# If no lat/lon given we don't want any results, but we still
# need distance and bearing columns so you can add, for example:
# .order("distance")
select(select_clause(nil, null_value, null_value)).where(false_condition)
end
}
I'm trying to use the near method of the Ruby geocoder gem in a Rails application with a postgres database.
#users = Contact.near(params[:search], params[:distance])
That line produces the 'operator does not exist error' you see below. The search parameter will be an address and the dist
In the Contact.rb model I do this
geocoded_by :full_address
def full_address
self.address + self.city + self.province + self.postal
end
Can you tell me what it is that I might be doing wrong? The error says, 'You might need to add explicit type casts.'
Address, city, province and postal are all strings.
PG::Error: ERROR: operator does not exist: numeric - character varying LINE 1: ...8.755864232 * 2 * ASIN(SQRT(POWER(SIN((43.7058645 - contacts... ^ HINT: No operator matches the given name and argument type(s). You might need to add explicit type casts. : SELECT contacts.*, 3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((43.7058645 - contacts.latitude) * PI() / 180 / 2), 2) + COS(43.7058645 * PI() / 180) * COS(contacts.latitude * PI() / 180) * POWER(SIN((-79.3687137 - contacts.longitude) * PI() / 180 / 2), 2))) AS distance, CAST(DEGREES(ATAN2( RADIANS(contacts.longitude - -79.3687137), RADIANS(contacts.latitude - 43.7058645))) + 360 AS decimal) % 360 AS bearing FROM "contacts" WHERE (contacts.latitude BETWEEN 43.676918143377826 AND 43.73481085662217 AND contacts.longitude BETWEEN -79.40875589075141 AND -79.3286715092486 AND 3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((43.7058645 - contacts.latitude) * PI() / 180 / 2), 2) + COS(43.7058645 * PI() / 180) * COS(contacts.latitude * PI() / 180) * POWER(SIN((-79.3687137 - contacts.longitude) * PI() / 180 / 2), 2))) <= '2') ORDER BY distance ASC LIMIT 1
In my case I have a model Product has_one Location
I use a geocoder gem to search location near a distance.
The request Location.near([0, 0], 100) is look like this:
SELECT locations.*, 6371.0 * 2 * ASIN(SQRT(POWER(SIN((0 - locations.latitude) * PI() / 180 / 2), 2) + COS(0 * PI() / 180) * COS(locations.latitude * PI() / 180) * POWER(SIN((1 - locations.longitude) * PI() / 180 / 2), 2) )) AS distance, CAST(DEGREES(ATAN2( RADIANS(longitude - 1), RADIANS(latitude - 0))) + 360 AS decimal) % 360 AS bearing FROM \"locations\" WHERE (6371.0 * 2 * ASIN(SQRT(POWER(SIN((0 - locations.latitude) * PI() / 180 / 2), 2) + COS(0 * PI() / 180) * COS(locations.latitude * PI() / 180) * POWER(SIN((1 - locations.longitude) * PI() / 180 / 2), 2) )) <= 20) ORDER BY distance
I want to do something like this:
Product.where(...).joins(:location).dosomething
How can I do it?
Location#near is a named scope? If so you can merge it with your Product scope using the & operator. I think this should work:
class Product
scope :near, lambda { |coord, dist| joins(:location) & Location.near(coord, dist) }
...
end
Then you can just use it like so:
Product.near([0, 0], 100)
Another possibility, since merging the scopes doesn't seem to work:
class Product
scope :near, lambda { |coord, dist| where(:id => Location.near(coord, dist).all.map(&:locationable_id) }
...
end
And usage like in the other answer:
Product.near([0, 0], 100)