PostgreSQL ActiveRecord Query Optimization - ruby-on-rails

I'm new to RoR and I need a query that returns the same instances as the following definition of queue :
#all = User.all
#membersOfActivity = User.where(id: User.joins(:memberships).where("activity_id = ?", session[:current_activity_id]))
#usersIAlreadyLiked = User.where(id: User.joins(:likes).where("activity_id = ? AND activity_likes_user IS NOT NULL", session[:current_activity_id]))
#notWanted = #membersOfActivity + #usersIAlreadyLiked
#queue = #all - #notWanted
However, I understand how inefficient the query I just wrote is for it downloads all the Users first to then select the complement. Do you have any idea on how to make it more efficient and directly select the complement of #notWanted? I tried several queries but none of those worked.
If you think you have a better solutions please let me know! Thanks!
EDIT :
Each user has many memberships, that connect a user with an activity. So each activity has also many membership. Each membership has a user_id and an activity_id.
Each user has many likes, and each like connects a user and an activity. Therefore each activity has also many likes. A like has a user_id, activity_id, user_liked_activity, and activity_liked_user.
user_liked_activity is NULL if the user did not express an opinion about the activity, TRUE is the user liked the activity, and FALSE if the user disliked the activity. Viceversa for activity_liked_user.

Here's a Rails 5 solution, using not syntax. At first glance, this appears to be similar to your original solution, but this runs only one query, built from the separate excludable categories (members and already_liked).
class User < ApplicationRecord
has_many :likes
has_many :memberships
def self.queue(current_activity_id)
members = User.joins(:memberships).where(memberships: {activity_id: current_activity_id})
already_liked = User.joins(:likes).where(likes: {activity_id: current_activity_id, activity_liked_user: [true, false]})
where.not(id: members).where.not(id: already_liked)
end
end
You will call this from your controller like so:
#queue = User.queue(session[:current_activity_id])
As I understand the problem, for the current session activity, we want a query that will return all users except those that belong to one of two categories: (1) Those who are members of the activity, represented by users having a membership linked to the current activity, and (2) those who already have a "liked" rating from the activity, represented by users having a "like" linked to the current activity for which activity_likes_user is either true or false (but not nil).
We establish ActiveRecord relations for each category, but we don't call anything that would actually trigger a query. Category 1 is members, and Category 2 is already_liked.
Now we build a query on the User model (this is implied, because we are calling from within a class method on the User class) asking for users where id is not one of the members and is not one of the already_liked.
There may be a more elegant solution, but this does work, and I believe it handles edge cases properly, though you should build tests to verify this.

Related

Rails includes method when to add

So I've read a lot about the rails includes method but I'm still a bit confused about what's the best situation to use it.
I have a situation where I have a user record and then this user is related to multiple models like client, player, game, team_player, team, server and server_center.
I need to display specific attributes from the related models in a view. I only need around 1-2 attributes from a specific model and I don't use the others.
I already added delegates for example to get the server.name from player I can use server_name but in this situation do I include all of the tables from which I need the attributes or is there something else I do because I only need a couple of attributes from the model.
My query is as follows at the moment:
#user_profile = User
.includes({:client => [:player, :team_player => [:team]]},
:game,
{:server_center => :server})
.where(game_id: #master.admin.games)
Includes ensures that all of the specified associations are loaded using the minimum possible number of queries.
Let say we have 2 models named User and Profile :
class User < ActiveRecord::Base
has_one :profile
end
class Profile < ActiveRecord::Base
belongs_to :user
end
If we are iterating through each of the users and display the name of each user were name field resides in Profile model which has a association with User model, we would normally have to retrieve the name with a separate database query each time. However, when using the includes method, it has already eagerly loaded the associated person table, so this block only required a single query.
without includes:
users = User.all
users.each do |user|
puts user.profile.name # need extra database query for each time we call name
end
with includes
# 1st query to get all users 2nd to get all profiles and loads to the memory
users = User.includes(:profile).all
users.each do |user|
puts user.profile.name # no extra query needed instead it loads from memory.
end
Eager Loading is used to prevent N+1 query problems. basically it does left outer join and this plays an important role in speeding up request response or optimizing the queries. eg: if we are having huge amount users and if we want to iterate through those users and their corresponding profile. no of time which we will be hitting database will be equals to number of users. but if we are using includes it will keep all profile into memory later when we iterate through the users it will fetch from this memory instead of querying.
Eager loading may not always be the best the cure for our N+1 queries for eg: if you are dealing with some complex queries preferably looks for some caching solutions like Russian Doll caching etc.. still both method has his own pros & cons end of the day it's up to you to determine the best approach.
one useful gem which helps to detect N+1 query is bullet

Sort by timestamp on has_many after scope

I have a User object, a Package object (User has_many packages) and then a LocationTracker (User has_many location_trackers), which acts as a join table between User and Package, but just tracks details such as the most recent package delivery.
I'd like to sort my Users based on the most recent package they sent. The LocationTracker has an attribute last_received_from_user
I can easily sort the users from a certain location by ordering by the last_received_from_user attribute, however I'd also like to have a global index page that shows all of the Users, sorted by the last package they delivered.
I'm having trouble grouping the users. I'm attempting to use a DISTINCT ON(last_received_from_user), but then it complains that the attribute isn't in the group, and when I add it to the group, it groups by that timestamp, which is obviously pretty unique, so I get duplicate users showing up.
My current code is as follows:
User.includes(:location_trackers)
.group("location_trackers.user_id, users.id")
.order("location_trackers.last_received_from_user #{order} NULLS LAST")
Any help is greatly appreciated!
EDIT:
I've got the last_received_from_user which allows me to sort users from a SINGLE location well. However, I need to be able to scope based on what could be a number of different options. For example, only show users in a certain area (Which could be compromised of a few locations), or order by ALL users for ALL locations. The attribute works great for a single user-location relationship, but fails when it comes to attempting to perform the search on more than 1 location.
I'd like to sort my Users based on the most recent package they sent
Wouldn't it be easier (and way more efficient) having an attribute like latest_delivery_location and using a callback on the User model like:
class User < ApplicationRecord
after_update :update_latest_delivery_location
private
def update_latest_delivery_location
update_attributes(
latest_delivery_location: location_trackers.last.last_received_from_user
)
end
end
Or updating such attribute after an order has been placed / dispatched. I'd go for this approach because is easier to maintain and, if you want it more performing you could always add an index on users.latest_delivery_location for sorting operations.

Getting a relationship limited by the current user without generating N queries in Rails

In the web app I'm building I have a model called Task and another called TaskStatus. Tasks are global and have many TaskStatus which also belong to a user (many users perform the same tasks and the status track that).
I want to display all the tasks and access the status of that task belonging to the current user without generating N queries (one per task). Does Rails offer a way to achieve this? I've been playing with scopes and includes to no-avail.
I know of I could just run the second query manually and store them in a hash indexed by the task id, but I'm trying to find, if possible, a more railsy way of doing it.
You can specify this filter with a seperate association using conditions:
has_many :current_user_task_statuses,
conditions: { user_id: proc { current_user.id } },
class_name: "TaskStatus"
Then you can preload all the wanted TaskStatuses with a constant amount of queries like this:
Task.all.includes(:current_user_task_statuses)
I'm not absolutely certain that the proc will work in the conditions, but give it a try. If it doesn't, try this:
has_many :current_user_task_statues,
proc { where('user_id = ?', current_user.id) },
class_name: 'TaskStatus'
Hope this helps!

Return nil object if after_initialize callback returns false

My Rails app has many models that form a hierarchy. For example: Retailer > Department > Product Category > Product > Review.
A business requirement is that high-authority users can "share" any individual element in the hierarchy with a new or existing "normal" user. Without having an object shared with them, normal users have no rights to see (or do anything else) with any object in any level of the hierarchy.
The sharing process includes a choice of whether the share grants permission to read-only, read-update, or full CRUD on the target object.
Sharing any object grants R/O, R/W or CRUD permission to that object and all lower level objects in the hierarchy, and R/O permission to all of the direct ancestors of the object. The object collection grows organically, so the permission system works by just logging the user_id, the object_id of the share, and the nature of the share (R/O, CRUD, etc). As the population of objects in this hierarchy grows all the time, it is impractical to create an explicit permission record in the DB for every user/object combination.
Instead, at the start of the user request cycle, ApplicationController gathers all the permission records (user X has CRUD permission to Department #5) and holds them in a hash in memory. A Permissions model knows how to evaluate the hash when any object is passed to it - Permission.allow?(:show, Department#5) would return true or false depending on the content of the user's permission hash.
Let's take, for example, the Department model:
# app/models/department.rb
class Department < ActiveRecord::Base
after_initialize :check_permission
private
def check_permission
# some code that returns true or false
end
end
When the check_permission method returns true, I want Department.first to bring back the first record in the database as normal, BUT, if check_permission returns false, I want to return nil.
Right now, I have a solution whereby default scopes trigger a permissions check, but this is causing 2X the number of queries, and for classes with a lot of objects, memory problems and time/performance issues are sure to be on the horizon.
My goal is to use after_initialize callbacks to pre-permission the objects.
It would appear however that after_initialize is unable to block the original object from being returned. It does allow me to reset the values of the attributes of the object, but not to dispense with it.
Anybody know how to achieve this?
EDIT:
Many thanks for all of the answers and comments offered so far; hopefully this extended version of the question clarifies things.
Basically you need to check for access rights (or permissions) before returning a database query result. And you are trying to integrate this logic into your models.
It is possible, but not with the design you described in your question. It is not clean to implement this directly in ActiveRecord adapter methods (such as first, all, last etc...). You need to rethink your design.
(skip to point 'D' if this is too much reading)
You have several choices, which all depend on the way your permissions are defined. Let's look at few cases:
A. A user have a list of departments he owns and only him can access them
You can simply implement this as a has_many/belongs_to association with Active Record Associations
B. Users and Departments are independent (in other words: no ownership such as described in the previous case) and permission can be set individually for each users and each departments.
Simply again, you can implement a has_and_belongs_to_many association with Active Record Associations. You will need to create web logic so the administrator of your application can add/edit/remove access rights.
C. More complex case: the existing authorization libraries
Most people will turn to authorization solutions such as cancan, pundit or other
D. When those authorization libraries are oversized for your needs (actually, my case in most of my projects), I found that implementing authorization through rails scoping answers all my needs.
Let's see it through a simple example. I want administrators to be able to access the whole database records ; and regular users to access only departments with status = open and only during operation hours (say 8am-6pm). I write a scope that implement my permission logic
# Class definition
class Department
scope :accessible_by -> (user) do
# admin user have all access, always
if user.is_admin?
all
# Regular user can access only 'open' departments, and only
# if their request is done between 8am and 6pm
elsif Time.now.hour >= 8 and Time.now.hour <= 18
where status: 'open'
# Fallback to return ActiveRecord empty result set
else
none
end
end
end
# Fetching without association
Department.accessible_by(current_user)
# Fetching through association
Building.find(5).departments.accessible_by(current_user)
Defining a scope obliges us to use it everywhere in our code. You can think of the risk to "forget" going through the scope and accessing directly the model (i.e writing Department.all instead of Department.accessible_by(current_user)). So that's why you must solidly test your permissions in your specs (at the controller or features level).
Note In this example we do not return nil when the permission fails (as you mentioned in your question), but an empty result set instead. It is generally better so you keep the ActiveRecord method chaining capability. But you could also raise an exception and rescue it from your controller then redirect to a 'not authorized' page for example.
That is not what the after_initialize callback is used for. Instead, you could just define a method that does the same thing. For example, put this in your Department model and it should achieve the results you are looking for:
def self.get_first
check_permission ? first : nil
end
UPDATE
I'm not exactly sure how safe something like this would be, but you could just override the all method as the other query methods are based off of it.
class Department < ActiveRecord::Base
def self.all
check_permission ? super : super.none
end
private
def self.check_permission
# some code that returns true or false
end
end
You are probably better off using some authorization framework though.
UPDATE 2
Thinking about this a little more, I strongly recommend using a different approach. You really shouldn't be overriding methods like all as there will surely be unintended side effects.
A practical alternative would be to create a has_and_belongs_to_many relationship between Department and User. Here is how you would set it up:
user.rb
class User < ActiveRecord::Base
has_and_belongs_to_many :departments
...
end
department.rb
class Department < ActiveRecord::Base
has_and_belongs_to_many :users
...
end
Then run these commands in your terminal:
rails g migration CreateJoinTableDepartmentsUsers departments users
rake db:migrate
Now you can add users to a department with #department.users << #user, or departments to a user with #user.departments << #department. This should achieve the functionality that you are looking for.
#user.departments will return only departments for that user, #user.departments.first will return the first department for that user or nil if it doesn't have any, and #user.departments.find(1) will return the corresponding department only if it belongs to the user or throw an exception otherwise.
You can use before_create callback to stop record creation if check permission is false. Just return false in check_permission filter and record will be not created.
class Department < ActiveRecord::Base
before_create :check_permission
private
def check_permission
# return false if permission is not allowed
end
end

Data Modeling Question for Rails

I have the following entities.
User
Event
Expense
Guest
A User is someone that is registered with my site. An event is something that a specific user can create and then invite people to (other users). An expense is created by a specific user for a specific event and can be associated with other users/guests of the event (like user A bought something for user B and user C and guest D). A guest is also created by a specific user for a specific event but once created for the event, the other users associated with that event can see the guest. A guest will never actually log into the system...it is just a placeholder for people that may have attended an event but whom are not registered on the site.
I am trying to wrap my head around the has_many and belongs_to attributes of these entities.
It is almost like in my many-to-many relationship tables I need another column which identifies the "type" of person (user or guest) so I can join properly to pull in names, etc. from the users and guests tables accordingly.
I have thought about creating a flag in the users table that says whether the record is a guest (and therefore not require that guest to register with the site, login, etc.) but that doesn't seem very clean.
Any thoughts are greatly appreciated.
Thanks!
Thomas
This sounds to me like you're naturally leaning towards what is called a polymorphic association. It might be useful for you to have a look at this Railscast by Ryan Bates: Polymorphic Associations
I agree with Gav Polymorphic just one table people with a column type and tada
class Person < ActiveRecord:Base
#methods and associations all people have
end
class User < Person
#methods and associations specific to users
end
class Guest < Person
#methods and associations specific to guests
end

Resources