Connections breaks includes - ruby-on-rails

I have a following setup:
class Product < ApplicationRecord
has_many :variants
end
class Variant < ApplicationRecord
belongs_to :product
end
Types::QueryType = GraphQL::ObjectType.define do
connection :products, Types::ProductType.connection_type do
resolve -> (obj, _, _) do
Product.all.includes(:variants)
end
end
end
Types::ProductType = GraphQL::ObjectType.define do
connection :variants, Types::VariantType.connection_type do
resolve -> (obj, _, _) { obj.variants }
end
end
And running a following query:
{
products {
edges {
nodes {
variants {
edges {
node {
id
}
}
}
}
}
}
}
produces following SQL queries:
Product Load (2.7ms) SELECT "products".* FROM "products" LIMIT $1 [["LIMIT", 25]]
Variant Load (8.6ms) SELECT "variants".* FROM "variants" WHERE "variants"."product_id" IN (1, 2, 3)
Variant Load (19.0ms) SELECT "variants".* FROM "variants" WHERE "variants"."product_id" = $1 LIMIT $2 [["product_id", 1], ["LIMIT", 25]]
Variant Load (13.6ms) SELECT "variants".* FROM "variants" WHERE "variants"."product_id" = $1 LIMIT $2 [["product_id", 2], ["LIMIT", 25]]
Variant Load (2.4ms) SELECT "variants".* FROM "variants" WHERE "variants"."product_id" = $1 LIMIT $2 [["product_id", 3], ["LIMIT", 25]]
As we can see in the sql output, includes works but graphql don't care and makes a n+1 anyway. Is that normal behaviour and i'm forced to use solutions like graphql-batch to fix that or something is not right with my setup? As far as i have seen all over the internet, using includes should be enough for such simple scenario and graphql should use the eager loaded data instead of producing the n+1. Have i done anything wrong in here?
I'm on graphql-ruby 1.7.9

I just received a reply on graphql-ruby issue tracker :
Hey, I noticed that LIMIT 25 is being applied to those queries. Do you
know where that's being applied? If you want to use the result from
the initial query, you should remove the LIMIT clause. (I'm guessing
that if you ask for .limit(25), ActiveRecord won't use a cached
relation.) Maybe you have a default_max_page_size? What happens if you
remove it?
So, long story short, i removed the default_max_page_size config from my schema and it resolved the issue.

Related

ActiveRecord::StatementInvalid: PG::UndefinedTable in many to many relation but table exists

I have an easy many to many relation and It doesn't work and I cannot understand why. I'm sure that is something so obvious... but..
class Content < ApplicationRecord
has_many :content_brands
has_many :brands, through: :content_brands
end
class ContentBrand < ApplicationRecord
belongs_to :content
belongs_to :brand
end
class Brand < ApplicationRecord
establish_connection Rails.application.config.brands_database_configuration
has_many :content_brands
has_many :contents, through: :content_brands
end
But
irb(main):002:0> Content.first.brands
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERRORE: la relazione "content_brands" non esiste
LINE 1: SELECT "brands".* FROM "brands" INNER JOIN "content_brands"...
^
: SELECT "brands".* FROM "brands" INNER JOIN "content_brands" ON "brands"."id" = "content_brands"."brand_id" WHERE "content_brands"."content_id" = $1 ORDER BY "brands"."name" ASC LIMIT $2
The table exists, I can query it
irb(main):006:0> ContentBrand.first.brand
ContentBrand Load (0.5ms) SELECT "content_brands".* FROM "content_brands" ORDER BY "content_brands"."id" ASC LIMIT $1 [["LIMIT", 1]]
Brand Load (27.4ms) SELECT "brands".* FROM "brands" WHERE "brands"."id" = $1 ORDER BY "brands"."name" ASC LIMIT $2 [["id", 1], ["LIMIT", 1]]
=> #<Brand id: 1, name: "Nokia", logo: "nokia.jpeg", created_at: "2016-12-08 15:50:48", updated_at: "2017-02-02 15:51:43", web_site: "http://www.nokia.it">
Why?
I'm getting crazy because the inverse relation works
Brand.first.contents
Brand Load (25.8ms) SELECT "brands".* FROM "brands" ORDER BY "brands"."name" ASC LIMIT $1 [["LIMIT", 1]]
Content Load (0.7ms) SELECT "contents".* FROM "contents" INNER JOIN "content_brands" ON "contents"."id" = "content_brands"."content_id" WHERE "content_brands"."brand_id" = $1 ORDER BY "contents"."published_at" DESC LIMIT $2 [["brand_id", 391], ["LIMIT", 11]]
=> #<ActiveRecord::Associations::CollectionProxy []>
irb(main):011:0>
Update: I forgot to tell you that Brand is on another database...
You can't setup associations to a model that is stored in another database in ActiveRecord. Which makes sense since you can't join another database in a single query in Postgres without jumping through some pretty serious hoops (Postgres_FDW). And with the polyglot nature of ActiveRecord this would just be too much complexity for a very limited use case.
If its in any way possible I would switch to a single database setup even if it means that you have to duplicate data.
If you look at the "inverse query" you can see that it works because its not a single query:
# queries the "brands" database
Brand Load (25.8ms) SELECT "brands".* FROM "brands" ORDER BY "brands"."name" ASC LIMIT $1 [["LIMIT", 1]]
# queries your main database
Content Load (0.7ms) SELECT "contents".* FROM "contents" INNER JOIN "content_brands" ON "contents"."id" = "content_brands"."content_id" WHERE "content_brands"."brand_id" = $1 ORDER BY "contents"."published_at" DESC LIMIT $2 [["brand_id", 391], ["LIMIT", 11]]
However this does not mean that the concept is feasible.

ActiveStorage order

Is there a way to order the blobs in ActiveStorage?
The following works
class Project < ApplicationRecord
has_many :attachments, as: :attachable
scope :with_attached_files, -> { includes(:attachments).merge(Attachment.with_attached_file.order('active_storage_blobs.filename')) }
end
However, it is case sensitive.
I have tried
scope :with_attached_files, -> { includes(:attachments).merge(Attachment.with_attached_file.order(Arel.sql('lower(active_storage_blobs.filename)'))) }
but it generates a SQL error.
PG::UndefinedTable - ERROR: missing FROM-clause entry for table
"active_storage_blobs"
The generated query is
SELECT "projects".* FROM "projects" WHERE "projects"."slug" = $1
ORDER BY lower(active_storage_blobs.filename) LIMIT $2 [["slug",
"aa0001-18"], ["LIMIT", 1]]
My current idea is to overwrite the default scope on ActiveStorageBlob but it doesn't seem to work so far.

Is there any way to access the Parent object whom called a singleton method?

Given the following functional snippet I'm having trouble reducing the database queries:
class User < ApplicationRecord
belongs_to :account
def self.do_something
self.find_each do |user|
puts "#{self.new.account.name}:#{user.name} did something"
end
end
end
class Account < ApplicationRecord
has_many :users
end
a = Account.first
puts 'starting'
a.users.do_something
Account Load (0.4ms) SELECT "accounts".* FROM "accounts" WHERE
"accounts"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
starting
Account Load (0.3ms) SELECT "accounts".* FROM "accounts" WHERE
"accounts"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Test:User did something
Account Load (0.3ms) SELECT "accounts".* FROM "accounts" WHERE
"accounts"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Test:User did something
Account Load (0.3ms) SELECT "accounts".* FROM "accounts" WHERE
"accounts"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Test:User did something
Account Load (0.3ms) SELECT "accounts".* FROM "accounts" WHERE
"accounts"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
Test:User did something
You can see that the Account model is being fetched from the database per user!
I was hoping to use something like self.account in the Singleton method to reference the original account, but the relationship obviously doesn't exist by default which is why I'm currently using self.new.account.
Is there anywhere else I can fetch the original Account model saved in a from inside self.do_something? I can obviously pass the account in a parameter, but that seems tedious especially if I may add arguments later...
Inside your find_each loop, you should be able to use user.account.
Outside that loop, I don't believe there's a documented / supported / won't-disappear-without-warning way to find the object. Depending on your Rails version, something like self.current_scope.proxy_association.owner might give you the answer you need... but do prefer user.account if at all possible: the more you use private APIs, the harder future upgrades can be.
Alternatively, consider using association extensions to define your do_something method inside the has_many association only -- if it's not suited to be called as User.do_something or User.where(name: "Bob").do_something (because those don't have an associated account), maybe it shouldn't be a top-level method after all.

Rails. Debugging a Uniqueness validation with scope that fails when object is updated

I have a model Prediction for which its instances have references to other models Participant and Match through the attributes :participant_id :match_id.
The idea is that a #participant can have only one #prediction per #match (every #participant is allowed to give maximum one prediction per #match).
So I've included the following validation in my Prediction model
validates :match_id, uniqueness: { scope: [:participant_id] }
The validation works OK to disallow #prediction objects to be created when there's already a #prediciton for the same #match and #participant. The problem is that it does, as well, prevent updating an object even though I'm not changing those assigned ids of the original object. Plus, it's letting me update only one instance of Prediction, all other's through an error.
What's puzzling is that it was working fine but for some reason, I've been unable to track, it stopped working and I have run out of ideas on how to debug. Could you provide any ideas? Below what I've checked/tried:
The relevant params of a failed request:
"prediction"=><ActionController::Parameters {"participant_id"=>"1", "match_id"=>"2", "local_score"=>"2", "visitant_score"=>"0"}
The error I'm getting:
#details={:match_id=>[{:error=>:taken, :value=>2}]},
#messages={:match_id=>["has already been taken"]}>
There are plenty predictions in the database with match_id: 2. However, this is happening despite having no other #prediction in the database with both match_id: 2 and participant_id: 1 except, of course, for the instance that throws this error while updating.
Funnily enough, if I try the same operation in the console (as below) it successfully updates.
#prediction = Prediction.find_by(participant_id: 1, match_id: 2)
#prediction.update_attributes(local_score: 8, visitant_score: 8)
Suggesting the problem is in the controller action:
app/controllers/predicitons_controller.rb
def update
respond_to do |format|
if #prediction.update(prediction_params)
format.json { render :show, status: :ok, location: #prediction }
format.js
else
#errorMessages = []
#prediction.errors.full_messages.each do |message|
#errorMessages << message
end
format.js {render template: 'predictions/update_error'}
end
end
end
I see no problem there. Additionally, it seems that the only request sent to the controller that updates successfully is:
"prediction"=><ActionController::Parameters {"participant_id"=>"1", "match_id"=>"1", "local_score"=>"1", "visitant_score"=>"1"}
And if I do these others, for example, it doesn't:
"prediction"=><ActionController::Parameters {"participant_id"=>"2", "match_id"=>"1", "local_score"=>"0", "visitant_score"=>"0"}
"prediction"=><ActionController::Parameters {"participant_id"=>"2", "match_id"=>"2", "local_score"=>"9", "visitant_score"=>"9"}
"prediction"=><ActionController::Parameters {"participant_id"=>"4", "match_id"=>"1", "local_score"=>"1", "visitant_score"=>"1"
All of these failed requests throw the "match_id already taken" error and update just fine in the console.
The controller request is touching the correct action with the method: :put. I've tried to change the syntax of the validation switching match_id for pool_id (which of course shouldn't make a difference) with no success.
Any, help would be appreciated.
app/models/prediction.rb
class MatchPrediction < ApplicationRecord
belongs_to :participant
belongs_to :match
validates :match_id, uniqueness: { scope: [:pool_participant_id] }
def correct_score?
if match.official_outcome
local_score == match.local_score && visitant_score == match.visitant_score ? true : false
end
end
def correct_outcome?
if match.official_outcome
predicted_outcome == match.official_outcome ? true : false
end
end
end
and server output pre rollback:
Started PUT "/pools/1/predictions/19" for 127.0.0.1 at 2018-03-29 18:17:11 +1100
Processing by PredictionsController#update as JS
Parameters: {"utf8"=>"✓", "authenticity_token"=>"SagJX+a7m0eCAdH7AdA0eYz6BVL1cesXYXOUkoe2FRynta6wyiWdskrC1007V1vyrIApPtdEQnVHWlzhSeJs5Q==", "prediction"=>{"participant_id"=>"4", "match_id"=>"2", "local_score"=>"7", "visitant_score"=>"0"}, "pool_id"=>"1", "id"=>"19"}
User Load (1.4ms) SELECT "users".* FROM "users" WHERE "users"."id" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["id", 4], ["LIMIT", 1]]
↳ /Users/andres/.rvm/gems/ruby-2.5.0#global/gems/rack-2.0.4/lib/rack/tempfile_reaper.rb:15
Prediction Load (0.4ms) SELECT "predictions".* FROM "predictions" WHERE "predictions"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
↳ app/controllers/predictions_controller.rb:86
(0.2ms) BEGIN
↳ app/controllers/predictions_controller.rb:55
Participant Load (0.2ms) SELECT "participants".* FROM "participants" WHERE "participants"."id" = $1 LIMIT $2 [["id", 4], ["LIMIT", 1]]
↳ app/controllers/predictions_controller.rb:55
Match Load (0.3ms) SELECT "matches".* FROM "matches" WHERE "matches"."id" = $1 LIMIT $2 [["id", 2], ["LIMIT", 1]]
↳ app/controllers/predictions_controller.rb:55
Prediction Exists (1.8ms) SELECT 1 AS one FROM "predictions" WHERE "predictions"."match_id" = $1 AND "predictions"."id" != $2 AND "predictions"."participant_id" = $3 LIMIT $4 [["match_id", 2], ["id", 1], ["participant_id", 4], ["LIMIT", 1]]
↳ app/controllers/predictions_controller.rb:55
Pool Load (0.5ms) SELECT "pools".* FROM "pools" WHERE "pools"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
↳ app/models/prediction.rb:35
Round Load (0.6ms) SELECT "rounds".* FROM "rounds" INNER JOIN "groups" ON "rounds"."id" = "groups"."round_id" WHERE "groups"."id" = $1 LIMIT $2 [["id", 1], ["LIMIT", 1]]
↳ app/models/prediction.rb:21
(0.2ms) ROLLBACK
↳ app/controllers/predictions_controller.rb:55
Embarrassing mistake really. I missed checking how the Predictions Controller was setting #prediction. I had entred the params to find the object incorrectly as #GorillaApe suggested:
def set_match_prediction
#prediction = Prediction.find(params[:pool_id])
end
instead of:
def set_match_prediction
#prediction = Prediction.find(params[:id])
end
this in turn was always setting the same object thus raising the validation error in all but one of the instances which, by chance, its :id coincided with :pool_id.

Handling a massive query in Rails

What's the best way to handle a large result set with Rails and Postgres? I didn't have a problem until today, but now I'm trying to return a 124,000 record object of #network_hosts, which has effectively DoS'd my development server.
My activerecord orm isn't the prettiest, but I'm pretty sure cleaning it up isn't going to help in relation to performance.
#network_hosts = []
#host_count = 0
#company.locations.each do |l|
if l.grace_enabled == nil || l.grace_enabled == false
l.network_hosts.each do |h|
#host_count += 1
#network_hosts.push(h)
#network_hosts.sort! { |x,y| x.ip_address <=> y.ip_address }
#network_hosts = #network_hosts.first(5)
end
end
end
In the end, I need to be able to return #network_hosts to the controller for processing into the view.
Is this something that Sidekiq would be able to help with, or is it going to be just as long? If Sidekiq is the path to take, how do I handle not having the #network_hosts object upon page load since the job is running asyncronously?
I believe you want to (1) get rid of all that looping (you've got a lot of queries going on) and (2) do your sorting with your AR query instead of in the array.
Perhaps something like:
NetworkHost.
where(location: Location.where.not(grace_enabed: true).where(company: #company)).
order(ip_address: :asc).
tap do |network_hosts|
#network_hosts = network_hosts.limit(5)
#host_count = network_hosts.count
end
Something like that ought to do it in a single DB query.
I had to make some assumptions about how your associations are set up and that you're looking for locations where grace_enabled isn't true (nil or false).
I haven't tested this, so it may well be buggy. But, I think the direction is correct.
Something to remember, Rails won't execute any SQL queries until the result of the query is actually needed. (I'll be using User instead of NetworkHost so I can show you the console output as I go)
#users = User.where(first_name: 'Random');nil # No query run
=> nil
#users # query is now run because the results are needed (they are being output to the IRB window)
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 LIMIT $2 [["first_name", "Random"], ["LIMIT", 11]]
# => #<ActiveRecord::Relation [...]>
#users = User.where(first_name: 'Random') # query will be run because the results are needed for the output into the IRB window
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 LIMIT $2 [["first_name", "Random"], ["LIMIT", 11]]
# => #<ActiveRecord::Relation [...]>
Why is this important? It allows you to store the query you want to run in the instance variable and not execute it until you get to a view where you can use some of the nice methods of ActiveRecord::Batches. In particular, if you have some view (or export function, etc.) where you are iterating the #network_hosts, you can use find_each.
# Controller
#users = User.where(first_name: 'Random') # No query run
# view
#users.find_each(batch_size: 1) do |user|
puts "User's ID is #{user.id}"
end
# User Load (0.5ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# User's ID is 1
# User Load (0.4ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 AND ("users"."id" > 1) ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# User's ID is 2
# User Load (0.3ms) SELECT "users".* FROM "users" WHERE "users"."first_name" = $1 AND ("users"."id" > 2) ORDER BY "users"."id" ASC LIMIT $2 [["first_name", "Random"], ["LIMIT", 1]]
# => nil
Your query is not executed until the view, where it will now load only 1,000 records (configurable) into memory at a time. Once it reaches the end of those 1,000 records, it will automatically run another query to fetch the next 1,000 records. So your memory is much more sane, at the cost of extra database queries (which are usually pretty quick)

Resources