I have two tables:-
Table 1. Activity
Table 2. ActivityAvailability
Relation Type: HasManyRelation (table 1 HasManyRelation to table2)
I couldn't able use withGraphJoined because it has a limit clause issue with pagination as the objection.js doc says. So I'm using withGraphFetched.
Activity.query().withGraphFetched('activityAvailabilityData').limit(10);
The main difference is that withGraphFetched uses multiple queries under the hood to fetch the result while withGraphJoined uses a single query and joins to fetch the results.
withGraphFetched under the hood queries
Query 1:
select `activity`.`id`, `name`, `type`, `city`, `state`, `address`, `latitude`, `longitude` from `activity` limit ?
Query 2: Here question mark will replace with above query activity IDs.
select `start_age`, `end_age`, `start_date`, `end_date`, `start_time`, `end_time` from `activity_availability` where `activity_availability`.`activity_id` in (?, ?, ?, ?)
Now first problem arries: If I want 10 records each time by limit.I will get if there is no where clause in second query.If incase any where clause is added in the child table (ActivityAvailability),there might be possibility it also eliminates few records from 10 and return.
So I solved it using joins along with withGraphFetched.
Activity.query().withGraphFetched('activityAvailabilityData').join('activity_availability', 'activity.id', '=', 'activity_availability.activity_id')
But this solution also has a drawback.
Whenever any where clause is added in the child table It should also be added in the parent table,just because I used joins.This is becoming difficult to manage.
So please let me know if there is another approach?
Related
I have a working SQL query for Postgres v10.
SELECT *
FROM
(
SELECT DISTINCT ON (title) products.title, products.*
FROM "products"
) subquery
WHERE subquery.active = TRUE AND subquery.product_type_id = 1
ORDER BY created_at DESC
With the goal of the query to do a distinct based on the title column, then filter and order them. (I used the subquery in the first place, as it seemed there was no way to combine DISTINCT ON with ORDER BY without a subquery.
I am trying to express said query in ActiveRecord.
I have been doing
Product.select("*")
.from(Product.select("DISTINCT ON (product.title) product.title, meals.*"))
.where("subquery.active IS true")
.where("subquery.meal_type_id = ?", 1)
.order("created_at DESC")
and, that works! But, it's fairly messy with the string where clauses in there. Is there a better way to express this query with ActiveRecord/Arel, or am I just running into the limits of what ActiveRecord can express?
I think the resulting ActiveRecord call can be improved.
But I would start improving with original SQL query first.
Subquery
SELECT DISTINCT ON (title) products.title, products.* FROM products
(I think that instead of meals there should be products?) has duplicate products.title, which is not necessary there. Worse, it misses ORDER BY clause. As PostgreSQL documentation says:
Note that the “first row” of each set is unpredictable unless ORDER BY is used to ensure that the desired row appears first
I would rewrite sub-query as:
SELECT DISTINCT ON (title) * FROM products ORDER BY title ASC
which gives us a call:
Product.select('DISTINCT ON (title) *').order(title: :asc)
In main query where calls use Rails-generated alias for the subquery. I would not rely on Rails internal convention on aliasing subqueries, as it may change anytime. If you do not take this into account you could merge these conditions in one where call with hash-style argument syntax.
The final result:
Product.select('*')
.from(Product.select('DISTINCT ON (title) *').order(title: :asc))
.where(subquery: { active: true, meal_type_id: 1 })
.order('created_at DESC')
i've some query to select all data based on skipped_play_id but when i'm execute the query it still return some duplicate skipped_play_id
user = User.first
skipped_plays=user.user_skipped_plays.select(:skipped_play_id,:created_at).group(:created_at,:skipped_play_id)
and why should i provide :created_at as one of group by argument if i only need query grouped by skipped_play_id. if i change my query to user.user_skipped_plays.select(:skipped_play_id,:created_at).group(:skipped_play_id)
it will return
ActiveRecord::StatementInvalid: PG::GroupingError: ERROR: column "user_skipped_plays.created_at" must appear in the GROUP BY clause or be used in an aggregate function
LINE 1: SELECT "user_skipped_plays"."skipped_play_id", "user_skippe...
^
: SELECT "user_skipped_plays"."skipped_play_id", "user_skipped_plays"."created_at" FROM "user_skipped_plays" WHERE "user_skipped_plays"."user_id" = $1 GROUP BY "user_skipped_plays"."skipped_play_id" ORDER BY "user_skipped_plays"."id" ASC LIMIT $2
from /Users/fourtyonestudio/.rvm/gems/ruby-2.3.1/gems/activerecord-5.0.0.1/lib/active_record/connection_adapters/postgresql_adapter.rb:657:in `prepare
i'll really appreciate any advice
I think you don't need GROUP BY in your code at all. If you need to select unique rows containing just skipped_play_id and created_at you can go with:
user.user_skipped_plays.select(:skipped_play_id, :created_at).uniq
But in such selection you will still get duplicated skipped_play_id if you have different rows with same skipped_play_id, but different created_at, because you take a combination. So, possibly, you just need to go with:
user.user_skipped_plays.select(:skipped_play_id).uniq
It depends on what you need. You can type in comments for my answer how do you plan to use this selection and I will help you to solve your issue
Please try this
User.first.user_skipped_plays.group_by { |t| t.skipped_play_id }
Let me know if you are getting correct data or not
I'm trying to combine a uniq statement with a select("distinct") statement in Active Record, and it results in two DISTINCT keywords, which of course leads to an invalid query. This is the simplest example I have come up with. (Mark that is is simplified in order to help you understand the problem - I'm not simply asking for how I get out distinct ids from a database.)
Product.all.uniq.select("distinct id").map(&:id)
This gives me this error message:
Product Load (0.7ms) SELECT DISTINCT distinct id FROM "products"
ActiveRecord::StatementInvalid: PG::SyntaxError: ERROR: syntax error at or near "distinct"
LINE 1: SELECT DISTINCT distinct id FROM "products"
^
: SELECT DISTINCT distinct id FROM "products"
Why do I get two DISTRINCT keywords here? Is there any way to avoid it? Using uniq twice works, but I need to do a select for one of the filters I'm implementing.
Edit: The select("distinct..") has to go before the uniq statement.
uniq already uses DISTINCT. Either use uniq or select("DISTINCT"). Moreover, you should use pluck, and not map over the records and select the id.
What you really want to use is
Product.pluck(:id)
or
Product.all.ids
What's not clear to me, is why you want to use distinct. How comes an ID has duplicate values?
If the field is different than an id, simply use
Product.select("DISTINCT(field)").map(&:field)
or even better
Product.uniq.pluck(:field)
Hence in your case
Product.uniq.pluck(:id)
# => SELECT DISTINCT "products"."id" FROM "products"
You can use uniq(false) to disable a previously used uniq scope. So your example would go like this:
scope = Product.all.uniq
scope.uniq(false).select("distinct id").map(&:id)
Source code documentation can be found here.
I've rewritten this question as my previous explanation was causing confusion.
In the SQL world, you have an initial record set that you apply a query to. The output of this query is the result set. Generally, the initial record set is an entire table of records and the result set is the records from the initial record set that match the query ruleset.
I have a use case where I need my application to occasionally operate on only a subset of records in a table. If a table has 10,000 records in it, I'd like my application to behave like only the first 1,000 records exist. These should be the same 1,000 records each time. In other words, I want the initial record set to be the first 1,000 devices in a table (when ordered by primary key), and the result set the resulting records from these first 1,000 devices.
Some solutions have been proposed, and it's revealed that my initial description was not very clear. To be more explicit, I am not trying to implement pagination. I'm also not trying to limit the number of results I receive (which .limit(1,000) would indeed achieve).
Thanks!
This is the line in your question that I don't understand:
This causes issues though with both of the calls, as limit limits the results of the query, not the database rows that the query is performed on.
This is not a Rails thing, this is a SQL thing.
Device.limit(n) runs SELECT * FROM device LIMIT n
Limit always returns a subset of the queried result set.
Would first(n) accomplish what you want? It will both order the result set ascending by the PK and limit the number of results returned.
SQL Statements can be chained together. So if you have your subset, you can then perform additional queries with it.
my_subset = Device.where(family: "Phone")
# SQL: SELECT * FROM Device WHERE `family` = "Phone"
my_results = my_subset.where(style: "Touchscreen")
# SQL: SELECT * FROM Device WHERE `family` = "Phone" AND `style` = "Touchscreen"
Which can also be written as:
my_results = Device.where(family: "Phone").where(style: "Touchscreen")
my_results = Device.where(family: "Phone", style: "Touchscreen")
# SQL: SELECT * FROM Device WHERE `family` = "Phone" AND `style` = "Touchscreen"
From your question, if you'd like to select the first 1,000 rows (ordered by primary key, pkey) and then query against that, you'll need to do:
my_results = Device.find_by_sql("SELECT *
FROM (SELECT * FROM devices ORDER BY pkey ASC LIMIT 1000)
WHERE `more_searching` = 'happens here'")
You could specifically ask for a set of IDs:
Device.where(id: (1..4).to_a)
That will construct a WHERE clause like:
WHERE id IN (1,2,3,4)
In rails 3.0.0, the following query works fine:
Author.where("name LIKE :input",{:input => "#{params[:q]}%"}).includes(:books).order('created_at')
However, when I input as search string (so containing a double colon followed by a dot):
aa:.bb
I get the following exception:
ActiveRecord::StatementInvalid: SQLite3::SQLException: ambiguous column name: created_at
In the logs the these are the sql queries:
with aa as input:
Author Load (0.4ms) SELECT "authors".* FROM "authors" WHERE (name LIKE 'aa%') ORDER BY created_at
Book Load (2.5ms) SELECT "books".* FROM "books" WHERE ("books".author_id IN (1,2,3)) ORDER BY id
with aa:.bb as input:
SELECT DISTINCT "authors".id FROM "authors" LEFT OUTER JOIN "books" ON "books"."author_id" = "authors"."id" WHERE (name LIKE 'aa:.bb%') ORDER BY created_at DESC LIMIT 12 OFFSET 0
SQLite3::SQLException: ambiguous column name: created_at
It seems that with the aa:.bb input, an extra query is made to fetch the distinct author id_s.
I thought Rails would escape all the characters. Is this expected behaviour or a bug?
Best Regards,
Pieter
The "ambiguous column" error usually happens when you use includes or joins and don't specify which table you're referring to:
"name LIKE :input"
Should be:
"authors.name LIKE :input"
Just "name" is ambiguous if your books table has a name column too.
Also: have a look at your development.log to see what the generated query looks like. This will show you if it's being escaped properly.
Replace
.includes(:books)
with
.preload(:books)
This should force activerecord to use 2 queries instead of the join.
Rails has 2 versions of includes: One which constructs a big query with joins (the 2nd of your 2 queries and thus more likely to result in ambiguous column references and one that avoids the joins in favour of a separate query per association.
Rails decides which strategy to used based on whether it thinks that your conditions, order etc refer to the included tables (since in that case the joins version is required). Where a condition is a string fragment that heuristic isn't very sophisticated - i seem to recall that it just scans the conditions for anything that might look like a column from another table (ie foo.bar) so having a literal of that form could fool it.
You can either qualify your column names so that it doesn't matter which includes strategy is used or you can use preload/eager_load instead of includes. These behave similarly to includes but force a specific include strategy rather than trying to guess which is most appropriate.