I've got a rather complicated SQL-Query whose results should be paginated and displayed to the user. I don't want to go into details here, but to put it simple I just select all columns of a model, and an additional column, that is used just for sorting purposes.
Note.select( ['notes.*', "<rather complicated clause> AS 'x'"] ).joins(...)...
After some .join()'s and a .group(), I finally call .order() and .paginate on the relation. If I order by any of the model's natural columns everything works fine, if I however order by the "artificial" column x, rails gives me the error:
no such column: x
This seems to occur because will_paginate seems to do a COUNT(*)-statement before getting the actual data, simply to get the amounts of data it has to process. This COUNT(*)-statement is (of course) generated from the original statement, which includes the ORDER BY x. The problem now is, that will_paginate keeps the ORDER BY in the statement but simply replaces the column definitions with COUNT(*) and thus cannot find the column x.
Using Rails 3 and will_paginate 3.0.pre2.
Is there any way to get around this?
thx for any help
You can disable the initial count query by passing in the value manually:
total_entries = Note.select(...).joins(...).count
Note.select( ... ).joins(...).group().paginate(:total_entries => total_entries)
The above is not a literal code sample and you may need to tweak your count query to get the correct results for your join. This should let you do a straight up complex query on the pagination.
Related
I need to get the last record with a certain value, but without loading it.
I have to do something like:
Thing.where(cool: true).where(created_at: Thing.where(cool: true).maximum(:created_at))
The above is a way to do it, but it does a 2nd query to get the maximum value first. I want to do it all in one query and get the SQL equivalent of something = max(etc).
Just before someone mentions it: .last doesn't work because it returns an object not a relation.
In order words, I need to do .last but returning a relation instead of objects.
Try this way:
Thing.where("cool=1 AND created_at=(SELECT MAX(created_at) FROM things WHERE cool=1)")
On Rails 7, when I tested your query, the database is queried only once.
How can I get my plucked array to work with the paginate method (which I believe only works on queryproxy objects)
My results are pulling up a few of the same nodes as there are multiple paths to it, so I added pluck and distinct like so..
current_user.friends....where()...params().pluck('DISTINCT e').paginate..
Is there another way around it? or would a change have to be made in the neo4j paginate gem?
Right now, this isn't doable using the paginate method. WillPaginate returns WillPaginate::Collection objects that are already populated from the database. We might be able to make it return something else and evaluate lazily but I'd have to play around with it more.
You can create Neo4j::Paginated objects directly, but these are just plucked results from QP.
# match stupid friends with awful events, return distinct events
query = current_user.friends(:f).where(stupid: true).events(e:).rel_where(expected_attendees: 0)
#bad_events = Neo4j::Paginated.create_from(query, 1, 15).pluck('distinct e')
create_from returns a Neo4j::Paginated object that delegates each and pluck to its the QueryProxy object fed to it. Note that it's going to paginate based on the end of the chain, so it's doing the first page with 15 per page based on the events. Also note that you can't do a distinct count.
Check https://github.com/neo4jrb/neo4j/blob/master/lib/neo4j/paginated.rb for more. It's pretty easy to read.
My question is about how to perform varying levels of search into a database while limiting the number of queries.
Let's start simple:
#companies = Company.where("active = ?", true)
Let's say we display records from this set. Then, we need:
#clientcompanies = #companies.where("client_id = ?", #client.id)
We display something from #clientcompanies. Then, we want to drill down further.
#searchcompanies = #clientcompanies.where("name LIKE ? OR notes LIKE ?", "#{params[:search]}%", "#{params[:search]}%")
Are these three statements the most efficient way to go about this?
If indeed the database is starting with the entire Company table each time around, is there a way to limit the scope so each of the above statements would take a shorter amount of time as the size of the set diminishes?
In case it matters, I'm running Rails 3 on both MySQL and PostgreSQL.
It doesn't get much more optimized then what you're already doing. Exactly zero of those statements will execute a SQL query until you try to iterate over the results. Calling methods like all, first, inspect, any?, each etc will be when the query is executed.
Each time you chain on a new where or other arel method, it appends to the sql query that it'll execute at the end. If, somewhere in the middle, you want to see the query that'll be executed you can do puts #searchcompanies.to_sql
Note that if you run these commands in the console each statement appears to run a SQL query only because the console automatically runs .inspect on the line you entered.
Hopefully I answered your question :)
There's a great railscast here: http://railscasts.com/episodes/239-activerecord-relation-walkthrough that explains how ActiveRelation works, and what you can do with it.
EDIT:
I may have mis-understood your question. You indicated that after each where call you were displaying information from the query. What's the use-case for this? Are you displaying all companies on the same page that you have filtered-out companies from a search? If you display something from that very first query then you will be pulling every single company row from your database (which is not going to be very scalable or performant at larger quantities of company entries).
Would it not make sense to only display information from the #searchcompanies variable?
I‘d like to ask I question that here that I think would be easy to some people.
Ok I have query that return records of two related tables. (One to many)
In this query I have about 3 to 4 calculated fields that are based on the fields from the 2 tables.
Now I want to have a group by clause for names and sum clause to sum the calculated fields but it ends up in error message saying:
“You tried to execute a query that is not part of aggregate function”
So I decided to just run the query without the totals *(ie no group by , sum etc,,,)
:
And then I created another query that totals my previous query. ( i.e. using group by clause for names and sum for calculated fields… no calculation here) This is fine ( I use to do this) but I don’t like having two queries just to get summary total. Is their any other way of doing this in the design view and create only one query?.
I would very much appreciate.
Thankyou:
JM
Sounds like the query is thinking the calculated fields need to be part of the grouping or something. You might need to look into sub-querying.
Can you post the sql (before and after). It would help in getting an understanding of what the issue is.
I am working on a ASP.NET MVC application where we have to write our own code for paging.
And I see elements getting repeated on different pages. This is happening because the order of elements in the IQueryable varies randomly and one have to query the database for each page.
I solved this problem by ordering the elements by date of creation and then the order I wanted. But I feels its heavy. Is there a better way to solve this problem.
If you fetch rows from a database via IQuerable and order on one column, if there are rows that have the same value in that column the order of those rows may vary, you need to do as you do now, that is order by two columns, first the one you are really interested in and then the second (like date), there should not be that big a performance hit since the sorting is done in the database.
But what need to do is specify the ordering like this:
Context.Products
.Where(p => p.ProductID > 100)
.OrderBy(p => p.CategoryID)
.ThenBy(p => p.Date).ToList();
Notice the ThenBy, that will generate the correct SQL.
No, that is the correct way. A database might give back rows in a non-determistic order if you do not tell it to order by something.
Have a look at: http://srtsolutions.com/blogs/billwagner/archive/2006/10/14/paging-output-with-linq2sql.aspx of Bill Wagner.
He uses Take() and Skip() to create paged output.