I have a table with almost 300K records in it. I run a simple select statement with a where clause on an indexed column ('type' is indexed):
SELECT *
FROM Asset_Spec
WHERE type = 'County'
That query is fast - about 1 second. Additionally I want to test against status:
SELECT *
FROM Asset_Spec
WHERE type = 'County'
AND status = 'Active'
The second one is VERY slow (minutes). Status is NOT indexed and in this particular case 99.9% of values in the db ARE 'Active'.
Any ideas how I can get better performance? We are compiling our own version of SQLite so I can tweak many settings (FYI - same performance on iOS pre-canned SQLite)
I looked at the query plan and the estimate for number of rows was off by an astounding amount. Asset_Spec (~2 rows) - actual number of rows is almost 300,000. Ran 'ANALYZE' - now the same query runs in 16ms.
the first thing I would try is using a subquery
SELECT * FROM
(SELECT *
FROM Asset_Spec
WHERE type = 'County')
WHERE status = 'Active'
and as Robert suggests, adding an index on any column you want to filter by is a good idea. I'd also consider changing fields Type and Status to be something other than string.
Any reason you need to select *?
Suggestions:
Do you need to retrieve multiple records? If all you need is the first record found, then add "limit 1" to the end of the query.
If you're just checking for the existence of a row, i.e. you only need to know that there is one row with status active, then "select 1" instead of "select *".
Related
For considerably long period of time I’ve been struggling the following problem. This is an example of data stored in the DB:
> show series
flights,cycleId=1535,cycleIdx=0,engineId=2,flightId=1696,flightIdx=0,type=fil
flights,cycleId=1535,cycleIdx=0,engineId=2,flightId=1696,flightIdx=0,type=std
flights,cycleId=1535,cycleIdx=0,engineId=2,flightId=1696,flightIdx=0,type=raw
...
and my intention is to select a specific one by using a query like this:
SELECT * FROM flights WHERE type='fil' AND engineId= '2' AND flightId = '1696' AND flightIdx = '0' AND cycleId = '1535' AND cycleIdx = '0'
Such query, however, yields always zero results. Zilch.
Selecting the first (and only) tag works fine:
SELECT * FROM flights WHERE cycleId = '1535'
but using this condition on any other tag, like for example
SELECT * FROM flights WHERE type='fil'
does never return a single row. Querying only the first tag and nothing else works.
Could you please give me a hint what am I doing wrong? From all I have found people are always selecting just by a single tag but never more. What is the part that I cannot see?
Many thanks for any ideas!
I believe I have discovered the reason: two keys from the tags made by mistake their way into the fields. I spotted the trouble when listing the tag and fields keys as
show tag keys
show field keys
Deleting all records does not remove the keys from these lists and the problem persists. One need to drop the entire database to restore the order of things.
I have a following issue:
I need to calculate difference between consecutive points where some arbitrary ID is equal. The following:
SELECT difference(value_field) FROM mesurementName WHERE "IdField" = '10'
Works, returns difference between each consecutive point with IdField BUT IdField is lost (only time is propagated to query result). In my case time is not unique (i.e. measurement may contain many points with same timestamp, but different IdField). So I tried:
SELECT difference(value_field), IdField FROM mesurementName WHERE "IdField" = '10'
which yields:
error parsing query: mixing aggregate and non-aggregate queries is not supported!!
My next attempt was using sub-query:
SELECT IdField, diff
FROM (
SELECT
difference(flow_val) as diff
FROM
mesurementA
WHERE "IdField" = '10'
)
Which resulted in always null value in IdField.
I'd like to ask you for help or suggestion how to solve issue. By the way, we are using InfluxDB 1.3, which is not supporting JOIN anymore
If anyone would stuck as I was, then solution is following:
SELECT difference(value_field) FROM mesurementName GROUP BY "IdField"
Above somehow implicitly add "IdField" to result series and is propagated to resulting measurements with INTO clause
So far I have a query with a result set (in a temp table) with several columns but I am only concerned with four. One is a customer ID(varchar), one is Date (smalldatetime), one is Amount(money) and the last is Type(char). I have multiple rows with the same custmer ID and want to evaluate them based on Date, Amount and Type. For example:
Customer ID Date Amount Type
A 1-1-10 200 blue
A 1-1-10 400 green
A 1-2-10 400 green
B 1-11-10 100 blue
B 1-11-10 100 red
For all occurrences of A I want to compare them to identify only one, first by earliest date, then by greatest Amount, then if still tied by comparing Types. I would then return one row for each customer.
I would provide some of the query but I am at home now after spending two days trying to get a correct result. It looks something like this:
(query to populate #tempTable)
GROUP BY customer_id
HAVING date_cd =
(SELECT MIN(date_cd)
FROM order_table ot
WHERE ot.customerID = #tempTable.customerID
)
OR date_cd IS NULL
I assume the HAVING would result in only one row per customer_id. This did not end up being the case since there were some ties there.
I am not sure I can do the OR - there are some with NULL values here - and it did not account for the step to the next comparison if they were all the same anyway. I am not seeing a way to avoid doing some row processing of the temp table with some kind of IF or WHERE loop.
As I write I am thinking maybe I use #tempTable.date_cd in the HAVING clause instead of looking at the original table. but that should return the same dates?
Am I on the right track or is there something missing? Suggestions? More info??
try below query :-
select * from #tempTable
GROUP BY customer_id
HAVING isnull(date_cd,"1900/01/01") =min(isnull(date_cd,"1900/01/01"))
I wanna update my table for all persons whoes activity lasted toooo long. The update should correct one time and for the subsequent rows I need to deal with new result. So thought about something like
UPDATE summary_table st
SET st.screen_on=newScreenOnValue
st.active_screen_on=st.active_screen_on-(st.screen_on-newScreenOnValue) --old-value minus thedifference
FROM (
SUB-SELECT with rowid, newScreenOnValue ... JOIN ... WHERE....
) nv
WHERE (st.rowid=nv.rowid)
I know that I can update the first and the second value directly, by rerunning the same query. But my problem is the costs of the subselect seems quite high and therefore wanna avoid a double-update resp. double-run of the same query.
The above SELECT is just a informal way of writting what I think I would like to get. I know that the st doesn't work, but I left it here for better understanding. When I try the above statement I always get back a SyntaxError at the position the FROM ends.
This can be achieved as follows:
UPDATE summary_table st
SET (st.screen_on, st.active_screen_on) =
((SELECT newScreenOnValue, st.active_screen_on-(st.screen_on-newScreenOnValue)
FROM ...
JOIN...
WHERE..))
[WHERE if any additional condition required];
The above query works perfectly fine on informix tried and tested until you make any errors in the FROM, JOIN, WHERE clauses.
Cheers !
Syntax error because a comma is missing between the first and second columns you're updating.
Never use ROWID's, they're volatile and also not used by default with IDS, unless you specify so.
Why are you using a subquery?
Ruby 1.9.2 / rails 3.1 / deploy onto heroku --> posgresql
Hi, Once a number of rows relating to an object goes over a certain amount, I wish to pull back every nth row instead. It's simply because the rows are used (in part) to display data for graphing, so once the number of rows returned goes above say 20, it's good to return every second one, and so forth.
This question seemed to point in the right direction:
ActiveRecord Find - Skipping Records or Getting Every Nth Record
Doing a mod on row number makes sense, but using basically:
#widgetstats = self.widgetstats.find(:all,:conditions => 'MOD(ROW_NUMBER(),3) = 0 ')
doesn't work, it returns an error:
PGError: ERROR: window function call requires an OVER clause
And any attempt to solve that with e.g. basing my OVER clause syntax on things I see in the answer on this question:
Row numbering in PostgreSQL
ends in syntax errors and I can't get a result.
Am I missing a more obvious way of efficiently returning every nth task or if I'm on the right track any pointers on the way to go? Obviously returning all the data and fixing it in rails afterwards is possible, but terribly inefficient.
Thank you!
I think you are looking for a query like this one:
SELECT * FROM (SELECT widgetstats.*, row_number() OVER () AS rownum FROM widgetstats ORDER BY id) stats WHERE mod(rownum,3) = 0
This is difficult to build using ActiveRecord, so you might be forced to do something like:
#widgetstats = self.widgetstats.find_by_sql(
%{
SELECT * FROM
(
SELECT widgetstats.*, row_number() OVER () AS rownum FROM widgetstats ORDER BY id
) AS stats
WHERE mod(rownum,3) = 0
}
)
You'll obviously want to change the ordering used and add any WHERE clauses or other modifications to suit your needs.
Were I to solve this, I would either just write the SQL myself, like the SQL that you linked to. You can do this with
my_model.connection.execute('...')
or just get the id numbers and find by id
ids = (1..30).step(2)
my_model.where(id => ids)