I have a simple SQL query in PostgreSQL 8.3 that grabs a bunch of comments. I provide a sorted list of values to the IN construct in the WHERE clause:
SELECT * FROM comments WHERE (comments.id IN (1,3,2,4));
This returns comments in an arbitrary order which in my happens to be ids like 1,2,3,4.
I want the resulting rows sorted like the list in the IN construct: (1,3,2,4).
How to achieve that?
You can do it quite easily with (introduced in PostgreSQL 8.2) VALUES (), ().
Syntax will be like this:
select c.*
from comments c
join (
values
(1,1),
(3,2),
(2,3),
(4,4)
) as x (id, ordering) on c.id = x.id
order by x.ordering
In Postgres 9.4 or later, this is simplest and fastest:
SELECT c.*
FROM comments c
JOIN unnest('{1,3,2,4}'::int[]) WITH ORDINALITY t(id, ord) USING (id)
ORDER BY t.ord;
WITH ORDINALITY was introduced with in Postgres 9.4.
No need for a subquery, we can use the set-returning function like a table directly. (A.k.a. "table-function".)
A string literal to hand in the array instead of an ARRAY constructor may be easier to implement with some clients.
For convenience (optionally), copy the column name we are joining to ("id" in the example), so we can join with a short USING clause to only get a single instance of the join column in the result.
Works with any input type. If your key column is of type text, provide something like '{foo,bar,baz}'::text[].
Detailed explanation:
PostgreSQL unnest() with element number
Just because it is so difficult to find and it has to be spread: in mySQL this can be done much simpler, but I don't know if it works in other SQL.
SELECT * FROM `comments`
WHERE `comments`.`id` IN ('12','5','3','17')
ORDER BY FIELD(`comments`.`id`,'12','5','3','17')
With Postgres 9.4 this can be done a bit shorter:
select c.*
from comments c
join (
select *
from unnest(array[43,47,42]) with ordinality
) as x (id, ordering) on c.id = x.id
order by x.ordering;
Or a bit more compact without a derived table:
select c.*
from comments c
join unnest(array[43,47,42]) with ordinality as x (id, ordering)
on c.id = x.id
order by x.ordering
Removing the need to manually assign/maintain a position to each value.
With Postgres 9.6 this can be done using array_position():
with x (id_list) as (
values (array[42,48,43])
)
select c.*
from comments c, x
where id = any (x.id_list)
order by array_position(x.id_list, c.id);
The CTE is used so that the list of values only needs to be specified once. If that is not important this can also be written as:
select c.*
from comments c
where id in (42,48,43)
order by array_position(array[42,48,43], c.id);
I think this way is better :
SELECT * FROM "comments" WHERE ("comments"."id" IN (1,3,2,4))
ORDER BY id=1 DESC, id=3 DESC, id=2 DESC, id=4 DESC
Another way to do it in Postgres would be to use the idx function.
SELECT *
FROM comments
ORDER BY idx(array[1,3,2,4], comments.id)
Don't forget to create the idx function first, as described here: http://wiki.postgresql.org/wiki/Array_Index
In Postgresql:
select *
from comments
where id in (1,3,2,4)
order by position(id::text in '1,3,2,4')
On researching this some more I found this solution:
SELECT * FROM "comments" WHERE ("comments"."id" IN (1,3,2,4))
ORDER BY CASE "comments"."id"
WHEN 1 THEN 1
WHEN 3 THEN 2
WHEN 2 THEN 3
WHEN 4 THEN 4
END
However this seems rather verbose and might have performance issues with large datasets.
Can anyone comment on these issues?
To do this, I think you should probably have an additional "ORDER" table which defines the mapping of IDs to order (effectively doing what your response to your own question said), which you can then use as an additional column on your select which you can then sort on.
In that way, you explicitly describe the ordering you desire in the database, where it should be.
sans SEQUENCE, works only on 8.4:
select * from comments c
join
(
select id, row_number() over() as id_sorter
from (select unnest(ARRAY[1,3,2,4]) as id) as y
) x on x.id = c.id
order by x.id_sorter
SELECT * FROM "comments" JOIN (
SELECT 1 as "id",1 as "order" UNION ALL
SELECT 3,2 UNION ALL SELECT 2,3 UNION ALL SELECT 4,4
) j ON "comments"."id" = j."id" ORDER BY j.ORDER
or if you prefer evil over good:
SELECT * FROM "comments" WHERE ("comments"."id" IN (1,3,2,4))
ORDER BY POSITION(','+"comments"."id"+',' IN ',1,3,2,4,')
And here's another solution that works and uses a constant table (http://www.postgresql.org/docs/8.3/interactive/sql-values.html):
SELECT * FROM comments AS c,
(VALUES (1,1),(3,2),(2,3),(4,4) ) AS t (ord_id,ord)
WHERE (c.id IN (1,3,2,4)) AND (c.id = t.ord_id)
ORDER BY ord
But again I'm not sure that this is performant.
I've got a bunch of answers now. Can I get some voting and comments so I know which is the winner!
Thanks All :-)
create sequence serial start 1;
select * from comments c
join (select unnest(ARRAY[1,3,2,4]) as id, nextval('serial') as id_sorter) x
on x.id = c.id
order by x.id_sorter;
drop sequence serial;
[EDIT]
unnest is not yet built-in in 8.3, but you can create one yourself(the beauty of any*):
create function unnest(anyarray) returns setof anyelement
language sql as
$$
select $1[i] from generate_series(array_lower($1,1),array_upper($1,1)) i;
$$;
that function can work in any type:
select unnest(array['John','Paul','George','Ringo']) as beatle
select unnest(array[1,3,2,4]) as id
Slight improvement over the version that uses a sequence I think:
CREATE OR REPLACE FUNCTION in_sort(anyarray, out id anyelement, out ordinal int)
LANGUAGE SQL AS
$$
SELECT $1[i], i FROM generate_series(array_lower($1,1),array_upper($1,1)) i;
$$;
SELECT
*
FROM
comments c
INNER JOIN (SELECT * FROM in_sort(ARRAY[1,3,2,4])) AS in_sort
USING (id)
ORDER BY in_sort.ordinal;
select * from comments where comments.id in
(select unnest(ids) from bbs where id=19795)
order by array_position((select ids from bbs where id=19795),comments.id)
here, [bbs] is the main table that has a field called ids,
and, ids is the array that store the comments.id .
passed in postgresql 9.6
Lets get a visual impression about what was already said. For example you have a table with some tasks:
SELECT a.id,a.status,a.description FROM minicloud_tasks as a ORDER BY random();
id | status | description
----+------------+------------------
4 | processing | work on postgres
6 | deleted | need some rest
3 | pending | garden party
5 | completed | work on html
And you want to order the list of tasks by its status.
The status is a list of string values:
(processing, pending, completed, deleted)
The trick is to give each status value an interger and order the list numerical:
SELECT a.id,a.status,a.description FROM minicloud_tasks AS a
JOIN (
VALUES ('processing', 1), ('pending', 2), ('completed', 3), ('deleted', 4)
) AS b (status, id) ON (a.status = b.status)
ORDER BY b.id ASC;
Which leads to:
id | status | description
----+------------+------------------
4 | processing | work on postgres
3 | pending | garden party
5 | completed | work on html
6 | deleted | need some rest
Credit #user80168
I agree with all other posters that say "don't do that" or "SQL isn't good at that". If you want to sort by some facet of comments then add another integer column to one of your tables to hold your sort criteria and sort by that value. eg "ORDER BY comments.sort DESC " If you want to sort these in a different order every time then... SQL won't be for you in this case.
Trying to fix my Informix query results format from a one to many relationship. My current query is using a JOIN but is creating a new line for every time there is a match to the JOIN ON condition. I should add the below is only an example, the real data is thousands of entries with about a 100 unique "category" entries so I cant hard code WHERE statements, it needs to read each entry and add if a match. I tried a GROUP_CONCAT however is just returned an error, guess its not a informix function, I also tried reading this thread but have yet been unable to get working. Show a one to many relationship as 2 columns - 1 unique row (ID & comma separated list)
Any help will be appreciated.
IBM/Informix-Connect Version 3.70.UC4
IBM/Informix LIBGLS LIBRARY Version 5.00.UC5
IBM Informix Dynamic Server Version 11.70.FC8W1
Tables
movie
name rating movie_id
rio g 1
horton g 2
blade r 3
lotr_1 pg13 4
lotr_2 pg13 5
paul_blart pg 6
category
cat_name id
kids 1
comedy 2
action 3
fantasy 4
category_member
movie_name cat_name catmem_id
lotr_1 action 1
lotr_1 fantasy 2
rio kids 3
rio comedy 4
When I use
#!/bin/bash
echo "SET isolation dirty read;
UNLOAD to /export/home/movie/movieDetail.unl DELIMITER ','
SELECT a.name, a.rating, b.cat_name
FROM movie a
LEFT JOIN category b ON b.movie_name = a.name
;" | dbaccess thedb;
What I get is
rio,g,kids
rio,g,comedy
lotr_1,pg13,action
lotr_1,pg13,fantasy
What I would like is
rio,g,kids,comedy
lotr_1,pg13,action,fantasy
Install the GROUP_CONCAT user-defined aggregate
You must install the GROUP_CONCAT user-defined aggregate from SO 715350 (referenced in your question) into your database. The GROUP_CONCAT aggregate is not defined by Informix, but can be added if you use the SQL from that question. One difference between that and a normal built-in function is that you need to install the aggregate in each database in the server where you need to use it. There might be a way to do a 'global install' (for all databases in a given server), but I've forgotten (or, more accurately, never learned) how to do it.
Writing your queries
With the sample database listed at the bottom:
The query in the question does not run:
SELECT a.name, a.rating, b.cat_name
FROM movie a
LEFT JOIN category b ON b.movie_name = a.name;
SQL -217: Column (movie_name) not found in any table in the query (or SLV is undefined).
This can be fixed by changing category to category_member. This produces:
SELECT a.name, a.rating, b.cat_name
FROM movie a
LEFT JOIN category_member b ON b.movie_name = a.name;
rio g kids
rio g comedy
horton g
blade r
lotr_1 pg13 action
lotr_1 pg13 fantasy
lotr_2 pg13
paul_blart pg
The LEFT JOIN appears to be unwanted. And using GROUP_CONCAT produces approximately the desired answer:
SELECT a.name, a.rating, GROUP_CONCAT(b.cat_name)
FROM movie a
JOIN category_member b ON b.movie_name = a.name
GROUP BY a.name, a.rating;
rio g kids,comedy
lotr_1 pg13 action,fantasy
If you specify the delimiter as ,, the commas in the data from the GROUP_CONCAT operator will be escaped to avoid ambiguity:
SELECT a.NAME, a.rating, GROUP_CONCAT(b.cat_name)
FROM movie a
JOIN category_member b ON b.movie_name = a.NAME
GROUP BY a.NAME, a.rating;
rio,g,kids\,comedy
lotr_1,pg13,action\,fantasy
Within standard Informix utilities, there isn't a way to avoid that; they don't leave the selected/unloaded data in an ambiguous format.
I'm not convinced that the database schema is very well organized. The Movie table is OK; the Category table is OK; but the Category_Member table would be more orthodox if it used the schema:
DROP TABLE IF EXISTS category_member;
CREATE TABLE category_member
(
movie_id INTEGER NOT NULL REFERENCES Movie(Movie_id),
category_id INTEGER NOT NULL REFERENCES Category(Id),
PRIMARY KEY(movie_id, category_id)
);
INSERT INTO category_member VALUES(4, 3);
INSERT INTO category_member VALUES(4, 4);
INSERT INTO category_member VALUES(1, 1);
INSERT INTO category_member VALUES(1, 2);
-- Use GROUP_CONCAT
SELECT a.NAME, a.rating, GROUP_CONCAT(c.cat_name)
FROM movie a
JOIN category_member b ON b.movie_id = a.movie_id
JOIN category c ON b.category_id = c.id
GROUP BY a.NAME, a.rating;
The output from this query is the same as from the previous one, but the joining is more orthodox.
Sample database
DROP TABLE IF EXISTS movie;
CREATE TABLE movie
(
name VARCHAR(20) NOT NULL UNIQUE,
rating CHAR(4) NOT NULL,
movie_id SERIAL NOT NULL PRIMARY KEY
);
INSERT INTO movie VALUES("rio", "g", 1);
INSERT INTO movie VALUES("horton", "g", 2);
INSERT INTO movie VALUES("blade", "r", 3);
INSERT INTO movie VALUES("lotr_1", "pg13", 4);
INSERT INTO movie VALUES("lotr_2", "pg13", 5);
INSERT INTO movie VALUES("paul_blart", "pg", 6);
DROP TABLE IF EXISTS category;
CREATE TABLE category
(
cat_name VARCHAR(10) NOT NULL UNIQUE,
id SERIAL NOT NULL PRIMARY KEY
);
INSERT INTO category VALUES("kids", 1);
INSERT INTO category VALUES("comedy", 2);
INSERT INTO category VALUES("action", 3);
INSERT INTO category VALUES("fantasy", 4);
DROP TABLE IF EXISTS category_member;
CREATE TABLE category_member
(
movie_name VARCHAR(20) NOT NULL,
cat_name VARCHAR(10) NOT NULL,
catmem_id SERIAL NOT NULL PRIMARY KEY
);
INSERT INTO category_member VALUES("lotr_1", "action", 1);
INSERT INTO category_member VALUES("lotr_1", "fantasy", 2);
INSERT INTO category_member VALUES("rio", "kids", 3);
INSERT INTO category_member VALUES("rio", "comedy", 4);
I have a mysql DB with rails, and a column "shorthand" (string) that I'd like to make unique across multiple tables. Is there a way I can do this without making a third table?
Expression
id
shorthand
...
etc
Variable
id
shorthand
...
etc
I want the values in the 'shorthand' columns of both tables to be unique between each other ie. a record shorthand value "xyz" in Expression would be rejected if a Variable with shorthand value "xyz" were to exist in the DB already.
Any thoughts appreciated, even "you have to use a third table" :)
Here an example using a third table:
-- TEMP SCHEMA for testing
DROP SCHEMA tmp CASCADE;
CREATE SCHEMA tmp ;
SET search_path=tmp;
CREATE TABLE shorthand
( shorthand varchar NOT NULL PRIMARY KEY
, one_or_two varchar NOT NULL
);
CREATE TABLE table_one
( one_id INTEGER NOT NULL PRIMARY KEY
, shorthand varchar NOT NULL REFERENCES shorthand(shorthand)
ON DELETE CASCADE ON UPDATE CASCADE
DEFERRABLE INITIALLY DEFERRED
, etc_one varchar
);
CREATE TABLE table_two
( two_id INTEGER NOT NULL PRIMARY KEY
, shorthand varchar NOT NULL REFERENCES shorthand(shorthand)
ON DELETE CASCADE ON UPDATE CASCADE
DEFERRABLE INITIALLY DEFERRED
, etc_two varchar
);
-- Trigger function for BOTH tables
CREATE FUNCTION set_one_or_two( ) RETURNS TRIGGER
AS $func$
BEGIN
IF (TG_OP = 'INSERT') THEN
INSERT INTO shorthand (shorthand, one_or_two)
VALUES(new.shorthand, TG_TABLE_NAME)
;
ELSEIF (TG_OP = 'UPDATE') THEN
UPDATE shorthand SET shorthand = new.shorthand
WHERE shorthand = old.shorthand
;
ELSEIF (TG_OP = 'DELETE') THEN
DELETE FROM shorthand
WHERE shorthand = old.shorthand
;
END IF;
RETURN NULL;
END
$func$ LANGUAGE plpgsql
;
-- Triggers for I/U/D
CREATE CONSTRAINT TRIGGER check_one
AFTER INSERT OR UPDATE OR DELETE
ON table_one
FOR EACH ROW
EXECUTE PROCEDURE set_one_or_two ( )
;
CREATE CONSTRAINT TRIGGER check_two
AFTER INSERT OR UPDATE OR DELETE
ON table_two
FOR EACH ROW
EXECUTE PROCEDURE set_one_or_two ( )
;
-- Some tests (incomplete)
INSERT INTO table_one (one_id,shorthand,etc_one) VALUES (1, 'one' , 'one' );
INSERT INTO table_two (two_id,shorthand,etc_two) VALUES (1, 'two' , 'two' );
SELECT * FROM shorthand;
\echo this should fail
INSERT INTO table_one (one_id,shorthand,etc_one) VALUES (11, 'two' , 'eleven' );
SELECT * FROM shorthand;
UPDATE table_one SET shorthand = 'eleven' WHERE one_id = 1;
SELECT * FROM shorthand;
I think this older article does exactly what you are looking for (simulating multi table constraints):
http://classes.soe.ucsc.edu/cmps180/Winter04/constraints.html
You might also like to investigate postgres CREATE CONSTRAINT TRIGGER using a function similar to the check_nojoin() function in the article.
http://www.postgresql.org/docs/9.0/static/sql-createconstraint.html
Once you have the exact SQL you need you can put it in your rails migration with execute "the required SQL"
An alternative approach is to use a third table 'shorthands' with columns 'shorthand' and 'src'. Define shorthand as the unique primary key on that table. On each of your other two tables define 'src' as a single char field defaulting to 'A' and 'B' on each table respecitively. Add a foreign key constraint on each of your two tables consisting of both 'shorthand' and 'src' and referencing table 'shorthands'. When inserting or updating rows in either of your two tables you need to ensure the 'shorthands' table is updated either explicity as part of your transaction or via a trigger and set both 'shorthand', and 'src' to the respective table ie 'A' or 'B'.
What the foreign key constraints do is ensure that the shorthand value exists in the shorthand's table for the respective src table but because of the uniqueness constraint on just the 'shorthand' column in the shorthand's table if the other table has already defined the shorthand value a key violation will occur thus guaranteeing uniqueness across two (or even more) tables.
Whatever you do, it is best to put the referential integrity into the database, not in orm/active record validations.
I have a table called messages and here is the table structure, I don’t want id is auto increment field but it should be a primary key for that table.
Here is table structure for messages
CREATE TABLE `messages` (
`id` INT(11) NOT NULL,
`user_id` INT(11) NOT NULL,
`text` VARCHAR(255) NOT NULL,
`source` VARCHAR(100),
`created_at` DATETIME DEFAULT NULL,
`updated_at` DATETIME DEFAULT NULL,
PRIMARY KEY (`id`)
);
while insert the data into table I am using below hash object
msg['id'] = 12345;
msg['user_id'] = 1;
msg['text'] = 'Hello world';
If I save this hash into messages table, id is not inserting
message = Message.new(msg);
message.save!
Rails is building insert sql with out id, so id value is not inserting messages table.
How insert the id value in table, This the insert sql rails build with out using id field
INSERT INTO `users` (`updated_at`, `user_id `, `text`, `created_at`) VALUES('2010-06-18 12:01:05', '1', 'Hello world', '2010-06-18 12:01:05');
Setting ID value is often useful when migrating from legacy data or - as I am doing right now - merging two apps while preserving FK integrity.
I just scratched my head for a while and it seems you have to set the PK value before calling save. After the record is saved, ActiveRecord ignores #id= or update_attribute . So while setting up the record from an attribute hash I use:
article = Article.new(attrs)
article.id = attrs["id"]
article.save!
You're working against the way rails works. ActiveRecord reserves the use of the id column and manages it for you.
Why should id not be an auto-incrementing column if it's the primary key?
Why do you need to control its value?
If you need an id column you can control yourself, add another one. It won't be the primary key, but you can make it a unique index too.