I was testing the following behaviour for datetime special register (stated here)
If the SQL statement in which a datetime special register is used is
in a user-defined function or stored procedure that is within the
scope of a trigger, Db2 uses the timestamp for the triggering SQL
statement to determine the special register value.
So i crated a table with a timestamp field, a stored procedure (native sql) that is inserting the same 10 rows to the table and the tamestamp column is given the value of "current timestamp". Then i created a trigger on some other table (after insert trigger).
The result is 10 rows with increasing timestamp. I expected the timestamp to be the same as in my interpretation the stored procedure was in the scope of a trigger.
Can you help me what this statement means?
create trigger date_check
after insert on test
for each row mode db2sql
call date_sp2()
create procedure date_sp2()
language sql
BEGIN
declare i smallint default 0;
my_loop: LOOP
insert into empty_char values('Y','Y','Y','Y',current date, current timestamp);
SET I = I + 1;
IF I = 10 THEN LEAVE my_loop;
END IF;
END LOOP my_loop;
END
My guess is that note 1 applies
https://www.ibm.com/support/knowledgecenter/SSEPEK_11.0.0/sqlref/src/tpc/db2z_currenttimestamp.html#fntarg_1
If this special register is used more than one time within a single SQL statement, or used with CURRENT DATE or CURRENT TIME within a single statement, all values are based on a single clock reading. ¹
¹ Except for the case of a non-atomic multiple row INSERT or MERGE statement.
Related
I want to write a stored procedure in DB2 database for UPDATE and DELETE operations.
I am able to do that by directly providing values in procedure, but I want to do it by passing dynamic values.
My Table Structure is -
create table emp2 (int_1 int, char_1 char(10))
Below is my stored procedure for UPDATE operation which I am able to run but, its not behaving as per expectations. Changes are not reflecting in DB even after passing correct parameters while calling stored procedure.
CREATE OR REPLACE PROCEDURE "DB2INST1"."UPDATE_1" (IN int_1 int, IN
char_1 char(10)) SPECIFIC UPDATE_1
LANGUAGE SQL
DYNAMIC RESULT SETS 1
BEGIN
update emp2 set char_1=char_1 where int_1=int_1;
END;
This is the my stored procedure for DELETE operation which I am able to run successfully, but it's deleting all rows from the database table instead of deleting a single row:
CREATE OR REPLACE PROCEDURE "DB2INST1"."DELETE_1" (IN int_1 int)
SPECIFIC DELETE_1
LANGUAGE SQL
DYNAMIC RESULT SETS 1
BEGIN
delete from emp2 where int_1=int_1;
END;
Please provide me syntax for creating stored procedure for UPDATE and DELETE operations by passing dynamic values in a DB2 database.
The problem is that you don't qualify your columns and parameters having the same names.
According to the References to SQL parameters, SQL variables, and global variables:
Names that are the same should be explicitly qualified. Qualifying a
name clearly indicates whether the name refers to a column, SQL
variable, SQL parameter, row variable field, or global variable. If
the name is not qualified, or qualified but still ambiguous, the
following rules describe whether the name refers to a column, an SQL
variable, an SQL parameter, or a global variable:
If the tables and views specified in an SQL routine body exist at the time the routine is created, the name is first checked as a column
name. If not found as a column, it is then checked as an SQL variable
in the compound statement, then checked as an SQL parameter, and then,
finally, checked as a global variable.
That is, the following statements changing all the table rows obviously are equivalent:
update emp2 set char_1 = char_1 where int_1 = int_1;
--and
update emp2 e set e.char_1 = e.char_1 where e.int_1 = e.int_1;
What you need is to rewrite this statement to the following, if you want to use the same column and parameter names (the routine name UPDATE_1 is used here for the parameter qualification).
update emp2 set char_1 = UPDATE_1.char_1 where int_1 = UPDATE_1.int_1
fiddle
Consider an enterprise that captures sensor data for different production facilities. per facility, we create an aggregation query that averages the values to 5min timeslots. This query exists out of a long list of with-clauses and writes data to a table (called aggregation_table).
Now my problem: currently we have n queries running that exactly run the same logic, the only thing that differs are table names (and sometimes column names but let's ignore that for now).
Instead of managing n different scripts that are basically the same, I would like to put it in a stored procedure that is able to work like this:
CALL aggregation_query(facility_name) -> resolve the different tables for that facility and then use them in the different with clauses
On top of that, instead of having this long set of clauses that give me the end-result, I would like to chunk them up in logical blocks that are parametrizable, So for example, if I call the aforementioned stored_procedure for facility A, I want to be able to pass / use this table name in these different functions, where the output can be re-used in the next statement (like you would do with with clauses).
Another argument of why I want to chunk this up in re-usable blocks is because we have many "derivatives" on this aggregation query, for example to manage historical data, to correct data or to have the sensor data on another aggregation level. As these become overly complex, it is much easier to manage them without having to copy paste and adjust these every time.
In the current set-up, it could be useful to know that I am only entitled to use plain BigQuery, As my team is not allowed to access the CI/CD / scheduling and repository. (meaning that I cannot solve the issue by having CI/CD that deploys the n different versions of the procedure and functions)
So in the end, I would like to end up with something like this using only bigquery:
CREATE OR REPLACE PROCEDURE
`aggregation_function`()
BEGIN
DECLARE
tablename STRING;
DECLARE
active_table_name STRING; ##get list OF tables CREATE TEMP TABLE tableNames AS
SELECT
table_catalog,
table_schema,
table_name
FROM
`catalog.schema.INFORMATION_SCHEMA.TABLES`
WHERE
table_name = tablename;
WHILE
(
SELECT
COUNT(*)
FROM
tableNames) >= 1 DO ##build dataset + TABLE name
SET
active_table_name = CONCAT('`',table_catalog,'.',table_schema,'.' ,table_name,'`'); ##use concat TO build string AND execute
EXECUTE IMMEDIATE '''
INSERT INTO
`aggregation_table_for_facility` (timeslot, sensor_name, AVG_VALUE )
WITH
STEP_1 AS (
SELECT
*
FROM
my_table_function_step_1(active_table_name,
parameter1,
parameter2) ),
STEP_2 AS (
SELECT
*
FROM
my_table_function_step_2(STEP_1,
parameter1,
parameter2) )
SELECT * FROM STEP_2
'''
USING active_table_name as active_table_name;
DELETE
FROM
tableNames
WHERE
table_name = tablename;
END WHILE
;
END
;
I was hoping someone could make a snippet on how I can do this in Standard SQL / Bigquery, so basically:
stored procedure that takes in a string variable and is able to use that as a table (partly solved in the approach above, but not sure if there are better ways)
(table) function that is able to take this table_name parameter as well and return back a table that can be used in the next with clause (or alternatively writes to a temp table)
I think below code snippets should provide you with some insights when dealing with procedures, inserts and execute immediate statements.
Here I'm creating a procedure which will insert values into a table that exists on the information schema. Also, as a value I want to return I use OUT active_table_name to return the value I assigned inside the procedure.
CREATE OR REPLACE PROCEDURE `project-id.dataset`.custom_function(tablename STRING,OUT active_table_name STRING)
BEGIN
DECLARE query STRING;
SET active_table_name= (SELECT CONCAT('`',table_catalog,'.',table_schema,'.' ,table_name,'`')
FROM `project-id.dataset.INFORMATION_SCHEMA.TABLES`
WHERE table_name = tablename);
#multine query can be handled by using ''' or """
Set query =
'''
insert into %s (string_field_0,string_field_1,string_field_2,string_field_3,string_field_4,int64_field_5)
with custom_query as (
select string_field_0,string_field_2,'169 BestCity',string_field_3,string_field_4,55677 from %s limit 1
)
select * from custom_query;
''';
# querys must perform operations and must be the last thing to perform
# pass parameters using format
execute immediate (format(query,active_table_name,active_table_name));
END
You can also use a loop to iterate trough records from a working table so it will execute the procedure and also be able to get the value from the procedure to use somewhere else.ie:A second procedure to perform a delete operation.
DECLARE tablename STRING;
DECLARE out_value STRING;
FOR record IN
(SELECT tablename from `my-project-id.dataset.table`)
DO
SET tablename = record.tablename;
LOOP
call `project-id.dataset`.custom_function(tablename,out_value);
select out_value;
END LOOP;
END FOR;
To recap, there are some restrictions such as the possibility to call procedures inside a execute immediate or to use execute immediate inside an execute immediate, to count a few. I think these snippets should help you dealing with your current situation.
For this sample I use the following documentation:
Data Manipulation Language
Dealing with outputs
Information Schema Tables
Execute Immediate
For...In
Loops
I will try to keep the query as short as possible. This involves 2 tables - lets call them staging_data and audit_data. STAGING_DATA has 3 columns:
user_no with data type number,
update_date_time with data type as date in DD-MON-YYYY HH24:MI:SS format
status_code which is varchar(1).
audit_data table also has the same 3 columns. The ask is to add 3 columns to audit_data table
seq_no (which will be unique to every user),
active_from (date type without the time format)
active_to (date type without the time format).
There is a procedure that inserts data from staging_data to audit_data.
Sample of the table audit_data
That data in audit table should look like :
For the next record for user_no 523(lets assume update_date_time is '23-Nov-2020 10:20') seq_no becomes 3, active_from_date becomes '23-Nov-2020', active_to becomes 31-Dec-99 and the active_to of user_no 523 with seq_no 2 becomes '22-Nov-2020'. So the data should look like this :
Highlighted the 3rd record which will be added later in light green.
So here goes my solution : I suggested to use row_number() over(partition by user_no) analytical function to get seq_no for each user. I wanted to create a view based on that but Boss doesn't want a view. He strictly wants to use a procedure. Procedure should check if the user_no exists (in this example 523). If exists then seq_no increases and active_to of the previous record for 523 changes to latest active_from - 1 date. I will be honest - I have no clue how to achieve this in Procedure. I understand I can create a cursor with the query I had in my mind for the view. But to add seq_no and change active_to date is something that has puzzled me. Can anyone please guide me in right direction/s? Also I apologise in advance if I have left out any other details. Its midnight here now and after 8 hours of racking my brain on this I am very hungry!
edit 11th Mar : here is the code for the procedure I wrote to insert data into the audit table for situation when a particular user_no has no record in audit table :
create or replace procedure test_aud IS
user_found_audit number;
lv_user_no AUDIT_DATA.user_no%TYPE;
cursor member_no is select distinct user_no from STAGING_DATA;
begin
open member_no;
loop
fetch member_no into lv_user_no;
exit when member_no%notfound;
select count(*) into user_found_audit from AUDIT_DATA where user_no = lv_user_no;
if user_found_audit = 0 then
insert into AUDIT_DATA(user_no, update_date_time,status_code, seq_no, last_update_date, active_from, active_to)
select user_no, update_date_time,status_code,row_number() over(partition by user_no order by UPDATE_DATE_TIME) as seqno,
to_char(trunc(update_date_time),'DD-MON-YYYY'),
to_char(trunc(update_date_time),'DD-MON-YYYY'),
lead(to_char(trunc(update_date_time)-1,'DD-MON-YYYY'),1,'31-DEC-99') over(PARTITION BY user_no ORDER BY UPDATE_DATE_TIME) from STAGING_DATA where user_no = lv_user_no;
commit;
else
dbms_output.put_line(lv_user_no||' exists in audit table');
-- to code the block when user_no exists, involves an update and insert
end if;
end loop;
close member_no;
end;
/
Well you need to collect a couple things. The latest stage row and the latest audit row. Then it is just a matter of generating the new audit information and updating the previous latest one. The following makes a couple simplifying assumptions:
Only the latest stage data for a given user_no needs processed as
all prior have been processed, However it does not assume the stage
table has been cleared.
The sequencing of 'Y' and 'N' status_codes are properly order in
that manner. In fact it does not even check the value.
It need not concern itself with the inherent race condition. The
condition is derives from seq_no being generated as Max()+1. This
structure virtually guarantees a duplicate will eventually be
created.
The nested procedure "establish_audit" does all the actual work. The rest are just supporting characters, including a couple just for debug purpose. See fiddle.
create or replace
procedure generate_stage_audit(user_no_in staging_data.user_no%type)
as
k_end_of_time constant date := date '9999-12-31';
l_latest_user_stage staging_data%rowtype;
l_latest_user_audit audit_data%rowtype;
procedure establish_audit
is
begin
insert into audit_data(user_no, update_date_time, status_code
,seq_no, active_from, active_to)
select l_latest_user_stage.user_no
, l_latest_user_stage.update_date_time
, l_latest_user_stage.status_code
, coalesce(l_latest_user_audit.seq_no,0) + 1
, trunc(l_latest_user_stage.update_date_time)
, k_end_of_time
from dual;
update audit_data
set active_to = trunc(l_latest_user_stage.update_date_time - 1)
where user_no = l_latest_user_audit.user_no
and seq_no = l_latest_user_audit.seq_no;
end establish_audit;
procedure retrieve_latest_stage
is
begin
select *
into l_latest_user_stage
from staging_data
where (user_no, update_date_time) =
( select user_no, max(update_date_time)
from staging_data
where user_no = user_no_in
group by user_no
);
end retrieve_latest_stage;
procedure retrieve_latest_audit
is
begin
select *
into l_latest_user_audit
from audit_data
where (user_no, seq_no) =
( select user_no, max(seq_no)
from audit_data
where user_no = user_no_in
group by user_no
);
exception
when no_data_found then
null;
end retrieve_latest_audit;
---- for debugging ---
procedure show_stage
is
begin
dbms_output.put_line('-------- Stage Row -------');
dbms_output.put_line(' user_no==>' || to_char(l_latest_user_stage.user_no));
dbms_output.put_line('update_date_time==>' || to_char(l_latest_user_stage.update_date_time));
dbms_output.put_line(' status_code==>' || to_char(l_latest_user_stage.status_code));
end show_stage;
procedure show_audit
is
begin
dbms_output.put_line('-------- Audit Row -------');
dbms_output.put_line(' user_no==>' || to_char(l_latest_user_audit.user_no));
dbms_output.put_line('update_date_time==>' || to_char(l_latest_user_audit.update_date_time));
dbms_output.put_line(' status_code==>' || to_char(l_latest_user_audit.status_code));
dbms_output.put_line(' seq_no==>' || to_char(l_latest_user_audit.seq_no));
dbms_output.put_line(' active_from==>' || to_char(l_latest_user_audit.active_from));
dbms_output.put_line(' active_to==>' || to_char(l_latest_user_audit.active_to));
end show_audit;
begin -- the main event
retrieve_latest_stage;
show_stage;
retrieve_latest_audit;
show_audit;
establish_audit;
end generate_stage_audit;
A couple warnings:
It seems you may be tempted to use string data type for the audit
columns Active_Form and Active_to as you are trying to declare then
"date type without the time". However there is no such data type in
Oracle; time is part of all dates. Do not do so, store them as
standard dates. (Note Dates are not stored in any format, but an
internal structure. Formats are strictly a visual representation).
Just throwaway the time with the format on the query or by setting
nls_date_format.
You may be tempted to convert call this through a trigger. Do not,
it will likely result in an "ORA-04091: Table is mutating"
exception.
I'm trying to obtain 2 different resultset from stored procedure, based on a single query. What I'm trying to do is that:
1.) return query result into OUT cursor;
2.) from this cursor results, get all longest values in each column and return that as second OUT
resultset.
I'm trying to avoid doing same thing twice with this - get data and after that get longest column values of that same data. I'm not sure If this is even possible, but If It is, can somebody show me HOW ?
This is an example of what I want to do (just for illustration):
CREATE OR REPLACE PROCEDURE MySchema.Test(RESULT OUT SYS_REFCURSOR,MAX_RESULT OUT SYS_REFCURSOR)
AS
BEGIN
OPEN RESULT FOR SELECT Name,Surname FROM MyTable;
OPEN MAX_RESULT FOR SELECT Max(length(Name)),Max(length(Surname)) FROM RESULT; --error here
END Test;
This example compiles with "ORA-00942: table or view does not exist".
I know It's a silly example, but I've been investigating and testing all sorts of things (implicit cursors, fetching cursors, nested cursors, etc.) and found nothing that would help me, specially when working with stored procedure returning multiple resultsets.
My overall goal with this is to shorten data export time for Excel. Currently I have to run same query twice - once for calculating data size to autofit Excel columns, and then for writing data into Excel.
I believe that manipulating first resultset in order to get second one would be much faster - with less DB cycles made.
I'm using Oracle 11g, Any help much appreciated.
Each row of data from a cursor can be read exactly once; once the next row (or set of rows) is read from the cursor then the previous row (or set of rows) cannot be returned to and the cursor cannot be re-used. So what you are asking is impossible as if you read the cursor to find the maximum values (ignoring that you can't use a cursor as a source in a SELECT statement but, instead, you could read it using a PL/SQL loop) then the cursor's rows would have been "used up" and the cursor closed so it could not be read from when it is returned from the procedure.
You would need to use two separate queries:
CREATE PROCEDURE MySchema.Test(
RESULT OUT SYS_REFCURSOR,
MAX_RESULT OUT SYS_REFCURSOR
)
AS
BEGIN
OPEN RESULT FOR
SELECT Name,
Surname
FROM MyTable;
OPEN MAX_RESULT FOR
SELECT MAX(LENGTH(Name)) AS max_name_length,
MAX(LENGTH(Surname)) AS max_surname_length
FROM MyTable;
END Test;
/
Just for theoretical purposes, it is possible to only read from the table once if you bulk collect the data into a collection then select from a table-collection expression (however, it is going to be more complicated to code/maintain and is going to require that the rows from the table are stored in memory [which your DBA might not appreciate if the table is large] and may not be more performant than compared to just querying the table twice as you'll end up with three SELECT statements instead of two).
Something like:
CREATE TYPE test_obj IS OBJECT(
name VARCHAR2(50),
surname VARCHAR2(50)
);
CREATE TYPE test_obj_table IS TABLE OF test_obj;
CREATE PROCEDURE MySchema.Test(
RESULT OUT SYS_REFCURSOR,
MAX_RESULT OUT SYS_REFCURSOR
)
AS
t_names test_obj_table;
BEGIN
SELECT Name,
Surname
BULK COLLECT INTO t_names
FROM MyTable;
OPEN RESULT FOR
SELECT * FROM TABLE( t_names );
OPEN MAX_RESULT FOR
SELECT MAX(LENGTH(Name)) AS max_name_length,
MAX(LENGTH(Surname)) AS max_surname_length
FROM TABLE( t_names );
END Test;
/
I'm moving from MySql to Postgres, and I noticed that when you delete rows from MySql, the unique ids for those rows are re-used when you make new ones. With Postgres, if you create rows, and delete them, the unique ids are not used again.
Is there a reason for this behaviour in Postgres? Can I make it act more like MySql in this case?
Sequences have gaps to permit concurrent inserts. Attempting to avoid gaps or to re-use deleted IDs creates horrible performance problems. See the PostgreSQL wiki FAQ.
PostgreSQL SEQUENCEs are used to allocate IDs. These only ever increase, and they're exempt from the usual transaction rollback rules to permit multiple transactions to grab new IDs at the same time. This means that if a transaction rolls back, those IDs are "thrown away"; there's no list of "free" IDs kept, just the current ID counter. Sequences are also usually incremented if the database shuts down uncleanly.
Synthetic keys (IDs) are meaningless anyway. Their order is not significant, their only property of significance is uniqueness. You can't meaningfully measure how "far apart" two IDs are, nor can you meaningfully say if one is greater or less than another. All you can do is say "equal" or "not equal". Anything else is unsafe. You shouldn't care about gaps.
If you need a gapless sequence that re-uses deleted IDs, you can have one, you just have to give up a huge amount of performance for it - in particular, you cannot have any concurrency on INSERTs at all, because you have to scan the table for the lowest free ID, locking the table for write so no other transaction can claim the same ID. Try searching for "postgresql gapless sequence".
The simplest approach is to use a counter table and a function that gets the next ID. Here's a generalized version that uses a counter table to generate consecutive gapless IDs; it doesn't re-use IDs, though.
CREATE TABLE thetable_id_counter ( last_id integer not null );
INSERT INTO thetable_id_counter VALUES (0);
CREATE OR REPLACE FUNCTION get_next_id(countertable regclass, countercolumn text) RETURNS integer AS $$
DECLARE
next_value integer;
BEGIN
EXECUTE format('UPDATE %s SET %I = %I + 1 RETURNING %I', countertable, countercolumn, countercolumn, countercolumn) INTO next_value;
RETURN next_value;
END;
$$ LANGUAGE plpgsql;
COMMENT ON get_next_id(countername regclass) IS 'Increment and return value from integer column $2 in table $1';
Usage:
INSERT INTO dummy(id, blah)
VALUES ( get_next_id('thetable_id_counter','last_id'), 42 );
Note that when one open transaction has obtained an ID, all other transactions that try to call get_next_id will block until the 1st transaction commits or rolls back. This is unavoidable and for gapless IDs and is by design.
If you want to store multiple counters for different purposes in a table, just add a parameter to the above function, add a column to the counter table, and add a WHERE clause to the UPDATE that matches the parameter to the added column. That way you can have multiple independently-locked counter rows. Do not just add extra columns for new counters.
This function does not re-use deleted IDs, it just avoids introducing gaps.
To re-use IDs I advise ... not re-using IDs.
If you really must, you can do so by adding an ON INSERT OR UPDATE OR DELETE trigger on the table of interest that adds deleted IDs to a free-list side table, and removes them from the free-list table when they're INSERTed. Treat an UPDATE as a DELETE followed by an INSERT. Now modify the ID generation function above so that it does a SELECT free_id INTO next_value FROM free_ids FOR UPDATE LIMIT 1 and if found, DELETEs that row. IF NOT FOUND gets a new ID from the generator table as normal. Here's an untested extension of the prior function to support re-use:
CREATE OR REPLACE FUNCTION get_next_id_reuse(countertable regclass, countercolumn text, freelisttable regclass, freelistcolumn text) RETURNS integer AS $$
DECLARE
next_value integer;
BEGIN
EXECUTE format('SELECT %I FROM %s FOR UPDATE LIMIT 1', freelistcolumn, freelisttable) INTO next_value;
IF next_value IS NOT NULL THEN
EXECUTE format('DELETE FROM %s WHERE %I = %L', freelisttable, freelistcolumn, next_value);
ELSE
EXECUTE format('UPDATE %s SET %I = %I + 1 RETURNING %I', countertable, countercolumn, countercolumn, countercolumn) INTO next_value;
END IF;
RETURN next_value;
END;
$$ LANGUAGE plpgsql;