An example of a Cursor & UPDATE statement in conjunction inside a procedure on Redshift - stored-procedures

Would you please provide an an example for a Redshift procedure where you have used a cursor and an UPDATE statement in conjunction? Is that even feasible, I couldn't find an example. I'm looking for a simple template code to learn how to have these 2 together in a single procedure on Redshift.
Here is an example use case:
I have a table like this:
CREATE TABLE test_tbl
(
Contactid VARCHAR(500),
sfdc_OppId_01 VARCHAR(500),
sfdc_OppId_02 VARCHAR(500),
sfdc_OppId_03 VARCHAR(500),
sfdc_OppId_04 VARCHAR(500),
sfdc_OppId_05 VARCHAR(500),
sfdc_OppId_06 VARCHAR(500)
)
I want to update each sfdc_OppId_xx with the relative value from another table; sfdc_tbl. Here is what sfdc_tbl looks like:
sfdc_contactId
sfdc_Opp_Id
AA123hgt
999999
AA123hgt
888888
AA123hgt
777777
AA123hgt
432567
AA123hgt
098765
AA123hgt
112789
So as you see, there are duplicate sfdc_contactid in the sfdc_tbl. My final goal is to list all the sfdc_Opp_Id for given contactid horizontally in the test_tbl. I shall not have duplicate contactid in the test_tbl.
INSERT INTO test_tbl (Contactid)
SELECT sfdc_contactId
FROM sfdc_tbl
GROUP BY sfdc_contactId
And here is what I'm trying to do:
CREATE OR REPLACE PROCEDURE testing_procedure (results INOUT refcursor)
AS
$$
BEGIN
OPEN cursor_testing FOR
SELECT
Ops.sfdc_Opp.id,
ROW_NUMBER () OVER(PARTITION BY Ops.sfdc_contactId ORDER BY sfdc_Opp_Id ) RWN
FROM sfdc_tbl Ops
INNER JOIN test_tbl tbl
ON Ops.sfdc_contactId = tbl.contactid;
UPDATE test_tbl
SET sfdc_Opp_01 = CASE WHEN cursor_testing.RWN = 1 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_02 = CASE WHEN cursor_testing.RWN = 2 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_03 = CASE WHEN cursor_testing.RWN = 3 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_04 = CASE WHEN cursor_testing.RWN = 4 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_05 = CASE WHEN cursor_testing.RWN = 5 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_06 = CASE WHEN cursor_testing.RWN = 6 THEN cursor_testing.sfdc_Ops_id ELSE NULL END
;
END;
$$
LANGUAGE plpgsql;
I keep getting an error
incorrect syntax at or near "cursor_testing"

I've answered a question with a similar solution. The SQL uses a cursor's data to INSERT into a table and this same path should work for UPDATE - How to join System tables or Information Schema tables with User defined tables in Redshift
That being said and looking at your code I really think you would be better off using a temp table rather than a cursor. The first thing to note is that a cursor is not a table. Your use pattern won't work AFAIK. You read a cursor row by row (or bunches) which is contrary to Redshift's columnar table storage. So you will need to loop on the rows from the cursor and perform N updates. This will be extremely slow! You would be querying columnar data, storing the results in a cursor as rows, reading these row one by one, and then performing a new query (UPDATE) for each row. Ick! Stay in "columnar land" and use a temp table.

Related

create columns in existing table and populate data using procedure

I will try to keep the query as short as possible. This involves 2 tables - lets call them staging_data and audit_data. STAGING_DATA has 3 columns:
user_no with data type number,
update_date_time with data type as date in DD-MON-YYYY HH24:MI:SS format
status_code which is varchar(1).
audit_data table also has the same 3 columns. The ask is to add 3 columns to audit_data table
seq_no (which will be unique to every user),
active_from (date type without the time format)
active_to (date type without the time format).
There is a procedure that inserts data from staging_data to audit_data.
Sample of the table audit_data
That data in audit table should look like :
For the next record for user_no 523(lets assume update_date_time is '23-Nov-2020 10:20') seq_no becomes 3, active_from_date becomes '23-Nov-2020', active_to becomes 31-Dec-99 and the active_to of user_no 523 with seq_no 2 becomes '22-Nov-2020'. So the data should look like this :
Highlighted the 3rd record which will be added later in light green.
So here goes my solution : I suggested to use row_number() over(partition by user_no) analytical function to get seq_no for each user. I wanted to create a view based on that but Boss doesn't want a view. He strictly wants to use a procedure. Procedure should check if the user_no exists (in this example 523). If exists then seq_no increases and active_to of the previous record for 523 changes to latest active_from - 1 date. I will be honest - I have no clue how to achieve this in Procedure. I understand I can create a cursor with the query I had in my mind for the view. But to add seq_no and change active_to date is something that has puzzled me. Can anyone please guide me in right direction/s? Also I apologise in advance if I have left out any other details. Its midnight here now and after 8 hours of racking my brain on this I am very hungry!
edit 11th Mar : here is the code for the procedure I wrote to insert data into the audit table for situation when a particular user_no has no record in audit table :
create or replace procedure test_aud IS
user_found_audit number;
lv_user_no AUDIT_DATA.user_no%TYPE;
cursor member_no is select distinct user_no from STAGING_DATA;
begin
open member_no;
loop
fetch member_no into lv_user_no;
exit when member_no%notfound;
select count(*) into user_found_audit from AUDIT_DATA where user_no = lv_user_no;
if user_found_audit = 0 then
insert into AUDIT_DATA(user_no, update_date_time,status_code, seq_no, last_update_date, active_from, active_to)
select user_no, update_date_time,status_code,row_number() over(partition by user_no order by UPDATE_DATE_TIME) as seqno,
to_char(trunc(update_date_time),'DD-MON-YYYY'),
to_char(trunc(update_date_time),'DD-MON-YYYY'),
lead(to_char(trunc(update_date_time)-1,'DD-MON-YYYY'),1,'31-DEC-99') over(PARTITION BY user_no ORDER BY UPDATE_DATE_TIME) from STAGING_DATA where user_no = lv_user_no;
commit;
else
dbms_output.put_line(lv_user_no||' exists in audit table');
-- to code the block when user_no exists, involves an update and insert
end if;
end loop;
close member_no;
end;
/
Well you need to collect a couple things. The latest stage row and the latest audit row. Then it is just a matter of generating the new audit information and updating the previous latest one. The following makes a couple simplifying assumptions:
Only the latest stage data for a given user_no needs processed as
all prior have been processed, However it does not assume the stage
table has been cleared.
The sequencing of 'Y' and 'N' status_codes are properly order in
that manner. In fact it does not even check the value.
It need not concern itself with the inherent race condition. The
condition is derives from seq_no being generated as Max()+1. This
structure virtually guarantees a duplicate will eventually be
created.
The nested procedure "establish_audit" does all the actual work. The rest are just supporting characters, including a couple just for debug purpose. See fiddle.
create or replace
procedure generate_stage_audit(user_no_in staging_data.user_no%type)
as
k_end_of_time constant date := date '9999-12-31';
l_latest_user_stage staging_data%rowtype;
l_latest_user_audit audit_data%rowtype;
procedure establish_audit
is
begin
insert into audit_data(user_no, update_date_time, status_code
,seq_no, active_from, active_to)
select l_latest_user_stage.user_no
, l_latest_user_stage.update_date_time
, l_latest_user_stage.status_code
, coalesce(l_latest_user_audit.seq_no,0) + 1
, trunc(l_latest_user_stage.update_date_time)
, k_end_of_time
from dual;
update audit_data
set active_to = trunc(l_latest_user_stage.update_date_time - 1)
where user_no = l_latest_user_audit.user_no
and seq_no = l_latest_user_audit.seq_no;
end establish_audit;
procedure retrieve_latest_stage
is
begin
select *
into l_latest_user_stage
from staging_data
where (user_no, update_date_time) =
( select user_no, max(update_date_time)
from staging_data
where user_no = user_no_in
group by user_no
);
end retrieve_latest_stage;
procedure retrieve_latest_audit
is
begin
select *
into l_latest_user_audit
from audit_data
where (user_no, seq_no) =
( select user_no, max(seq_no)
from audit_data
where user_no = user_no_in
group by user_no
);
exception
when no_data_found then
null;
end retrieve_latest_audit;
---- for debugging ---
procedure show_stage
is
begin
dbms_output.put_line('-------- Stage Row -------');
dbms_output.put_line(' user_no==>' || to_char(l_latest_user_stage.user_no));
dbms_output.put_line('update_date_time==>' || to_char(l_latest_user_stage.update_date_time));
dbms_output.put_line(' status_code==>' || to_char(l_latest_user_stage.status_code));
end show_stage;
procedure show_audit
is
begin
dbms_output.put_line('-------- Audit Row -------');
dbms_output.put_line(' user_no==>' || to_char(l_latest_user_audit.user_no));
dbms_output.put_line('update_date_time==>' || to_char(l_latest_user_audit.update_date_time));
dbms_output.put_line(' status_code==>' || to_char(l_latest_user_audit.status_code));
dbms_output.put_line(' seq_no==>' || to_char(l_latest_user_audit.seq_no));
dbms_output.put_line(' active_from==>' || to_char(l_latest_user_audit.active_from));
dbms_output.put_line(' active_to==>' || to_char(l_latest_user_audit.active_to));
end show_audit;
begin -- the main event
retrieve_latest_stage;
show_stage;
retrieve_latest_audit;
show_audit;
establish_audit;
end generate_stage_audit;
A couple warnings:
It seems you may be tempted to use string data type for the audit
columns Active_Form and Active_to as you are trying to declare then
"date type without the time". However there is no such data type in
Oracle; time is part of all dates. Do not do so, store them as
standard dates. (Note Dates are not stored in any format, but an
internal structure. Formats are strictly a visual representation).
Just throwaway the time with the format on the query or by setting
nls_date_format.
You may be tempted to convert call this through a trigger. Do not,
it will likely result in an "ORA-04091: Table is mutating"
exception.

Most Efficient Version of PLSQL Stored Procedure

I am writing a PL/SQL stored procedure which will be called from within a .NET application.
My stored procedure must return
the count of values in a table of part revisions, based on an input part number,
the name of the lowest revision level currently captured in this table for the input part number
the name of the revision level for a particular unit in the database associated with this part number and an input unit ID.
The unit's revision level name is captured within a separate table with no direct relationship to the part revision table.
Relevant data structure:
Table Part has columns:
Part_ID int PK
Part_Number varchar2(30)
Table Part_Revisions:
Revision_ID int PK
Revision_Name varchar2(100)
Revision_Level int
Part_ID int FK
Table Unit:
Unit_ID int PK
Part_ID int FK
Table Unit_Revision:
Unit_ID int PK
Revision_Name varchar2(100)
With that said, what is the most efficient way for me to query these three data elements into a ref cursor for output? I am considering the following option 1:
OPEN cursor o_Return_Cursor FOR
SELECT (SELECT COUNT (*)
FROM Part_Revisions pr
inner join PART pa on pa.part_id = pr.part_id
WHERE PA.PART_NO = :1 )
AS "Cnt_PN_Revisions",
(select pr1.Revision_Name from Part_Revisions pr1
inner join PART pa1 on pa1.part_id = pr1.part_id
WHERE PA.PART_NO = :1 and pr1.Revision_Level = 0)
AS "Input_Revison_Level",
(select ur.Revision_Name from Unit_Revision ur
WHERE ur.Unit_ID = :2) as "Unit_Revision"
FROM DUAL;
However, Toad's Explain Plan returns Cost:2 Cardinality: 1, which I suspect is due to me using DUAL in my main query. Comparing that to option 2:
select pr.Revision_Name, (select count(*)
from Part_Revisions pr1
where pr1.part_id = pr.part_id) as "Count",
(select ur.Revision_Name
from Unit_Revision ur
where ur.Unit_ID = :2) as "Unit_Revision"
from Part_Revisions pr
inner join PART pa on pa.part_id = pr.part_id
WHERE PA.PART_NO = :1 and pr.Revision_Level = 0
Essentially I don't really know how to compare the results from my execution plans, to chose the best design. I have also considered a version of option 1, where instead of joining twice to the Part table, I select the Part_ID into a local variable, and simply query the Part_Revisions table based on that value. However, this is not something I can use the Explain Plan to analyze.
Your description and select statements look different... I based the procedure on the SQL statements.
PROCEDURE the_proc
(
part_no_in IN NUMBER
, revision_level_in IN NUMBER
, unit_id_in IN NUMBER
, part_rev_count_out OUT NUMBER
, part_rev_name_out OUT VARCHAR2
, unit_rev_name_out OUT VARCHAR2
)
AS
BEGIN
SELECT COUNT(*)
INTO part_rev_count_out
FROM part pa
WHERE pa.part_no = part_no_in
AND EXISTS
(
SELECT 1
FROM part_revisions pr
WHERE pa.part_id = pr.part_id
);
SELECT pr1.revision_name
INTO part_rev_name_out
FROM part_revisions pr1
WHERE pr1.revision_level = revision_level_in
AND EXISTS
(
SELECT 1
FROM part pa1
WHERE pa1.part_id = pr1.part_id
AND pa.part_no = part_no_in
);
SELECT ur.revision_name
INTO unit_rev_name_out
FROM unit_revision ur
WHERE ur.unit_id = unit_id_in;
END the_proc;
It looks like you are obtaining scalar values. Rather than return a cursor, just return the values using clean sql statements. I have done this numerous times from .net, it works fine.
Procedure get_part_info(p_partnum in part.part_number%type
, ret_count out integer
, ret_revision_level out part_revisions.revision_level%type
, ret_revision_name out part_revisions.revision_name%type) as
begin
select count(*) into ret_count from ....;
select min(revision_level) into ret_revision_level from ...;
select revision_name in ret_revision_name...;
return;
end;

Stored Procedures reading a value of Min()

I am trying to find the minimum value of a primary key column of type (int) in a particular Table
A portion of my Stored Procedure Code:
IF NOT EXISTS
(
SELECT *
FROM Table
)
BEGIN
SELECT *
FROM Table
END
ELSE
BEGIN
SELECT Min(ColumnOne)
FROM Table
END
This is my main code after reading:
if (!reader.Read())
return "EMPTY TABLE";
else
return reader.GetInt32(0).ToString();
My ExecuteReader has no problem but when I got an exception at the statement
reader.GetInt32(0).ToString()
I believe I extract the information wrongly when my tables have more than one entry. What is the correct function I should call from reader to get the number??
i didn't get your Question.
AS you specify min() Value in Question And you wrote max() Function in T-SQL Script.
You Can try below if you want to retrieve next Val of a Column
Select isnull(max(ColumnOne),0)+1 FROM Table
Above Query will return you
1 in case Table is empty
else max current Value+1(Next Available Value)
from Table.

temp table creation in informix 4GL SE

How should I create and populate data in a temp table? then how can I print data retrieved on temp table? I saw example like this; but no further help. SO I need some help.
SELECT DISTINCT * FROM Table INTO TEMP Temp1;
You select the data just like you would from any other table:
SELECT * INTO r_temp1.* FROM Temp1
or defining cursors for such a statement, and using a FOREACH loop, or ...
The trick is that you need to know at compile time what the columns in Temp1 are going to be. In this case, you could use:
DEFINE r_temp1 RECORD LIKE Table.*
In more general cases, you'll probably assemble a record by hand from the relevant bits and pieces:
DEFINE r_temp1 RECORD
col1 LIKE Table1.ColumnA,
col2 LIKE Table2.ColumnB,
...
colN LIKE TableZ.ColumnZ
END RECORD
I created the following function that can be called passing in the temp-table name:
function unload_temp(l_table)
define
l_table char(20),
l_file char(20),
str char(512)
let l_file = l_table clipped, ".out"
let str = "select * from ",l_table clipped
whenever error continue
prepare stmt_table_exists from str
whenever error stop
#Return if table does not exist
if sqlca.sqlcode = -206
then
return
end if
unload to l_file delimiter "|" str
end function #unload_temp()
This will create a file called tableName.out.
This is extremely helpful when debugging and you want to see how a temp table looks in middle running a program.

Self reference update on insert trigger in Informix

I'm extracting data from various sources into one table. In this new table, there's a field called lineno. This field value is should be in sequence based on company code and batch number. I've wrote the following procedure
CREATE PROCEDURE update_line(company CHAR(4), batch CHAR(8), rcptid CHAR(12));
DEFINE lineno INT;
SELECT Count(*)
INTO lineno
FROM tmp_cb_rcpthdr
WHERE cbrh_company = company
AND cbrh_batchid = batch;
UPDATE tmp_cb_rcpthdr
SET cbrh_lineno = lineno + 1
WHERE cbrh_company = company
AND cbrh_batchid = batch
AND cbrh_rcptid = rcptid;
END PROCEDURE;
This procedure will be called using the following trigger
CREATE TRIGGER tmp_cb_rcpthdr_ins INSERT ON tmp_cb_rcpthdr
REFERENCING NEW AS n
FOR EACH ROW
(
EXECUTE PROCEDURE update_line(n.company, cbrh_batchid, cbrh_rcptid)
);
However, I got the following error
SQL Error = -747 Table or column matches object referenced in
triggering statement.
From oninit.com, I learn that the error caused by a triggered SQL statement acts on the triggering table which in this case is the UPDATE statement.
So my question is, how do I solve this problem? Is there any work around or better solution?
I think the design needs to be reconsidered. For a start, what happens if some rows get deleted from tmp_cb_rcpthdr ? The COUNT(*) query will result in duplicate lineno values.
Even if this is an ETL only process, and you can be confident the data won't be manipulated from elsewhere, performance will be an issue, and will only get worse the more data you have for any one combination of company and batch_id.
Is it necessary for the lineno to increment from zero, or is it just to maintain the original load order? Because if it's the latter, a SEQUENCE or a SERIAL field on the table will achieve the same end, and be a lot more efficient.
If you must generate lineno in this way, I would suggest you create a second control table, keyed on company and batch_id, that tracks the current lineno value, ie: (untested)
CREATE PROCEDURE update_line(company CHAR(4), batch CHAR(8));
DEFINE lineno INT;
SELECT cbrh_lineno INTO lineno
FROM linenoctl
WHERE cbrh_company = company
AND cbrh_batchid = batch;
UPDATE linenoctl
SET cbrh_lineno = lineno + 1
WHERE cbrh_company = company
AND cbrh_batchid = batch;
-- A test that no other process has grabbed this record
-- might need to be considered here, ie cbrh_lineno = lineno
RETURN lineno + 1
END PROCEDURE;
Then use it as follows:
CREATE TRIGGER tmp_cb_rcpthdr_ins INSERT ON tmp_cb_rcpthdr
REFERENCING NEW AS n
FOR EACH ROW
(
EXECUTE PROCEDURE update_line(n.company, cbrh_batchid) INTO cbrh_lineno
);
See the IDS documentation for more on using calculated values with triggers.

Resources