I'm using a procedure to insert rows into a table. I want to return the count of rows inserted, but I can't figure out how to turn on the variable substitution from inside a procedure. I've tried all of the following, but they all return errors:
snowflake.execute({sqlText: '!set variable_substitution=True'});
snowflake.execute({sqlText: 'set variable_substitution=True'});
snowflake.execute({sqlText: '-o variable_substitution=True'});
How do I turn this option on so I can run "select &__rowcount;" and get my count back?
Here's code for a test procedure, if that helps:
CREATE OR REPLACE PROCEDURE TEST_OF_GETTING_COUNTS()
RETURNS VARCHAR
LANGUAGE javascript
CALLED ON NULL INPUT
AS
$$
// Turn on variable substitution
snowflake.execute(
{sqlText: '!set variable_substitution=True'}
);
// Prepare SQL to identify tables to be updated
snowflake.execute({sqlText: 'SELECT 1'});
// Now get the count of rows selected
var getCount = snowflake.createStatement({sqlText: "SELECT &__ROWCOUNT;"});
var rowCountResultSet = getCount.execute();
while (rowCount.next()) {
rowCount= rowCountResultSet.getColumnValue(1);
}
// Turn off variable substitution
snowflake.execute({sqlText: '!set variable_substitution=False'});
return rowCount;
$$;
CALL TEST_OF_GETTING_COUNTS();
NickW, in his comment above, gave me a link to a page in Snowflake's documentation that lists all the JavaScript methods available for the snowflake, Statement, ResultSet, and SfDate objects. It gave me exactly what I needed: the getRowCount() method.
Related
I have a snowflake table being used to store records of sql being executed by a stored procedure and any error messages. The records in this table are being saved as a string with special chars escaped with javascripts escape('log') function. I then want to create a view to this log table that will return the records in an easily readable format.
My first attempt at this was to create an additional stored procedure to query the log table, pass the record to the unescape() function, then return it. Calling this procedure works as intended but we can't then create a view of this data say with something like
create view log_view as
select (call UNESCAPE_PROC());
The other idea was to use a UDF rather than a stored procedure. However this also fails as we can't execute sql code with a javascript UDF. This post touches on this idea.
My question is how can I record these executed statements in a table in such a way as they can be later viewed in a plain text readable format. I've outlined my attempts below but perhaps my approach is flawed, open to any suggestions.
Minimal working example below
Sample log table and procedure to write a statement to said table
create or replace table event_table(
event varchar,
event_stamp timestamp
);
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+escape(to_log)+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ escape(to_log)
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Stored procedure to return readable log records
CREATE OR REPLACE PROCEDURE UNESCAPE_PROC()
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP will select a chosen column from event_table table, pass it to javascripts unescape() fn and return it'
EXECUTE AS CALLER
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$;
call UNESCAPE_PROC();
Which returns the records in a readable form as expected
However this of course wont work as part of a view eg.
-- define a view calling this procedure??
create view log_view as
select (call UNESCAPE_PROC());
Javascript user defined function can be used in a view like this, however it cannot be used to execute sql as in the stored procedures
-- use a UDF instead
CREATE OR REPLACE FUNCTION UNESCAPE_UDF()
RETURNS string
LANGUAGE JAVASCRIPT
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$
;
select UNESCAPE_UDF();
Stored procedures will solve one half of my problem for me, whilst UDF's will solve the other half. How can I combine the functionality of these two methods to solve this issue?
A much cleaner approach using parameters binding:
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', '${str_stamp}');`;
sql = `INSERT INTO event_table (event,event_stamp)
VALUES(?, try_to_timestamp(?));`;
var stmnt = snowflake.createStatement({sqlText: sql, binds:[to_log, str_stamp]});
stmnt.execute();
return "logged: "+ to_log
$$;
Call:
call insert_to_event(current_timestamp());
-- logged: insert into dummy_table values(2, 'Bill', '2022-02-03T17:45:44.140Z');
select * from event_table;
Found a solution/workaround.
Rather than using javascripts escape/unescape functions to remove special chars from the logs, we use a regex replace eg.
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
to_log = to_log.replace(/[`~!##$%^&*|+=?'"<>\{\}\[\]\\\/]/gi, '');
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+to_log+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ to_log
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Which writes to the log table in an easily readable format with no need for additional stored procedures/UDF's.
I am using the Snowflake Cloud Database, please help me with a tool to debugging the procedures or functions.
The following Snowflake Javascript Stored Procedure is a template I use to get started on a new Stored Procedures. It contains plenty of debugging tricks, such as:
it has a "where am I?" variable which gives you a understanding of where in the code you are
it gathers information in an array as the process moves along
it returns that array to the standard output of the call command
it has a "good start" of an exception block, who's contents also get pushed out to standard output on a call of the stored procedure, should it fail.
Something I've been meaning to add is to set a query tag in the code as well, that'd be helpful when reviewing query history, to easily identify the SQL commands that were used in the execution of the Stored Procedure.
This "ties into" the final "debugging trick" - you should always review the query history (actual queries your code executed) when developing stored procedures in a development or test environment, particularly when you are building dynamic SQL statements. Reviewing your query history is a must-do and will show you exactly the commands run and the order of operations of them running.
Here's the code with the sample table it uses, I hope it helps...Rich
CREATE OR REPLACE TABLE test_scripts (
load_seq number,
script varchar(2000)
);
INSERT INTO test_scripts values
(1, 'SELECT current_timestamp();'),
(2, 'SELECT current_warehouse();'),
(3, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(4, 'SELECT current_date();'),
(5, 'SELECT current_account();'),
(6, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(7, 'SELECT ''RICH'';');
select * from test_scripts;
CREATE OR REPLACE PROCEDURE sp_test(p1 varchar, p2 varchar)
RETURNS ARRAY
LANGUAGE javascript
EXECUTE AS caller
AS
$$
//note: you can change the RETURN to VARCHAR if needed
// but the array "looks nice"
try {
var whereAmI = 1;
var return_array = [];
var counter = 0;
var p1_str = "p1: " + P1
var p2_str = "p2: " + P2
var load_seq = P1;
var continue_flag = P2;
whereAmI = 2;
return_array.push(p1_str)
return_array.push(p2_str)
whereAmI = 3;
//which SQL do I want to run?
if (continue_flag=="YES") {
return_array.push("query 1")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq >= " + load_seq + " order by 1, 2;";
}
else {
return_array.push("query 2")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq = " + load_seq + " order by 1, 2;";
}
whereAmI = 4;
//begin the run of grabbing the commands
var stmt = snowflake.createStatement( {sqlText: sqlquery} );
var rs = stmt.execute();
whereAmI = 5;
// Loop through the results, processing one row at a time...
while (rs.next()) {
counter = counter + 1;
var tmp_load_seq = rs.getColumnValue(1);
var tmp_script = rs.getColumnValue(2);
var tmp_rs = snowflake.execute({sqlText: tmp_script});
tmp_rs.next();
var tmp_col1 = tmp_rs.getColumnValue(1);
return_array.push("tmp_col1: " + tmp_col1)
}
whereAmI = 6;
return_array.push("end process - counter: " + counter)
return return_array;
}
catch (err) {
return_array.push("error found")
return_array.push(whereAmI)
return_array.push(err)
return return_array;
}
$$;
CALL sp_test(3, 'NO');
I do not believe there is any editor / debugger for stored procedures for Snowflake. Few options:
You can break your code to smaller parts and try to troubleshoot
Use a log table and insert into log table often, so you can look at the log table to find out what went wrong
unfortunately there isn't one environment to rule them all
1. write your SQL in a Worksheet or Editor
2. write your SPROC code in a JS enabled editor
3. merge them together in a Worksheet or Editor
4. Unit test in SPROCS as shown above by #Rich Murmane
I normally just write SPROCS in a Worksheet but it isnt optimal
Logging is your friend here, as there is no debugger. In general finding and using a debugger for db stored procedures is hard to pull off. Not impossible, just unlikely.
This is a decent alternative:
CREATE or replace PROCEDURE do_log(MSG STRING)
RETURNS STRING
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
//see if we should log - checks for do_log = true session variable
try{
var foo = snowflake.createStatement( { sqlText: `select $do_log` } ).execute();
} catch (ERROR){
return; //swallow the error, variable not set so don't log
}
foo.next();
if (foo.getColumnValue(1)==true){ //if the value is anything other than true, don't log
try{
snowflake.createStatement( { sqlText: `create temp table identifier ($log_table) if not exists (ts number, msg string)`} ).execute();
snowflake.createStatement( { sqlText: `insert into identifier ($log_table) values (:1, :2)`, binds:[Date.now(), MSG] } ).execute();
} catch (ERROR){
throw ERROR;
}
}
$$
;
Then in the stored procedure, you want to debug add a log function at the top:
function log(msg){
snowflake.createStatement( { sqlText: `call do_log(:1)`, binds:[msg] } ).execute();
}
Then above the call to the stored procedure:
set do_log = true; --true to enable logging, false (or undefined) to disable
set log_table = 'my_log_table'; --The name of the temp table where log messages go
Then in the actual stored procedure you need to add some logging lines:
log('this is another log message');
Then call the stored procedure as you would normally. Then select from my_log_table.
Important note: this uses a temp table, so you won't be able to read from that logging table in a different Snowflake connection. This means if you're using the Worksheet editor you need to keep all this stuff on the same sheet.
"Borrowed" from: https://community.snowflake.com/s/article/Snowflake-Stored-Procedure-Logging
I have a db2 stored procedure that takes in some parameters, gets some data from somewhere and then returns a result set through a cursor.
Now I want to write a table function in db2, that will call this stored procedure, read from the result set and return the data in the result set as a table (eventually I want to use this table function in a join).
I would like to know if this is permitted in db2 (we're using DB2 v10.5), i.e. execute a stored procedure in a table function and fetch and read from the result set from the stored procedure. If so, what is the right syntax for calling the stored procedure and reading the result set inside a table function in db2? Thanks!
Yes, it's possible. See the example below.
--#SET TERMINATOR #
CREATE OR REPLACE PROCEDURE TEST_PROC(P_TABSCHEMA VARCHAR(128))
DYNAMIC RESULT SETS 1
READS SQL DATA
BEGIN
DECLARE C1 CURSOR WITH HOLD WITH RETURN FOR
SELECT TABSCHEMA, TABNAME, COLCOUNT
FROM SYSCAT.TABLES
WHERE TABSCHEMA=P_TABSCHEMA;
OPEN C1;
END#
--CALL TEST_PROC('SYSCAT')#
CREATE OR REPLACE FUNCTION TEST_PROC(P_TABSCHEMA VARCHAR(128))
RETURNS TABLE (
TABSCHEMA VARCHAR(128)
, TABNAME VARCHAR(128)
, COLCOUNT INT
)
READS SQL DATA
BEGIN
DECLARE SQLSTATE CHAR(5);
DECLARE V_TABSCHEMA VARCHAR(128);
DECLARE V_TABNAME VARCHAR(128);
DECLARE V_COLCOUNT INT;
DECLARE V1 RESULT_SET_LOCATOR VARYING;
CALL TEST_PROC(P_TABSCHEMA);
ASSOCIATE RESULT SET LOCATOR (V1) WITH PROCEDURE TEST_PROC;
ALLOCATE C1 CURSOR FOR RESULT SET V1;
L1: LOOP
FETCH C1 INTO V_TABSCHEMA, V_TABNAME, V_COLCOUNT;
IF SQLSTATE<>'00000' THEN LEAVE L1; END IF;
PIPE(V_TABSCHEMA, V_TABNAME, V_COLCOUNT);
END LOOP L1;
CLOSE C1;
RETURN;
END#
SELECT * FROM TABLE(TEST_PROC('SYSCAT'))#
You need to create the DB2 table-function as follows:
CREATE FUNCTION database_schema.function_name ( IN_PARTID VARCHAR(1000) )
RETURNS TABLE ( PARTNO CHAR(25), PARTDS CHAR(30), QUANTITY INT )
BEGIN
RETURN SELECT PARTNO , PARTDS , CASE WHEN QUANTITY > 0 THEN QUANTITY ELSE 0 END QUANTITY
FROM
(
SELECT PARTNO
,MAX(PARTDS) AS PARTDS
,SUM(QUANTITY) AS QUANTITY
FROM database_schema.table_name
WHERE 1=1
AND PARTID = (CAST(IN_PARTID AS INT))
GROUP BY PARTNO
) AA;
END;
Then invoke the table-function as join or straight SQL:
SELECT partno,partds,quantity
FROM TABLE(database_schema.function_name('parameter_1'))
I have a stored procedure, in which I want to simply store result of a select statement in an output parameter and return that , how can i do that.
I would appreciate if you give me the right syntax of it, since i am new to DB and Sybase specially, that's why i am just giving u a pseudo code for that..
/pseudo code
create my_proc(in_param i,out_param o1,out_param o2){
.....other select and insert statements
.....
if(xyz=true){
o1 = select * from emplyees
}
return o1,o2
}
You don't need output parameters to return result of query, try below code
create procedure proc1
(
#val1 integer
)
as
begin
select * from emplyees
end
/pseudo code
create my_proc(in_param int,out_param1 int out,out_param2 int out)
BEGIN
.....other select and insert statements
if(xyz=true)BEGIN
select out_param1=e.col1,out_param1=2=e.col2 from emplyees e
END
END
Modify datatypes accordingly
Thanks,
Gopal
Here's my SQL Server stored procedure :
ALTER PROCEDURE [dbo].[SearchUser]
(#Text NVARCHAR(100),
#TotalRows INT = 0 OUTPUT)
AS
BEGIN
SELECT #TotalRows=1000
SELECT * from Users
END
And my C# code
using (var context = new TestDBEntities())
{
var outputParameter = new ObjectParameter("TotalRows", typeof(Int32));
context.SearchUser("", outputParameter);
Response.Write(outputParameter.Value);
}
However outputParameter.Value always is null.
Could anybody tell me why?
Output parameters filled by its actual values during the execution of the stored procedure.
But table-valued stored procedure actually get executed only in moment when you're trying to iterate resulting recordset, but not calling a wrapper method.
So, this DOES'T work:
using (var context = new TestDBEntities())
{
var outputParameter = new ObjectParameter("TotalRows", typeof(Int32));
context.SearchUser("", outputParameter);
// Paremeter value is null, because the stored procedure haven't been executed
Response.Write(outputParameter.Value);
}
This DOES:
using (var context = new TestDBEntities())
{
var outputParameter = new ObjectParameter("TotalRows", typeof(Int32));
// Procedure does not executes here, we just receive a reference to the output parameter
var results = context.SearchUser("", outputParameter);
// Forcing procedure execution
results.ToList();
// Parameter has it's actual value
Response.Write(outputParameter.Value);
}
When you're working with stored procedures what don't return any recordset, they execute immediately after a method call, so you have actual value in output parameter.
We had a simular issue due to defered excecution our unit tests failed. In short if you have a stored proc that does NOT return anything you need to be sure to set the response type as 'None' when set as 'None' it will be excecuted when called and not defered.
In case you return anything (E.g. Scalar type of String results) it will excecute it when you use the result even if that .Count() or .ToList() is outside of the method that contains the function call.
So try not to force excecution if not need, when needed it should excecute but be sure to declare it correctly or it might not work.
I have same problem before. The main reason I think that the entities framework has the bug in case the user stored procedure has output parameter and return a result set. For example:
ALTER PROCEDURE [dbo].[SearchTest]
(
#RowTotal INT = 0 OUTPUT,
#RowCount INT = 0 OUTPUT
)
AS
BEGIN
SET NOCOUNT ON
SELECT * FROM SomeThing
SELECT #RowTotal = 1233, #RowCount = 5343
END
However if you change the user stored procedure as following, you can get the output params
ALTER PROCEDURE [dbo].[SearchTest]
(
#RowTotal INT = 0 OUTPUT,
#RowCount INT = 0 OUTPUT
)
AS
BEGIN
SET NOCOUNT ON
SELECT #RowTotal = 1233, #RowCount = 5343
END
You can workaround as following:
ALTER PROCEDURE [dbo].[SearchTest]
AS
BEGIN
DECLARE #RowTotal INT, #RowCount INT
SET NOCOUNT ON
SELECT #RowTotal = 1233, #RowCount = 5343
SELECT #RowTotal AS RowTotal, #RowCount AS RowCount, s.*
FROM SomeThing s
END
If anybody has better solution, please tell me