I have written a stored procedure which will be called from python. The stored procedure needs to insert the variant data into my table if the id doesn't exist or update the existing variant data where there is a match for the id. The id will be passed the way the variant data is, but for now I am just trying to get it working with a hardcoded id. The stored procedure gets called successfully from python, but then nothing gets inserted or updated in the stored procedure and the stored procedure doesn't give me an error. I am not sure if I am doing something wrong or the...
if exists (select * from my_database_table where my_variant_data:id::varchar = '123456')
... part is being ignored because it isn't supported. I haven't been able to find anything in the documentation to prove or disprove this. Does anyone know?
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var insert_update_query = `
if exists (select * from my_database_table where my_variant_data:id::varchar = '123456')
begin
update my_database_table SET my_variant_data = parse_json(:1)) WHERE my_variant_data:id::varchar = '123456'
end
else
begin
insert into my_database_table(my_variant_data) select (parse_json(:1));
end
`
var result = "";
try {
var sql_insert_update_query = snowflake.createStatement({
sqlText: insert_update_query
});
var insert_update_query_result = sql_insert_update_query.execute();
result += "\n Query succeeded";
} catch (err) {
result += "\n Query failed failed: " + err.code + "\n State: " + err.state;
result += "\n Message: " + err.message;
result += "\n Stack Trace:\n" + err.stackTraceTxt;
}
return result;
$$
;
I have tested the insert and update parts of the query in the stored procedure individually and they work fine.
Insert - works as expected.
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var sql_command = "insert into my_database_table(my_variant_data) select (parse_json(:1));";
var sql = snowflake.createStatement( {sqlText: sql_command, binds:[JSON.stringify(variant_data)]});
var resultSet = sql.execute();
return sql_command;
$$
;
Update - works as expected.
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var sql_command = "UPDATE my_database_table SET my_variant_data = parse_json(:1)) WHERE my_variant_data:id::varchar = '123456'";
var sql = snowflake.createStatement( {sqlText: sql_command, binds:[JSON.stringify(variant_data)]});
var resultSet = sql.execute();
$$
;
Given the CODE executed needs to be valid runs on the console SQL, which this if is not, and it is fundamentally a MERGE command I would suggest flipping the code into a MERGE:
MERGE INTO my_database_table USING my_database_table
ON my_variant_data:id::varchar = '123456'
WHEN MATCHED THEN UPDATE SET my_variant_data = parse_json(:1))
WHEN NOT MATCHED THEN INSERT (my_variant_data) VALUES (parse_json(:1));
otherwise if you are want it in SP space, then I would be inclinded to break the code into a SELECT x INTO varaible FROM blar pattern and then have the IF be in SP and pick between the two blocks of SQL to run. But given it's just a merge, I would again still, do a merge.
I am kinda stuck with the limitations of UDFs/stored Procedures in Snowflake and wonder what I am missing. I feel like there has to be a solution, but I can't see it right now, so maybe som new input from stackoverflow can help me.
I want something (UDF/Procedure) that has an input and returns a table. I created a stored procedure that generates the SQl-Statement. But in the stored procedure I can only return a single value and no table.
My stored procedure is the following:
CREATE OR REPLACE PROCEDURE DWH.TEMP.getAccounts(NODE_NAME VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS '
var maxLevel = 0;
function executeSQL(SQL ) {
// Prepare statement.
var stmt = snowflake.createStatement(
{
sqlText: SQL
}
);
// Execute Statement
var res = stmt.execute();
res.next();
return res.getColumnValue(1);
}
maxLevel= executeSQL(" SELECT MAX(LEVEL) as Max_Level FROM DWH.MART.VDIM_ACCOUNT");
var sqlString = "SELECT DISTINCT AccountID FROM DWH.MART.VDIM_ACCOUNT WHERE ";
var sqlWhere = "";
for (var i = 0;i<=maxLevel;i++)
{
sqlWhere += "Group_L"+ i +" in (\'"+NODE_NAME+"\')";
if(i<maxLevel)
{
sqlWhere += " OR ";
}
}
sqlString += sqlWhere;
return sqlString;
'
If I execute this with a node, for example with call GETACCOUNTS('Test') I get the result:
SELECT DISTINCT AccountID
FROM dwh.mart.vdim_account
WHERE group_l0 IN ( 'TEST' )
OR group_l1 IN ( 'TEST' )
OR group_l2 IN ( 'TEST' )
OR group_l3 IN ( 'TEST' )
OR group_l4 IN ( 'TEST' )
OR group_l5 IN ( 'TEST' )
But I want the result of above SQL-Statement returnes as a table or list. How do I do this in Snowflake?
I am stuck because I need the for loop to dynamically generate my SQL-Statement. I can only use loops in JS UDFs, but I can only execute SQL in SQL UDFs.
I'm using a procedure to insert rows into a table. I want to return the count of rows inserted, but I can't figure out how to turn on the variable substitution from inside a procedure. I've tried all of the following, but they all return errors:
snowflake.execute({sqlText: '!set variable_substitution=True'});
snowflake.execute({sqlText: 'set variable_substitution=True'});
snowflake.execute({sqlText: '-o variable_substitution=True'});
How do I turn this option on so I can run "select &__rowcount;" and get my count back?
Here's code for a test procedure, if that helps:
CREATE OR REPLACE PROCEDURE TEST_OF_GETTING_COUNTS()
RETURNS VARCHAR
LANGUAGE javascript
CALLED ON NULL INPUT
AS
$$
// Turn on variable substitution
snowflake.execute(
{sqlText: '!set variable_substitution=True'}
);
// Prepare SQL to identify tables to be updated
snowflake.execute({sqlText: 'SELECT 1'});
// Now get the count of rows selected
var getCount = snowflake.createStatement({sqlText: "SELECT &__ROWCOUNT;"});
var rowCountResultSet = getCount.execute();
while (rowCount.next()) {
rowCount= rowCountResultSet.getColumnValue(1);
}
// Turn off variable substitution
snowflake.execute({sqlText: '!set variable_substitution=False'});
return rowCount;
$$;
CALL TEST_OF_GETTING_COUNTS();
NickW, in his comment above, gave me a link to a page in Snowflake's documentation that lists all the JavaScript methods available for the snowflake, Statement, ResultSet, and SfDate objects. It gave me exactly what I needed: the getRowCount() method.
i'm trying to get timestamp value as result of stored procedure.
but getting error .
error message:- SQL Error [100132] [P0000]: JavaScript execution error: Incorrect Timestamp returned.
CREATE OR REPLACE PROCEDURE simple_stored_procedure_example(awesome_argument VARCHAR)
returns TIMESTAMPNTZ
language javascript
as
$$
var cmd = "SELECT EXTRACT_END_TIME FROM EXPLARITY_DB.EXPLARITY_SCHEMA.DEMO_CONTROL_TABLE WHERE PIPELINE_NAME =:1";
var sql = snowflake.createStatement(
{sqlText: cmd,
binds: ['awesome_argument']
}
);
var result1 = sql.execute();
return result1;
$$;
CALL simple_stored_procedure_example('pipeline1');
It can be done like this:
create or replace table tst_tbl(c1 timestamp);
insert into tst_tbl values ('2021-02-02 10:00:00.000');
create or replace procedure my_test(myarg VARCHAR)
returns TIMESTAMP
language javascript
as
$$
var cmd = "SELECT c1 FROM tst_tbl";
var sql = snowflake.createStatement({sqlText: cmd});
var resultSet = sql.execute();
resultSet.next();
my_date = resultSet.getColumnValue(1);
return my_date;
$$;
call my_test('test');
I get 1 row back as expected.
I am using the Snowflake Cloud Database, please help me with a tool to debugging the procedures or functions.
The following Snowflake Javascript Stored Procedure is a template I use to get started on a new Stored Procedures. It contains plenty of debugging tricks, such as:
it has a "where am I?" variable which gives you a understanding of where in the code you are
it gathers information in an array as the process moves along
it returns that array to the standard output of the call command
it has a "good start" of an exception block, who's contents also get pushed out to standard output on a call of the stored procedure, should it fail.
Something I've been meaning to add is to set a query tag in the code as well, that'd be helpful when reviewing query history, to easily identify the SQL commands that were used in the execution of the Stored Procedure.
This "ties into" the final "debugging trick" - you should always review the query history (actual queries your code executed) when developing stored procedures in a development or test environment, particularly when you are building dynamic SQL statements. Reviewing your query history is a must-do and will show you exactly the commands run and the order of operations of them running.
Here's the code with the sample table it uses, I hope it helps...Rich
CREATE OR REPLACE TABLE test_scripts (
load_seq number,
script varchar(2000)
);
INSERT INTO test_scripts values
(1, 'SELECT current_timestamp();'),
(2, 'SELECT current_warehouse();'),
(3, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(4, 'SELECT current_date();'),
(5, 'SELECT current_account();'),
(6, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(7, 'SELECT ''RICH'';');
select * from test_scripts;
CREATE OR REPLACE PROCEDURE sp_test(p1 varchar, p2 varchar)
RETURNS ARRAY
LANGUAGE javascript
EXECUTE AS caller
AS
$$
//note: you can change the RETURN to VARCHAR if needed
// but the array "looks nice"
try {
var whereAmI = 1;
var return_array = [];
var counter = 0;
var p1_str = "p1: " + P1
var p2_str = "p2: " + P2
var load_seq = P1;
var continue_flag = P2;
whereAmI = 2;
return_array.push(p1_str)
return_array.push(p2_str)
whereAmI = 3;
//which SQL do I want to run?
if (continue_flag=="YES") {
return_array.push("query 1")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq >= " + load_seq + " order by 1, 2;";
}
else {
return_array.push("query 2")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq = " + load_seq + " order by 1, 2;";
}
whereAmI = 4;
//begin the run of grabbing the commands
var stmt = snowflake.createStatement( {sqlText: sqlquery} );
var rs = stmt.execute();
whereAmI = 5;
// Loop through the results, processing one row at a time...
while (rs.next()) {
counter = counter + 1;
var tmp_load_seq = rs.getColumnValue(1);
var tmp_script = rs.getColumnValue(2);
var tmp_rs = snowflake.execute({sqlText: tmp_script});
tmp_rs.next();
var tmp_col1 = tmp_rs.getColumnValue(1);
return_array.push("tmp_col1: " + tmp_col1)
}
whereAmI = 6;
return_array.push("end process - counter: " + counter)
return return_array;
}
catch (err) {
return_array.push("error found")
return_array.push(whereAmI)
return_array.push(err)
return return_array;
}
$$;
CALL sp_test(3, 'NO');
I do not believe there is any editor / debugger for stored procedures for Snowflake. Few options:
You can break your code to smaller parts and try to troubleshoot
Use a log table and insert into log table often, so you can look at the log table to find out what went wrong
unfortunately there isn't one environment to rule them all
1. write your SQL in a Worksheet or Editor
2. write your SPROC code in a JS enabled editor
3. merge them together in a Worksheet or Editor
4. Unit test in SPROCS as shown above by #Rich Murmane
I normally just write SPROCS in a Worksheet but it isnt optimal
Logging is your friend here, as there is no debugger. In general finding and using a debugger for db stored procedures is hard to pull off. Not impossible, just unlikely.
This is a decent alternative:
CREATE or replace PROCEDURE do_log(MSG STRING)
RETURNS STRING
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
//see if we should log - checks for do_log = true session variable
try{
var foo = snowflake.createStatement( { sqlText: `select $do_log` } ).execute();
} catch (ERROR){
return; //swallow the error, variable not set so don't log
}
foo.next();
if (foo.getColumnValue(1)==true){ //if the value is anything other than true, don't log
try{
snowflake.createStatement( { sqlText: `create temp table identifier ($log_table) if not exists (ts number, msg string)`} ).execute();
snowflake.createStatement( { sqlText: `insert into identifier ($log_table) values (:1, :2)`, binds:[Date.now(), MSG] } ).execute();
} catch (ERROR){
throw ERROR;
}
}
$$
;
Then in the stored procedure, you want to debug add a log function at the top:
function log(msg){
snowflake.createStatement( { sqlText: `call do_log(:1)`, binds:[msg] } ).execute();
}
Then above the call to the stored procedure:
set do_log = true; --true to enable logging, false (or undefined) to disable
set log_table = 'my_log_table'; --The name of the temp table where log messages go
Then in the actual stored procedure you need to add some logging lines:
log('this is another log message');
Then call the stored procedure as you would normally. Then select from my_log_table.
Important note: this uses a temp table, so you won't be able to read from that logging table in a different Snowflake connection. This means if you're using the Worksheet editor you need to keep all this stuff on the same sheet.
"Borrowed" from: https://community.snowflake.com/s/article/Snowflake-Stored-Procedure-Logging