i'm trying to get timestamp value as result of stored procedure.
but getting error .
error message:- SQL Error [100132] [P0000]: JavaScript execution error: Incorrect Timestamp returned.
CREATE OR REPLACE PROCEDURE simple_stored_procedure_example(awesome_argument VARCHAR)
returns TIMESTAMPNTZ
language javascript
as
$$
var cmd = "SELECT EXTRACT_END_TIME FROM EXPLARITY_DB.EXPLARITY_SCHEMA.DEMO_CONTROL_TABLE WHERE PIPELINE_NAME =:1";
var sql = snowflake.createStatement(
{sqlText: cmd,
binds: ['awesome_argument']
}
);
var result1 = sql.execute();
return result1;
$$;
CALL simple_stored_procedure_example('pipeline1');
It can be done like this:
create or replace table tst_tbl(c1 timestamp);
insert into tst_tbl values ('2021-02-02 10:00:00.000');
create or replace procedure my_test(myarg VARCHAR)
returns TIMESTAMP
language javascript
as
$$
var cmd = "SELECT c1 FROM tst_tbl";
var sql = snowflake.createStatement({sqlText: cmd});
var resultSet = sql.execute();
resultSet.next();
my_date = resultSet.getColumnValue(1);
return my_date;
$$;
call my_test('test');
I get 1 row back as expected.
Related
I have written a stored procedure which will be called from python. The stored procedure needs to insert the variant data into my table if the id doesn't exist or update the existing variant data where there is a match for the id. The id will be passed the way the variant data is, but for now I am just trying to get it working with a hardcoded id. The stored procedure gets called successfully from python, but then nothing gets inserted or updated in the stored procedure and the stored procedure doesn't give me an error. I am not sure if I am doing something wrong or the...
if exists (select * from my_database_table where my_variant_data:id::varchar = '123456')
... part is being ignored because it isn't supported. I haven't been able to find anything in the documentation to prove or disprove this. Does anyone know?
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var insert_update_query = `
if exists (select * from my_database_table where my_variant_data:id::varchar = '123456')
begin
update my_database_table SET my_variant_data = parse_json(:1)) WHERE my_variant_data:id::varchar = '123456'
end
else
begin
insert into my_database_table(my_variant_data) select (parse_json(:1));
end
`
var result = "";
try {
var sql_insert_update_query = snowflake.createStatement({
sqlText: insert_update_query
});
var insert_update_query_result = sql_insert_update_query.execute();
result += "\n Query succeeded";
} catch (err) {
result += "\n Query failed failed: " + err.code + "\n State: " + err.state;
result += "\n Message: " + err.message;
result += "\n Stack Trace:\n" + err.stackTraceTxt;
}
return result;
$$
;
I have tested the insert and update parts of the query in the stored procedure individually and they work fine.
Insert - works as expected.
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var sql_command = "insert into my_database_table(my_variant_data) select (parse_json(:1));";
var sql = snowflake.createStatement( {sqlText: sql_command, binds:[JSON.stringify(variant_data)]});
var resultSet = sql.execute();
return sql_command;
$$
;
Update - works as expected.
create or replace procedure my_stored_procedure("variant_data" variant)
returns string
language javascript
strict
execute as owner
as
$$
var sql_command = "UPDATE my_database_table SET my_variant_data = parse_json(:1)) WHERE my_variant_data:id::varchar = '123456'";
var sql = snowflake.createStatement( {sqlText: sql_command, binds:[JSON.stringify(variant_data)]});
var resultSet = sql.execute();
$$
;
Given the CODE executed needs to be valid runs on the console SQL, which this if is not, and it is fundamentally a MERGE command I would suggest flipping the code into a MERGE:
MERGE INTO my_database_table USING my_database_table
ON my_variant_data:id::varchar = '123456'
WHEN MATCHED THEN UPDATE SET my_variant_data = parse_json(:1))
WHEN NOT MATCHED THEN INSERT (my_variant_data) VALUES (parse_json(:1));
otherwise if you are want it in SP space, then I would be inclinded to break the code into a SELECT x INTO varaible FROM blar pattern and then have the IF be in SP and pick between the two blocks of SQL to run. But given it's just a merge, I would again still, do a merge.
I am creating a stored procedure in snowflake. I want to update ORG_ID column OF EMPLOYEE table with value 'ZZ'. Here is my code, not sre where am I going wrong.
CREATE OR REPLACE PROCEDURE NEW(VAL VARCHAR)
RETURNS string
LANGUAGE JAVASCRIPT
AS
$$
let sql_command = `update EMPLOYEE set ORG_ID = ${VAL}`;
snowflake.execute({sqlText: sql_command});
return 'success';
$$;
call NEW('ZZ');
Variables should be bound:
CREATE OR REPLACE PROCEDURE NEW(VAL VARCHAR)
RETURNS string
LANGUAGE JAVASCRIPT
AS
$$
let sql_command = `update EMPLOYEE set ORG_ID = :1`;
snowflake.execute({sqlText: sql_command, binds: [VAL]});
return 'success';
$$;
Related: Binding Variables
String interpolation could also be used but it is prone to SQL Injection :
let sql_command = `update EMPLOYEE set ORG_ID = '${VAL}'`;
I have a snowflake table being used to store records of sql being executed by a stored procedure and any error messages. The records in this table are being saved as a string with special chars escaped with javascripts escape('log') function. I then want to create a view to this log table that will return the records in an easily readable format.
My first attempt at this was to create an additional stored procedure to query the log table, pass the record to the unescape() function, then return it. Calling this procedure works as intended but we can't then create a view of this data say with something like
create view log_view as
select (call UNESCAPE_PROC());
The other idea was to use a UDF rather than a stored procedure. However this also fails as we can't execute sql code with a javascript UDF. This post touches on this idea.
My question is how can I record these executed statements in a table in such a way as they can be later viewed in a plain text readable format. I've outlined my attempts below but perhaps my approach is flawed, open to any suggestions.
Minimal working example below
Sample log table and procedure to write a statement to said table
create or replace table event_table(
event varchar,
event_stamp timestamp
);
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+escape(to_log)+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ escape(to_log)
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Stored procedure to return readable log records
CREATE OR REPLACE PROCEDURE UNESCAPE_PROC()
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP will select a chosen column from event_table table, pass it to javascripts unescape() fn and return it'
EXECUTE AS CALLER
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$;
call UNESCAPE_PROC();
Which returns the records in a readable form as expected
However this of course wont work as part of a view eg.
-- define a view calling this procedure??
create view log_view as
select (call UNESCAPE_PROC());
Javascript user defined function can be used in a view like this, however it cannot be used to execute sql as in the stored procedures
-- use a UDF instead
CREATE OR REPLACE FUNCTION UNESCAPE_UDF()
RETURNS string
LANGUAGE JAVASCRIPT
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$
;
select UNESCAPE_UDF();
Stored procedures will solve one half of my problem for me, whilst UDF's will solve the other half. How can I combine the functionality of these two methods to solve this issue?
A much cleaner approach using parameters binding:
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', '${str_stamp}');`;
sql = `INSERT INTO event_table (event,event_stamp)
VALUES(?, try_to_timestamp(?));`;
var stmnt = snowflake.createStatement({sqlText: sql, binds:[to_log, str_stamp]});
stmnt.execute();
return "logged: "+ to_log
$$;
Call:
call insert_to_event(current_timestamp());
-- logged: insert into dummy_table values(2, 'Bill', '2022-02-03T17:45:44.140Z');
select * from event_table;
Found a solution/workaround.
Rather than using javascripts escape/unescape functions to remove special chars from the logs, we use a regex replace eg.
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
to_log = to_log.replace(/[`~!##$%^&*|+=?'"<>\{\}\[\]\\\/]/gi, '');
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+to_log+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ to_log
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Which writes to the log table in an easily readable format with no need for additional stored procedures/UDF's.
I am executing
CALL LOAD_CUSTDIM_HOUSTONS:
This does the below:
I am running this code to get unique Customer Details such as Consumer No and Customer Name from STG_HOUSTON table and Get Max(CUST_DImKEY) from the DIM_CUSTOMER Key for the CUSTOMER Name from above i.e Houston's.
Next I am checking if the Max key value is null i mean no match for the CUSTOMER so Key value first time to be always 1 next time it will be max value + 1.
Once we get this bind this value and the select where clause will get the unique customer and insert into Customer Dimension table.
Stored procedure looks like this:
create or replace procedure LOAD_CUSTDIM_HOUSTONS()
returns varchar not null
language javascript
as
$$
var return_value = "";
try
{
var SQL_SMT = "SELECT DISTINCT CONSUMER_NUMBER,CUSTOMER_NAME FROM STG_HOUSTONS ORDER BY CONSUMER_NUMBER ASC";
var stmt1 = snowflake.createStatement(
{
sqlText: SQL_SMT
}
);
/* Creates result set */
var res1 = stmt1.execute();
while (res1.next())
{
var var_ConsumNo=res1.getColumnValue(1);
var var_custname=res1.getColumnValue(2);
return_value =" Inside While";
var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
var res2 = t_stmt.execute();
res2.next();
var_Custkey =res2.getColumnValue(1)
return_value= return_value + var_Custkey;
if (var_CustKey == null)
{
var_CustdimKey =1
}
else
{
var_CustdimKey=var_CustKey+1
}
return_value= return_value + var_CustdimKey.toString();
var SQL_INSDIMCUS="INSERT INTO DIM_CUSTOMER(CUSTOMER_KEY,CUSTOMER_ID,CUSTOMER_NAME,CONSUMER_ID,CONSUMER_NAME,CUSTOMER_IND,TODATE)" + " SELECT DISTINCT :1, CUSTOMER_NUMBER,CUSTOMER_NAME,CONSUMER_NUMBER,CONSUMER_SHPPNG_FNAME + ' ' + CONSUMER_SHPPNG_LNAME AS CONSUMER_NAME, FALSE, NULL FROM STG_HOUSTONS WHERE CONSUMER_NUMBER = :2 ;";
var stmt = snowflake.createStatement({sqlText: SQL_INSDIMCUS,binds: [var_CustdimKey,var_ConsumNo]} );
var res3 = stmt.execute();
result = "COMPLETED SUCCESSFULLY!";
}
}
catch (err)
{ result = "Failed!";
var time_st = snowflake.execute( {sqlText: `SELECT CURRENT_TIMESTAMP;`} );
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
});
}
return return_value;
$$
;
This is the function I am using it to get max(DIM CUST KEY).
create or replace function GET_CUSTDIMKEYS ( CUSTOMERNAME varchar )
returns bigint
as 'Select max(CUSTOMER_KEY)
from PUBLIC.DIM_CUSTOMER
where CUSTOMER_NAME = CUSTOMERNAME ';
I tried with Binding variable / direct substitution of the values to dynamically create the SQL and execute it.
Using DBeaver21.1.1 IDE to create snowflake stored procedure and functions
This is the execution log Created by the IDE.
2021-10-05 20:44:48.030 - SQL Error [100183] [P0000]: Execution error in store procedure LOAD_CUSTDIM_HOUSTONS:
Unsupported type for binding argument 2undefined
At Snowflake.execute, line 49 position 14
org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [100183] [P0000]: Execution error in store procedure LOAD_CUSTDIM_HOUSTONS:
Unsupported type for binding argument 2undefined
At Snowflake.execute, line 49 position 14
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCPreparedStatementImpl.executeStatement(JDBCPreparedStatementImpl.java:208)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:510)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$0(SQLQueryJob.java:441)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:428)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:813)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3280)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:118)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:116)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:4624)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Line 49 is:
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
The error message:
Unsupported type for binding argument 2undefined
means the variable passed into the binding argument is not defined.
This tells me that the variable "err" was not initialized properly. I have seen some cases that under certain conditions, like when calling a function that does not exist on an object, though exception is thrown, the argument passed into the catch block is still NULL.
You need to look into the query history for this particular session, and find out which one of the queries that ran from the SP failed, and find out why.
And in the catch block, try to check the "err" object before using its attributes, something like:
if (err) {
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
});
} else {
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS','my code', 'my state','my error', 'my stack']
});
}
This can avoid such failure.
This is in addition to Eric's answer. You should not wrap strings in single quotes when used as bind variables:
var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
This should be:
//var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
If you wrap a string type bind variable in single quotes, the single quotes become part of the variable. The practical effect is that instead of running a statement like this:
SELECT GET_CUSTDIMKEYS('MyCustomer'); -- Looks for MyCustomer
You run a statement like this:
SELECT GET_CUSTDIMKEYS('\'MyCustomer\''); -- Looks for 'MyCustomer'
The issue is first the second result set should have a while loop which was missing,
And rest of the code proceeds will solve the issue and also we have to make sure the sql statement variables, resultset variables with in the procedures are not named the same or used multiple times. And also declare or initialize variables used in the stored procedure
lessons learnt by me and happy learning.
var var_Custkey=0;
while (res2.next())
{
var_Custkey =res2.getColumnValue(1)
return_value= return_value + var_Custkey;
if (var_CustKey == 0)
{
var_CustdimKey =1
}
else
{
var_CustdimKey=var_CustKey+1
}
return_value= return_value + var_CustdimKey.toString();
var SQL_INSDIMCUS="INSERT INTO
DIM_CUSTOMER(CUSTOMER_KEY,CUSTOMER_ID,CUSTOMER_NAME,CONSUMER_ID,CONSUMER_NAME,CUSTOMER_IND,TODATE)" + " SELECT DISTINCT :1, CUSTOMER_NUMBER,CUSTOMER_NAME,CONSUMER_NUMBER,CONSUMER_SHPPNG_FNAME + ' ' + CONSUMER_SHPPNG_LNAME AS CONSUMER_NAME, FALSE, NULL FROM STG_HOUSTONS WHERE CONSUMER_NUMBER = :2 ;";
var stmt = snowflake.createStatement({sqlText: SQL_INSDIMCUS,binds: [var_CustdimKey,var_ConsumNo]} );
var res3 = stmt.execute();
}
I am trying to create a stored procedure in Snowflake that pivot's the data in a derived table. The pivot columns are dynamic in nature. I have found a way to do this through parameter passing. I am trying to do the same without passing any parameters but the code is not working.
My approach has been to apply a loop at the table from which I am extracting the columns into a variable. Then pass this variable in the pivot. The while loop by itself seems to be working fine. The error comes when this variable is being passed to the pivot.
The code I am using:
CREATE OR REPLACE PROCEDURE LOOP_EXMPL_5()
returns varchar
language javascript
as
$$
var column1 = 'qwerty';
var command = `SELECT ATTR_NAME FROM TBL_DIM`;
var stmt = snowflake.createStatement({sqlText: command});
var rs = stmt.execute();
while (rs.next())
{
var column1 = column1.concat(",","'",rs.getColumnValue(1),"'");
}
var column2 = column1
var command_1 = `CREATE OR REPLACE VIEW HIERARCHY_VIEW_2 AS SELECT * FROM (SELECT MSTR.PROD_CODE AS
PROD_CODE,DIM.ATTR_NAME AS ATTR_NAME,MSTR.ATTR_VALUE AS ATTR_VALUE FROM TBL_DIM DIM INNER JOIN
TBL_MSTR MSTR ON DIM.ATTR_KEY=MSTR.ATTR_KEY ) Q
PIVOT (MAX (Q.ATTR_VALUE) FOR Q.ATTR_NAME IN ('${column2}'))
AS P
ORDER BY P.PROD_CODE;`;
var stmt_1 = snowflake.createStatement({sqlText: command_1});
var rs_1 = stmt_1.execute();
return 'success'
$$;
The error I am getting:
Execution error in store procedure LOOP_EXMPL_5: SQL compilation error: syntax error line 2 at position 73 unexpected 'Region'. At Statement.execute, line 16 position 21.
The variable value that is being passed:
qwerty,'Region','Sub-Region','Division','Company-Product','Company-Mfg','Company-Ship From to Customer','Business Unit','Category','Sub-Category','Segment','Sub-Segment','Brand','Aggregate Brand','Sub-Brand','PPG'
I will remove the qwerty part in the SQL somehow.
Here is the working code. Thanks a lot for your help Felipe and Greg.
CREATE OR REPLACE PROCEDURE LOOP_EXMPL_9()
returns varchar
language javascript
as
$$
var column1 = "";
var command = `SELECT ATTR_NAME FROM TBL_DIM`;
var stmt = snowflake.createStatement({sqlText: command});
var rs = stmt.execute();
while (rs.next())
{
if (column1 != "") column1 += ",";
column1 += `'${rs.getColumnValue (1)}'`;
}
var column2 = column1;
var command_1 = `CREATE OR REPLACE VIEW HIERARCHY_VIEW_2 AS SELECT * FROM (SELECT
MSTR.PROD_CODE AS PROD_CODE,DIM.ATTR_NAME AS ATTR_NAME,MSTR.ATTR_VALUE AS ATTR_VALUE
FROM TBL_DIM DIM INNER JOIN TBL_MSTR MSTR ON DIM.ATTR_KEY=MSTR.ATTR_KEY ) Q
PIVOT (MAX (Q.ATTR_VALUE) FOR Q.ATTR_NAME IN (${column2}))
AS P
ORDER BY P.PROD_CODE;`;
var stmt_1 = snowflake.createStatement({sqlText: command_1});
var rs_1 = stmt_1.execute();
return 'success'
$$;
For your objectives, see:
https://medium.com/snowflake/dynamic-pivots-in-sql-with-snowflake-c763933987c
How to pivot on dynamic values in Snowflake
To debug the current problem: column1.concat(",","'",rs.getColumnValue(1),"'"); is not escaping and quoting the column names correctly.
Set the UDF to return that value and then the value of var command_1 so you can debug appropriately.
A good way to escape the columns:
select '\\''
|| listagg(distinct pivot_column, '\\',\\'') within group (order by pivot_column)
|| '\\''
And then use:
for pivot_column in (${col_list}))
You can start with an empty string:
var column1 = "";
You can then concatenate the column list like this:
if (column1 != "") column1 += ",";
column1 += `"${rs.getColumnValue(1)}"`);
The reason you're getting the SQL syntax error is that the column names are in single quotes when they should be in double quotes.