Snowflake - using binding variables - stored-procedures

I am trying to call the below procedure and I'm getting an unexpected ':' error due to binding variable.
If I am using simple JS concatenation instead,its working. But please let me know if there is any alternative using binding variables.Also in which all part of a query binding variables will work?
CREATE OR REPLACE PROCEDURE test_proc ()
RETURNS STRING
LANGUAGE JAVASCRIPT
AS
$$
var V_TEMP = '123'
var V_SQL = `CREATE TABLE TEST_TABLE:1 AS
SELECT A.FIRST_NAME,
A.LAST_NAME
FROM
MARKET A
WHERE first_name>3000;`
var EXEC_V_SQL = snowflake.createStatement(
{
sqlText: V_SQL, binds: [ V_TEMP ]
}
)
var result1 = EXEC_V_SQL.execute();
$$; ```

The issue is you are using a bind variable for an object name without using the IDENTIFIER() function. Try something like this instead:
CREATE OR REPLACE PROCEDURE test_proc ()
RETURNS STRING
LANGUAGE JAVASCRIPT
AS
$$
var V_TEMP = 'TEST_TABLE123'
var V_SQL = `CREATE TABLE IDENTIFIER(:1) AS
SELECT A.FIRST_NAME,
A.LAST_NAME
FROM
MARKET A
WHERE first_name>3000;`
var EXEC_V_SQL = snowflake.createStatement(
{
sqlText: V_SQL, binds: [ V_TEMP ]
}
)
var result1 = EXEC_V_SQL.execute();
$$;

Related

Execution error in stored procedure : Unsupported type for binding argument 2undefined

I am executing
CALL LOAD_CUSTDIM_HOUSTONS:
This does the below:
I am running this code to get unique Customer Details such as Consumer No and Customer Name from STG_HOUSTON table and Get Max(CUST_DImKEY) from the DIM_CUSTOMER Key for the CUSTOMER Name from above i.e Houston's.
Next I am checking if the Max key value is null i mean no match for the CUSTOMER so Key value first time to be always 1 next time it will be max value + 1.
Once we get this bind this value and the select where clause will get the unique customer and insert into Customer Dimension table.
Stored procedure looks like this:
create or replace procedure LOAD_CUSTDIM_HOUSTONS()
returns varchar not null
language javascript
as
$$
var return_value = "";
try
{
var SQL_SMT = "SELECT DISTINCT CONSUMER_NUMBER,CUSTOMER_NAME FROM STG_HOUSTONS ORDER BY CONSUMER_NUMBER ASC";
var stmt1 = snowflake.createStatement(
{
sqlText: SQL_SMT
}
);
/* Creates result set */
var res1 = stmt1.execute();
while (res1.next())
{
var var_ConsumNo=res1.getColumnValue(1);
var var_custname=res1.getColumnValue(2);
return_value =" Inside While";
var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
var res2 = t_stmt.execute();
res2.next();
var_Custkey =res2.getColumnValue(1)
return_value= return_value + var_Custkey;
if (var_CustKey == null)
{
var_CustdimKey =1
}
else
{
var_CustdimKey=var_CustKey+1
}
return_value= return_value + var_CustdimKey.toString();
var SQL_INSDIMCUS="INSERT INTO DIM_CUSTOMER(CUSTOMER_KEY,CUSTOMER_ID,CUSTOMER_NAME,CONSUMER_ID,CONSUMER_NAME,CUSTOMER_IND,TODATE)" + " SELECT DISTINCT :1, CUSTOMER_NUMBER,CUSTOMER_NAME,CONSUMER_NUMBER,CONSUMER_SHPPNG_FNAME + ' ' + CONSUMER_SHPPNG_LNAME AS CONSUMER_NAME, FALSE, NULL FROM STG_HOUSTONS WHERE CONSUMER_NUMBER = :2 ;";
var stmt = snowflake.createStatement({sqlText: SQL_INSDIMCUS,binds: [var_CustdimKey,var_ConsumNo]} );
var res3 = stmt.execute();
result = "COMPLETED SUCCESSFULLY!";
}
}
catch (err)
{ result = "Failed!";
var time_st = snowflake.execute( {sqlText: `SELECT CURRENT_TIMESTAMP;`} );
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
});
}
return return_value;
$$
;
This is the function I am using it to get max(DIM CUST KEY).
create or replace function GET_CUSTDIMKEYS ( CUSTOMERNAME varchar )
returns bigint
as 'Select max(CUSTOMER_KEY)
from PUBLIC.DIM_CUSTOMER
where CUSTOMER_NAME = CUSTOMERNAME ';
I tried with Binding variable / direct substitution of the values to dynamically create the SQL and execute it.
Using DBeaver21.1.1 IDE to create snowflake stored procedure and functions
This is the execution log Created by the IDE.
2021-10-05 20:44:48.030 - SQL Error [100183] [P0000]: Execution error in store procedure LOAD_CUSTDIM_HOUSTONS:
Unsupported type for binding argument 2undefined
At Snowflake.execute, line 49 position 14
org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [100183] [P0000]: Execution error in store procedure LOAD_CUSTDIM_HOUSTONS:
Unsupported type for binding argument 2undefined
At Snowflake.execute, line 49 position 14
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCPreparedStatementImpl.executeStatement(JDBCPreparedStatementImpl.java:208)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:510)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$0(SQLQueryJob.java:441)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:428)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:813)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3280)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:118)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:116)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:4624)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Line 49 is:
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
The error message:
Unsupported type for binding argument 2undefined
means the variable passed into the binding argument is not defined.
This tells me that the variable "err" was not initialized properly. I have seen some cases that under certain conditions, like when calling a function that does not exist on an object, though exception is thrown, the argument passed into the catch block is still NULL.
You need to look into the query history for this particular session, and find out which one of the queries that ran from the SP failed, and find out why.
And in the catch block, try to check the "err" object before using its attributes, something like:
if (err) {
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS',err.code, err.state,err.message, err.stackTraceTxt]
});
} else {
snowflake.execute({
sqlText: `insert into PROC_ERROR_LOG
VALUES (?,?,?,?,?)`
,binds: ['LOAD CUSTOMER DIMENSION HOUSTONS','my code', 'my state','my error', 'my stack']
});
}
This can avoid such failure.
This is in addition to Eric's answer. You should not wrap strings in single quotes when used as bind variables:
var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
This should be:
//var_custname = "'" + var_custname + "'";
var t_SQLstmt = " SELECT GET_CUSTDIMKEYS(?)" ;
var t_stmt = snowflake.createStatement ({sqlText: t_SQLstmt,binds: [var_custname]});
If you wrap a string type bind variable in single quotes, the single quotes become part of the variable. The practical effect is that instead of running a statement like this:
SELECT GET_CUSTDIMKEYS('MyCustomer'); -- Looks for MyCustomer
You run a statement like this:
SELECT GET_CUSTDIMKEYS('\'MyCustomer\''); -- Looks for 'MyCustomer'
The issue is first the second result set should have a while loop which was missing,
And rest of the code proceeds will solve the issue and also we have to make sure the sql statement variables, resultset variables with in the procedures are not named the same or used multiple times. And also declare or initialize variables used in the stored procedure
lessons learnt by me and happy learning.
var var_Custkey=0;
while (res2.next())
{
var_Custkey =res2.getColumnValue(1)
return_value= return_value + var_Custkey;
if (var_CustKey == 0)
{
var_CustdimKey =1
}
else
{
var_CustdimKey=var_CustKey+1
}
return_value= return_value + var_CustdimKey.toString();
var SQL_INSDIMCUS="INSERT INTO
DIM_CUSTOMER(CUSTOMER_KEY,CUSTOMER_ID,CUSTOMER_NAME,CONSUMER_ID,CONSUMER_NAME,CUSTOMER_IND,TODATE)" + " SELECT DISTINCT :1, CUSTOMER_NUMBER,CUSTOMER_NAME,CONSUMER_NUMBER,CONSUMER_SHPPNG_FNAME + ' ' + CONSUMER_SHPPNG_LNAME AS CONSUMER_NAME, FALSE, NULL FROM STG_HOUSTONS WHERE CONSUMER_NUMBER = :2 ;";
var stmt = snowflake.createStatement({sqlText: SQL_INSDIMCUS,binds: [var_CustdimKey,var_ConsumNo]} );
var res3 = stmt.execute();
}

UDF/Procedure with control flow that returns a table

I am kinda stuck with the limitations of UDFs/stored Procedures in Snowflake and wonder what I am missing. I feel like there has to be a solution, but I can't see it right now, so maybe som new input from stackoverflow can help me.
I want something (UDF/Procedure) that has an input and returns a table. I created a stored procedure that generates the SQl-Statement. But in the stored procedure I can only return a single value and no table.
My stored procedure is the following:
CREATE OR REPLACE PROCEDURE DWH.TEMP.getAccounts(NODE_NAME VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS OWNER
AS '
var maxLevel = 0;
function executeSQL(SQL ) {
// Prepare statement.
var stmt = snowflake.createStatement(
{
sqlText: SQL
}
);
// Execute Statement
var res = stmt.execute();
res.next();
return res.getColumnValue(1);
}
maxLevel= executeSQL(" SELECT MAX(LEVEL) as Max_Level FROM DWH.MART.VDIM_ACCOUNT");
var sqlString = "SELECT DISTINCT AccountID FROM DWH.MART.VDIM_ACCOUNT WHERE ";
var sqlWhere = "";
for (var i = 0;i<=maxLevel;i++)
{
sqlWhere += "Group_L"+ i +" in (\'"+NODE_NAME+"\')";
if(i<maxLevel)
{
sqlWhere += " OR ";
}
}
sqlString += sqlWhere;
return sqlString;
'
If I execute this with a node, for example with call GETACCOUNTS('Test') I get the result:
SELECT DISTINCT AccountID
FROM dwh.mart.vdim_account
WHERE group_l0 IN ( 'TEST' )
OR group_l1 IN ( 'TEST' )
OR group_l2 IN ( 'TEST' )
OR group_l3 IN ( 'TEST' )
OR group_l4 IN ( 'TEST' )
OR group_l5 IN ( 'TEST' )
But I want the result of above SQL-Statement returnes as a table or list. How do I do this in Snowflake?
I am stuck because I need the for loop to dynamically generate my SQL-Statement. I can only use loops in JS UDFs, but I can only execute SQL in SQL UDFs.

return timestamp value in store procedure

i'm trying to get timestamp value as result of stored procedure.
but getting error .
error message:- SQL Error [100132] [P0000]: JavaScript execution error: Incorrect Timestamp returned.
CREATE OR REPLACE PROCEDURE simple_stored_procedure_example(awesome_argument VARCHAR)
returns TIMESTAMPNTZ
language javascript
as
$$
var cmd = "SELECT EXTRACT_END_TIME FROM EXPLARITY_DB.EXPLARITY_SCHEMA.DEMO_CONTROL_TABLE WHERE PIPELINE_NAME =:1";
var sql = snowflake.createStatement(
{sqlText: cmd,
binds: ['awesome_argument']
}
);
var result1 = sql.execute();
return result1;
$$;
CALL simple_stored_procedure_example('pipeline1');
It can be done like this:
create or replace table tst_tbl(c1 timestamp);
insert into tst_tbl values ('2021-02-02 10:00:00.000');
create or replace procedure my_test(myarg VARCHAR)
returns TIMESTAMP
language javascript
as
$$
var cmd = "SELECT c1 FROM tst_tbl";
var sql = snowflake.createStatement({sqlText: cmd});
var resultSet = sql.execute();
resultSet.next();
my_date = resultSet.getColumnValue(1);
return my_date;
$$;
call my_test('test');
I get 1 row back as expected.

Snowflake Stored Procedure Dynamic Column Pivot

I am trying to create a stored procedure in Snowflake that pivot's the data in a derived table. The pivot columns are dynamic in nature. I have found a way to do this through parameter passing. I am trying to do the same without passing any parameters but the code is not working.
My approach has been to apply a loop at the table from which I am extracting the columns into a variable. Then pass this variable in the pivot. The while loop by itself seems to be working fine. The error comes when this variable is being passed to the pivot.
The code I am using:
CREATE OR REPLACE PROCEDURE LOOP_EXMPL_5()
returns varchar
language javascript
as
$$
var column1 = 'qwerty';
var command = `SELECT ATTR_NAME FROM TBL_DIM`;
var stmt = snowflake.createStatement({sqlText: command});
var rs = stmt.execute();
while (rs.next())
{
var column1 = column1.concat(",","'",rs.getColumnValue(1),"'");
}
var column2 = column1
var command_1 = `CREATE OR REPLACE VIEW HIERARCHY_VIEW_2 AS SELECT * FROM (SELECT MSTR.PROD_CODE AS
PROD_CODE,DIM.ATTR_NAME AS ATTR_NAME,MSTR.ATTR_VALUE AS ATTR_VALUE FROM TBL_DIM DIM INNER JOIN
TBL_MSTR MSTR ON DIM.ATTR_KEY=MSTR.ATTR_KEY ) Q
PIVOT (MAX (Q.ATTR_VALUE) FOR Q.ATTR_NAME IN ('${column2}'))
AS P
ORDER BY P.PROD_CODE;`;
var stmt_1 = snowflake.createStatement({sqlText: command_1});
var rs_1 = stmt_1.execute();
return 'success'
$$;
The error I am getting:
Execution error in store procedure LOOP_EXMPL_5: SQL compilation error: syntax error line 2 at position 73 unexpected 'Region'. At Statement.execute, line 16 position 21.
The variable value that is being passed:
qwerty,'Region','Sub-Region','Division','Company-Product','Company-Mfg','Company-Ship From to Customer','Business Unit','Category','Sub-Category','Segment','Sub-Segment','Brand','Aggregate Brand','Sub-Brand','PPG'
I will remove the qwerty part in the SQL somehow.
Here is the working code. Thanks a lot for your help Felipe and Greg.
CREATE OR REPLACE PROCEDURE LOOP_EXMPL_9()
returns varchar
language javascript
as
$$
var column1 = "";
var command = `SELECT ATTR_NAME FROM TBL_DIM`;
var stmt = snowflake.createStatement({sqlText: command});
var rs = stmt.execute();
while (rs.next())
{
if (column1 != "") column1 += ",";
column1 += `'${rs.getColumnValue (1)}'`;
}
var column2 = column1;
var command_1 = `CREATE OR REPLACE VIEW HIERARCHY_VIEW_2 AS SELECT * FROM (SELECT
MSTR.PROD_CODE AS PROD_CODE,DIM.ATTR_NAME AS ATTR_NAME,MSTR.ATTR_VALUE AS ATTR_VALUE
FROM TBL_DIM DIM INNER JOIN TBL_MSTR MSTR ON DIM.ATTR_KEY=MSTR.ATTR_KEY ) Q
PIVOT (MAX (Q.ATTR_VALUE) FOR Q.ATTR_NAME IN (${column2}))
AS P
ORDER BY P.PROD_CODE;`;
var stmt_1 = snowflake.createStatement({sqlText: command_1});
var rs_1 = stmt_1.execute();
return 'success'
$$;
For your objectives, see:
https://medium.com/snowflake/dynamic-pivots-in-sql-with-snowflake-c763933987c
How to pivot on dynamic values in Snowflake
To debug the current problem: column1.concat(",","'",rs.getColumnValue(1),"'"); is not escaping and quoting the column names correctly.
Set the UDF to return that value and then the value of var command_1 so you can debug appropriately.
A good way to escape the columns:
select '\\''
|| listagg(distinct pivot_column, '\\',\\'') within group (order by pivot_column)
|| '\\''
And then use:
for pivot_column in (${col_list}))
You can start with an empty string:
var column1 = "";
You can then concatenate the column list like this:
if (column1 != "") column1 += ",";
column1 += `"${rs.getColumnValue(1)}"`);
The reason you're getting the SQL syntax error is that the column names are in single quotes when they should be in double quotes.

Trying to pass parameter as binding variable in snowflake statement

Below is my stored procedure, I'm not sure as to why it keeps throwing an error. The error I get is
SQL compilation error: syntax error line XX at position XX unexpected '?'.
I have followed the documentation here but it does not seem to work for me.
This is what I have:
CREATE OR REPLACE PROCEDURE spExample(INPUT_TABLE VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
AS
$$
result = "";
try {
var sql_cmd = "SELECT * FROM ?;";
var sql_stmt = snowflake.createStatement({sqlText: sql_cmd, binds:[INPUT_TABLE]});
sql_stmt.execute();
} catch(err) {
result += "Message: " + err.message;
}
return result;
$$;
Have I made a mistake somewhere?
Above answers subject your code to sql injection attack. And of course you can bind a table name to a variable in snowflake.
Do
var sql_cmd = "SELECT * FROM IDENTIFIER('?');";
It seems I had the exact same misunderstanding as the OP. It was good to find this answer.
In any case, it's a lot more flexible & more readable to use JavaScript template literals using backticks (instead of using single quotes or double quotes). They allow you to use expression interpolation in the format of
`Some text here. ${expression} Some more text here.`
Just fill in with your variable or variables (or expression).
Here is what I tried and it executed perfectly:
CREATE OR REPLACE PROCEDURE spExample(INPUT_TABLE VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
AS
$$
result = "";
try {
var sql_cmd = "SELECT * FROM IDENTIFIER(?);";
var sql_stmt = snowflake.createStatement({sqlText: sql_cmd, binds:[INPUT_TABLE]});
sql_stmt.execute();
} catch(err) {
result += "Message: " + err.message;
}
return result;
$$;
Call spExample('PAIDBILLS');
For me, this worked, first Quotes then brackets. I think its different for different type of query , other answers worked for me for select but not for SHOW
var sql_cmd = "SHOW WAREHOUSES like '(?)';"
CREATE OR REPLACE PROCEDURE spExample(INPUT_TABLE VARCHAR)
RETURNS varchar
LANGUAGE JAVASCRIPT
execute as owner
AS
$$
result = "";
try {
var sql_cmd = "SHOW WAREHOUSES like '(?)';";
var sql_stmt = snowflake.execute({sqlText: sql_cmd, binds:[INPUT_TABLE]});
} catch(err) {
result += "Message: " + err.message;
}
return result;
$$;
Call spExample('WREHOUSE NAME');
actually yes.
Bind variable is just that, a variable. So you can do a
SELECT * FRMO MY_TABLE WHERE MY_COLUMN=?
but you can't use bind to substitute for commands or column or table names. You can however use simple JS concatenation, like
var sql_cmd = "SELECT * FROM "+INPUT_TABLE;

Resources