Save Sparql results into virtuoso conductor - save

I want to save results of a Sparql query :
SELECT ?items ?head
FROM <http://localhost:8890/diplomatiki#>
WHERE {
?exeiId <http://localhost:8890/schemas/diplomatiki/itemid> ?items;
<http://localhost:8890/schemas/diplomatiki/headid> ?head
}
into a table or as a linked Data locally (virtuoso conductor), How can Ι do it.

you could create your own procedure.
like:
create procedure
INSERT_SPARQL_RESULT
(
in query VARCHAR
)
{
DECLARE state, msg, descs, rows any;
exec('SPARQL ' || query, state, msg, vector (), 0, descs, rows);
declare i INTEGER;
for(i:=0;i<LENGTH(rows);i:=i+1){
INSERT INTO tablename VALUES (rows[i][0], rows[i][1]);
}
};
It is important to add 'SPARQL ' before your query.
After successfully creating your procedure:
CALL INSERT_SPARQL_RESULT('
SELECT ?items ?head
FROM <http://localhost:8890/diplomatiki#>
WHERE {
?exeiId <http://localhost:8890/schemas/diplomatiki/itemid> ?items;
<http://localhost:8890/schemas/diplomatiki/headid> ?head
}');

Related

update a column inside a stored procedure

I have two snowflake tables as below
TEST1
create OR REPLACE table TEST1 (
id varchar(100),
name varchar(100),
org VARCHAR(64)
);
INSERT INTO TEST1 values (100, 'ABC', null);
INSERT INTO TEST1 values (200, 'XYZ', null);
INSERT INTO TEST1 values (300, 'VBN', null);
CONTROL
create OR REPLACE table CONTROL (
KEY_COLUMNS VARCHAR,
TABLE_NAME VARCHAR,
org VARCHAR
);
INSERT INTO CONTROL values ('id,name, address','TEST1','Z');
INSERT INTO CONTROL values ('id,name, address','TEST2','T');
I have created a stored procedure which updates column 'org' in TEST1 table with the values from CONTROL table (column org) where table name is TEST1.
CREATE OR REPLACE PROCEDURE TEST(source_tbl VARCHAR)
RETURNS string
LANGUAGE SQL
AS
$$
DECLARE
query1 STRING;
BEGIN
QUERY1 := 'update TEST1 set ORG = (select org from CONTROL WHERE TABLE_NAME = ''source_tbl'');';
EXECUTE IMMEDIATE :QUERY1;
RETURN :QUERY1;
END;
$$;
call TEST((select TABLE_NAME from CONTROL WHERE TABLE_NAME = 'TEST1'));
expected output
My output
The parameter should be bound variable instead of passing as the string constant ''source_tbl'':
CREATE OR REPLACE PROCEDURE TEST(source_tbl VARCHAR)
RETURNS string
LANGUAGE SQL
AS
$$
DECLARE
query1 STRING;
BEGIN
QUERY1 := 'update TEST1 set ORG = (select org from CONTROL WHERE TABLE_NAME = ?);';
EXECUTE IMMEDIATE :QUERY1 USING (SOURCE_TBL);
RETURN :QUERY1;
END;
$$
Related: EXECUTE IMMEDIATE:
EXECUTE IMMEDIATE '<string_literal>'
[ USING (bind_variable_1 [, bind_variable_2 ...] ) ] ;

Executing sql from a javascipt UDF

I have a snowflake table being used to store records of sql being executed by a stored procedure and any error messages. The records in this table are being saved as a string with special chars escaped with javascripts escape('log') function. I then want to create a view to this log table that will return the records in an easily readable format.
My first attempt at this was to create an additional stored procedure to query the log table, pass the record to the unescape() function, then return it. Calling this procedure works as intended but we can't then create a view of this data say with something like
create view log_view as
select (call UNESCAPE_PROC());
The other idea was to use a UDF rather than a stored procedure. However this also fails as we can't execute sql code with a javascript UDF. This post touches on this idea.
My question is how can I record these executed statements in a table in such a way as they can be later viewed in a plain text readable format. I've outlined my attempts below but perhaps my approach is flawed, open to any suggestions.
Minimal working example below
Sample log table and procedure to write a statement to said table
create or replace table event_table(
event varchar,
event_stamp timestamp
);
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+escape(to_log)+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ escape(to_log)
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Stored procedure to return readable log records
CREATE OR REPLACE PROCEDURE UNESCAPE_PROC()
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP will select a chosen column from event_table table, pass it to javascripts unescape() fn and return it'
EXECUTE AS CALLER
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$;
call UNESCAPE_PROC();
Which returns the records in a readable form as expected
However this of course wont work as part of a view eg.
-- define a view calling this procedure??
create view log_view as
select (call UNESCAPE_PROC());
Javascript user defined function can be used in a view like this, however it cannot be used to execute sql as in the stored procedures
-- use a UDF instead
CREATE OR REPLACE FUNCTION UNESCAPE_UDF()
RETURNS string
LANGUAGE JAVASCRIPT
AS
$$
unescape_sql =
`select event from event_table`
var errs_res = [];
try {
all_logs = snowflake.execute(
{ sqlText: unescape_sql }
);
// iterate over all columns
while (all_logs.next()) {
errs_res.push(all_logs.getColumnValue(1));
}
return unescape(errs_res)
}
catch(err){
return "something went wrong: " + err
}
$$
;
select UNESCAPE_UDF();
Stored procedures will solve one half of my problem for me, whilst UDF's will solve the other half. How can I combine the functionality of these two methods to solve this issue?
A much cleaner approach using parameters binding:
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', '${str_stamp}');`;
sql = `INSERT INTO event_table (event,event_stamp)
VALUES(?, try_to_timestamp(?));`;
var stmnt = snowflake.createStatement({sqlText: sql, binds:[to_log, str_stamp]});
stmnt.execute();
return "logged: "+ to_log
$$;
Call:
call insert_to_event(current_timestamp());
-- logged: insert into dummy_table values(2, 'Bill', '2022-02-03T17:45:44.140Z');
select * from event_table;
Found a solution/workaround.
Rather than using javascripts escape/unescape functions to remove special chars from the logs, we use a regex replace eg.
create or replace procedure insert_to_event(stamp string)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
COMMENT = 'SP to log an event message with timestamp to event_table'
EXECUTE AS CALLER
AS
$$
// some variables to log in our event table
var str_stamp = (new Date()).toISOString();
to_log = `insert into dummy_table values(2, 'Bill', `+STAMP+`);`;
to_log = to_log.replace(/[`~!##$%^&*|+=?'"<>\{\}\[\]\\\/]/gi, '');
sql =
`INSERT INTO event_table (
event,
event_stamp
)
VALUES
('`+to_log+`', to_timestamp('`+str_stamp+`'));`;
var stmnt = snowflake.createStatement({ sqlText: sql });
stmnt.execute();
return "logged: "+ to_log
$$;
call insert_to_event(current_timestamp());
select * from event_table;
Which writes to the log table in an easily readable format with no need for additional stored procedures/UDF's.

Getting a count of rows inserted in a Snowflake procedure

I'm using a procedure to insert rows into a table. I want to return the count of rows inserted, but I can't figure out how to turn on the variable substitution from inside a procedure. I've tried all of the following, but they all return errors:
snowflake.execute({sqlText: '!set variable_substitution=True'});
snowflake.execute({sqlText: 'set variable_substitution=True'});
snowflake.execute({sqlText: '-o variable_substitution=True'});
How do I turn this option on so I can run "select &__rowcount;" and get my count back?
Here's code for a test procedure, if that helps:
CREATE OR REPLACE PROCEDURE TEST_OF_GETTING_COUNTS()
RETURNS VARCHAR
LANGUAGE javascript
CALLED ON NULL INPUT
AS
$$
// Turn on variable substitution
snowflake.execute(
{sqlText: '!set variable_substitution=True'}
);
// Prepare SQL to identify tables to be updated
snowflake.execute({sqlText: 'SELECT 1'});
// Now get the count of rows selected
var getCount = snowflake.createStatement({sqlText: "SELECT &__ROWCOUNT;"});
var rowCountResultSet = getCount.execute();
while (rowCount.next()) {
rowCount= rowCountResultSet.getColumnValue(1);
}
// Turn off variable substitution
snowflake.execute({sqlText: '!set variable_substitution=False'});
return rowCount;
$$;
CALL TEST_OF_GETTING_COUNTS();
NickW, in his comment above, gave me a link to a page in Snowflake's documentation that lists all the JavaScript methods available for the snowflake, Statement, ResultSet, and SfDate objects. It gave me exactly what I needed: the getRowCount() method.

How to debug a Snowflake stored procedure?

I am using the Snowflake Cloud Database, please help me with a tool to debugging the procedures or functions.
The following Snowflake Javascript Stored Procedure is a template I use to get started on a new Stored Procedures. It contains plenty of debugging tricks, such as:
it has a "where am I?" variable which gives you a understanding of where in the code you are
it gathers information in an array as the process moves along
it returns that array to the standard output of the call command
it has a "good start" of an exception block, who's contents also get pushed out to standard output on a call of the stored procedure, should it fail.
Something I've been meaning to add is to set a query tag in the code as well, that'd be helpful when reviewing query history, to easily identify the SQL commands that were used in the execution of the Stored Procedure.
This "ties into" the final "debugging trick" - you should always review the query history (actual queries your code executed) when developing stored procedures in a development or test environment, particularly when you are building dynamic SQL statements. Reviewing your query history is a must-do and will show you exactly the commands run and the order of operations of them running.
Here's the code with the sample table it uses, I hope it helps...Rich
CREATE OR REPLACE TABLE test_scripts (
load_seq number,
script varchar(2000)
);
INSERT INTO test_scripts values
(1, 'SELECT current_timestamp();'),
(2, 'SELECT current_warehouse();'),
(3, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(4, 'SELECT current_date();'),
(5, 'SELECT current_account();'),
(6, 'SELECT COUNT(*) FROM snowflake.account_usage.tables;'),
(7, 'SELECT ''RICH'';');
select * from test_scripts;
CREATE OR REPLACE PROCEDURE sp_test(p1 varchar, p2 varchar)
RETURNS ARRAY
LANGUAGE javascript
EXECUTE AS caller
AS
$$
//note: you can change the RETURN to VARCHAR if needed
// but the array "looks nice"
try {
var whereAmI = 1;
var return_array = [];
var counter = 0;
var p1_str = "p1: " + P1
var p2_str = "p2: " + P2
var load_seq = P1;
var continue_flag = P2;
whereAmI = 2;
return_array.push(p1_str)
return_array.push(p2_str)
whereAmI = 3;
//which SQL do I want to run?
if (continue_flag=="YES") {
return_array.push("query 1")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq >= " + load_seq + " order by 1, 2;";
}
else {
return_array.push("query 2")
var sqlquery = "SELECT * FROM test_scripts WHERE load_seq = " + load_seq + " order by 1, 2;";
}
whereAmI = 4;
//begin the run of grabbing the commands
var stmt = snowflake.createStatement( {sqlText: sqlquery} );
var rs = stmt.execute();
whereAmI = 5;
// Loop through the results, processing one row at a time...
while (rs.next()) {
counter = counter + 1;
var tmp_load_seq = rs.getColumnValue(1);
var tmp_script = rs.getColumnValue(2);
var tmp_rs = snowflake.execute({sqlText: tmp_script});
tmp_rs.next();
var tmp_col1 = tmp_rs.getColumnValue(1);
return_array.push("tmp_col1: " + tmp_col1)
}
whereAmI = 6;
return_array.push("end process - counter: " + counter)
return return_array;
}
catch (err) {
return_array.push("error found")
return_array.push(whereAmI)
return_array.push(err)
return return_array;
}
$$;
CALL sp_test(3, 'NO');
I do not believe there is any editor / debugger for stored procedures for Snowflake. Few options:
You can break your code to smaller parts and try to troubleshoot
Use a log table and insert into log table often, so you can look at the log table to find out what went wrong
unfortunately there isn't one environment to rule them all
1. write your SQL in a Worksheet or Editor
2. write your SPROC code in a JS enabled editor
3. merge them together in a Worksheet or Editor
4. Unit test in SPROCS as shown above by #Rich Murmane
I normally just write SPROCS in a Worksheet but it isnt optimal
Logging is your friend here, as there is no debugger. In general finding and using a debugger for db stored procedures is hard to pull off. Not impossible, just unlikely.
This is a decent alternative:
CREATE or replace PROCEDURE do_log(MSG STRING)
RETURNS STRING
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS $$
//see if we should log - checks for do_log = true session variable
try{
var foo = snowflake.createStatement( { sqlText: `select $do_log` } ).execute();
} catch (ERROR){
return; //swallow the error, variable not set so don't log
}
foo.next();
if (foo.getColumnValue(1)==true){ //if the value is anything other than true, don't log
try{
snowflake.createStatement( { sqlText: `create temp table identifier ($log_table) if not exists (ts number, msg string)`} ).execute();
snowflake.createStatement( { sqlText: `insert into identifier ($log_table) values (:1, :2)`, binds:[Date.now(), MSG] } ).execute();
} catch (ERROR){
throw ERROR;
}
}
$$
;
Then in the stored procedure, you want to debug add a log function at the top:
function log(msg){
snowflake.createStatement( { sqlText: `call do_log(:1)`, binds:[msg] } ).execute();
}
Then above the call to the stored procedure:
set do_log = true; --true to enable logging, false (or undefined) to disable
set log_table = 'my_log_table'; --The name of the temp table where log messages go
Then in the actual stored procedure you need to add some logging lines:
log('this is another log message');
Then call the stored procedure as you would normally. Then select from my_log_table.
Important note: this uses a temp table, so you won't be able to read from that logging table in a different Snowflake connection. This means if you're using the Worksheet editor you need to keep all this stuff on the same sheet.
"Borrowed" from: https://community.snowflake.com/s/article/Snowflake-Stored-Procedure-Logging

DB2 - how to call a stored procedure that returns a result set in another user defined table function

I have a db2 stored procedure that takes in some parameters, gets some data from somewhere and then returns a result set through a cursor.
Now I want to write a table function in db2, that will call this stored procedure, read from the result set and return the data in the result set as a table (eventually I want to use this table function in a join).
I would like to know if this is permitted in db2 (we're using DB2 v10.5), i.e. execute a stored procedure in a table function and fetch and read from the result set from the stored procedure. If so, what is the right syntax for calling the stored procedure and reading the result set inside a table function in db2? Thanks!
Yes, it's possible. See the example below.
--#SET TERMINATOR #
CREATE OR REPLACE PROCEDURE TEST_PROC(P_TABSCHEMA VARCHAR(128))
DYNAMIC RESULT SETS 1
READS SQL DATA
BEGIN
DECLARE C1 CURSOR WITH HOLD WITH RETURN FOR
SELECT TABSCHEMA, TABNAME, COLCOUNT
FROM SYSCAT.TABLES
WHERE TABSCHEMA=P_TABSCHEMA;
OPEN C1;
END#
--CALL TEST_PROC('SYSCAT')#
CREATE OR REPLACE FUNCTION TEST_PROC(P_TABSCHEMA VARCHAR(128))
RETURNS TABLE (
TABSCHEMA VARCHAR(128)
, TABNAME VARCHAR(128)
, COLCOUNT INT
)
READS SQL DATA
BEGIN
DECLARE SQLSTATE CHAR(5);
DECLARE V_TABSCHEMA VARCHAR(128);
DECLARE V_TABNAME VARCHAR(128);
DECLARE V_COLCOUNT INT;
DECLARE V1 RESULT_SET_LOCATOR VARYING;
CALL TEST_PROC(P_TABSCHEMA);
ASSOCIATE RESULT SET LOCATOR (V1) WITH PROCEDURE TEST_PROC;
ALLOCATE C1 CURSOR FOR RESULT SET V1;
L1: LOOP
FETCH C1 INTO V_TABSCHEMA, V_TABNAME, V_COLCOUNT;
IF SQLSTATE<>'00000' THEN LEAVE L1; END IF;
PIPE(V_TABSCHEMA, V_TABNAME, V_COLCOUNT);
END LOOP L1;
CLOSE C1;
RETURN;
END#
SELECT * FROM TABLE(TEST_PROC('SYSCAT'))#
You need to create the DB2 table-function as follows:
CREATE FUNCTION database_schema.function_name ( IN_PARTID VARCHAR(1000) )
RETURNS TABLE ( PARTNO CHAR(25), PARTDS CHAR(30), QUANTITY INT )
BEGIN
RETURN SELECT PARTNO , PARTDS , CASE WHEN QUANTITY > 0 THEN QUANTITY ELSE 0 END QUANTITY
FROM
(
SELECT PARTNO
,MAX(PARTDS) AS PARTDS
,SUM(QUANTITY) AS QUANTITY
FROM database_schema.table_name
WHERE 1=1
AND PARTID = (CAST(IN_PARTID AS INT))
GROUP BY PARTNO
) AA;
END;
Then invoke the table-function as join or straight SQL:
SELECT partno,partds,quantity
FROM TABLE(database_schema.function_name('parameter_1'))

Resources