Processing aborted due to error Snowflake - stored-procedures

I've encountered this error:
Execution error in stored procedure: SQL execution internal error: Processing aborted due to error at Snowflake.execute
when running this script:
CREATE OR REPLACE PROCEDURE DATES_TABLE (INITIALDATE VARCHAR, FINALDATE VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
var DATESDIFF = (Date.parse(formatDate(FINALDATE)) - Date.parse(formatDate(INITIALDATE)))/ (1000 * 3600 * 24);
snowflake.execute(
{
sqlText: ` CREATE OR REPLACE TEMPORARY TABLE TEMP_DATE_RANGE AS SELECT DATE FROM (
SELECT
CAST(DATEADD (DAY, DatesDiff.n, :1) AS DATE) AS DATE
FROM (
SELECT
ROW_NUMBER() OVER (ORDER BY 1) - 1
FROM
TABLE (generator (rowcount => :3))) DatesDiff (n)
); `,
binds: [formatDate(INITIALDATE), formatDate(FINALDATE), DATESDIFF]
}
);
function formatDate(date) {
var d = new Date(date),
month = '' + (d.getMonth() + 1),
day = '' + d.getDate(),
year = d.getFullYear();
if (month.length < 2)
month = '0' + month;
if (day.length < 2)
day = '0' + day;
return [year, month, day].join('-');
}
$$
;
CALL DATES_TABLE('2021-04-01','2021-05-24');
Which when ran outside of the stored procedure, creates a table with dates between the range inputted.
Any idea why this is happening, how to sort it out?

The problem is in binding a variable to TABLE (generator (rowcount => :3)), as Snowflake expects a constant there.
Instead, you could do something like:
SELECT ROW_NUMBER() OVER (ORDER BY 1) - 1 AS rn
FROM TABLE (generator (rowcount => 1000))
QUALIFY rn < :2
I did some cleanup, and this works:
CREATE OR REPLACE PROCEDURE DATES_TABLE (INITIALDATE VARCHAR, FINALDATE VARCHAR)
RETURNS VARCHAR
LANGUAGE JAVASCRIPT
EXECUTE AS CALLER
AS
$$
var DATESDIFF = (Date.parse(formatDate(FINALDATE)) - Date.parse(formatDate(INITIALDATE)))/ (1000 * 3600 * 24);
snowflake.execute(
{
sqlText: `
CREATE OR REPLACE TEMPORARY TABLE TEMP_DATE_RANGE AS
SELECT CAST(DATEADD(DAY, DatesDiff.rn, :1) AS DATE) AS DATE
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY 1) - 1 AS rn
FROM TABLE (generator (rowcount => 1000))
QUALIFY rn < :2
)
;`
, binds: [formatDate(INITIALDATE), DATESDIFF]
}
);
function formatDate(date) {
var d = new Date(date),
month = '' + (d.getMonth() + 1),
day = '' + d.getDate(),
year = d.getFullYear();
if (month.length < 2)
month = '0' + month;
if (day.length < 2)
day = '0' + day;
return [year, month, day].join('-');
}
$$
;
CALL DATES_TABLE('2021-04-01','2021-05-24');
select * from TEMP_DATE_RANGE;
For a shorter way of generating a sequence of dates, see my answer to https://stackoverflow.com/a/66449068/132438.

Related

What am i doing wrong in writing this Snowflake function with variables?

Another user was helping me with this problem, but i'm having trouble executing it, i'm getting error:
syntax error line 3 at position 8 unexpected 'time'. syntax error line 3 at position 23 unexpected ':'. (line 3)
it seems i'm either declaring the variables wrong, actually i'm sure of that because when i comment out "time", it gives me an error with "curDay".
Here is the function i'm trying to execute;
CREATE OR REPLACE FUNCTION DB_BI_DEV.RAW_CPMS_AAR.cfn_GetShiftIDFromDateTime (dateTime TIMESTAMP_NTZ(9), shiftCalendarID int)
RETURNS table (shiftID int)
AS
$$
DECLARE
time time TIME(:dateTime);
curDay int;
prvDay int;
shiftID int;
BEGIN
SELECT TOP 1
ID,
DATEDIFF( day, BeginDate, :dateTime ) % PeriodInDays + 1,
( :curDay + PeriodInDays - 2 ) % PeriodInDays + 1
INTO :shiftCalendarID, :curDay, :prvDay
FROM RAW_CPMS_AAR.ShiftCalendar
WHERE ID = :shiftCalendarID
OR ( :shiftCalendarID IS NULL
AND Name = 'Factory'
AND BeginDate <= :dateTime )
ORDER BY BeginDate DESC;
SELECT ID into :shiftID
FROM Shift
WHERE ShiftCalendarID = #shiftCalendarID
AND ( ( FromDay = :curDay AND FromTimeOfDay <= :time AND TillTimeOfDay > :time )
OR ( FromDay = :curDay AND FromTimeOfDay >= TillTimeOfDay AND FromTimeOfDay <= :time )
OR ( FromDay = :prvDay AND FromTimeOfDay >= TillTimeOfDay AND TillTimeOfDay > :time )
);
END;
$$
This may need some small changes, but should be close to what you require:
CREATE OR REPLACE FUNCTION DB_BI_DEV.RAW_CPMS_AAR.cfn_GetShiftIDFromDateTime (dateTime TIMESTAMP_NTZ(9), shiftCalendarID int)
RETURNS table (shiftID int)
AS
$$
WITH T0 (ShiftCalendarID, CurDay, PrvDay)
AS (
SELECT TOP 1
ID AS ShiftCalendarID,
DATEDIFF( day, BeginDate, :dateTime ) % PeriodInDays + 1 AS CurDay,
( CurDay + PeriodInDays - 2 ) % PeriodInDays + 1 AS PrvDay
FROM RAW_CPMS_AAR.ShiftCalendar
WHERE ID = :shiftCalendarID
OR ( :shiftCalendarID IS NULL
AND Name = 'Factory'
AND BeginDate <= :dateTime )
ORDER BY BeginDate DESC
),
T1 (TimeValue)
AS (
SELECT TIME_FROM_PARTS(
EXTRACT(HOUR FROM :datetime),
EXTRACT(MINUTE FROM :datetime),
EXTRACT(SECOND FROM :datetime))
)
)
SELECT ID as shiftID
FROM Shift, T0, T1
WHERE Shift.ShiftCalendarID = T0.ShiftCalendarID
AND ( ( FromDay = T0.CurDay AND FromTimeOfDay <= T1.TimeValue AND TillTimeOfDay > T1.TimeValue )
OR ( FromDay = T0.CurDay AND FromTimeOfDay >= TillTimeOfDay AND FromTimeOfDay <= T1.TimeValue )
OR ( FromDay = T0.PrvDay AND FromTimeOfDay >= TillTimeOfDay AND TillTimeOfDay > T1.TimeValue )
);
$$
To add on to Dave Weldon's solution - in UDF bodies, there are no colons in front of parameters. Instead of :shiftCalendarID it is shiftCalendarID and :dateTime is just dateTime. Colons are needed in Stored Procedures, because the parameters are treated as string literal constants, but this is not the case with User Defined Functions.

Store the result of sql and process it in informix

We have a view which contains 2 columns: pattern_start_time, pattern_end_time.
The select query in the function will convert it to minutes and using that result we are processing to get the shift unused coverage.The function is getting created but the processing is not happening and getting the below error:
SQLError[IX000]:Routine (my_list) cant be resolved.
Also please enter image description heresuggest to loop till the length of the result.
CREATE function myshifttesting(orgid int) returning int;
DEFINE my_list LIST( INTEGER not null );
DEFINE my_list1 LIST( INTEGER not null );
define i, j, sub, sub1 int;
define total int;
TRACE ON;
TRACE 'my testing starts';
INSERT INTO TABLE( my_list )
select
((extend(current, year to second) + (dots.v_shift_coverage.pattern_start_time - datetime(00:00) hour to minute) - current)::interval minute(9) to minute)::char(10)::INTEGER
from
dots.v_shift_coverage
where
org_guid = orgid;
INSERT INTO TABLE( my_list1 )
select
((extend(current, year to second) + (dots.v_shift_coverage.pattern_end_time - datetime(00:00) hour to minute) - current)::interval minute(9) to minute)::char(10)::INTEGER
from
dots.v_shift_coverage
where
org_guid = orgid;
let sub = 0;
let sub1 = 0;
let total = 0;
for j = 0 to 4
if (my_list(j) < my_list1(j))
then
if (my_list(j + 1) > my_list1(j))
then
let sub = sub + my_list(j + 1) - my_list1(j);
end if;
end if;
end for
if (my_list(0) > my_list1(4))
then
let sub1 = my_list(0) - my_list1(4);
end if;
let total = sub + sub1;
return total;
end function;
The error that you are receiving is because my_list(j) is not valid Informix syntax to access a LIST element. Informix is interpreting my_list(j) as a call to a function named mylist.
You can use a temporary table to "emulate" an array with your logic, something like this:
CREATE TABLE somedata
(
letter1 CHAR( 2 ),
letter2 CHAR( 2 )
);
INSERT INTO somedata VALUES ( 'a1', 'a2' );
INSERT INTO somedata VALUES ( 'b1', 'b2' );
INSERT INTO somedata VALUES ( 'c1', 'c2' );
INSERT INTO somedata VALUES ( 'd1', 'd2' );
INSERT INTO somedata VALUES ( 'e1', 'e2' );
DROP FUNCTION IF EXISTS forloop;
CREATE FUNCTION forloop()
RETURNING CHAR( 2 ) AS letter1, CHAR( 2 ) AS letter2;
DEFINE number_of_rows INTEGER;
DEFINE iterator INTEGER;
DEFINE my_letter1 CHAR( 2 );
DEFINE my_letter2 CHAR( 2 );
-- Drop temp table if it already exists in the session
DROP TABLE IF EXISTS tmp_data;
CREATE TEMP TABLE tmp_data
(
tmp_id SERIAL,
tmp_letter1 CHAR( 2 ),
tmp_letter2 CHAR( 2 )
);
-- Insert rows into the temp table, serial column will be the access key
INSERT INTO tmp_data
SELECT 0,
d.letter1,
d.letter2
FROM somedata AS d
ORDER BY d.letter1;
-- Get total rows of temp table
SELECT COUNT( * )
INTO number_of_rows
FROM tmp_data;
FOR iterator = 1 TO number_of_rows
SELECT d.tmp_letter1
INTO my_letter1
FROM tmp_data AS d
WHERE d.tmp_id = iterator;
-- Check if not going "out of range"
IF iterator < number_of_rows THEN
SELECT d.tmp_letter2
INTO my_letter2
FROM tmp_data AS d
WHERE d.tmp_id = iterator + 1;
ELSE
-- iterator + 1 is "out of range", return to the beginning
SELECT d.tmp_letter2
INTO my_letter2
FROM tmp_data AS d
WHERE d.tmp_id = 1;
END IF;
RETURN my_letter1, my_letter2 WITH RESUME;
END FOR;
END FUNCTION;
-- Running the function
EXECUTE FUNCTION forloop();
-- Results
letter1 letter2
a1 b2
b1 c2
c1 d2
d1 e2
e1 a2
5 row(s) retrieved.

Is there a faster way of to generate the required output than using a one-to-many join in Proc SQL?

I require an output that shows the total number of hours worked in a rolling 24 hour window. The data is currently stored such that each row is one hourly slot (for example 7-8am on Jan 2nd) per person and how much they worked in that hour stored as "Hour". What I need to create is another field that is the sum of the most recent 24 hourly slots (inclusive) for each row. So for the 7-8am example above I would want the sum of "Hour" across the 24 rows: Jan 1st 8-9am, Jan 1st 9-10am... Jan 2nd 6-7am, Jan 2nd 7-8am.
Rinse and repeat for each hourly slot.
There are 6000 people, and we have 6 months of data, which means the table has 6000 * 183 days * 24 hours = 26.3m rows.
I am currently done this using the code below, which works on a sample of 50 people very easily, but grinds to a halt when I try it on the full table, somewhat understandably.
Does anyone have any other ideas? All date/time variables are in datetime format.
proc sql;
create table want as
select x.*
, case when Hours_Wrkd_In_Window > 16 then 1 else 0 end as Correct
from (
select a.ID
, a.Start_DTTM
, a.End_DTTM
, sum(b.hours) as Hours_Wrkd_In_Window
from have a
left join have b
on a.ID = b.ID
and b.start_dttm > a.start_dttm - (24 * 60 * 60)
and b.start_dttm <= a.start_dttm
where datepart(a.Start_dttm) >= &report_start_date.
and datepart(a.Start_dttm) < &report_end_date.
group by ID
, a.Start_DTTM
, a.End_DTTM
) x
order by x.ID
, x.Start_DTTM
;quit;
The most performant DATA step solution most likely involves a ring-array to track the 1hr time slots and hours worked within. The ring will allow a rolling aggregate (sum and count) to be computed based on what goes into and out of the ring.
If you have a wide SAS license, look into the procedures in SAS/ETS (Econometrics and Time Series). Proc EXPAND might have some rolling aggregate capability.
This sample DATA Step code took <10s (WORK folder on SSD) to run on simulated data for 6k people with 6months of complete coverage of 1hr time slots.
data have(keep=id start_dt end_dt hours);
do id = 1 to 6000;
do start_dt
= intnx('dtmonth', datetime(), -12)
to intnx('dtmonth', datetime(), -6)
by dhms(0,1,0,0)
;
end_dt = start_dt + dhms(0,1,0,0);
hours = 0.25 * floor (5 * ranuni(123)); * 0, 1/4, 1/2, 3/4 or 1 hour;
output;
end;
end;
format hours 5.2;
run;
/* %let log= ; options obs=50 linesize=200; * submit this (instead of next) if you want to log the logic; */
%let log=*; options obs=max;
data want2(keep=id start_dt end_dt hours hours_rolling_sum hours_rolling_cnt hours_out_:);
array dt_ring(24) _temporary_;
array hr_ring(24) _temporary_;
call missing (of dt_ring(*));
call missing (of hr_ring(*));
if 0 then set have; * prep pdv column order;
hours_rolling_sum = 0;
hours_rolling_cnt = 0;
label hours_rolling_sum = 'Hours worked in prior 24 hours';
index = 0;
do until (last.id);
set have;
by id start_dt;
index + 1;
if index > 24 then index = 1;
hours_out_sum = 0;
hours_out_cnt = 0;
do clear = 1 by 1 until (clear=0);
if sum (dt_ring(index), 0) = 0 then do;
* index is first go through ring array, or hit a zeroed slot;
&log putlog 'NOTE: ' index= 'clear for empty ring item. ';
clear = 0;
end;
else
if start_dt - dt_ring(index) >= %sysfunc(dhms(0,24,0,0)) then do;
&log putlog / 'NOTE: ' index= 'reducting and zeroing.' /;
hours_out_sum + hr_ring(index);
hours_out_cnt + 1;
hours_rolling_sum = hours_rolling_sum - hr_ring(index);
hours_rolling_cnt = hours_rolling_cnt - 1;
dt_ring(index) = 0;
hr_ring(index) = 0;
* advance item to next item, that might also be more than 24 hours ago;
index = index + 1;
if index > 24 then index = 1;
end;
else do;
&log putlog / 'NOTE: ' index= 'back off !' /;
* index was advanced to an item within 24 hours, back off one;
index = index - 1;
if index < 1 then index = 24;
clear = 0;
end;
end; /* do clear */
dt_ring(index) = start_dt;
hr_ring(index) = hours;
hours_rolling_sum + hours;
hours_rolling_cnt + 1;
&log putlog 'NOTE: ' index= 'overlaying and aggregating.' / 'NOTE: ' start_dt= hours= hours_rolling_sum= hours_rolling_cnt=;
output;
end; /* do until */
format hours_rolling_sum 5.2 hours_rolling_cnt 2.;
format hours_out_sum 5.2 hours_out_cnt 2.;
run;
options obs=max;
When reviewing the results you should notice the delta for hours_rolling_sum is +(hours in slot) - (hours_out_sum{which is hrs removed from ring})
If you must use SQL, I would suggest following #jspascal and index the table, but rearrange the query to left join original data to inner-joined subselect (so that SQL will do an index involved hash join on the ids) . For same amount of few people it should faster than original query, but still be too slow for doing all 6K.
proc sql;
create index id on have;
create index id_slot on have (id, start_dt);
quit;
proc sql _method;
reset inobs=50; * limit data so you can see the _method;
create table want as
select
have.*
, case
when ROLLING.HOURS_WORKED_24_HOUR_PRIOR > 16
then 1
else 0
end as REVIEW_TIME_CLOCKING_FLAG
from
have
left join
(
select
EACH_SLOT.id
, EACH_SLOT.start_dt
, count(*) as SLOT_COUNT_24_HOUR_PRIOR
, sum(PRIOR_SLOT.hours) as HOURS_WORKED_24_HOUR_PRIOR
from
have as EACH_SLOT
join
have as PRIOR_SLOT
on
EACH_SLOT.ID = PRIOR_SLOT.ID
and EACH_SLOT.start_dt - PRIOR_SLOT.start_dt between 0 and %sysfunc(dhms(0,24,0,0))-0.1
group by
EACH_SLOT.id, EACH_SLOT.start_dt
) as ROLLING
on
have.ID = ROLLING.ID
and have.start_dt = ROLLING.start_dt
order by
id, start_dt
;
%put NOTE: SQLOOPS = &SQLOOPS;
quit;
The inner join is pyramid-like and still involves a lot of internal looping.
A compound index on the columns being accessed in the joined table - id + start_dttm + hours - would be useful if there isn't one already.
Using msglevel=i will print some diagnostics about how the query is executed. It may give some additional hints.

How to have sum function of a field of MB and GB values in MVC

My field values are like:
810.9 MB
1.2 GB
395.1 MB
982.3 MB
7.7 GB
149.4 MB
10.0 GB
429.1 MB
3.1 GB
and I want to sum this column in gb in my ASP.NET MVC controller.
But I have no idea how to do this.
You could try something like this:
-- This is your "raw" input - just all the strings in your example
DECLARE #input TABLE (Measure VARCHAR(50))
INSERT INTO #input ( Measure )
VALUES ('810.9 MB'), ('1.2 GB'), ( '395.1 MB'), ( '982.3 MB'), ( '7.7 GB'), ( '149.4 MB'), ( '10.0 GB'), ( '429.1 MB'), ( '3.1 GB')
-- Now declare a separate table that contains (1) the raw value, (2) the contained *numerical* value, and (3) the unit of measure
DECLARE #Storage TABLE (Measure VARCHAR(50), NumValue DECIMAL(20,4), Unit VARCHAR(10))
-- Fill your raw input into that "working table"
INSERT INTO #Storage (Measure, NumValue, Unit)
SELECT
Measure,
NumMeasure = CAST(SUBSTRING(Measure, 1, CHARINDEX(' ', Measure)) AS DECIMAL(20, 2)),
Unit = SUBSTRING(Measure, CHARINDEX(' ', Measure) + 1, 9999)
FROM
#input
SELECT * FROM #Storage
-- when you select from that "working" table, you can now easily *SUM* the numerical values,
-- and show them on screen whichever way you want - as "xxx MB" or "yyyy GB" or whatever - up to you
SELECT
SUM(CASE Unit
WHEN 'MB' THEN NumValue * 1000000
WHEN 'GB' THEN NumValue * 1000000000
ELSE NumValue
END),
CAST(SUM(CASE Unit
WHEN 'MB' THEN NumValue * 1000000
WHEN 'GB' THEN NumValue * 1000000000
ELSE NumValue
END) / 1000000000.0 AS VARCHAR(25)) + ' GB'
FROM
#Storage
Update:
If you want to do this in C# code, try this:
foreach(var item in list)
{
// split "item" into two parts
string[] parts = item.Split(' ');
// parts[0] should be a decimal value
decimal numValue = 0.0m;
if (decimal.TryParse(parts[0], out numValue))
{
decimal convertedValue = 0.0m;
if(parts[1] == "MB")
{
convertedValue = numValue * 1000000;
}
else if (parts[1] == "GB")
{
convertedValue = numValue * 1000000000;
}
}
}

How to use a numeric parameter in a stored procedure?

Good morning,
I'm having troubles to integrate a working query into a stored procedure.
My main issue, is that I'm using a WHILE loop with an integer, and that the stored procedure is having troubles with it.
My working query/code is the following:
CREATE TABLE #tempScaffStandingByTime
(
TotalStanding INT,
MonthsAgo INTEGER
)
DECLARE #StartDate DATETIME = null
DECLARE #months INTEGER = 12
use [Safetrak-BradyTechUK]
WHILE #months >= 0
BEGIN
SET #StartDate = DATEADD(mm, -12 + #months, DATEADD(mm, 0, DATEADD(mm,
DATEDIFF(mm,0,GETDATE()-1), 1)))
INSERT INTO #tempScaffStandingByTime
select TOP 1 COUNT(*) OVER () AS TotalRecords, #months
from asset a
join wshhistory h on h.assetid = a.uid
where a.deleted = 0 and h.assetstate <> 6
AND (dbo.getdecommissiondate(a.uid) > #StartDate)
group by a.uid
SET #months -= 3
END
SELECT * FROM #tempScaffStandingByTime
DROP TABLE #tempScaffStandingByTime
This results in the input which I want:
I then tried to import this code into my stored procedure
DECLARE #Query varchar (8000)
, #Account varchar (100) = 'BradyTechUK'
SET #Account = 'USE [Safetrak-' + #Account + ']'
/************************************************************************************/
/********** Create Table to hold data ***********************************************/
CREATE TABLE #tempScaffStandingByTime
(
TotalStanding INT,
MonthsAgo INTEGER
)
/************************************************************************************/
/********** Populate temp table with data *******************************************/
DECLARE #StartDate DATETIME = null
DECLARE #months INTEGER = 12
SET #Query= +#Account+ '
WHILE '+#months+' >= 0
BEGIN
SET '+#StartDate+' = DATEADD(mm, -12 + ('+#months+', DATEADD(mm, 0, DATEADD(mm, DATEDIFF(mm,0,GETDATE()-1), 1)))
INSERT INTO #tempScaffStandingByTime
select TOP 1 COUNT(*) OVER () AS TotalRecords, '+#months+'
from asset a
join wshhistory h on h.assetid = a.uid
where a.deleted = 0 and h.assetstate <> 6
AND (dbo.getdecommissiondate(a.uid) > '+#StartDate+')
group by a.uid
SET '+#months+' -= 3
END'
EXEC (#Query)
/************************************************************************************/
/********** Select Statement to return data to sp ***********************************/
Select TotalStanding
, MonthsAgo
FROM #tempScaffStandingByTime
/************************************************************************************/
DROP TABLE #tempScaffStandingByTime
But when loading the stored procedure in SSRS I get a conversion error.
I can convert my parameter to a string, but then it won't count anymore.
I searched far and wide on the internet, tried various things, but can't get it to work.
Any help is appreciated, thanks!
It looks like your problem original comes from the lack of conversion between the INT in #months and your VARCHAR in #Accounts and #Query.
You need to cast the months to a string to append it, otherwise the SQL Server thinks you're trying to do addition.
However you'll also have problems with the usage of your #StartDate variable as you try to set it via your dynamic SQL which is not going to work like that - and as with #months it'll have an incorrect datatype, but I'm also unsure why it's written as is, and what you're tying to achieve with it
I think - from what you're writing - that this might be what you're looking for?
SET #Query= #Account + '
DECLARE #StartDate DATETIME = null
DECLARE #months VARCHAR(2) = ''12''
WHILE #months >= 0
BEGIN
SET #StartDate = DATEADD(mm, -12 + #months, DATEADD(mm, 0, DATEADD(mm, DATEDIFF(mm,0,GETDATE()-1), 1)))
INSERT INTO #tempScaffStandingByTime
select TOP 1 COUNT(*) OVER () AS TotalRecords, #months, #StartDate
from asset a
join wshhistory h on h.assetid = a.uid
where a.deleted = 0 and h.assetstate <> 6
AND (dbo.getdecommissiondate(a.uid) > #StartDate)
group by a.uid
SET #months -= 3
END'
EXEC (#Query)

Resources