I was wondering if someone could help me with the error message I am getting from Snowflake. I am trying to create a stored procedure that will loop through 125 files in S3 and copy into the corresponding tables in Snowflake. The names of the tables are the same names as the csv files. In the example I only have 2 file names set up (if someone knows a better way than having to liste all 125, that will be extremely. helpful) .
The error message I am getting is the following:
syntax error line 5 at position 11 unexpected '1'.
syntax error line 6 at position 22 unexpected '='. (line 4)
CREATE OR REPLACE PROCEDURE load_data_S3(file_name VARCHAR,table_name VARCHAR)
RETURNS VARCHAR
LANGUAGE SQL
AS
$$
BEGIN
FOR i IN 1 to 2 LOOP
CASE i
WHEN 1 THEN
SET file_name = 'file1.csv';
SET table_name = 'FILE1';
WHEN 2 THEN
SET file_name = 'file2.csv';
SET table_name = 'FILE2';
--WILL LIST THE REMAINING 123 WHEN STATEMENTS
ELSE
-- Do nothing
END CASE;
COPY INTO table_name
FROM #externalstg/file_name
FILE_FORMAT = (type='csv');
END LOOP;
RETURN 'Data loaded successfully';
END;
$$;
There are various ways to list the files in a stage (see the post here). You can loop through the resultset and run COPY INTO on each record
I am programming an instrument that uses Lua with extensions for accessing the instrument controls and displays.
The instrument has a command to read the analyzer plotted points for frequency and amplitude:
:analyzer:trace:frequency?
and
:analyzer:trace:amplitude?
Whenever either of the above is executed, 768 values are returned (frequency or amplitude). This is an embedded test system running Lua for user apps - I can't change the command.
I tried to use table.pack() to put these returned values into a table, but the syntax of the command with the ":" causes an error.
Code I tried --
freq = {};
freq = table.pack(:analyzer:trace:frequency?);
Error message is -- tests.lua:2428: unexpected symbol near ':'
Whenever the :analyzer:trace:frequency? is run stand alone (from PuTTY) or as a line in the Lua code, there is no error.
PuTTY receives 768 frequency values, each separated by a comma.
Looking for ways to direct the return into an array/table or to wrap the command so that it will execute similarly as above.
Thanks
I am using a shell script to extract the data from 'extr' table. The extr table is a very big table having 410 columns. The table has 61047 rows of data. The size of one record is around 5KB.
I the script is as follows:
#!/usr/bin/ksh
sqlplus -s \/ << rbb
set pages 0
set head on
set feed off
set num 20
set linesize 32767
set colsep |
set trimspool on
spool extr.csv
select * from extr;
/
spool off
rbb
#-------- END ---------
One fine day the extr.csv file was having 2 records with incorrect number of columns (i.e. one record with more number of columns and other with less). Upon investigation I came to know that the two duplicate records were repeated in the file. The primary key of the records should ideally be unique in file but in this case 2 records were repeated. Also, the shift in the columns was abrupt.
Small example of the output file:
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|154|H
5003|A7A|AAB|249.67|AAB|153.33|205|R
5004|A8A|269|F
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
Here the primary key records for 5003 and 5004 have reappeared in place of 5007 and 5008. Also the duplicate reciords have shifted the records of 5007 and 5008 by appending/cutting down their columns.
Need your help in analysing why this happened? Why the 2 rows were extracted multiple times? Why the other 2 rows were missing from the file? and Why the records were shifted?
Note: This script is working fine since last two years and has never failed except for one time (mentioned above). It ran successfully during next run. Recently we have added one more program which accesses the extr table with cursor (select only).
I reproduced a similar behaviour.
;-> cat input
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
See the input file as your database.
Now I write a script that accesses "the database" and show some random freezes.
;-> cat writeout.sh
# Start this script twice
while IFS=\| read a b c d e f; do
# I think you need \c for skipping \n, but I do it different one time
echo "$a|$b|$c|$d|" | tr -d "\n"
(( sleeptime = RANDOM % 5 ))
sleep ${sleeptime}
echo "$e|$f"
done < input >> output
EDIT: Removed cat input | in script above, replaced by < input
Start this script twice in the background
;-> ./writeout.sh &
;-> ./writeout.sh &
Wait until both jobs are finished and see the result
;-> cat output
5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|200|F
5003|A3A|AAB|153.33|5001|A1A|AAB|190.00|105|A
5002|A2A|ABB|180.00|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|200|F
5003|A3A|AAB|153.33|258|G
5006|A6A|ABB|147.89|154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|205|R
5004|A4A|ABB|261.50|269|F
5005|A5A|AAB|243.00|258|G
5006|A6A|ABB|147.89|215|F
154|H
5009|A9A|AAB|368.00|358|S
5010|AAA|ABB|245.71|215|F
When I edit the last line of writeout.sh into done > output I do not see the problem, but that might be due to buffering and the small amount of data.
I still don't know exactly what happened in your case, but it really seems like 2 progs writing simultaneously to the same script.
A job in TWS could have been restarted manually, 2 scripts in your masterscript might write to the same file or something else.
Preventing this in the future can be done using some locking / checks (when the output file exists, quit and return errorcode to TWS).
I recently migrated from Delphi 7 with SQL Server 2000 to Delphi 2010 with SQL Server 2008. I am using dbExpress.
After installing the new version I have found that the on sites that have a lot of data that system has become slow and unstable.
Can any one tell me if there is an issue between dbExpress and SQL Server 2008? Please help!!!!!
By performing a profiler trace you can see if you have any bottlenecks on SQL Server. Your default profiler trace (include TextData for RPC:Completed) should be good enough to start with.
The profiler trace can be analysed to see what takes the longest time. You can easily load the trace into a table and analyse it there. Note that when loaded into a table, the duration column is in microseconds. See the function fn_trace_gettable for a quicker way of loading a trace file into a table.
A common cause for poor performance, especially after a major change, is bad indexing.
Since SQL Server 2005 the optimiser stores within in-memory structures the indices it would like to have seen. These can be accessed with the dynamic management views sys.dm_db_missing_index_details, sys.dm_db_missing_index_groups and sys.dm_db_missing_index_groups_stats.
Here is a simple sample SQL to create your own missing index report, including the basic code to generate the missing index.
select
d.statement
, d.equality_columns
, d.inequality_columns
, d.included_columns
, s.user_seeks Seeks
, s.last_user_seek
, cast (s.avg_total_user_cost as decimal (9,2)) Cost
, s.avg_user_impact [%]
, 'CREATE INDEX MissingIndex_ ON ' + d.statement + '('
+ case when equality_columns IS NOT NULL then equality_columns else '' end
+ case when equality_columns IS NOT NULL AND inequality_columns IS NOT NULL then ', ' else '' end
+ case when inequality_columns IS NOT NULL then inequality_columns else '' end
+ ')'
+ case when included_columns IS NOT NULL then ' INCLUDE (' + included_columns + ')' else '' end
AS SQL
from sys.dm_db_missing_index_details d
INNER JOIN sys.dm_db_missing_index_groups g ON d.index_handle = g.index_handle
INNER JOIN sys.dm_db_missing_index_group_stats s ON g.index_group_handle = s.group_handle
I want to display the progress of a calculation done with a DO-loop, on the console screen. I can print out the progress variable to the terminal like this:
PROGRAM TextOverWrite_WithLoop
IMPLICIT NONE
INTEGER :: Number, Maximum = 10
DO Number = 1, MAXIMUM
WRITE(*, 100, ADVANCE='NO') REAL(Number)/REAL(Maximum)*100
100 FORMAT(TL10, F10.2)
! Calcultations on Number
END DO
END PROGRAM TextOverWrite_WithLoop
The output of the above code on the console screen is:
10.00 20.00 30.00 40.00 50.00 60.00 70.00 80.00
90.00 100.00
All on the same line, wrapped only by the console window.
The ADVANCE='No' argument and the TL10 (tab left so many spaces) edit descriptor works well to overwrite text on the same line, e.g. the output of the following code:
WRITE(*, 100, ADVANCE='NO') 100, 500
100 FORMAT(I3, 1X, TL4, I3)
Is:
500
Instead of:
100 500
Because of the TL4 edit descriptor.
From these two instances one can conclude that the WRITE statement cannot overwrite what has been written by another WRITE statement or by a previous execution of the same WRITE satement (as in a DO-loop).
Can this be overcome somehow?
I am using the FTN95 compiler on Windows 7 RC1. (The setup program of the G95 compiler bluescreens Windows 7 RC1, even thought it works fine on Vista.)
I know about the question Supressing line breaks in Fortran 95 write statements, but it does not work for me, because the answer to that question means new ouput is added to the previous output on the same line; instead of new output overwriting the previous output.
Thanks in advance.
The following should be portable across systems by use of ACHAR(13) to encode the carriage return.
character*1 creturn
! CODE::
creturn = achar(13) ! generate carriage return
! other code ...
WRITE( * , 101 , ADVANCE='NO' ) creturn , i , npoint
101 FORMAT( a , 'Point number : ',i7,' out of a total of ',i7)
There is no solution to this question within the scope of the Fortran standards. However, if your compiler understand backslash in Fortran strings (GNU Fortran does if you use the option -fbackslash), you can write
write (*,"(A)",advance="no") "foo"
call sleep(1)
write (*,"(A)",advance="no") "\b\b\bbar"
call sleep(1)
write (*,"(A)",advance="no") "\b\b\bgee"
call sleep(1)
write (*,*)
end
This uses the backslash character (\b) to erase previously written characters on that line.
NB: if your compiler does not understand advance="no", you can use related non-standard tricks, such as using the $ specifier in the format string.
The following worked perfectly using g95 fortran:
NF = NF + 1
IF(MOD(NF,5).EQ.0) WRITE(6,42,ADVANCE='NO') NF, ' PDFs'//CHAR(13)
42 FORMAT(I6,A)
gave:
5 PDFs
leaving the cursor at the #1 position on the same line. On the next update,
the 5 turned into a 10. ASCII 13 (decimal) is a carriage return.
OPEN(6,CARRIAGECONTROL ='FORTRAN')
DO I=1,5
WRITE(6,'(1H+" ",I)') I
ENDDO