I am using vsql.exe on an external Vertica database for which I don't have any administrative access. I use some views with simple SELECT+FROM+WHERE queries.
These queries 90% of the time work just fine, but some times, randomly, I get this error:
ERROR 3326: Execution time exceeded run time cap of 00:00:45
The strange thing is that this error can happen way after those 45 seconds, even after 3 minutes. I've been told this is related to having different resource pools, but anyway I don't want to dig into that.
The problem is that when this occurs, vsql.exe returns errorlevel 0 and there is (apparently almost) no way to know this failed.
The output of the query is stored in a csv file. When it succeeds, it ends with (#### rows). But when it fails with this error, it just stops at any point of the csv, and its resulting size is around half of what's expected. This is of course not what you would expect when an error occurs, like no output or an empty one.
If there is a connection error or if the query has syntax errors, the errorlevel is not 0, so in those cases it behaves as expected.
I've tried many things, like increasing the timeout or adding -v ON_ERROR_STOP=ON to the vsql.exe parameters, but none of that helped.
I've googled a lot and found many people having this error, but the solutions are mostly related to increasing the timeouts, not related to the errorlevel returned.
Any help will be greatly appreciated.
TL;DR: how can I detect an error 3326 in a batch file like this?
#echo off
vsql.exe -h <hostname> -U <user> -w <pwd> -o output.cs -Ac "SELECT ....;"
echo %errorlevel% is always 0
if errorlevel 1 echo Error!! But this is never displayed.
Now that's really unexpected to me. I don't have Windows available just now, but trying on my Mac - at first just triggering a deliberate error:
$ vsql -h zbook -d sbx -U dbadmin -w $VSQL_PASSWORD -v ON_ERROR_STOP=ON -Ac "select * from foobarfoo"
ERROR 4566: Relation "foobarfoo" does not exist
$ echo $?
1
With ON_ERROR_STOP set to ON, this should be the behaviour everywhere.
Could you try what I did above through Windows, just with echo %ERRORLEVEL% instead of echo $?, just from the Windows command prompt and not in a batch file?
Next test: I run on resource pool general in my little test database, so I temporarily modify it to a runtime cap of 30 sec, run a silly query that will take over 30 seconds with ON_ERROR_STOP set to ON, collect the value returned by vsql and set the runtime cap of general back to NONE. I also have the %VSQL_* % env variables set so I don't have to repeat them all the time:
rem Windows way to set environment variables for vsql:
set VSQL_HOST=zbook
set VSQL_DATABASE=sbx
set VSQL_USER=dbadmin
set VSQL_PASSWORD=***masked***
Now for the test (backslashes, in Linux/MacOs escape a new line, which enables you to "word wrap" a shell command. Use the caret (^) in Windows for that):
marco ~/1/Vertica/supp $ # set a runtime cap
marco ~/1/Vertica/supp $ vsql -i -c \
"alter resource pool general runtimecap '00:00:30'"
ALTER RESOURCE POOL
Time: First fetch (0 rows): 116.326 ms. All rows formatted: 116.730 ms
marco ~/1/Vertica/supp $ vsql -v ON_ERROR_STOP=ON -iAc \
"select count(*) from one_million_rows a cross join one_million_rows b"
ERROR 3326: Execution time exceeded run time cap of 00:00:30
marco ~/1/Vertica/supp $ # test the return code
marco ~/1/Vertica/supp $ echo $?
1
marco ~/1/Vertica/supp $ # clear the runtime cap
marco ~/1/Vertica/supp $ vsql -i -c \
"alter resource pool general runtimecap NONE "
ALTER RESOURCE POOL
Time: First fetch (0 rows): 11.148 ms. All rows formatted: 11.383 ms
So it works in my case. Your line:
if errorlevel 1 echo Error!! But this is never displayed.
... never echoes anything because the previous line, with echo will return 0 to the shell, overriding the previous errorlevel.
Try it command by command on your Windows command prompt, and see what happens. Just echo %errorlevel%, without evaluating it.
And I notice that you are trying to export to CSV format. Then, try this:
Format the output unaligned (-A)
set the field separator to comma (-F ',')
remove the footer '(n rows)' (-P footer)
limit the output to 5 rows in the query for test
(I show the output before redirecting to file):
marco ~/1/Vertica/supp $ vsql -A -F ',' -P footer -c "select * from one_million_rows limit 5"
id,id_desc,dob,category,busid,revenue
0,0,1950-01-01,1,====== boss ========,0.000
1,-1,1950-01-02,2,kbv-000001kbv-000001,0.010
2,-2,1950-01-03,3,kbv-000002kbv-000002,0.020
3,-3,1950-01-04,4,kbv-000003kbv-000003,0.030
4,-4,1950-01-05,5,kbv-000004kbv-000004,0.040
Not aligning is much faster than aligning.
Then, as you spend most time in the fetching of the rows (that's because you get a timeout in the middle of an output file write process), try fetching more rows at a time than the default 1000. You will need to play with the value, depending on the network settings at your site until you get your best value:
-v ROWS_AT_A_TIME=10000
Once you're happy with the tested output, try this command (change the SELECT for your needs, of course ....):
marco ~/1/Vertica/supp $ vsql -A -F ',' -P footer \
-v ON_ERROR_STOP=ON -v ROWS_AT_A_TIME=10000 -o one_million_rows.csv \
-c "select * from one_million_rows"
marco ~/1/Vertica/supp $ wc -l one_million_rows.csv
1000001 one_million_rows.csv
The table actually contains one million rows. Note the line count in the file: 1,000,001. That's the title line included, but the footer (1000000 rows) removed.
I'm still pretty new to batch but I wanted to try out some different functionalities within the same batch program. Essentially, I am attempting to parse through a .txt file of hostnames and ping them 1 time each, and return a pass or fail by utilizing a find query for the specific ping result. The second find query is redundant but left in. A string of "timed out" indicates that the host is down. I have been able to achieve this with this bulky code, but my actual ping statistics are no longer being written to this log file.
I'm not too familiar with the command pipeline character but it may just be incompatible for this specific use case. If I remove the piped command for the find query, it is able to write the output of the ping command just fine, but then I lose the utility of a return pass or fail value. Is this just a simple syntax error? I've tried moving the location of the actual write to file argument on the line itself but it hasn't worked. Also, why are the errorlevel values inverted (line 21)? I can't seem to return what I'm looking for.
Is there anyway I can get all these parts to play nice together? I apologize if the answer is quite obvious...
#echo off
setlocal
set hosts=temp.txt
set count=0
echo.
echo Parsing File: %hosts%
echo.
echo Start > C:\Users\___\Downloads\pingLOG.txt
for /F "tokens=*" %%i in (%hosts%) do (
set /A count=count+1
echo+
echo.
echo [+] Pinging: %%i
echo [+] Pinging: %%i >> C:\Users\___\Downloads\pingLOG.txt
echo.
ping -n 1 "%%i" | find /I "timed out" >> C:\Users\___\Downloads\pingLOG.txt
if errorlevel == 1 (
echo Pass
) else (
echo Fail
)
echo.
echo %TIME% >> C:\Users\___\Downloads\pingLOG.txt
)
echo.
echo %count% Hosts Scanned
find /c "timed out" C:\Users\___\Downloads\pingLOG.txt
echo.
pause
You can't filter the output for a certain string and expect the complete output (eliminating unwanted parts is the very reason for filtering).
To accomplish your goal, you need a temporary file (the complete output) and filter that file, so you have both variants (filtered and unfiltered) (suboptimal, but well...):
ping -n 1 "%%i" >"%temp%\temp.tmp"
find /I "timed out" "%temp%\temp.tmp" >nul && set "status=Fail" || set "status=Pass"
type "%temp%\temp.tmp" >> "C:\Users\___\Downloads\pingLOG.txt"
echo %status% >> "C:\Users\___\Downloads\pingLOG.txt"
I use windows 7 pro service pack 1.
I have the following code in the post build event :
SET VAR1=BLABLA
ECHO %VAR1% > Test.txt
It wont work. In the file, i get «command echo activated» (translated from french).
Embarcadero documentation says that i can use any valid dos command in those events.
If i just use :
ECHO BLABLA > Test.txt
It works, no problem. Is this a bug or there is a problem with % character ? % is an ascii char so i dont even know what could be the problem.
ty for your help.
% is used in cmd to delimit variablenames when the value of the variable is required, hence echo %var1% > test.txt will write the current value of the environment variable var1 to the file.
If var1 is not defined at the time, it will report the echo status (Echo is on/off`.
This can be circumvented by using echo(%var1% - the ( modifies echo's behaviour to not report the echo status if the arguments are resolved to nothing.
If you want to echo a literal % then you need to escape the % with another %. cmd normally uses ^ to escape symbols with a special meaning - % is the exception; %% to echo a literal %.
BTW - the space between the string to be echoed and the redirector will be output to the file. To prevent this, use > test.txt echo %var1% Note that > creates a file anew. >> will create or append if the file already exists. The space between the redirector and the filename is optional.
However, it's important when using batch to post exactly the code that's in use.
SET VAR1=BLABLA
ECHO %VAR1% > Test.txt
will work happily.
SET VAR1 = BLABLA
ECHO %VAR1% > Test.txt
will not because this latter code sets a variable named "var1Space"
On my 10.2.1 system, I've tried the code as published.
The actual code that's executed is
SET VAR1=BLABLA&ECHO %VAR1% > Test.txt
not
SET VAR1=BLABLA
ECHO %VAR1% > Test.txt
as shown in the "Build events commands" window.
This will not work because the entire line is executed as published on the "build events" page - SET VAR1=BLABLA&ECHO %VAR1% > Test.txt which will be interpreted by cmd after cmd performs its standard parsing routine.
cmd replaces any %var% with the actual value at parse time, not at run time hence as var1 has no value when the line SET VAR1=BLABLA&ECHO %VAR1% > Test.txt is parsed, the code is executed as SET VAR1=BLABLA&ECHO > Test.txt hence the problem encountered.
To cure this, you need to use
SET VAR1=BLABLA&call ECHO %%VAR1%% > Test.txt
where cmd will execute the parsed-ECHO command in a subshell. % is the escape character for % so the subshell executes ECHO %VAR1% > Test.txt after var1 has been set.
I'd suggest you raise this as a problem with EMBT. Batch commands cannot be strung together with & without side-effects. The code entered into the "Build events commands" window should be executed without reformatting - just written to a (temporary) batch file and the batch file then executed.
No doubt the eager downvoters will support the resolution of this problem.
Hello everyone I'm having an issue with this script. I've just begun work on it and it is supposed to look for entries previously generated by another script I made.
The gist of the thing is that the log has entries like:
makefile_1786878:/home/user/project
the format is filename_inode:/originaldirectory/
and this script is supposed to take a parameter and look for its exact match in the log
if [ $# -eq 0 ]
then
echo "No filename has been provided. Please enter a filename to restore!"
exit 1
fi
echo You have entered $1
echo Looking for $1 in the list of items deleted by safe_rm...
restoredfile=$(grep ^$1 $HOME/.restore.info)
echo $restoredfile
The problem I'm having is, if the user entered "mak" or "make" or "makefi" as a parameter it will incorrectly look up this entry
I want it to specifically get the exact match for this, I don't know how to force grep to do that
Try either one of these and see if it'll work for you:
grep -w "makefile"
grep "\<makefile\>"
If that work, then just change your grep to:
grep either one of those with the $1 parameter inside.
I have written a shell script which executing some of the mysql commands.
I want to run that shell script from rails side. And want to get result back of commands to rails from shell script.
How may i handle it?
EDIT:
script.sh
mysql -u root -pmysql << eof
SELECT TABLE_NAME AS "Table Name", table_rows AS "Quant of Rows", ROUND((data_length + index_length)/1024/1024,2) AS "Total Size Mb" FROM information_schema.TABLES WHERE information_schema.TABLES.table_schema='database_name';
eof
This is my script. How may i return SELECT query result to rails ?
You can do something like this, it will block until the script is finished and any output that it sent to standard out will be contained in the output variable.
output = `/my/script.sh`
puts output
Use backticks ` `
`/path/to/script` #backticks
Edit
To get the output of a mysql statement, use the -e flag
Eg. mysql -e "SELECT * from information_schema"