I'm trying to run a command on influxDB using -execute from the cli
influx -execute 'select * from test_measurement where time > \‘2020-01-13T16:22:00Z\’ and time < \‘2020-01-13T16:22:30Z\’ -username uname -password pwd
The query doesn't run as I'm unable to escape the single quotes(') for time condition. Can someone help with the syntax please.
Could you try:
influx -execute "select * from test_measurement where time > '2020-01-13T16:22:00Z' and time < '2020-01-13T16:22:30Z'" -username uname -password pwd
time > 'date str'
would not work in influx cli because that is not a valid operator for string fields... I think you want a numeric value in the form of nanoseconds or whatever the precision of your db instance.
For string fields, one could use a regex match like
=~ /2020-01-*/
for all january 2020 matches, for instance.
Related
I am using vsql.exe on an external Vertica database for which I don't have any administrative access. I use some views with simple SELECT+FROM+WHERE queries.
These queries 90% of the time work just fine, but some times, randomly, I get this error:
ERROR 3326: Execution time exceeded run time cap of 00:00:45
The strange thing is that this error can happen way after those 45 seconds, even after 3 minutes. I've been told this is related to having different resource pools, but anyway I don't want to dig into that.
The problem is that when this occurs, vsql.exe returns errorlevel 0 and there is (apparently almost) no way to know this failed.
The output of the query is stored in a csv file. When it succeeds, it ends with (#### rows). But when it fails with this error, it just stops at any point of the csv, and its resulting size is around half of what's expected. This is of course not what you would expect when an error occurs, like no output or an empty one.
If there is a connection error or if the query has syntax errors, the errorlevel is not 0, so in those cases it behaves as expected.
I've tried many things, like increasing the timeout or adding -v ON_ERROR_STOP=ON to the vsql.exe parameters, but none of that helped.
I've googled a lot and found many people having this error, but the solutions are mostly related to increasing the timeouts, not related to the errorlevel returned.
Any help will be greatly appreciated.
TL;DR: how can I detect an error 3326 in a batch file like this?
#echo off
vsql.exe -h <hostname> -U <user> -w <pwd> -o output.cs -Ac "SELECT ....;"
echo %errorlevel% is always 0
if errorlevel 1 echo Error!! But this is never displayed.
Now that's really unexpected to me. I don't have Windows available just now, but trying on my Mac - at first just triggering a deliberate error:
$ vsql -h zbook -d sbx -U dbadmin -w $VSQL_PASSWORD -v ON_ERROR_STOP=ON -Ac "select * from foobarfoo"
ERROR 4566: Relation "foobarfoo" does not exist
$ echo $?
1
With ON_ERROR_STOP set to ON, this should be the behaviour everywhere.
Could you try what I did above through Windows, just with echo %ERRORLEVEL% instead of echo $?, just from the Windows command prompt and not in a batch file?
Next test: I run on resource pool general in my little test database, so I temporarily modify it to a runtime cap of 30 sec, run a silly query that will take over 30 seconds with ON_ERROR_STOP set to ON, collect the value returned by vsql and set the runtime cap of general back to NONE. I also have the %VSQL_* % env variables set so I don't have to repeat them all the time:
rem Windows way to set environment variables for vsql:
set VSQL_HOST=zbook
set VSQL_DATABASE=sbx
set VSQL_USER=dbadmin
set VSQL_PASSWORD=***masked***
Now for the test (backslashes, in Linux/MacOs escape a new line, which enables you to "word wrap" a shell command. Use the caret (^) in Windows for that):
marco ~/1/Vertica/supp $ # set a runtime cap
marco ~/1/Vertica/supp $ vsql -i -c \
"alter resource pool general runtimecap '00:00:30'"
ALTER RESOURCE POOL
Time: First fetch (0 rows): 116.326 ms. All rows formatted: 116.730 ms
marco ~/1/Vertica/supp $ vsql -v ON_ERROR_STOP=ON -iAc \
"select count(*) from one_million_rows a cross join one_million_rows b"
ERROR 3326: Execution time exceeded run time cap of 00:00:30
marco ~/1/Vertica/supp $ # test the return code
marco ~/1/Vertica/supp $ echo $?
1
marco ~/1/Vertica/supp $ # clear the runtime cap
marco ~/1/Vertica/supp $ vsql -i -c \
"alter resource pool general runtimecap NONE "
ALTER RESOURCE POOL
Time: First fetch (0 rows): 11.148 ms. All rows formatted: 11.383 ms
So it works in my case. Your line:
if errorlevel 1 echo Error!! But this is never displayed.
... never echoes anything because the previous line, with echo will return 0 to the shell, overriding the previous errorlevel.
Try it command by command on your Windows command prompt, and see what happens. Just echo %errorlevel%, without evaluating it.
And I notice that you are trying to export to CSV format. Then, try this:
Format the output unaligned (-A)
set the field separator to comma (-F ',')
remove the footer '(n rows)' (-P footer)
limit the output to 5 rows in the query for test
(I show the output before redirecting to file):
marco ~/1/Vertica/supp $ vsql -A -F ',' -P footer -c "select * from one_million_rows limit 5"
id,id_desc,dob,category,busid,revenue
0,0,1950-01-01,1,====== boss ========,0.000
1,-1,1950-01-02,2,kbv-000001kbv-000001,0.010
2,-2,1950-01-03,3,kbv-000002kbv-000002,0.020
3,-3,1950-01-04,4,kbv-000003kbv-000003,0.030
4,-4,1950-01-05,5,kbv-000004kbv-000004,0.040
Not aligning is much faster than aligning.
Then, as you spend most time in the fetching of the rows (that's because you get a timeout in the middle of an output file write process), try fetching more rows at a time than the default 1000. You will need to play with the value, depending on the network settings at your site until you get your best value:
-v ROWS_AT_A_TIME=10000
Once you're happy with the tested output, try this command (change the SELECT for your needs, of course ....):
marco ~/1/Vertica/supp $ vsql -A -F ',' -P footer \
-v ON_ERROR_STOP=ON -v ROWS_AT_A_TIME=10000 -o one_million_rows.csv \
-c "select * from one_million_rows"
marco ~/1/Vertica/supp $ wc -l one_million_rows.csv
1000001 one_million_rows.csv
The table actually contains one million rows. Note the line count in the file: 1,000,001. That's the title line included, but the footer (1000000 rows) removed.
I have a log file that looks like this:
May 25 05:34:16 server sshd[1203]: Received disconnect from 192.0.2.2 port 39102:11
May 25 05:34:16 server sshd[1203]: Disconnected from 192.0.2.1 port 39102
Now i want to extract all of the ip addresses and the date/time strings at the beginning using grep. I already know how to get the ips:
grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}' /log.txt
and the dates/times:
grep -o '[A-Z][a-z][a-z] [0-3][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]' /log.txt
but i don't know how to get both at the same time in a format like:
May 25 05:34:16 192.0.2.1
I've read something like:
grep -oE 'match1|match2' /log.txt
but that doesn't seem to work.
Printing two matches in the single line is easier with awk, following will print date(by printing $1,$2,$3 and all the valid IP address.
gawk '{match($0,/[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/,a);split(a[0],b,".")} b[1]<=255&& b[2]<=255 && b[3]<=255 && b[4]<=255 &&length(a[0]){print $1,$2,$3, a[0]}' log_file
May 25 05:34:16 192.0.2.2
May 25 05:34:16 192.0.2.1
Explanation: First use match function to capture all the strings having format of digit.digit.digit.digit and store them into an array called "a" , then split the captured array(a) by dot(.) characters and check if each is <= 255 to ensure the IP address is valid.
Note that: GNU awk is used here.
Also note that, the regex you mentioned will also print invalid IP addresses (Eg: 333.222.555.666).
You could use your 2 patterns in a capturing group and use those in the replacement using sed:
sed -i -E 's#^([A-Z][a-z][a-z] [0-3][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]).* ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*$#\1 \2#g' log.txt
That will match:
^ Start of string
([A-Z][a-z][a-z] [0-3][0-9] [0-9][0-9]:[0-9][0-9]:[0-9][0-9]) Your date/time like pattern
.* Match any char 0+ times
([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) Match space followed your ip like pattern
.* Match any char 0+ times
$ End of string
Result
May 25 05:34:16 192.0.2.2
May 25 05:34:16 192.0.2.1
With any awk in any shell on any UNIX box:
$ awk '{print $1, $2, $3, $(NF-2)}' file
May 25 05:34:16 192.0.2.2
May 25 05:34:16 192.0.2.1
I use Jenkins ver. 1.522 and I want to pass a long string with spaces and quotes as a parameter in the parameterized build section. The job only runs a python script.
My problem is that I can't find a way to escape my string so that jenkins passes it correctly to the script.
Assuming...
string: fixVersion in ("foo") AND issuetype in (Bug, Improvement) AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key DESC
variable name: bar
script name: coco.py
When I run the script in the terminal, everything is fine: python coco.py --option 'fixVersion in ("foo") AND issuetype in (Bug, Improvement) AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key DESC'
When I run the same script with jenkins using the parametrized build and try to escape the variable so it end up taken as one parameter by the py script it is oddly espacped by jenkins.
In my jenkins job I call the script: python coco.py --option \'${BAR}\'
and it ends up as:
python coco.py --option '"fixVersion' in '('\''foo'\'')' AND issuetype in '(Bug,' 'Improvement)' in '(Production,' 'Stage)' AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key 'DESC"'
I also tried \"${BAR}\", \"$BAR\",\'$BAR\'
What it the right way do acheive it?
Try
python coco.py --option "${BAR}"
Alternatively, if you need the single quotes surrounding everything
python coco.py --option \'"${BAR}"\'
In the cases you listed, bash will treat the spaces as delimiters. Putting the double quotes around a variable will preserve the whitespace in a string. Example
aString='foo bar'
for x in $aString; do echo $x; done
# foo
# bar
for x in "$aString"; do echo $x; done
# foo bar
I am using Jenkins v1.606 and ran into this same issue!
The issue that I saw passing user defined string params containing spaces into an execution shell would not properly format the string (only with a parameter that had 1 or more spaces). What you have to watch out for is reviewing the 'output' log. Jenkins will not properly display the string param value within the log.
Example (correct format for containing spaces):
docker exec -i container-base /bin/bash -c "cd /container/path/to/code/ && ./gradlew test_xml -P DISPLAY_NAME='${DISPLAY_NAME}' -P USERNAME='${USERNAME}' -P SERVER_NAME='${SERVER_NAME}'"
Jenkins Output of string (notice the string values format):
+ docker exec -i container-base /bin/bash -c 'cd /container/path/to/code/ && ./gradlew test_xml -P DISPLAY_NAME='\''VM10 USER D33PZ3R0'\'' -P USERNAME='\''d33pz3r0#stackoverflow.com'\'' -P SERVER_NAME='\''stackoverflow.com'\'''
Conclusion:
In my example, the literal command was encapsulated with <">, followed by surrounding the parameters with <'> to escape the literal cmd string and control the Jenkins string syntax. Remember not to just watch your Jenkins output log as it lead me wrong for an entire day while I fought with this! This should be the same for your issue as well, you do not need to escape with \' or other escape characters. Hope this helps!!
I have written a shell script which executing some of the mysql commands.
I want to run that shell script from rails side. And want to get result back of commands to rails from shell script.
How may i handle it?
EDIT:
script.sh
mysql -u root -pmysql << eof
SELECT TABLE_NAME AS "Table Name", table_rows AS "Quant of Rows", ROUND((data_length + index_length)/1024/1024,2) AS "Total Size Mb" FROM information_schema.TABLES WHERE information_schema.TABLES.table_schema='database_name';
eof
This is my script. How may i return SELECT query result to rails ?
You can do something like this, it will block until the script is finished and any output that it sent to standard out will be contained in the output variable.
output = `/my/script.sh`
puts output
Use backticks ` `
`/path/to/script` #backticks
Edit
To get the output of a mysql statement, use the -e flag
Eg. mysql -e "SELECT * from information_schema"
Supposing I have:
an Informix database named "my_database"
a table named "my_table" with the columns "col_1", "col_2" and "col_3":
I can extract the contents of the table by creating a my_table.sql script like:
unload to "my_table.txt"
select * from my_table;
and invoking dbaccess from the command line:
dbaccess my_database my_table.sql
This will produce the my_table.txt file with contents like:
value_a1|value_a2|value_a3
value_b1|value_b2|value_b3
Now, what do I have to do if I want to obtain the column names in the my_table.txt? Like:
col_1|col_2|col_3
value_a1|value_a2|value_a3
value_b1|value_b2|value_b3
Why you don't use dbschema?
To get schema of one table (without -t parameter show all database)
dbschema -d [DBName] -t [DBTable] > file.sql
To get schema of one stored procedure
dbschema -d [DBName] -f [SPName] > file.sql
None of the standard Informix tools put the column names at the top of the output as you want.
The program SQLCMD (not the Microsoft newcomer - the original one, available from the IIUG Software Archive) has the ability to do that; use the -H option for the column headings (and -T to get the column types).
sqlcmd -U -d my_database -t my_table -HT -o my_table.txt
sqlunload -d my_database -t my_table -HT -o my_table.txt
SQLCMD also can do CSV output if that's what you need (but — bug — it doesn't format the column names or column types lines correctly).
Found an easier solution. Place the headers in one file say header.txt (it will contain a single line "col_1|col_2|col_3") then to combine the header file and your output file run:
cat header.txt my_table.txt > my_table_wth_head.txt