basically i'm doing a curl and grepping some stuff.
But, i want set the output of this curl to a variable, to then use it on another curl.
e.g:
curl -u asd:asd http://zzz:123/aa/aa.aaa?cmd=ls | grep -B1 -E '<bbb>[4-7]\d{8,}' | grep yyy | tail -n 1 | sed -n -e 's/.*<xxx>\(.*\)<\/xxx>.*/\1/p')
but then I want set the output to a var and use it:
RUN aaa=$(previous curl) && curl -u asd:asd http://$aaa.com
tried with ${aaa}, with "$aaa", etc... didn't work. any solutions?
UPDATE:
something wrong is happening in previous curl 'cause doesn't return the value. probably for not doing the curl
I fear you will not be able to acheive this, because from my understanding RUN statement is to execute a command. To store value you'll have use SET.
For me the following workaround helped
RUN export aaa=$(curl -u asd:asd http://$aaa.com);echo aaa;
You can add the downsteam commands that will use the variable aaa towards right of the semicolon
I'm trying to find lines that start with 'query' or the following sign: '>' but I don't know how to do this.
If this is the dataset:
query=345
query=4565
brink=980
>ehlhdhdk
>blonk
I want to only preserve the lines 1,2,4 and 5.
I have tried: grep -e 'query=' filename.txt||grep -F '>' filename.txt > newfile.txt
and:
cat filename.txt | grep -e 'query='||grep -F '>' > newfile.txt.
But these do not work and they do not output the newfile.txt and instead they just output into the terminal.
You can use
grep -E '^(query|>)' filename.txt > newfile.txt
Details
^ - start of string
(query|>) - a capturing group that matches either
query - query
| - or
> - a > char.
See the online demo:
#!/bin/bash
s='query=345
query=4565
brink=980
>ehlhdhdk
>blonk'
grep -E '^(query|>)' <<< "$s"
Output:
query=345
query=4565
>ehlhdhdk
>blonk
Use multiple patterns with option -e:
grep -e '^>' -e '^query' file
Output:
query=345
query=4565
>ehlhdhdk
>blonk
I am trying to fetch the secret and workdir of a jenkins node in a script.
curl -L -s -u user:password -X GET https://cje.example.com/computer/jnlpAgentTest/slave-agent.jnlp
This gives
<jnlp codebase="https://cje.example.com/computer/jnlpAgentTest/" spec="1.0+">
<information>
<title>Agent for jnlpAgentTest</title>
<vendor>Jenkins project</vendor>
<homepage href="https://jenkins-ci.org/"/>
</information>
<security>
<all-permissions/>
</security>
<resources>
<j2se version="1.8+"/>
<jar href="https://cje.example.com/jnlpJars/remoting.jar"/>
</resources>
<application-desc main-class="hudson.remoting.jnlp.Main">
<argument>b8c80148ce36de10c9358384fac9e28fbba941055a9a6ab2277e75ddc29a8744</argument>
<argument>jnlpAgentTest</argument>
<argument>-workDir</argument>
<argument>/tmp/jnlpAgenttest</argument>
<argument>-internalDir</argument>
<argument>remoting</argument>
<argument>-url</argument>
<argument>https://cje.example.com/</argument>
</application-desc>
</jnlp>
Here I want to fetch the secret and workDir, secret can be fetched using this command
curl -L -s -u admin:password -H "Jenkins-Crumb:dac7ce5614f8cb32a6ce75153aaf2398" -X GET https://owood.ci.cloudbees.com/computer/jnlpAgentTest/slave-agent.jnlp | sed "s/.*<application-desc main-class=\"hudson.remoting.jnlp.Main\"><argument>\([a-z0-9]*\).*/\1/"
b8c80148ce36de10c9358384fac9e28fbba941055a9a6ab2277e75ddc29a8744
But I couldn't find a way to fetch workDir value, here it is /tmp/jnlpAgenttest which exists immediately after the tag -workDir
You can use
sed -n '/-workDir/{n;s/.*>\([^<>]*\)<.*/\1/p}'
sed -n -e '/-workDir/{n' -e 's/.*>\([^<>]*\)<.*/\1/p' -e '}'
See the online demo.
Details:
/-workDir/ - find the line with -workDir tag
n - reads the next line into pattern space discarding the previous one read
s/.*>\([^<>]*\)<.*/\1/ - matches any text up to >, then captures into Group 1 any zero or more chars other than < and >, then matches < and the rest of the string and replaces with Group 1 value
p - print the result of substitution.
I am having issues executing the following query. Is there any way to maintain the upper case letters with the psql query? I have tried quotes which does not work and I have tried single quotes, in which case I get a syntax error. NOTE: It appears the columns were create using quotes, fine, but how do I reference quotes from within the command then?)
psql bash CLI
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "\copy table_name(fileName, time) FROM $OUTPUT_FILE WITH DELIMITER AS ',' NULL 'null'"
Result:
ERROR: column "filename" of relation "table_name" does not exist
PostgreSQL treats an identifier such as a table name as lowercase unless you quote it.
You said though that double quotes didn't work. That's probably because you didn't get the quoting right. Quoting in the shell is hard. You have to end your double quote string surrounding the entire query just to start a single quote string that contains the double quote:
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME -c "\copy table_name("'"fileName"'", time) FROM $OUTPUT_FILE WITH DELIMITER AS ',' NULL 'null'"
You can leverage the fact that psql also accepts the command through standard input, which allows you to write it as follows without worrying about the quotes:
psql -h $DB_HOST -p $DB_PORT -U $DB_USER -d $DB_NAME <<SQL
\copy table_name("fileName", time) FROM $OUTPUT_FILE WITH DELIMITER AS ',' NULL 'null'
SQL
I'm trying to automate a JMX command using a tool called JMXterm.
The script looks like so:
#!/bin/bash
beanstemp="/tmp/jmx_beans"
read -r -p "Enter server name in FQDN " scrapername
bean1="BrokerName=localhost,Connection=Scraper_$scrapername,ConnectorName=openwire,Type=Connection"
echo beans -d $domain | $cmd > $beanstemp
idnum=$(grep "ID_$scrapername" $beanstemp | grep openwire | awk -F- '{print $2"-"$3}')
idname="$scrapername-$idnum"
bean2="BrokerName=localhost,Connection=ID_"$idname"2_0,ConnectorName=openwire,Type=Connection"
domain="org.apache.activemq"
server="scrapermq.sj.peer39.com:1099"
cmd="java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l $server"
echo run stop -b $bean1 -d $domain | $cmd -l $server
echo run stop -b $bean2 -d $domain | $cmd -l $server
When I run the script I get the proper response for the first command but the second one fails.
Here's the output of sh -x:
itaig#iganot-lt:~$ sh -x jmx.sh
+ beanstemp=/tmp/jmx_beans
+ read -r -p Enter server name in FQDN scrapername
Enter server name in FQDN selvmnj27.domain.company.com
+ bean1=BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
+ echo beans -d
+
+ + grep openwire
grep ID_selvmnj27.domain.company.com+ awk -F- {print $2"-"$3}
/tmp/jmx_beans
+ idnum=
+ idname=selvmnj27.domain.company.com-
+ bean2=BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
+ domain=org.apache.activemq
+ server=scrapermq.sj.peer39.com:1099
+ cmd=java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.sj.peer39.com:1099
+ echo+ run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connectionjava -jar -d Downloads/jmxterm-1.0-alpha-4-uber.jar -l org.apache.activemq scrapermq.sj.peer39.com:1099 -l
scrapermq.sj.peer39.com:1099
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#calling operation stop of mbean org.apache.activemq:BrokerName=localhost,Connection=Scraper_selvmnj27.domain.company.com,ConnectorName=openwire,Type=Connection
#operation returns:
null
$>+ + echo runjava stop -b -jar Downloads/jmxterm-1.0-alpha-4-uber.jar BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -l scrapermq.sj.peer39.com:1099 -l -d scrapermq.sj.peer39.com:1099
org.apache.activemq
Welcome to JMX terminal. Type "help" for available commands.
$>run stop -b BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection -d org.apache.activemq
#InstanceNotFoundException: org.apache.activemq:BrokerName=localhost,Connection=ID_selvmnj27.domain.company.com-2_0,ConnectorName=openwire,Type=Connection
$>itaig#iganot-lt:~$
"Operation returns: null" is the response I'm looking for.
The problem begins with this line:
echo beans -d $domain | $cmd > $beanstemp
When it runs the output of the command should be written to the $beanstemp file but the file stays empty.
When I run the command manually from terminal it works:
itaig#iganot-lt:~$ echo beans -d org.apache.activemq | java -jar Downloads/jmxterm-1.0-alpha-4-uber.jar -l scrapermq.domain.company.com:1099 > /tmp/jmx_beans
Welcome to JMX terminal. Type "help" for available commands.
$>beans -d org.apache.activemq
#domain = org.apache.activemq:
$>itaig#iganot-lt:~$ tail -5 /tmp/jmx_beans
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj28.domain.company.com,consumerId=ID_selvmnj28.domain.company.com-45117-1431866382308-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj29.domain.company.com,consumerId=ID_selvmnj29.domain.company.com-52961-1431866382264-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_-1_1,destinationName=topic_//ActiveMQ.Advisory.TempQueue_topic_//ActiveMQ.Advisory.TempTopic,destinationType=Topic,persistentMode=Non-Durable
org.apache.activemq:BrokerName=localhost,Type=Subscription,clientId=Scraper_selvmnj30.domain.company.com,consumerId=ID_selvmnj30.domain.company.com-33036-1431866396456-0_416_1_1,destinationName=SCRAPER_VIDEO_INPUT,destinationType=Queue,persistentMode=Non-Durable
itaig#iganot-lt:~$
So my question is: What am I doing wrong and why does the command work when running manually from terminal but not from the script?
Thanks in advance
Ok I found the problem.
Certain lines included variables which have not been declared in time.
After re-organizing the order of variable declaration the problem has been solved.