I am using below snippet in my jenking groovy file, where I am getting null response.
def Services= sh(script: "curl -s --header \"PRIVATE-TOKEN: ${gittoken}\" ${url}| jq -r .${Servicename}[]", returnStdout: true)
file which I am downloading is like below.
{
"FrameWork1": [
"sample/sample1",
"sample/sample2"
]
}
Basically I am getting values of Framework1
if I give below one I am getting the first value of Framework object
Working one:###############
def Services= sh(script: "curl -s --header \"PRIVATE-TOKEN: ${gittoken}\" ${url}| jq -r .${Servicename}[1]", returnStdout: true)
jq filter should be .FrameWork1[] to get a list of strings
https://jqplay.org/s/3iCv-off4ep
Don't use shell parameter expansion to generate the jq program, pass the shell parameters as arguments to jq instead (then you do not have to worry about contexts, quoting, escaping, etc.)
... | jq -r --arg svc "$Servicename" '.[$svc][]'
Related
I have this line here that is inside of a groovy function and i am trying to return a cluster ID
EMR_ID = sh(returnStdout: true, script: "aws emr list-clusters --active --profile \'${PROFILE}\' | jq -r '.Clusters[] | select (.Name=="\'${ENV}\'-emr-cluster") | .Id'")
Without the environmental variables, this works fine.
so the $PROFILE param is set to dev and the $ENV is set to aws-dev.
I keep getting the error that complains about the $ENV var, saying that an unexpected bracket is showing up
It is rarely a good idea to pass environment variables using string-interpolation. There are much better ways.
One you might consider is modifying your invocation of jq as follows:
jq -r --arg env "${ENV}" '.Clusters[] | select (.Name=="\($env)-emr-cluster") | .Id'
Here, $env is an ordinary jq variable, or more accurately perhaps, a defined constant.
After realising that the jenkins snippet generator was actually useful I used this:
sh '''jq -r --arg env "${ENV}" \'.Clusters[] | select (.Name=="\\($env)-emr-cluster") | .Id\' '''
and it worked!
I have an envs.json file with content:
{
"dev-cc1": { "url": "https://my-url.com" },
"dev-cc2": { "url": "https://my-url.com" }
}
I would like to fetch urls based on an environment variable $ENV, but every other related article I came across didn't seem to work, or I just read it wrong. I think I'm missing something super trivial...
Things I tried:
export ENV="dev-cc1"
jq --arg ENV "$ENV" -n '."env.ENV"' envs.json
Because my env variable has a dash in it, I quoted around it, and then tried to reference it inside. However jq returns back null.
The hardcoded query works (jq --arg ENV "$ENV" '."dev-cc1"' envs.json), and I've also confirmed that the environment variable is passed in correctly jq --arg ENV "$ENV" -n 'env.ENV'.
I tried a ton of different ways to substitute this env var in but none worked.. Could anyone please give this a second pair of eyes?
Try
ENV="dev-cc1" jq '.[env.ENV]' envs.json
or
export ENV="dev-cc1"
jq '.[env.ENV]' envs.json
or using a jq variable:
ENV="dev-cc1"
jq --arg ENV "$ENV" '.[$ENV]' envs.json
OK, this might be a silly question. I've got the test.json file:
{
"timestamp": 1234567890,
"report": "AgeReport"
}
What I want to do is to extract timestamp and report values and store them in some env variables:
export $(cat test.json | jq -r '#sh "TIMESTAMP=\(.timestamp) REPORT=\(.report)"')
and the result is:
echo $TIMESTAMP $REPORT
1234567890 'AgeReport'
The problem is that those single quotes break other commands.
How can I get rid of those single quotes?
NOTE: I'm gonna leave the accepted answer as is, but see #Inian's answer for a better solution.
Why make it convoluted with using eval and have a quoting mess? Rather simply emit the variables by joining them with NULL (\u0000) and read it back in the shell environment
{
IFS= read -r -d '' TIMESTAMP
IFS= read -r -d '' REPORT
} < <(jq -r '(.timestamp|tostring) + "\u0000" + .report + "\u0000"' test.json)
This makes your parsing more robust by making the fields joined by NULL delimiter, which can't be part of your string sequence.
From the jq man-page, the #sh command converts its input to be
escaped suitable for use in a command-line for a POSIX shell.
So, rather than attempting to splice the output of jq into the shell's export command which would require carefully removing some quoting, you can generate the entire commandline inside jq, and then execute it with eval:
eval "$(
cat test.json |\
jq -r '#sh "export TIMESTAMP=\(.timestamp) REPORT=\(.report)"'
)"
I've tried various iterations of using either ", ' and ` to enclose a curl query to an instance of jira in order to get all issues for a particular fix Version.
curl -D- -u username:password -X POST -H "Content-Type: application/json" -d '{"jql":"project = PROJ AND fixVersion=Version-1.2.3"}' "https://thejirainstall.com/jira/rest/api/2/search"
However, using this and a couple of other change on fixVersion such as:
fixVersion="Version-1.2.3"
or
fixVersion=\"Version-1.2.3\"
or
fixVersion=Version-1\u002e2\u002e3
Add and remove quotes at will.
The ones that don't fail outright return:
{"errorMessages":["Error in the JQL Query: '\\.' is an illegal JQL escape sequence. The valid escape sequences are \\', \\\", \\t, \\n, \\r, \\\\, '\\ ' and \\uXXXX. (line 1, character 38)"],"errors":{}}
How do I either escape periods . or add another set of quotes?
Ok, so it turns out that Jira doesn't permit version names in jql syntax. The version id must be used instead.
And, in order to get the version id you must parse the result from https://thejirainstall.com/jira/rest/api/2/project/ON/versions?
This now means that I have to use a JSON parser anyway. So, now I'm using jq via homebrew install jq
My current solution is to write a bash script as below:
JIRA_FIXVERSION
fixVersionQuery='https://thejirainstall.com/jira/rest/api/2/project/ON/versions?';
myJSONResponse=`curl -u username:password -X GET -H "Content-Type: application/json" --insecure --silent $fixVersionQuery |jq '.[] | {id,name} | select(.name=="Version-1.2.3" | .["id"]'`;
echo $myJSONResponse;
I have a script with which I POST data to a server using cURL. When I use an HTML form to POST the same data, the POST looks something like this and all is well:
description=Something&name=aName&xml=wholeBiunchOfData&xslt=moreData
The XML and XSLT are large and change; I would prefer to maintain them in external files. However, the following does not work as I expect;
curl --cookie cjar --cookie-jar cjar --location --output NUL ^
--data "name=aName&description=Something" ^
--data "xml=#localFile.xml" ^
--data "xslt=#localFile.xslt" ^
http://someUrl.html
I have tried various combinations of the # and local files without success. How do I POST the contents of a file?
Looking at the man page it looks like the --data #file syntax does not permit for a variable name, it must be in the file. http://paulstimesink.com/2005/06/29/http-post-with-curl/. You could also try using a backtick
curl --cookie cjar --cookie-jar cjar --location --output NUL ^
--data "name=aName&description=Something" ^
--data "xml=`cat localFile.xml`" ^
--data "xslt=`cat someFile.xml`" ^
http://someUrl.html
I'd recommend trying the following:
curl --cookie cjar --cookie-jar cjar --location --output NUL ^
--data "name=aName&description=Something" ^
--data-urlencode "xml#localFile.xml" ^
--data-urlencode "xslt#localFile.xslt" ^
http://someUrl.html
XML (including stylesheets) will need to be URL-encoded before being made part of a URL.
You can also use --trace-ascii - as an additional parameter to dump the input and output to standard out for further debugging, and you can find more information on the main man page.
Hope this helps!