I have this line here that is inside of a groovy function and i am trying to return a cluster ID
EMR_ID = sh(returnStdout: true, script: "aws emr list-clusters --active --profile \'${PROFILE}\' | jq -r '.Clusters[] | select (.Name=="\'${ENV}\'-emr-cluster") | .Id'")
Without the environmental variables, this works fine.
so the $PROFILE param is set to dev and the $ENV is set to aws-dev.
I keep getting the error that complains about the $ENV var, saying that an unexpected bracket is showing up
It is rarely a good idea to pass environment variables using string-interpolation. There are much better ways.
One you might consider is modifying your invocation of jq as follows:
jq -r --arg env "${ENV}" '.Clusters[] | select (.Name=="\($env)-emr-cluster") | .Id'
Here, $env is an ordinary jq variable, or more accurately perhaps, a defined constant.
After realising that the jenkins snippet generator was actually useful I used this:
sh '''jq -r --arg env "${ENV}" \'.Clusters[] | select (.Name=="\\($env)-emr-cluster") | .Id\' '''
and it worked!
Related
I am using below snippet in my jenking groovy file, where I am getting null response.
def Services= sh(script: "curl -s --header \"PRIVATE-TOKEN: ${gittoken}\" ${url}| jq -r .${Servicename}[]", returnStdout: true)
file which I am downloading is like below.
{
"FrameWork1": [
"sample/sample1",
"sample/sample2"
]
}
Basically I am getting values of Framework1
if I give below one I am getting the first value of Framework object
Working one:###############
def Services= sh(script: "curl -s --header \"PRIVATE-TOKEN: ${gittoken}\" ${url}| jq -r .${Servicename}[1]", returnStdout: true)
jq filter should be .FrameWork1[] to get a list of strings
https://jqplay.org/s/3iCv-off4ep
Don't use shell parameter expansion to generate the jq program, pass the shell parameters as arguments to jq instead (then you do not have to worry about contexts, quoting, escaping, etc.)
... | jq -r --arg svc "$Servicename" '.[$svc][]'
I have an envs.json file with content:
{
"dev-cc1": { "url": "https://my-url.com" },
"dev-cc2": { "url": "https://my-url.com" }
}
I would like to fetch urls based on an environment variable $ENV, but every other related article I came across didn't seem to work, or I just read it wrong. I think I'm missing something super trivial...
Things I tried:
export ENV="dev-cc1"
jq --arg ENV "$ENV" -n '."env.ENV"' envs.json
Because my env variable has a dash in it, I quoted around it, and then tried to reference it inside. However jq returns back null.
The hardcoded query works (jq --arg ENV "$ENV" '."dev-cc1"' envs.json), and I've also confirmed that the environment variable is passed in correctly jq --arg ENV "$ENV" -n 'env.ENV'.
I tried a ton of different ways to substitute this env var in but none worked.. Could anyone please give this a second pair of eyes?
Try
ENV="dev-cc1" jq '.[env.ENV]' envs.json
or
export ENV="dev-cc1"
jq '.[env.ENV]' envs.json
or using a jq variable:
ENV="dev-cc1"
jq --arg ENV "$ENV" '.[$ENV]' envs.json
OK, this might be a silly question. I've got the test.json file:
{
"timestamp": 1234567890,
"report": "AgeReport"
}
What I want to do is to extract timestamp and report values and store them in some env variables:
export $(cat test.json | jq -r '#sh "TIMESTAMP=\(.timestamp) REPORT=\(.report)"')
and the result is:
echo $TIMESTAMP $REPORT
1234567890 'AgeReport'
The problem is that those single quotes break other commands.
How can I get rid of those single quotes?
NOTE: I'm gonna leave the accepted answer as is, but see #Inian's answer for a better solution.
Why make it convoluted with using eval and have a quoting mess? Rather simply emit the variables by joining them with NULL (\u0000) and read it back in the shell environment
{
IFS= read -r -d '' TIMESTAMP
IFS= read -r -d '' REPORT
} < <(jq -r '(.timestamp|tostring) + "\u0000" + .report + "\u0000"' test.json)
This makes your parsing more robust by making the fields joined by NULL delimiter, which can't be part of your string sequence.
From the jq man-page, the #sh command converts its input to be
escaped suitable for use in a command-line for a POSIX shell.
So, rather than attempting to splice the output of jq into the shell's export command which would require carefully removing some quoting, you can generate the entire commandline inside jq, and then execute it with eval:
eval "$(
cat test.json |\
jq -r '#sh "export TIMESTAMP=\(.timestamp) REPORT=\(.report)"'
)"
I am running into an issue where I am trying to run the following command:
aws ecs list-task-definitions | grep Foo-Task-Testing | awk -F '/' '{print $2}'
This returns exactly what I am looking for which is just the task definition name.
When running the command in the CLI with just grep i get this:
"arn:aws:ecs:us-east-1:xxxxxxxxxx:task-definition/Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1",
When including Awk, I get:
Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1"
However, when I try to add this to my Jenkins pipeline:
ecsTaskDefinitionName = Foo-Task-Testing
ecsTaskDefinition = sh(returnStdout: true, script: "aws ecs list-task-definitions | grep $ecsTaskDefinitionName | awk -F '/' '{print \$2}'").trim()
I always get this error message:
/home/jenkins/workspace/foo_test_PR-828#tmp/durable-a5ce4670/script.sh: 1: /home/jenkins/workspace/foo_test_PR-828#tmp/durable-a5ce4670/script.sh: Syntax error: Unterminated quoted string
I have a feeling this has to do with how I am using Awk in Groovy but I can't seem to find enough examples online to confirm this. Can anyone either provide a way of doing this in Groovy w/o using Awk or any experienced Groovy programmers can tell me the correct way of passing Awk?
You can avoid the need for awk with grep -o:
... | grep -o Foo-Task-Testing.*
returns
Foo-Task-Testing-TaskDefinition-OYBZ78KBUI57:1
(-o only returns the match, .* greedily matches everything after)
I'm trying to set up a shell script that will start a screen session (or rejoin an existing one) only if it is invoked from an interactive shell. The solution I have seen is to check if $- contains the letter "i":
#!/bin/sh -e
echo "Testing interactivity..."
echo 'Current value of $- = '"$-"
if [ `echo \$- | grep -qs i` ]; then
echo interactive;
else
echo noninteractive;
fi
However, this fails, because the script is run by a new noninteractive shell, invoked as a result of the #!/bin/sh at the top. If I source the script instead of running it, it works as desired, but that's an ugly hack. I'd rather have it work when I run it.
So how can I test for interactivity within a script?
Give this a try and see if it does what you're looking for:
#!/bin/sh
if [ $_ != $0 ]
then
echo interactive;
else
echo noninteractive;
fi
The underscore ($_) expands to the absolute pathname used to invoke the script. The zero ($0) expands to the name of the script. If they're different then the script was invoked from an interactive shell. In Bash, subsequent expansion of $_ gives the expanded argument to the previous command (it might be a good idea to save the value of $_ in another variable in order to preserve it).
From man bash:
0 Expands to the name of the shell or shell script. This is set
at shell initialization. If bash is invoked with a file of com‐
mands, $0 is set to the name of that file. If bash is started
with the -c option, then $0 is set to the first argument after
the string to be executed, if one is present. Otherwise, it is
set to the file name used to invoke bash, as given by argument
zero.
_ At shell startup, set to the absolute pathname used to invoke
the shell or shell script being executed as passed in the envi‐
ronment or argument list. Subsequently, expands to the last
argument to the previous command, after expansion. Also set to
the full pathname used to invoke each command executed and
placed in the environment exported to that command. When check‐
ing mail, this parameter holds the name of the mail file cur‐
rently being checked.
$_ may not work in every POSIX compatible sh, although it probably works in must.
$PS1 will only be set if the shell is interactive. So this should work:
if [ -z "$PS1" ]; then
echo noninteractive
else
echo interactive
fi
try tty
if tty 2>&1 |grep not ; then echo "Not a tty"; else echo "a tty"; fi
man tty :
The tty utility writes the name of the terminal attached to standard
input to standard output. The name that is written is the string
returned by ttyname(3). If the standard input is not a terminal, the
message ``not a tty'' is written.
You could try using something like...
if [[ -t 0 ]]
then
echo "Interactive...say something!"
read line
echo $line
else
echo "Not Interactive"
fi
The "-t" switch in the test field checks if the file descriptor given matches a terminal (you could also do this to stop the program if the output was going to be printed to a terminal, for example). Here it checks if the standard in of the program matches a terminal.
Simple answer: don't run those commands inside ` ` or [ ].
There is no need for either of those constructs here.
Obviously I can't be sure what you expected
[ `echo \$- | grep -qs i` ]
to be testing, but I don't think it's testing what you think it's testing.
That code will do the following:
Run echo \$- | grep -qs i inside a subshell (due to the ` `).
Capture the subshell's standard output.
Replace the original ` ` expression with a string containing that output.
Pass that string as an argument to the [ command or built-in (depending on your shell).
Produce a successful return code from [ only if that string was nonempty (assuming the string didn't look like an option to [).
Some possible problems:
The -qs options to grep should cause it to produce no output, so I'd expect [ to be testing an empty string regardless of what $- looks like.
It's also possible that the backslash is escaping the dollar sign and causing a literal 'dollar minus' (rather than the contents of a variable) to be sent to grep.
On the other hand, if you removed the [ and backticks and instead said
if echo "$-" | grep -qs i ; then
then:
your current shell would expand "$-" with the value you want to test,
echo ... | would send that to grep on its standard input,
grep would return a successful return code when that input contained the letter i,
grep would print no output, due to the -qs flags, and
the if statement would use grep's return code to decide which branch to take.
Also:
no backticks would replace any commands with the output produced when they were run, and
no [ command would try to replace the return code of grep with some return code that it had tried to reconstruct by itself from the output produced by grep.
For more on how to use the if command, see this section of the excellent BashGuide.
If you want to test the value of $- without forking an external process (e.g. grep) then you can use the following technique:
if [ "${-%i*}" != "$-" ]
then
echo Interactive shell
else
echo Not an interactive shell
fi
This deletes any match for i* from the value of $- then checks to see if this made any difference.
(The ${parameter/from/to} construct (e.g. [ "${-//[!i]/}" = "i" ] is true iff interactive) can be used in Bash scripts but is not present in Dash, which is /bin/sh on Debian and Ubuntu systems.)