While tryting to execute an ansible-playbook from jenkins execute shell, extra quotes are being added by jenkins which causes the ansible-playbook execution to fail. Any work around on this ?
/usr/local/bin/ansible-playbook -i $env $role -e"var1=$var1, var2=$var2, var3=$var3"
The output of the above is :
/usr/local/bin/ansible-playbook -i env-value role-value '-evar1=var1-value, var2=var2-value, var3=var3-value'
If i escape quotes as follows:
/usr/local/bin/ansible-playbook -i $env $role -e\"var1=$var1, var2=$var2, var3=$var3\"
the output of the above is :
/usr/local/bin/ansible-playbook -i env-value role-value '-e"var1=var1-value,' 'var2=var2-value,' 'var3=var3-value"'
What I would do to avoid this quoting issue is to use a -e for every argument:
extra_args+="-e var1=$var1 -e var2=$var2 -e var3=$var3"
ansible-playbook -i $env $role $extra_args
It's also useful when specifying an optional argument that's read in from a jenkins parameter, for example:
if [[ -z $var1 ]]; then
extra_args+="-e var1=$var1"
fi
Also note that if you specify multiple extra vars using -e on the command line you should separate them with spaces, not commas.
Related
Use the command flag looked like a solution but it doesn't work
Inside the following shell:
nix shell github:nixos/nixpkgs/nixpkgs-unstable#hello
the path contain a directory with an executable hello
I've tried this:
nix shell github:nixos/nixpkgs/nixpkgs-unstable#hello --command echo $PATH
I can't see the hello executable
My eyes are not the problem.
diff <( echo $PATH ) <( nix shell github:nixos/nixpkgs/nixpkgs-unstable#hello --command echo $PATH)
It see no difference. It means that the printed path doesn't not contains hello.
Why?
The printed path does not contain hello because if your starting PATH was /nix/var/nix/profiles/default/bin:/run/current-system/sw/bin, then you just ran:
nix shell 'github:nixos/nixpkgs/nixpkgs-unstable#hello' --command \
echo /nix/var/nix/profiles/default/bin:/run/current-system/sw/bin
That is to say, you passed your original path as an argument to the nix shell command, instead of passing it a reference to a variable for it to expand later.
The easiest way to accomplish what you're looking for is:
nix shell 'github:nixos/nixpkgs/nixpkgs-unstable#hello' --command \
sh -c 'echo "$PATH"'
The single quotes prevent your shell from expanding $PATH before a copy of sh invoked by nix is started.
Of course, if you really don't want to start any kind of child shell, then you can run a non-shell tool to print environment variables:
nix shell 'github:nixos/nixpkgs/nixpkgs-unstable#hello' --command \
env | grep '^PATH='
I am running into a problem where my shell script doesn't work if the variable is not set/set to empty string.
For this I am using the alpine image
docker run -dt alpine
docker exec -it <container> sh
Here is the problematic code:
x=""
sh -c "if [ "$x" != "required" ]; then sed; fi"
When x is not set, I get the error:
sh: required: unknown operand
This seems to only be the problem on an empty string. If I set x="lkajsdfasl", it will work just fine.
This just breaks down on an empty string/not set.
Due to the way my docker-compose is setup, I can only use sh and have to use sh -c
It's a quoting issue. Use single quotes around the command like this
sh -c 'if [ "$x" != "required" ]; then sed; fi'
I can run this command from the command line to get the value for the latest release in a GitHub repo:
curl --silent "https://api.github.com/repos/MyOrg/MyRepo/releases/latest"|grep "tag_name"|sed -E 's/."([^"]+)"./\1/'
I'd like to use this in a Jenkinsfile, but I can't seem to properly escape the special characters.
pipeline {
agent any
stages {
stage('Test') {
steps {
sh """
latest=`curl --silent \\\"https://api.github.com/repos/MyOrg/MyRepo/releases/latest\\\"|
grep \\\"tag_name\\\"|
sed -E \\\'s/.*\\\"([^\\\"]+)\\\".*/\1/\\\'`
""".stripIndent()
}
}
}
}
When run in a pipeline, I get the following error:
[Pipeline] sh
curl --silent "https://api.github.com/repos/MyOrg/MyRepo/releases/latest"
grep "tag_name"
sed -E 's/.*"
sed: -e expression #1, char 1: unknown command: `''
instead of wrapping the large expression in backticks, try instead wrapping it in
$(command_1 | command_2 | command_3)
Also add, at the start of the shell command:
set -x;
and you'll get debug output in the console output (possibly). You should then see how your script is being evaluated.
1) I am running a docker container with following cmd (passing few env variables with -e option)
$ docker run --name=xyz -d -e CONTAINER_NAME=xyz -e SSH_PORT=22 -e NWMODE=HOST -e XDG_RUNTIME_DIR=/run/user/0 --net=host -v /mnt:/mnt -v /dev:/dev -v /etc/sysconfig/network-scripts:/etc/sysconfig/network-scripts -v /:/hostroot/ -v /etc/hostname:/etc/host_hostname -v /etc/localtime:/etc/localtime -v /var/run/docker.sock:/var/run/docker.sock --privileged=true cf3681e04bfb
2) After running the container as above, i check the env variable NWMODE inside the container, and it shows correctly as shown below :
$ docker exec -it xyz bash
$ env | grep NWMODE
NWMODE=HOST
3) Now, i created a sample service 'b' shown below which executes a script b.sh (where i try to access NWMODE) :
root#ubuntu16:/etc/systemd/system# cat b.service
[Unit]
Description=testing service b
[Service]
ExecStart=/bin/bash /etc/systemd/system/b.sh
root#ubuntu16:/etc/systemd/system# cat b.sh
#!/bin/bash`
systemctl import-environment
echo "NWMODE:" $NWMODE`
4) Now if i start service 'b' and see its logs, it shows that it is not able to access NWMODE env variable
$ systemctl start b
$ journalctl -fu b
...
systemd[1]: Started testing service b.
bash[641]: NWMODE: //blank for $NWMODE here`
5) Now rather than having 'systemctl import-environment' in b.sh, if i do following then the b.service logs show the correct value of NWMODE env variable:
$ systemctl import-environment
$ systemctl start b
Though the step 5 above works i can't go for it, as all the services in my system will be started automatically by systemd. In that case, can anyone please let me know how can i access the environment variables (passed using 'docker run...' cmd above) in a service file (say for e.g. in b.sh above). Can this be achieved somehow with systemctl import-environment or there is some other way ?
systemd unsets all environment variables to provide a clean environment. Afaik that is intended to be a security feature.
Workaround: Create a file /etc/systemd/system.conf.d/myenvironment.conf:
[Manager]
DefaultEnvironment=CONTAINER_NAME=xyz NWMODE=HOST XDG_RUNTIME_DIR=/run/user/0
systemd will set the environment variables declared in this file.
You can set up an ENTRYPOINT script that automatically creates this file before running systemd. Example:
RUN echo '#! /bin/bash \n\
echo "[Manager] \n\
DefaultEnvironment=$(while read -r Line; do echo -n "$Line" ; done < <(env) \n\
" >/etc/systemd/system.conf.d/myenvironment.conf \n\
exec /lib/systemd/systemd \n\
' >/usr/local/bin/setmyenv && chmod +x /usr/bin/setmyenv
ENTRYPOINT /usr/bin/setmyenv
Instead of creating the script within Dockerfile you can store it outside and add it with COPY:
#! /bin/bash
echo "[Manager]
DefaultEnvironment=$(while read -r Line; do echo -n "$Line" ; done < <(env)
" >/etc/systemd/system.conf.d/myenvironment.conf
exec /lib/systemd/systemd
TL;DR
Run the the command using bash, first store the docker environment variables to a file (or just pipe them two awk), extract & export the variable and finally run your main script.
ExecStart=/bin/bash -c "cat /proc/1/environ | tr '\0' '\n' > /home/env_file; export MY_ENV_VARIABLE=$(awk -F= -v key="MY_ENV_VARIABLE" '$1==key {print $2}' /home/env_file); /usr/bin/python3 /usr/bin/my_python_script.py"
Whatever #mviereck is saying is true, still I have found another solution to this problem.
My use case is to pass an environment variable to my system-d container in the Docker run command (docker run -e MY_ENV_VARIABLE="some_val") and use that in the python script that is run through the system-d unit file.
According to this post (https://forums.docker.com/t/where-are-stored-the-environment-variables/65762) the container environment variables can be found in the running process /proc/1/environ inside the container. Performing a cat does show that the environment variable MY_ENV_VARIABLE=some_val does exist, though in some mangled form.
$ cat /proc/1/environ
HOSTNAME=271fbnd986bdMY_ENV_VARIABLE=some_valcontainer=dockerLC_ALL=CDEBIAN_FRONTEND=noninteractiveHOME=/rootroot#271fb0d986bd
The main task now would be to extract MY_ENV_VARIABLE="some_val" value and pass it to the ExecStart directive in the system-d unit file.
(extraction code referenced from How to grep for value in a key-value store from plain text)
# this outputs a nice key,value pair
$ cat /proc/1/environ | tr '\0' '\n'
HOSTNAME=861f23cd1b33
MY_ENV_VARIABLE=some_val
container=docker
LC_ALL=C
DEBIAN_FRONTEND=noninteractive
HOME=/root
# we can store this in a file for use, too
$ cat /proc/1/environ | tr '\0' '\n' > /home/env_var_file
# we can then reuse the file to extract the value of interest against a key
$ awk -F= -v key="MY_ENV_VARIABLE" '$1==key {print $2}' /home/env_file
some_val
Now in the ExecStart directive in the system-d unit file we can do this:
[Service]
Type=simple
ExecStart=/bin/bash -c "cat /proc/1/environ | tr '\0' '\n' > /home/env_file; export MY_ENV_VARIABLE=$(awk -F= -v key="MY_ENV_VARIABLE" '$1==key {print $2}' /home/env_file); /usr/bin/python3 /usr/bin/my_python_script.py"
I'm trying to do this:
run "echo -n 'foo' > bar.txt"
and the contents of bar.txt ends up being:
-n foo \n
(With \n representing an actual newline)
I use run for other commands like rm -rf and, to my knowledge, it works fine.
I just found this in man echo:
Some shells may provide a builtin echo command which is similar or identical to this utility. Most notably, the builtin echo in sh(1) does not accept the -n option. Consult the builtin(1) manual page.
My version of bash has an echo builtin but seems to be respecting the -n flag. It looks like the shell on your deployment machine doesn't, in which case using the full path to the echo binary might do what you want here:
run "/bin/echo -n 'foo' > bar.txt"
It appears as though the -n flag isn't being interpreted as a flag by the shell. If, from the command line, one executes echo -Y hi, the output will be -Y hi.