in a Dockerfile I'm trying to capture a build arg variable inside a file with no luck.
Now I'm doing something such as:
RUN echo '\n\
enableSemantics( "$DOMAIN_NAME" );\n\
include_once "$IP/LocalSettings.local.php"; ' >> /var/www/w/LocalSettings.php
and I would like to replace $DOMAIN_NAME with DOMAIN_NAME build ARG (but not $IP, which is not...)
What should I change?
Try inverting the quotes :
ARG DOMAIN_NAME
RUN echo "\n\
enableSemantics( '$DOMAIN_NAME' );\n\
include_once '$IP/LocalSettings.local.php'; " >> /var/www/w/LocalSettings.php
I don't know the exact reason , why this works , but most probably , its how Docker resolves the variables , i.e it ignores variables between double quotes.
Also you can skip the $ in this way : \$
Related
In bash when I try to use autocompletion in double-quoted strings it works:
E.g.
echo "My home directory is $HO<t>"
# expands to
echo "My home directory is $HOME"
But when I try the same thing in zsh it just does nothing.
At the same time it works outside strings:
echo My\ home\ directory\ is\ $HO<t>
# expands to
echo My\ home\ directory\ is\ $HOME
Is it possible to make it work the same as bash?
Zsh completion has a bug where completion of parameter names fails if a partial parameter name is followed by a ".
As a workaround, just remove the final " from your string before pressing Tab. Then, Zsh's completion will work correctly and it will automatically add the final " for you.
When I use any command with sudo the environment variables are not there. For example after setting HTTP_PROXY the command wget works fine without sudo. However if I type sudo wget it says it can't bypass the proxy setting.
First you need to export HTTP_PROXY. Second, you need to read man sudo, and look at the -E flag. This works:
$ export HTTP_PROXY=foof
$ sudo -E bash -c 'echo $HTTP_PROXY'
Here is the quote from the man page:
-E, --preserve-env
Indicates to the security policy that the user wishes to preserve their
existing environment variables. The security policy may return an error
if the user does not have permission to preserve the environment.
The trick is to add environment variables to sudoers file via sudo visudo command and add these lines:
Defaults env_keep += "ftp_proxy http_proxy https_proxy no_proxy"
taken from ArchLinux wiki.
For Ubuntu 14, you need to specify in separate lines as it returns the errors for multi-variable lines:
Defaults env_keep += "http_proxy"
Defaults env_keep += "https_proxy"
Defaults env_keep += "HTTP_PROXY"
Defaults env_keep += "HTTPS_PROXY"
For individual variables you want to make available on a one off basis you can make it part of the command.
sudo http_proxy=$http_proxy wget "http://stackoverflow.com"
You can also combine the two env_keep statements in Ahmed Aswani's answer into a single statement like this:
Defaults env_keep += "http_proxy https_proxy"
You should also consider specifying env_keep for only a single command like this:
Defaults!/bin/[your_command] env_keep += "http_proxy https_proxy"
A simple wrapper function (or in-line for loop)
I came up with a unique solution because:
sudo -E "$#" was leaking variables that was causing problems for my command
sudo VAR1="$VAR1" ... VAR42="$VAR42" "$#" was long and ugly in my case
demo.sh
#!/bin/bash
function sudo_exports(){
eval sudo $(for x in $_EXPORTS; do printf '%q=%q ' "$x" "${!x}"; done;) "$#"
}
# create a test script to call as sudo
echo 'echo Forty-Two is $VAR42' > sudo_test.sh
chmod +x sudo_test.sh
export VAR42="The Answer to the Ultimate Question of Life, The Universe, and Everything."
export _EXPORTS="_EXPORTS VAR1 VAR2 VAR3 VAR4 VAR5 VAR6 VAR7 VAR8 VAR9 VAR10 VAR11 VAR12 VAR13 VAR14 VAR15 VAR16 VAR17 VAR18 VAR19 VAR20 VAR21 VAR22 VAR23 VAR24 VAR25 VAR26 VAR27 VAR28 VAR29 VAR30 VAR31 VAR32 VAR33 VAR34 VAR35 VAR36 VAR37 VAR38 VAR39 VAR40 VAR41 VAR42"
# clean function style
sudo_exports ./sudo_test.sh
# or just use the content of the function
eval sudo $(for x in $_EXPORTS; do printf '%q=%q ' "$x" "${!x}"; done;) ./sudo_test.sh
Result
$ ./demo.sh
Forty-Two is The Answer to the Ultimate Question of Life, The Universe, and Everything.
Forty-Two is The Answer to the Ultimate Question of Life, The Universe, and Everything.
How?
This is made possible by a feature of the bash builtin printf. The %q produces a shell quoted string. Unlike the parameter expansion in bash 4.4, this works in bash versions < 4.0
Add code snippets to /etc/sudoers.d
Don't know if this is available in all distros, but in Debian-based distros, there is a line at or near the tail of the /etc/sudoers file that includes the folder /etc/sudoers.d. Herein, one may add code "snippets" that modify sudo's configuration. Specifically, they allow control over all environment variables used in sudo.
As with /etc/sudoers, these "code snippets" should be edited using visudo. You can start by reading the README file, which is also a handy place for keeping any notes you care to make:
$ sudo visudo -f /etc/sudoers.d/README
# files for your snippets may be created/edited like so:
$ sudo visudo -f /etc/sudoers.d/20_mysnippets
Read the "Command Environment" section of 'man 5 sudoers'
Perhaps the most informative documentation on environment configuration in sudo is found in the Command environment section of man 5 sudoers. Here, we learn that a sudoers environment variables that are blocked by default may be "whitelisted" using the env_check or env_keep options; e.g.
Defaults env_keep += "http_proxy HTTP_PROXY"
Defaults env_keep += "https_proxy HTTPS_PROXY"
Defaults env_keep += "ftp_proxy FTP_PROXY"
And so, in the OP's case, we may "pass" the sudoer's environment variables as follows:
$ sudo visudo -f /etc/sudoers.d/10_myenvwlist
# opens the default editor for entry of the following lines:
Defaults env_keep += "http_proxy HTTP_PROXY"
Defaults env_keep += "https_proxy HTTPS_PROXY"
# and any others deemed useful/necessary
# Save the file, close the editor, and you are done!
Get your bearings from '# sudo -V'
The OP presumably discovered the missing environment variable in sudo by trial-and-error. However, it is possible to be proactive: A listing of all environment variables, and their allowed or denied status is available (and unique to each host) from the root prompt as follows:
# sudo -V
...
Environment variables to check for safety:
...
Environment variables to remove:
...
Environment variables to preserve:
...
Note that once an environment variable is "whitelisted" as above, it will appear in subsequent listings of sudo -V under the "preserve" listing.
If you have the need to keep the environment variables in a script you can put your command in a here document like this. Especially if you have lots of variables to set things look tidy this way.
# prepare a script e.g. for running maven
runmaven=/tmp/runmaven$$
# create the script with a here document
cat << EOF > $runmaven
#!/bin/bash
# run the maven clean with environment variables set
export ANT_HOME=/usr/share/ant
export MAKEFLAGS=-j4
mvn clean install
EOF
# make the script executable
chmod +x $runmaven
# run it
sudo $runmaven
# remove it or comment out to keep
rm $runmaven
I am trying to write a dockerfile in which I add a few java-options to a script called envvars.
To achieve that I want to append a few text-lines to said file like so:
RUN echo "JAVA_OPTS=$JAVA_OPTS -Djavax.net.ssl.trustStore=${CERT_DIR}/${HOSTNAME}_truststore.jks" >> ${BIN_DIR}/envvars
RUN echo "JAVA_OPTS=$JAVA_OPTS -Djavax.net.ssl.trustStorePassword=${PWD_TRUSTSTORE}" >> ${BIN_DIR}/envvars
RUN echo "export JAVA_OPTS" >> ${BIN_DIR}/envvars
The issue here is, that I want the misc. placeholders ${varname} (those with curly braces) to be replaced during execution of the docker build command while the substring '$JAVA_OPTS' (i.e. those without braces) should be echoed and thus added to the envvars file verbatim, i.e. in the end the result in the /usr/local/apache2/bin/envvars file should read:
...
JAVA_OPTS=$JAVA_OPTS -Djavax.net.ssl.trustStore=/usr/local/apache2/cert/myserver_truststore.jks
JAVA_OPTS=$JAVA_OPTS -Djavax.net.ssl.trustStorePassword=my_secret
export JAVA_OPTS
How can one escape a $-sign from variable substitution in dockerfiles?
I found hints to use \$ or $$ but neither worked for me.
In case that matters (which I hope/expect not to): I am building the image using "Docker Desktop" on Windows 10 but I would expect the dockerfile to be agnostic of that.
first you need to add this # escape=` to your Dockerfile since \ is an escape charachter in the Dockerfile . then you can use \$ to escape the dollar sign in the RUN section
Example:
# escape=`
RUN echo "JAVA_OPTS=\$JAVA_OPTS -Djavax.net.ssl.trustStore=${CERT_DIR}/${HOSTNAME}_truststore.jks" >> ${BIN_DIR}/envvars
that will be JAVA_OPTS=$JAVA_OPTS in your env file
The Debian control file has a line like this (among many others):
Version: 1.1.0
We are using jenkins to build our application as a .deb package. in Jenkins we are doing something like this:
cp -r $WORKSPACE/p1.1/ourap/scripts/ourapp_debian $TARGET/
cd $TARGET
fakeroot dpkg-deb --build ourapp_debian
We would like to do shomething like this in our control file:
Packages: ourapp
Version: 1.1.$BUILD_NUMBER
but obviously this is not possible.
So we need something like a sed script to find the line starting with Version: and replace anything after it with a constant plus the BUILD_NUMBER env var which Jenkins creates.
We have tried things like this:
$ sed -i 's/xxx/$BUILD_NUMBER/g' control
then put "Version: xxx" in our file, but this doesn't work, and there must be a better way?
Any ideas?
We don't use the change-log, as this package will be installed on servers which no one has access to. the change logs are word docs given to the customer.
We don't use or need any of the Debian helper tools.
Create two files:
f.awk
function vp(s) { # return 1 for a string with version info
return s ~ /[ \t]*Version:/
}
function upd() { # an example of version number update function
v[3] = ENVIRON["BUILD_NUMBER"]
}
vp($0) {
gsub("[^.0-9]", "") # get rid of everything but `.' and digits
split($0, v, "[.]") # split version info into array `v' elements
upd()
printf "Version: %s.%s.%s\n", v[1], v[2], v[3]
next # done with this line
}
{ # print the rest without modifications
print
}
f.example
rest1
Version: 1.1.0
rest2
Run the command
BUILD_NUMBER=42 awk -f f.awk f.example
Expected output is
rest1
Version: 1.1.42
rest2
With single quote:
sed -ri "s/(Version.*\.)[0-9]*/\1$BUILD_NUMBER/g" <control file>
OR
sed -ni "/Version/{s/[0-9]*$/$BUILD_NUMBER/};p" <control file>
I use Jenkins ver. 1.522 and I want to pass a long string with spaces and quotes as a parameter in the parameterized build section. The job only runs a python script.
My problem is that I can't find a way to escape my string so that jenkins passes it correctly to the script.
Assuming...
string: fixVersion in ("foo") AND issuetype in (Bug, Improvement) AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key DESC
variable name: bar
script name: coco.py
When I run the script in the terminal, everything is fine: python coco.py --option 'fixVersion in ("foo") AND issuetype in (Bug, Improvement) AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key DESC'
When I run the same script with jenkins using the parametrized build and try to escape the variable so it end up taken as one parameter by the py script it is oddly espacped by jenkins.
In my jenkins job I call the script: python coco.py --option \'${BAR}\'
and it ends up as:
python coco.py --option '"fixVersion' in '('\''foo'\'')' AND issuetype in '(Bug,' 'Improvement)' in '(Production,' 'Stage)' AND resolution = Fixed ORDER BY resolution ASC, assignee ASC, key 'DESC"'
I also tried \"${BAR}\", \"$BAR\",\'$BAR\'
What it the right way do acheive it?
Try
python coco.py --option "${BAR}"
Alternatively, if you need the single quotes surrounding everything
python coco.py --option \'"${BAR}"\'
In the cases you listed, bash will treat the spaces as delimiters. Putting the double quotes around a variable will preserve the whitespace in a string. Example
aString='foo bar'
for x in $aString; do echo $x; done
# foo
# bar
for x in "$aString"; do echo $x; done
# foo bar
I am using Jenkins v1.606 and ran into this same issue!
The issue that I saw passing user defined string params containing spaces into an execution shell would not properly format the string (only with a parameter that had 1 or more spaces). What you have to watch out for is reviewing the 'output' log. Jenkins will not properly display the string param value within the log.
Example (correct format for containing spaces):
docker exec -i container-base /bin/bash -c "cd /container/path/to/code/ && ./gradlew test_xml -P DISPLAY_NAME='${DISPLAY_NAME}' -P USERNAME='${USERNAME}' -P SERVER_NAME='${SERVER_NAME}'"
Jenkins Output of string (notice the string values format):
+ docker exec -i container-base /bin/bash -c 'cd /container/path/to/code/ && ./gradlew test_xml -P DISPLAY_NAME='\''VM10 USER D33PZ3R0'\'' -P USERNAME='\''d33pz3r0#stackoverflow.com'\'' -P SERVER_NAME='\''stackoverflow.com'\'''
Conclusion:
In my example, the literal command was encapsulated with <">, followed by surrounding the parameters with <'> to escape the literal cmd string and control the Jenkins string syntax. Remember not to just watch your Jenkins output log as it lead me wrong for an entire day while I fought with this! This should be the same for your issue as well, you do not need to escape with \' or other escape characters. Hope this helps!!