Problem exporting environment variable in Makefile - docker

I would like to export the environment variable in the Makefile. In this case, is to get the IP for debugging with docker
Makefile
start:
export XDEBUG_REMOTE_HOST=$$(/sbin/ip route|awk '/kernel.*metric/ { print $$9 }') \
; docker-compose up -d
Update from answers:
version: '3.5'
services:
php:
build:
context: .
dockerfile: docker/php/Dockerfile
environment:
- XDEBUG_CONFIG="idekey=docker"
- XDEBUG_REMOTE_HOST=${XDEBUG_REMOTE_HOST}
output:
$ make start
export XDEBUG_REMOTE_HOST=$(/sbin/ip route|awk '/kernel.*metric/ { print $9 }') \
; docker-compose up -d
Starting service_php ... done
$ docker-compose exec php bash
WARNING: The XDEBUG_REMOTE_HOST variable is not set. Defaulting to a blank string.

You need to make sure the variable assignment and the docker command run in the same shell. Trivially, put them in the same rule:
start:
XDEBUG_REMOTE_HOST=$$(/sbin/ip route|awk '/kernel.*metric/ { print $$9 }') \
docker-compose up -d
I took out the # because it's probably simply a bad idea, especially if you need to understand what's going on here. You can use make -s once your Makefile is properly tested if you don't want to see what it's doing.
The purpose of export is to expose a variable to subprocesses, but that's not necessary here. Instead, we use the shell's general
variable=value anothervar=anothervalue command
syntax to set the value of a variable for the duration of a single command.
If the internals of docker-compose require the variable to be exported, then of course, you can do that too:
start:
export XDEBUG_REMOTE_HOST=$$(/sbin/ip route|awk '/kernel.*metric/ { print $$9 }') \
; docker-compose up -d
Notice how the backslash at the end of the first line of the command list joins the two commands on a single logical line, so they get passed to the same shell instance, and the ; command separator is required to terminate the first command. (I put the semicolon at beginning of line as an ugly reminder to the reader that this is all one command line.)
Specifically for docker-compose, the customary way to set a variable from the command line is with a specific named option;
start:
docker-compose up -e XDEBUG_REMOTE_HOST=$$(/sbin/ip route|awk '/kernel.*metric/ { print $$9 }') -d
There are other ways to solve this such as the GNU Make .ONESHELL directive but this is simple and straightforward, and portable to any Make.

If you assume that the route exists when make is first invoked, you can assign a make variable as opposed to a shell variable as follows:
export XDRH_MAKE_VAR:=$(shell /sbin/ip route|awk '/kernel.*metric/ { print $$9 }')
start:
#echo XDHR_MAKE_VAR=$(XDRH_MAKE_VAR)
XDEBUG_REMOTE_HOST=$(XDRH_MAKE_VAR) docker-compose up -d
XDRH_FILE:
echo $(XDRH_MAKE_VAR) > $#
someother_target:
XDEBUG_REMOTE_HOST=$(XDRH_MAKE_VAR) some_other_command
command_that_uses_it_as_param $(XDRH_MAKE_VAR)
NOTE_does_not_work:
XDEBUG_REMOTE_HOST=$(XDRH_MAKE_VAR) echo $$XDEBUG_REMOTE_HOST
The last one does not work, because the bash shell will expand $XDEBUG_REMOTE_HOST before assigning it (See here). Also, the variable is set at make parse time, so if any of your rules effect the route, then this will not be reflected in its value.
If you want to access the value in the shell afterwards, you would want to do something like:
bash> make start XDRH_FILE
bash> XDEBUG_REMOTE_HOST=`cat XDRH_FILE`
bash> docker-compose exec php bash

Related

Build a docker container for a "custom" program [duplicate]

I am new to the docker world. I have to invoke a shell script that takes command line arguments through a docker container.
Ex: My shell script looks like:
#!bin/bash
echo $1
Dockerfile looks like this:
FROM ubuntu:14.04
COPY ./file.sh /
CMD /bin/bash file.sh
I am not sure how to pass the arguments while running the container
with this script in file.sh
#!/bin/bash
echo Your container args are: "$#"
and this Dockerfile
FROM ubuntu:14.04
COPY ./file.sh /
ENTRYPOINT ["/file.sh"]
you should be able to:
% docker build -t test .
% docker run test hello world
Your container args are: hello world
Use the same file.sh
#!/bin/bash
echo $1
Build the image using the existing Dockerfile:
docker build -t test .
Run the image with arguments abc or xyz or something else.
docker run -ti --rm test /file.sh abc
docker run -ti --rm test /file.sh xyz
There are a few things interacting here:
docker run your_image arg1 arg2 will replace the value of CMD with arg1 arg2. That's a full replacement of the CMD, not appending more values to it. This is why you often see docker run some_image /bin/bash to run a bash shell in the container.
When you have both an ENTRYPOINT and a CMD value defined, docker starts the container by concatenating the two and running that concatenated command. So if you define your entrypoint to be file.sh, you can now run the container with additional args that will be passed as args to file.sh.
Entrypoints and Commands in docker have two syntaxes, a string syntax that will launch a shell, and a json syntax that will perform an exec. The shell is useful to handle things like IO redirection, chaining multiple commands together (with things like &&), variable substitution, etc. However, that shell gets in the way with signal handling (if you've ever seen a 10 second delay to stop a container, this is often the cause) and with concatenating an entrypoint and command together. If you define your entrypoint as a string, it would run /bin/sh -c "file.sh", which alone is fine. But if you have a command defined as a string too, you'll see something like /bin/sh -c "file.sh" /bin/sh -c "arg1 arg2" as the command being launched inside your container, not so good. See the table here for more on how these two options interact
The shell -c option only takes a single argument. Everything after that would get passed as $1, $2, etc, to that single argument, but not into an embedded shell script unless you explicitly passed the args. I.e. /bin/sh -c "file.sh $1 $2" "arg1" "arg2" would work, but /bin/sh -c "file.sh" "arg1" "arg2" would not since file.sh would be called with no args.
Putting that all together, the common design is:
FROM ubuntu:14.04
COPY ./file.sh /
RUN chmod 755 /file.sh
# Note the json syntax on this next line is strict, double quotes, and any syntax
# error will result in a shell being used to run the line.
ENTRYPOINT ["file.sh"]
And you then run that with:
docker run your_image arg1 arg2
There's a fair bit more detail on this at:
https://docs.docker.com/engine/reference/run/#cmd-default-command-or-options
https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
With Docker, the proper way to pass this sort of information is through environment variables.
So with the same Dockerfile, change the script to
#!/bin/bash
echo $FOO
After building, use the following docker command:
docker run -e FOO="hello world!" test
What I have is a script file that actually runs things. This scrip file might be relatively complicated. Let's call it "run_container". This script takes arguments from the command line:
run_container p1 p2 p3
A simple run_container might be:
#!/bin/bash
echo "argc = ${#*}"
echo "argv = ${*}"
What I want to do is, after "dockering" this I would like to be able to startup this container with the parameters on the docker command line like this:
docker run image_name p1 p2 p3
and have the run_container script be run with p1 p2 p3 as the parameters.
This is my solution:
Dockerfile:
FROM docker.io/ubuntu
ADD run_container /
ENTRYPOINT ["/bin/bash", "-c", "/run_container \"$#\"", "--"]
If you want to run it #build time :
CMD /bin/bash /file.sh arg1
if you want to run it #run time :
ENTRYPOINT ["/bin/bash"]
CMD ["/file.sh", "arg1"]
Then in the host shell
docker build -t test .
docker run -i -t test
I wanted to use the string version of ENTRYPOINT so I could use the interactive shell.
FROM docker.io/ubuntu
...
ENTRYPOINT python -m server "$#"
And then the command to run (note the --):
docker run -it server -- --my_server_flag
The way this works is that the string version of ENTRYPOINT runs a shell with the command specified as the value of the -c flag. Arguments passed to the shell after -- are provided as arguments to the command where "$#" is located. See the table here: https://tldp.org/LDP/abs/html/options.html
(Credit to #jkh and #BMitch answers for helping me understand what's happening.)
Another option...
To make this works
docker run -d --rm $IMG_NAME "bash:command1&&command2&&command3"
in dockerfile
ENTRYPOINT ["/entrypoint.sh"]
in entrypoint.sh
#!/bin/sh
entrypoint_params=$1
printf "==>[entrypoint.sh] %s\n" "entry_point_param is $entrypoint_params"
PARAM1=$(echo $entrypoint_params | cut -d':' -f1) # output is 1 must be 'bash' it will be tested
PARAM2=$(echo $entrypoint_params | cut -d':' -f2) # the real command separated by &&
printf "==>[entrypoint.sh] %s\n" "PARAM1=$PARAM1"
printf "==>[entrypoint.sh] %s\n" "PARAM2=$PARAM2"
if [ "$PARAM1" = "bash" ];
then
printf "==>[entrypoint.sh] %s\n" "about to running $PARAM2 command"
echo $PARAM2 | tr '&&' '\n' | while read cmd; do
$cmd
done
fi

Access files written in docker volumes from the host

I have a docker container writing logfiles to a name volume.
From the host I want to analyce the logfiles and search for given log messages. But when I access the folder which 'docker inspect VOLUMNAME' gives, I get strange behavior, which I do not understand.
e.g. following command does give empty lines as output:
user#docker-host-01:~/docker-server-env/otaya-designdb$ sudo bash -c "for logfile in /var/lib/docker/volumes/design-db-logs/_data/*/*; do echo ${logfile}; done"
user#docker-host-01:~/docker-server-env/otaya-designdb$
What could be the reason?
Your local shell is expanding the variable expansion inside the double quotes before the loop happens. Change the double quotes to single quotes.
That is, when you run
sudo bash -c "for ... ; do echo ${logfile}; done"
first your local shell replaces the variable reference with whatever your local environment has set for $logfile, probably nothing
sudo bash -c 'for ...; do echo ; done'
and then it runs that command. If you change this to single quotes initially
sudo bash -c 'for ... ; do echo ${logfile}; done'
it will avoid this expansion.
You can see this just by putting the word echo at the front of the command: the shell will do its expansion, and then echo will print out the command that would have run.

Best practice to include a bash script in a Docker image

I'm creating a Dockerfile that needs to execute a command, let's call it foo
In order to execute foo, I need to create a .cfc in current directory with token information to call this foo service.
So basically I should do something like
ENV FOO_TOKEN token
ENV FOO_HOST host
ENV FOO_SHARED_DIRECTORY directory
ENV LIBS_TARGET target
and then put the first three variables in a .cfg file and then launch a command using the last variable as target.
Given that if run more than one CMD in a Dockerfile, only the last one will be considered, how should I do that?
My ideal execution is docker run -e "FOO_TOKEN=aaaaaaa" -e "FOO_HOST=myhost" -e "FOO_SHARED_DIRECTORY=Shared" -e "LIBS_TARGET=target/scala-2.11/*.jar" -it --rm --name my-ci-deploy foo/foo:latest
If you wanted to keep everything in the Dockerfile (something I think is rather desirable), you can do something nasty like:
ENV SCRIPT=IyEvdXNyL2Jpbi9lbnYgYmFzaApwZG9fc3Fsc3J2PTAKc3Vkbz0KdmVuZG9yPSQoIGxzYl9yZWxlYXNlIC1p
RUN echo -n "$SCRIPT" | base64 -d | /usr/bin/env bash
Where the contents of SCRIPT= are derived by piping your shell script thusly:
cat my_script.sh | base64 --wrap=0
You may have to adjust the /usr/bin/env bash if you have a really minimal (Alpine) setup.

Docker compose args return of linux command

I am trying to set an arg for docker compose using an output of linux command as my example:
args:
ID_GITLAB: $(id -u $USER)
but when I run my compose I get following error:
ERROR: Invalid interpolation format for "build" option in service "gpc-fontes-ci": "$(id -u $USER)"
Just do
USER_ID=$(id -u $USER) docker-compose
With the Compose file using a regular variable
args:
ID_GITLAB: $USER_ID
You need to escape $ character with another $, e.g.
args:
ID_GITLAB: $$(id -u $USER)
It's the same rule for command.
See: Variable substitution.
You can use a $$ (double-dollar sign) when your configuration needs a literal dollar sign. This also prevents Compose from interpolating a value, so a $$ allows you to refer to environment variables that you don’t want processed by Compose.
For example:
web:
build: .
command: "$$VAR_NOT_INTERPOLATED_BY_COMPOSE"

How to set PS1 in Docker Container

I want to set $PS1 environment variable to the container. It helps me to identify multilevel or complex docker environment setup. Currently docker container prompts with:
root#container-id#
If I can change it as following , I can identify the container by looking at the $PS1 prompt itself.
[Level-1]root#container-id#
I did experiments by exporting $PS1 by making my own image (Dockerfile), .profile file etc. But it's not reflecting.
I had the same problem but in docker-compose context.
Here is how I managed to make it work:
# docker-compose.yml
version: '3'
services:
my_service:
image: my/image
environment:
- "PS1=$$(whoami):$$(pwd) $$ "
Just pass PS1 value as an environment variable in docker-compose.yml configuration file.
Notice how dollars signs need to be escaped to prevent docker-compose from interpolating values (documentation).
This Dockerfile sets PS1 by doing:
RUN echo 'export PS1="[\u#docker] \W # "' >> /root/.bash_profile
We use a similar technique for tracking inputs and outputs in complex container builds.
https://github.com/ianmiell/shutit/blob/master/shutit_global.py#L1338
This line represents the product of hard-won experience dealing with docker/(p)expect combinations:
"SHUTIT_BACKUP_PS1_%s=$PS1 && PS1='%s' && unset PROMPT_COMMAND"
Backing up the prompt is handy if you want to revert, setting the PS1 with PS1= sets the PS1, and unsetting the PROMPT_COMMAND removes any nasty surprises with the terminal being reset etc.. for the expect.
If the question is about how to ensure it's set when you run the container up (as opposed to building), then you may need to add something to your .bashrc / .profile files depending on how you run up your container. As far as I know there's no way to ensure it with a dockerfile directive and make it persist.
I normally create /home/USER/.bashrc or /root/.bashrc, depending on who the USER of the Dockerfile is. That works well. I've tried
ENV PS1 '# '
but that never worked for me.
Here's a way to set the PS1 when you run the container:
docker run -it \
python:latest \
bash -c "echo \"export PS1='[python:latest] \w$ '\" >> ~/.bashrc && bash"
I made a little wrapper script, to be able to run any image with my custom prompt:
#!/usr/bin/env bash
# ~/bin/docker-run
set -eu
image=$1
docker run -it \
-v $(pwd):/opt/app
-w /opt/app ${image} \
bash -c "echo \"export PS1='[${image}] \w$ '\" >> ~/.bashrc && bash"
In debian 9, for running bash, this worked:
RUN echo 'export PS1="[\$ENV_VAR] \W # "' >> /root/.bashrc
It's generally running as root and I generally know I am in docker, so I wanted to have a prompt that indicated what the container was, so I used an environment variable. And I guess the bash I use loads .bashrc preferentially.
Try setting environment variables using docker options
Example:
docker run \
-ti \
--rm \
--name ansibleserver-debug \
-w /githome/axel-ansible/ \
-v /home/lordjea/githome/:/githome/ \
-e "PS1=DEBUG$(pwd)# " \
lordjea/priv:311 bash
docker --help
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Run a command in a new container
Options:
...
-e, --env list Set environment variables
...
You should set that in .profile, not .bashrc.
Just open .profile from your root or home and replace PS1='\u#\h:\w\$ ' with PS1='\e[33;1m\u#\h: \e[31m\W\e[0m\$ ' or whatever you want.
Note that you need to restart your container.
On my MAC I have an alias named lxsh that will start a bash shell using the ubuntu image in my current directory (details). To make the shell's prompt change, I mounted a host file onto /root/.bash_aliases. It's a dirty hack, but it works. The full alias:
alias lxsh='echo "export PS1=\"lxsh[\[$(tput bold)\]\t\[$(tput sgr0)\]\w]\\$\[$(tput sgr0)\] \"" > $TMPDIR/a5ad217e-0f2b-471e-a9f0-a49c4ae73668 && docker run --rm --name lxsh -v $TMPDIR/a5ad217e-0f2b-471e-a9f0-a49c4ae73668:/root/.bash_aliases -v $PWD:$PWD -w $PWD -it ubuntu'
The below solution assumes that you've used Dockerfile USER to set a non-root Linux user for Bash.
What you might have tried without success:
ENV PS1='[docker]$' ## may not work
Using ENV to set PS1 can fail because the value can be overridden by default settings in a preexisting .bashrc when an interactive shell is started. Some Linux distributions are opinionated about PS1 and set it in an initial .bashrc for each user (Ubuntu does this, for example).
The fix is to modify the Dockerfile to set the desired value at the end of the user's .bashrc -- overriding any earlier settings in the script.
FROM ubuntu:20.04
# ...
USER myuser ## the username
RUN echo "PS1='\n[ \u#docker \w ]\n$ '" >>.bashrc

Resources