I want to pass into docker run all the environment variables I've configured in the Travis web UI.
I'm able to run env > .env to save them to a file and then pass that into docker via --env-file .env.
Unfortunately, this also overrides system ones such as PATH that interfere with the container.
I'm able to filter out PATH using env | grep -vE "^(PATH=)" > .env but I'm wondering whether there's a way to get just the Travis ones?
Here's my .travis.yml:
language: bash
sudo: required
services:
- docker
before_install:
- env | grep -vE "^(PATH=)" > .env
install:
- docker build -t mycompany/myapp .
script:
- docker run -i --env-file .env mycompany/myapp nosetests
after_success:
- echo "SUCCESS!"
I don't recommend passing all your environment vars, but if you whitelist them by prefixing them with something like, say, TRAVIS_ you could do something like:
export TRAVIS_WUT=foo
export TRAVIS_FOO=asdf
docker run $(printenv | grep -E '^TRAVIS_' | sed 's/TRAVIS_/-e /g')
# would run -> docker run -e FOO=asdf -e WUT=foo something
Related
I'm attempting to have my .gitlab-ci.yml file use an image off the Gitlab container registry. I have successfully uploaded the Dockerfile to the registry and I can pull the image from the registry on my local machine and build a container just fine. However, when using the image for my .gitlab-ci.yml file, I get this error:
Authenticating with credentials from job payload (GitLab Registry)
standard_init_linux.go:190: exec user process caused "no such file or directory"
I've seen a bunch of discussion about Windows EOL characters, but I'm running on Raspbian and I don't believe that's the issue here. However, I'm pretty new at this and can't figure out what the issue is. I appreciate any help.
.gitlab-ci.yml file:
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
stages:
- test-version
test:
stage: test-version
image: registry.gitlab.com/my/project/test:latest
script:
- python --version
test.Dockerfile (which is in the registry as registry.gitlab.com/my/project/test:latest):
ARG base_img="python:3.6"
FROM ${base_img}
# Install Python packages
RUN pip install --upgrade pip
Edit:
Another thing to note is that if I change the image in the .gitlab-ci.yml file to just python:3.6, then it runs just fine. It's only when I attempt to link my image in the registry.
As you confirmed in the comments, gitlab.com/my/project is a private repository, so that one cannot directly use docker pull or the image: property with registry.gitlab.com/my/project/test:latest.
However, you should be able to adapt your .gitlab-ci.yml by using the image: docker:latest and manually running docker commands (including docker login).
This relies on the so-called Docker-in-Docker (dind) approach, and it is supported by GitLab CI.
Here is a generic template of .gitlab-ci.yml relying on this idea:
stages:
- test-version
test:
stage: test-version
image: docker:latest
services:
- docker:dind
variables:
# GIT_STRATEGY: none # uncomment if "git clone" is unneeded
IMAGE: "registry.gitlab.com/my/project/test:latest"
before_script:
# - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
# or better
- echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
script:
- docker pull "$IMAGE"
- |
docker run --rm -v "$PWD:/build" -w /build "$IMAGE" /bin/bash -c "
export PS4='+ \e[33;1m(\$0 # line \$LINENO) \$\e[0m ' # optional
set -ex # mandatory
## TODO insert your multi-line shell script here ##
echo \"One comment\" # quotes must be escaped here
: A better comment
python --version
echo $PWD # interpolated outside the container
echo \$PWD # interpolated inside the container
## (cont'd) ##
" "$CI_JOB_NAME"
- echo done
This leads to a bit more boilerplate, but this is generic so you can just replace the IMAGE definition and replace the TODO area with your own Bash script, just ensuring that the two items are fulfilled:
If your shell code contains some double quotes, you need to escape them, because the whole code is surrounded by docker run … " and " (the last variable "$CI_JOB_NAME" is a detail, it is optional and just allows one to override the $0 variable referenced within the Bash variable PS4)
If your shell code contains local variables, they need to be escaped (cf. the \$PWD above), otherwise these variables will be resolved prior running the docker run … "$IMAGE" /bin/sh -c "…" command itself.
I'm creating a Dockerfile that needs to execute a command, let's call it foo
In order to execute foo, I need to create a .cfc in current directory with token information to call this foo service.
So basically I should do something like
ENV FOO_TOKEN token
ENV FOO_HOST host
ENV FOO_SHARED_DIRECTORY directory
ENV LIBS_TARGET target
and then put the first three variables in a .cfg file and then launch a command using the last variable as target.
Given that if run more than one CMD in a Dockerfile, only the last one will be considered, how should I do that?
My ideal execution is docker run -e "FOO_TOKEN=aaaaaaa" -e "FOO_HOST=myhost" -e "FOO_SHARED_DIRECTORY=Shared" -e "LIBS_TARGET=target/scala-2.11/*.jar" -it --rm --name my-ci-deploy foo/foo:latest
If you wanted to keep everything in the Dockerfile (something I think is rather desirable), you can do something nasty like:
ENV SCRIPT=IyEvdXNyL2Jpbi9lbnYgYmFzaApwZG9fc3Fsc3J2PTAKc3Vkbz0KdmVuZG9yPSQoIGxzYl9yZWxlYXNlIC1p
RUN echo -n "$SCRIPT" | base64 -d | /usr/bin/env bash
Where the contents of SCRIPT= are derived by piping your shell script thusly:
cat my_script.sh | base64 --wrap=0
You may have to adjust the /usr/bin/env bash if you have a really minimal (Alpine) setup.
I have a docker-compose.yml that references ${host_repo_dir}. Trying to run the service defined there as follows:
docker-compose run -e host_repo_dir=$(pwd) http-api
Output:
WARNING: The host_repo_dir variable is not set. Defaulting to a blank string.
What gives? I think I'm following https://docs.docker.com/compose/reference/run/.
my environment:
docker version: 18.09.3
docker-compose version: 1.24.0
docker for windows, accessed from WSL
Is this a WSL problem?
Update: setting the env vars in advance sort-of works, but seemingly not always:
echo "host repo dir: ${host_repo_dir}, repo name: ${repository_name}, docker repo: ${docker_repository}"
docker-compose run \
-e host_repo_dir \
-e repository_name \
-e docker_repository \
${repository_name} || ( cd ${previous_directory} ; exit 3 )
output:
host repo dir: /c/Users/muellmi1/projects/payoff-2015/http-api, repo name: http-api, docker repo: localhost.localdomain
WARNING: The docker_repository variable is not set. Defaulting to a blank string.
docker_repository was set beforehand by $1 at the beginning of this script. I now understand that it is required to export the variables beforehand. So what worked for me was running export [VARNAME] for each of the variables later passed to docker-compose.
Assuming host_repo_dir is already set and exported, you can do.
docker-compose run -e host_repo_dir http-api
I have a line in my Dockerfile:
&& echo "xdebug.remote_host=192.168.0.216" >> /usr/local/etc/php/conf.d/xdebug.ini`
I want to make the IP dynamic. How would I get the host IP in there?
You need to use build-time variables (–build-arg).
This flag allows you to pass the build-time variables that are accessed like regular environment variables in the RUN instruction of the Dockerfile.
So, Dockerfile is modified to:
ARG IP_ADDRESS
RUN ... && echo "xdebug.remote_host=$IP_ADDRESS" >> /usr/local/etc/php/conf.d/xdebug.ini`
And you just need to define build-time variable IP_ADDRESS during image building:
$ docker build --build-arg IP_ADDRESS=<IP_ADDRESS> .
If you use docker-compose:
1. Create file .env with the following content:
IP_ADDRESS="<IP_ADDRESS>"
You can make it every time like (example is for a linux machine):
IP_ADDRESS=$(ip a | grep <interface> | grep inet | awk '{print $2}' | awk -F'/' '{print $1}')
echo "IP_ADDRESS=$IP_ADDRESS" > .env
2. Use the following docker-compose.yaml to build your image:
version: '3'
services:
myservice:
build:
context: .
args:
IP_ADDRESS: ${IP_ADDRESS}
3. Build the above image:
docker-compose build
There's no simple in built way to get the Docker host IP (unless you are using Docker for Mac)
Entrypoint
It's best not to set a Docker host IP at build time, otherwise the image will be tied to the host it was built on and won't work anywhere else.
An ENTRYPOINT can be used to do the config setup based on an environment variable and then pass through all commands to the container:
#!/bin/sh
if [ -n "$IP_ADDRESS" ]; then
echo "xdebug.remote_host=$IP_ADDRESS" >> /usr/local/etc/php/conf.d/xdebug.ini
else
echo "No environment variable IP_ADDRESS set for xdebug"
fi
exec "$#"
Then run with:
docker run -e IP_ADDRESS=192.168.51.5 me/app-debug
Docker for Mac
On Docker for Mac 17.12+ you can use the host name docker.for.mac.host.internal
Xdebug
Another option is setting xdebug.remote_connect_back = 1 so you don't need a specific remote_host for xdebug.
Build
Nicolay's answer covers the build time setup.
I am using Docker 17.04.0-ce, build 4845c56 with docker-compose 1.12.0, build b31ff33 on Ubuntu 16.04.2 LTS. I simply want to pass an environment variable and display it from my script running in a container. I am doing this according to the documentation https://docs.docker.com/compose/compose-file/#environment . The problem is that the variable is not passed to the container.
My docker-compose.yml file:
env-file-test:
build: .
dockerfile: Dockerfile
environment:
- DEMO_VAR
My Dockerfile:
FROM alpine
COPY docker-start.sh /
CMD ["/docker-start.sh"]
And the docker-start.sh file:
#!/bin/sh
echo "DEMO_VAR Var Passed in: $DEMO_VAR"
I try to set the variable in my current terminal session and pass it to the container:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo docker-compose up
Starting envfiletest_env-file-test_1
Attaching to envfiletest_env-file-test_1
env-file-test_1 | DEMO_VAR Var Passed in:
envfiletest_env-file-test_1 exited with code 0
So you can see that the variable DEMO_VAR is empty!
I also tried using variables in docker-compose.yml like this: DEMO_VAR=${DEMO_VAR} but then when I run sudo docker-compose up, I get a warning: "WARNING: The DEMO_VAR variable is not set. Defaulting to a blank string.".
What am I doing wrong? What should I do to pass the variable to the container?
I found a solution. Answering my own question...
The problem was with the sudo command. It turned out that it does not pass environment variables by default. There are some possible solutions:
Use sudo -E. Demo:
$ export DEMO_VAR=aabbdd
$ echo $DEMO_VAR
aabbdd
$ sudo -E docker-compose up
env-file-test_1 | DEMO_VAR Var Passed in: aabbdd
Use sudo VAR=value:
sudo DEMO_VAR=$DEMO_VAR docker-compose up
Add environment variables to the sudoers file (https://stackoverflow.com/a/8636711)
Use docker without sudo (https://askubuntu.com/questions/477551/how-can-i-use-docker-without-sudo)
you should use ENV in your Dockerfile, and avoid export.
See the doc
https://docs.docker.com/engine/reference/builder/#env