Is it possible for a dockerfile / docker compose file to access the host env vars at build time when running docker-compose build app or do you have to manually pass them into the build command?
Environment variables from the host that you want to use at build time would need to be passed into the build command with the --build-arg flag described here.
Related
I want to add zabbix data source into my grafana in kunernetes, for that I created a custom image using this dockerfile and added
ARG GF_INSTALL_PLUGINS="alexanderzobnin-zabbix-app"
Then build the image and ran.
But when I logged to that docker container and run grafana-cli plugins ls, it shows nothing.
How can I create a docker image with zabbix datasource into that?
Since the base image use the GF_INSTALL_PLUGINS environment variable at run-time, it is best for you to set it during the run of the image
While running in docker run -e GF_INSTALL_PLUGINS="alexanderzobnin-zabbix-app" ..., if you use docker-compose or kubernetes then you should pass that value in environment variables.
If you want to install the plugin in the image you can use below statement
RUN grafana-cli plugins install $GF_INSTALL_PLUGINS
But this will not work if you volume mount /var/lib/grafana-plugins to a folder on host
What I'd like to do is this:
FROM mcr.microsoft.com/dotnet/core/aspnet:3.0-alpine
# more stuff
But, this needs to run on an isolated clean build machine which does not have internet access, so I need to route it through a local mirror server (e.g. Artifactory or Nexus or another similar thing)
If the docker image were hosted on docker hub (e.g. FROM ubuntu) then the --registry-mirrors docker configuration option would solve it for us.
BUT because microsoft have decided not to use docker hub, the registry mirror thing isn't working either.
After much effort, I've set up a a custom domain mirror for mcr.microsoft.com and so now I can do this:
FROM microsoft-docker-mirror.local.domain/dotnet/core/aspnet:3.0-alpine
# more stuff
That works. But we have remote workers who may not be on the local office LAN and can't see my local mirror server. What I now want to do is vary it depending on environment. E.g.
ENV MICROSOFT_DOCKER_REPO mcr.microsoft.com
FROM ${MICROSOFT_DOCKER_REPO}/dotnet/core/aspnet:3.0-alpine
My isolated build machine would set the MICROSOFT_DOCKER_REPO environment variable, and everyone else's machines would use the default mcr.microsoft.com
Anyway. Adding the ENV line to the dockerfile results in docker throwing this error:
Error response from daemon: No build stage in current context
There seem to be lots of references to the fact that the FROM line must be the first line in the file, before even any comments...
How can I reference an environment variable in my FROM statement? Or alternatively, how can I make registry mirrors work for things that aren't docker hub?
Thanks!
"ARG is the only instruction that may precede FROM in the Dockerfile" - Dockerfile reference.
ARG MICROSOFT_DOCKER_REPO=mcr.microsoft.com
FROM ${MICROSOFT_DOCKER_REPO}/dotnet/core/aspnet:3.0-alpine
Build with a non-default MICROSOFT_DOCKER_REPO using the --build-arg: docker build --rm --build-arg MICROSOFT_DOCKER_REPO=repo.example.com -t so:58196638 .
Note: you can pass the --build-arg from the hosts environment i.e. MICROSOFT_DOCKER_REPO=repo.example.com docker build --rm --build-arg MICROSOFT_DOCKER_REPO=${MICROSOFT_DOCKER_REPO} -t so:58196638 .
I'm working on a dotnet core docker container (not aspnet), I'd like to specify configuration options for it through appsettings.json. These values will eventually be filled in through environment variables in kubernetes.
However, for local development, how do we easily pass in these settings without storing them in the container?
You can map local volumes to docker -v local_path:container_path.
If you gonna use kubernetes you can use ConfigMap as well.
You can pass env variables while running the container with -e flag of the command docker run.
With this method, you’ll have to pass each variable in the command line. For example, docker run -e VAR1=value1 -e VAR2=value2
If this gets cumbersome, you can write these values to an env file and use this file like so, docker run --env-file=filename
For reference, you can check out the official docs.
I have the Docker configuration task and I need to execute in docker container a bash script which uses Bamboo variables.
Is there a way to pass all Bamboo variables to docker container?
I have lots of Bamboo plans with quite a few different variables in them so putting all variables in container environment variables is not an option.
Of course, I can dump them into file in one task and parse the variables from file in docker task, but I was hoping to find an easier solution.
Thanks!
What version of Bamboo are you using?
There are a problem with bamboo variables is docker containers in some versions of Bamboo, but was fixed in Bamboo 6.1.0:
Unable to use variables in Container name field in Run docker task
Workaround:
Create a Script Task that runs before the Docker Task.
Run commands like
echo "export sourcepath=$ini_source_path" > scriptname.sh
chmod +x scriptname.sh
The Docker Task will be map the ${bamboo.working.directory} to the Docker \data volume.
So the just created scriptname.sh script is available in the Docker container.The script will be executed, and will set the variable correctly.
More info in this post:
How to send bamboo variables from Bamboo script to docker container?
I think the only way is what you mention already - dump the env variables to file in dedicated task earlier in the flow. Say, you do it like so:
#!/bin/bash
env | grep ^bamboo_ > my_env_file
Note the strict regex preventing dumping variables such as PATH.
Then in the docker task, in "Additional arguments" add the following
--env-file=my_env_file
You can use the env_file option using compose:
web:
env_file:
- your-variables.env
or use docker run --env-file=your-variables.env ....
The .env file is a simple key value text file:
# my env file
BAMBOO_ENV=development
I want to automate my build process and need to pass an environment variable to run some of the commands in the Dockerfile. I was wondering if there was any way to do this in Dockerhub. I know docker cloud has something like this, but I was wondering whether the functionality was there in Dockerhub since there is the --build-args argument in the cli for normal building.
Set up Automated builds
Docker Hub (https://hub.docker.com) can automatically build images from source code in an external repository and automatically push the built image to your Docker repositories which will be hosted under your Docker Hub repositories account Eg: https://cloud.docker.com/u/binbash/repository/list
When you set up automated builds (also called autobuilds), you create a list of branches and tags that you want to build into Docker images. When you push code to a source code branch (currently only GitHub / Bitbucket are supported) for one of those listed image tags, the push uses a webhook to trigger a new build, which produces a Docker image. The built image is then pushed to the Docker Hub registry.
For detailed implementation steps please refer to https://docs.docker.com/docker-hub/builds/
Environment variables for builds
You can set the values for environment variables (actually they are mapped to build ARG values - docker build --build-arg - to be exclusively used at build-time - https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg).
NOT to be confused with the environment values, ENV VARS, used by your service at runtime (docker run --env MYVAR1=foo - https://docs.docker.com/v17.12/edge/engine/reference/commandline/run/#set-environment-variables--e-env-env-file)
These Environment Variables configured from the Docker Hub UI are used in your build processes when you configure an automated build. Add your build environment variables by clicking the plus sign next to the Build environment variables section, and then entering a variable name and the value.
When you set variable values from the Docker Hub UI, they can be used by the commands you set in hooks files (THIS IS VERY IMPORTANT and will be extended below), but they are stored so that only users who have admin access to the Docker Hub repository can see their values. This means you can use them to safely store access tokens or other information that should remain secret.
Build hook examples (to implement Docker Hub UI Env vars)
Adding variables from the auto-build’s web UI makes them available inside the hooks. In the hook, you’ll have to use that value to set a custom build arg using --build-arg. Finally, you have to use this custom build arg inside your Dockerfile to manually set an environment variable using ENV command or export.
Example:
Say your want an environment variable TERRAFORM_VERSION='0.12.0-beta2' in your build environment
Step 1.
Add this in the auto-build’s web UI for ‘build environment variables’
Step 2.
Create a custom build hook i.e create a folder called hooks in the same directory as your Dockerfile. Inside the hooks folder, create a file called build. This creates the custom build hook. Docker will use this to build your image. Contents of build:
#!/bin/bash
docker build -t $IMAGE_NAME --build-arg TERRAFORM_VERSION=$TERRAFORM_VERSION .
NOTE: Here $TERRAFORM_VERSION is coming from the web UI.
Step3:
In your Dockerfile
ARG TERRAFORM_VERSION
ENV TERRAFORM_VERSION $TERRAFORM_VERSION
NOTE: Here $TERRAFORM_VERSION is coming from the custom build args in your bash script file named build.
Complete example: https://github.com/binbashar/public-docker-images/tree/master/terraform-resources
That's it! It should work now. Probably renaming ‘build environment variables’ to ‘custom hook environment variables’ in Docker Hub will ease the understanding of this concept in the official documentation (https://docs.docker.com/docker-hub/builds/advanced/).
Extra Points!
There are a number of key environment arguments set upon launching a build script, all of which you can use in your hooks and which can all be useful in making custom build-args.
SOURCE_BRANCH: the name of the branch or the tag that is currently being tested.
SOURCE_COMMIT: the SHA1 hash of the commit being tested.
COMMIT_MSG: the message from the commit being tested and built.
DOCKER_REPO: the name of the Docker repository being built.
DOCKERFILE_PATH: the Dockerfile currently being built.
DOCKER_TAG: the Docker repository tag being built.
IMAGE_NAME: the name and tag of the Docker repository being built. (This variable is a combination of DOCKER_REPO:DOCKER_TAG.)
An example:
Step 1. Create the Dockerfile:
ARG NODE_VERSION
FROM node:$NODE_VERSION
Step 2. Create the hooks/build file:
#!/bin/bash
NODE_VERSION=$(echo $DOCKER_TAG | cut -d "-" -f2)
if [ $DOCKER_TAG == "latest" ]
then
docker build . --build-arg NODE_VERSION=${DOCKER_TAG} -t ${IMAGE_NAME}
else
docker build . --build-arg NODE_VERSION=${NODE_VERSION} -t ${IMAGE_NAME}
fi
Source: github.com/SamuelA