docker: Variables in two stage build - docker

I am using a two stage build for a docker image;
I want to make a value in my second stage configurable, so I was thinking of using ARG.
However I am not sure the --build-arg command line option applies to other than the first stage, so I came up with this
### First stage
FROM some_base_image
ARG MYUSERNAME=foo
### Second stage
FROM another_base_image
ARG MYUSERNAME=$MYUSERNAME
but that didn't work;
any suggestions how to pass via the build command line some --build-args that should be usable from the second stage?

ARG instructions have a scope, it's described here in the docs: https://docs.docker.com/engine/reference/builder/#scope
An ARG instruction goes out of scope at the end of the build stage where it was defined. To use an arg in multiple stages, each stage must include the ARG instruction.
So you can use an ARG in multiple stages, you just need to use the ARG instruction again as shown in the example:
FROM busybox
ARG SETTINGS
RUN ./run/setup $SETTINGS
FROM busybox
ARG SETTINGS
RUN ./run/other $SETTINGS

Related

Dockerfile build ARG in COPY --from=

I am trying to set up a build process for a project and am running into an issue with using arg in the COPY command.
Part of the process is the build of a library into an image, that is used by multiple other images. The problem occurs in the following lines:
ARG BUILD_CONFIG=dev
COPY --from=company/mu_library:${BUILD_CONFIG} /some/path /other/path
According to the error message, the ${BUILD_CONFIG} is never translated into dev. When adding an echo line beforehand, the echo prints dev.
ARG BUILD_CONFIG=dev
RUN echo ${BUILD_CONFIG}
COPY --from=company/mu_library:${BUILD_CONFIG} /some/path /other/path
Does anyone have an idea how to go around it without creating duplicate stages in the dockerfile that all point to separate tags?
EDIT:
Minimal Dockerfile
FROM node:12.15:0 as prod
ARG BUILD_CONFIG=dev
RUN echo ${BUILD_CONFIG}
COPY --from=test/test-library:${BUILD_CONFIG} /work/dist /work/library/dist
CMD[ "bash" ]
Error:
invalid from flag value test/test-library:${BUILD_CONFIG}: invalid reference format
At last check, you can't use a build arg there, but you can use it in a top level from line using the multi-stage syntax. Then, you also need to define the build arg at the top level:
ARG BUILD_CONFIG=dev
FROM test/test-library:${BUILD_CONFIG} as test-library
FROM node:12.15:0 as prod
ARG BUILD_CONFIG
RUN echo ${BUILD_CONFIG}
COPY --from=test-library /work/dist /work/library/dist
CMD[ "bash" ]

Docker ARG tricky behavior

Docker build tricky behavior:
Dockerfile
ARG php_version="7.2"
ARG flavor="stretch"
FROM php:${php_version}-fpm-${flavor}
#ARG php_version="7.2"
ENV php_v $php_version
CMD echo $php_v
If you build it and run:
docker build -t args:1.0 .
docker run -it --name testargs args:1.0
Output is empty string instead of the expected "7.2"
But if the third ARG line is uncommented we get "7.2"
Why does it behave this way?
Each FROM in a Dockerfile represents a new build stage and an ARG declared before the FROM is not available for use in the newer build stages starting with another FROM.
To quote the relevant section of the doc:
An ARG declared before a FROM is outside of a build stage, so it can’t
be used in any instruction after a FROM.

docker pass build arg to the shell script which needs to be run during the docker building

The problem is i cannot get the docker build arg value in the shell script while running the docker build.
My docker build command:
DOCKER_BUILDKIT=1 docker build --no-cache --progress=plain \
-t test \
--build-arg WHL_PATH=/fake/path \
.
Dockerfile
ARG WHL_PATH
FROM python:3.8.8
COPY test.sh .
RUN ./test.sh $WHL_PATH
and in the test.sh the "$1" is empty...., if in the Dockerfile i put some constant value then i will be able to see that value in the $1, but with docker build arg or set the build arg as ENV VAR are always empty...
Where am i doing wrong, how should i achieve this?
Docker version 20.10.5, build 55c4c88
Build args are scoped. Before the first FROM step they only apply to the FROM steps to adjust the image you use. Within each stage, an ARG step applies to the remaining steps within that stage. So the fix is to reorder your steps:
FROM python:3.8.8
COPY test.sh .
ARG WHL_PATH
RUN ./test.sh $WHL_PATH
Oops, i never realised the position of the ARG instruction matters, basically:
any ARG before the first FROM can be used in any FROM line
any ARG within a build stage (after a FROM) can be used in that build stage
After i moved the ARG WHL_PATH after the line FROM xxx it works perfectly, hope this can save some of your time in the future.
And i was inspired by this answer actually: https://stackoverflow.com/a/50292255/7658313

Passing environment variables across stages in docker multistage image

I have a docker image which has 4 lower layers.
I want to reduce the size of my current image layer using multistage, but this causes a loss of environment, port and cmd config properties across the stages. Is there a way to pass on such config variables across stages in Dockerfile.
You can do one of the following
Use a base container and set the environment values there
FROM alpine:latest as base
ARG version_default
ENV version=$version_default
FROM base
RUN echo ${version}
FROM base
RUN echo ${version}
Other way is to use ARGS as below. There is some repetition but it becomes more centralised
ARG version_default=v1
FROM alpine:latest as base1
ARG version_default
ENV version=$version_default
RUN echo ${version}
RUN echo ${version_default}
FROM alpine:latest as base2
ARG version_default
RUN echo ${version_default}
Note examples copied from https://github.com/moby/moby/issues/37345

Select ENV/ARG in Dockerfile depending on build ARG

I currently have two Dockerfiles that are identical except for some ENV vars at the beginning that have different values. I want to combine them into one Dockerfile and select the ENV vars depending on one build-arg / ARG instead.
I tried something like this with $target being either WIN or UNIX:
FROM alpine
ARG target
ARG VAR1_WIN=Value4Win
ARG VAR1_UNIX=Value4Unix
ARG VAR1=VAR1_$target
ARG VAR1=${!VAR1}
RUN echo $VAR1
But it throws an error: failed to process "${!VAR1}": missing ':' in substitution
I tried a lot but I'm unable to double expand $VAR1.
How do I do this correctly? Thx.
For the conditional syntax, there is a pattern you can use with a multi-stage build:
# ARG defined before the first FROM can be used in FROM lines
ARG target
# first base image for WIN target
FROM alpine as base-WIN
# switching to ENV since ARG doesn't pass through to the next stage
ENV VAR1=Value4Win
# second base image for UNIX target
FROM alpine as base-UNIX
ENV VAR1=Value4Unix
# select one of the above images based on the value of target
FROM base-${target} as release
RUN echo $VAR1
The double expand syntax ${!VAR} only works in bash, while Dockerfile is not parsed by shell. According to the docker manual, Dockerfile does not support double expand.
Note that alpine use ash instead of bash, so it does not support ${!VAR} either. You have to use eval + echo. Try RUN VAR1="$(eval "echo \$$VAR1")"; echo $VAR1.
If you can't pass Value4Win or Value4Unix directly via --build-arg, here is one way:
FROM alpine
ARG target
ARG VAR1=Value4${target}
RUN echo $VAR1
Doing a docker build --build-arg target=WIN . gives:
Step 4/4 : RUN echo $VAR1
---> Running in 6e94bc28d459
Value4WIN

Resources