I'm building a Docker image using multiple build args, and was wondering if it was possible to pass them to docker build as a file, in the same way --env-file can be pased to docker run. The env file will be parsed by docker run automatically and the variables made available in the container.
Is it possible to specify a file of build arguments in the same way?
There's no such an option, at least for now. But if you have too many build args and want to save it in a file, you can archive it as follows:
Save the following shell to buildargs.sh, make it executable and put it in your PATH:
#!/bin/bash
awk '{ sub ("\\\\$", " "); printf " --build-arg %s", $0 } END { print "" }' $#
Build your image with argfile like:
docker build $(buildargs.sh argfile) -t your_image .
This code is safe for build-arg's that contain spaces and special characters:
for arg in buildarg1 buildarg2 ; do opts+=(--build-arg "$arg") ; done
...
docker run ... "${opts[#]}"
Just substitute buildarg1 and so on with your build-arg's escaped.
Using linux you can create a file (example: arg_file) with the variables declared:
ARG_VAL_1=Hello
ARG_VAL_2=World
Execute the source command on that file:
source arg_file
Then build a docker image using that variables run this command:
docker build \
--build-arg "ARG_VAL_1=$ARG_VAL_1" \
--build-arg "ARG_VAL_2=$ARG_VAL_2" .
Related
Running
docker run -t -i -w=[absolute_work_dir] [docker_image] [executable]
is fine. However when I set the workdir using variable in a PowerShell script (.ps1):
$WORKDIR = [absolute_work_dir]
docker run -t -i -w=$WORKDIR [docker_image] [executable]
it gave the error:
docker: Error response from daemon: the working directory '$WORKDIR' is invalid, it needs to be an absolute path.
What is possibly wrong?
You could assemble a string with your variable, then execute the string as a command.
$DOCKRUNSTR = "docker run -t -i -w=" + $WORKDIR + " [docker_image] [executable]"
& $DOCKRUNSTR
The & will tell powershell to run that string as a command.
Edit: Inside a .ps1 file try Invoke-Expression instead of &
Maybe there's a better powershell super user solution, but this seems like it could get the job done for you.
I have a Dockerfile in which files in a directory are downloaded:
RUN wget https://www.classe.cornell.edu/~cesrulib/downloads/tarballs/ -r -l1 --no-parent -A tgz \
--cut=99 -nH -nv --show-progress --progress=bar:force:noscroll
I know that there is exactly one file here of the form "bmad_dist_YYYY_MMDD.tgz" where "YYYY_MMDD" is a date. For example, the file might be named "bmad_dist_2020_0707.tgz". I want to set a bash variable to the file name without the ".tgz" extension. If this was outside of docker I could use:
FULLNAME=$(ls -1 bmad_dist_*.tgz)
BMADDIST="${FULLNAME%.*}"
So I tried in the dockerfile:
ENV FULLNAME $(ls -1 bmad_dist_*.tgz)
ENV BMADDIST "${FULLNAME%.*}"
But this does not work. Is it possible to do what I want?
Shell expansion does not happen in Dockerfile ENV. Then workaround that you can try is to pass the name during Docker build.
Grab the filename during build name and discard the file or you can try --spider for wget to just get the filename.
ARG FULLNAME
ENV FULLNAME=${FULLNAME}
Then pass the full name dynamically during build time.
For example
docker build --build-args FULLNAME=$(wget -nv https://upload.wikimedia.org/wikipedia/commons/5/54/Golden_Gate_Bridge_0002.jpg 2>&1 |cut -d\" -f2) -t my_image .
The ENV ... ... syntax is mainly for plaintext content, docker build arguments, or other environment variables. It does not support a subshell like your example.
It is also not possible to use RUN export ... and have that variable defined in downstream image layers.
The best route may be to write the name to a file in the filesystem and read from that file instead of an environment variable. Or, if an environment variable is crucial, you could set an environment variable from the contents of that file in an ENTRYPOINT script.
When I run this command on my gitlab pipeline
docker build --build-arg NPM_TOKEN=${NPM_TOKEN} --tag $REGISTRY_IMAGE/web-public:$CI_COMMIT_SHA --tag $REGISTRY_IMAGE/web-public:$CI_COMMIT_REF_NAME packages/web-public
it fails with
build requires exactly 1 argument
It looks to me like I am actually passing one argument, the path; packages/web-public. Flags are not arguments as far as I know.
What am I missing here?
This is the structure of my project
Quote your variables. Something in those variables is expanding to be more than the single arg to the flag.
docker build --build-arg "NPM_TOKEN=${NPM_TOKEN}" --tag "$REGISTRY_IMAGE/web-public:$CI_COMMIT_SHA" --tag "$REGISTRY_IMAGE/web-public:$CI_COMMIT_REF_NAME" packages/web-public
You can also echo that command to see how the variables are expanding, e.g.
echo docker build ...
from https://docs.docker.com/engine/reference/commandline/build/
docker build [OPTIONS] PATH | URL | -
It looks like there's something wrong with your PATH. Try using the absolute path or change to the directory containing the Dockerfile and use .
see also: "docker build" requires exactly 1 argument(s)
My issue was that I had a multi line script entry, eg
script:
- >
docker build \
--network host \
-t ${CI_REGISTRY}/kylehqcom/project/image:latest \
....
As soon as I added to a single line, we were all ok. So I guess the line breaks got "entered" after the first line which meant that the subsequent lines were ignored and the error was returned. Also note, that I CI linted via the GitLab ui and all was syntactically correct.
I have the following line in my Dockerfile which is supposed to capture the display number of the host:
RUN DISPLAY_NUMBER="$(echo $DISPLAY | cut -d. -f1 | cut -d: -f2)" && echo $DISPLAY_NUMBER
When I tried to build the Dockerfile, the DISPLAY_NUMBER is empty. But however when I run the same command directly in the terminal I get the see the result. Is there anything that I'm doing wrong here?
Commands specified with RUN are executed when the image is built. There is no display during build hence the output is empty.
You can exchange RUN with ENTRYPOINT then the command is executed when the docker starts.
But how to forward the hosts display to the container is another matter entirely.
Host environment variables cannot be passed during build, only at run-time.
Only build args can be specified by:
first "declaring the arg"
ARG DISPLAY_NUMBER
and then running
docker build . --no-cache -t disp --build-arg DISPLAY_NUMBER=$DISPLAY_NUMBER
You can work around this issue using the envsubst trick
RUN echo $DISPLAY_NUMBER
And on the command line:
envsubst < Dockerfile | docker build . -f -
Which will rewrite the Dockerfile in memory and pass it to Docker with the environment variable changed.
Edit: Note that this solution is pretty useless though, because you probably
want to do this during run-time anyways, because this value should depend on not on where the image is built, but rather where it is run.
I would personally move that logic into your ENTRYPOINT or CMD script.
According to the documentation, it's possible to define multiple args for the flag --build-arg, but I can't find out how. I tried the following:
docker build -t essearch/ess-elasticsearch:1.7.6 --build-arg number_of_shards=5 number_of_replicas=2 --no-cache .
=> This returns an error.
I also tried:
docker build -t essearch/ess-elasticsearch:1.7.6 --build-arg number_of_shards=5,number_of_replicas=2 --no-cache .
=> This sets one variable, number_of_shards, to the value "5,number_of_replicas=2"
Any idea how I can define multiple arguments?
Use --build-arg with each argument.
If you are passing two argument then add --build-arg with each argument like:
docker build \
-t essearch/ess-elasticsearch:1.7.6 \
--build-arg number_of_shards=5 \
--build-arg number_of_replicas=2 \
--no-cache .
The above answer by pl_rock is correct, the only thing I would add is to expect the ARG inside the Dockerfile if not you won't have access to it. So if you are doing
docker build -t essearch/ess-elasticsearch:1.7.6 --build-arg number_of_shards=5 --build-arg number_of_replicas=2 --no-cache .
Then inside the Dockerfile you should add
ARG number_of_replicas
ARG number_of_shards
I was running into this problem, so I hope I help someone (myself) in the future.
If you want to use environment variable during build. Lets say setting username and password.
username= Ubuntu
password= swed24sw
Dockerfile
FROM ubuntu:16.04
ARG SMB_PASS
ARG SMB_USER
# Creates a new User
RUN useradd -ms /bin/bash $SMB_USER
# Enters the password twice.
RUN echo "$SMB_PASS\n$SMB_PASS" | smbpasswd -a $SMB_USER
Terminal Command
docker build --build-arg SMB_PASS=swed24sw --build-arg SMB_USER=Ubuntu . -t IMAGE_TAG
It's a shame that we need multiple ARG too, it results in multiple layers and slows down the build because of that, and for anyone also wondering that, currently there is no way to set multiple ARGs per one line.
In case you want to pass automatically build arguments from a specific file, you can do it this way :
docker build $(cat .my-env-file-name | while read line; do out+="--build-arg $line"; done; echo $out; out="") .
A way to pass in build arguments from a file using xargs is as follows:
cat .MY_ENV_FILE | xargs printf -- '--build-arg %s\n' | xargs docker build -t MY_TAG .