I am trying to make different commands execute depending on what OS my original host is. Part of the process involves a docker build, so I do not think that using the $(OS) string will help.
My current idea is set the environment variable with a uname in my make file and pass it as an environment variable to docker compose
copy:
cp docker-compose.override.yml.dist docker-compose.override.yml
cp .env.dist .env
dev: copy restart
docker-compose exec cli sh
create: export TARGET=$(shell sh -c uname)
create: copy restart
TARGET="$(TARGET)" docker-compose exec -T cli make build
echo $(TARGET)
echo $(TARGET)
build: export TARGET=$(shell sh -c uname)
build:
ifeq ($(TARGET),Darwin)
cp terra/static.go.dist terra/static.go
go run builder/main.go
rm -rf coverage.out
rm -rf dist/${CLI_VERSION}/osx
mkdir -p dist/${CLI_VERSION}/osx
GOOS=darwin GOARCH=amd64 CGO_ENABLED=0 go build -a -installsuffix cgo -o dist/${CLI_VERSION}/osx/mjolnir
ls -la dist/${CLI_VERSION}/osx/mjolnir
endif
Unfortunately, this fails with following output:
TARGET="Darwin" docker-compose exec -T cli make build
make: Nothing to be done for 'build'.
echo Darwin
Darwin
echo Darwin
Darwin
I will appreciate any pointers as to what I am doing wrong.
A typical setup here is to have a separate Make target for each target platform.
TARGET := $(shell uname)
build: build-$(TARGET)
build-Darwin:
...
GOOS=darwin go build ...
Once you have that, you can explicitly specify that build target in your command.
create: copy restart
docker-compose run cli make build-$(TARGET)
You can also pass Make variables as command-line arguments, which will pass through the layers of Docker more easily than environment variables.
create: copy restart
docker-compose run cli make build TARGET=$(TARGET)
Related
As it stands, my Dockerfile works as written below, but currently I have to run the two commented lines in order to pull, compile, and deploy my application to the server. I tried creating a shell script to run those commands using ADD and ENTRYPOINT, but when I run (using the docker commands below) the shell script runs and then the container exits.
What/How do I modify (I'm assuming, the docker run command) to fix this?
Is there an easier way to import libraries than the multiple URLS for RPM? I tried using YUM, but I wasn't sure how to set up my repo for installing anything.
Dockerfile
FROM registry.access.redhat.com/jboss-eap-7/eap71-openshift
USER root
RUN rpm -i [the URLS of the 40 libraries I need for SVN]
ADD subversion_installer_1.14.1.sh /home/svn_installer.sh
RUN yes | /home/svn_installer.sh
USER jboss
ARG REPO_USER
ARG REPO_PW
ARG REPO_URL
ENV REPO_USER=$REPO_USER
ENV REPO_PW=$REPO_PW
ENV REPO_URL=$REPO_URL
#RUN svn export --username="$REPO_USER" --password="$REPO_PW" "$REPO_URL" /usr/svn/myapp
#RUN /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-1.el7_6.x86_64/bin/jar -cvf $JBOSS_HOME/standalone/deployments/myapp.war /usr/svn/myapp
Docker commands
docker build . -t myapp:latest
docker run -d -p 8080:8080 -p 9990:9990 --env-file=svnvars.cfg myapp:latest
Found out what I was doing wrong. I was trying to use
/opt/eap/bin/standalone.sh
as the last command in my entrypoint script.
I discovered this was wrong by calling
docker images inspect myapp:latest
where I found
"Cmd": [
"/opt/eap/bin/openshift-launch.sh"
],
I was calling the wrong command. So I fixed this by replacing the command in my shell script and changing my ENTRYPOINT to CMD.
Here are the corrected files:
Dockerfile
FROM registry.access.redhat.com/jboss-eap-7/eap71-openshift
USER root
RUN rpm -i [too many libraries]
ADD subversion_installer_1.14.1.sh /home/svn_installer.sh
ADD svnvars.cfg /var/svn/svnvars.cfg
RUN yes | /home/svn_installer.sh
USER jboss
ARG REPO_USER
ARG REPO_PW
ARG REPO_URL
ENV REPO_USER=$REPO_USER
ENV REPO_PW=$REPO_PW
ENV REPO_URL=$REPO_URL
ADD entrypoint.sh /home/entrypoint.sh
CMD /home/entrypoint.sh
entrypoint.sh
#!/bin/bash
svn export --username="$REPO_USER" --password="$REPO_PW" "$REPO_URL" /usr/svn/myapp
cd /usr/svn/myapp
ant war
/opt/eap/bin/openshift-launch.sh
Is it possible to conditionally set an ENV variable in a Dockerfile based on the value of a build ARG?
Ex: something like
ARG BUILDVAR=sad
ENV SOMEVAR=if $BUILDVAR -eq "SO"; then echo "hello"; else echo "world"; fi
Update: current usage based on Mario's answer:
ARG BUILD_ENV=prod
ENV NODE_ENV=production
RUN if [ "${BUILD_ENV}" = "test" ]; then export NODE_ENV=development; fi
However, running with --build-arg BUILD_ENV=test and then going onto the host, I still get
docker run -it mycontainer bin/bash
[root#brbqw1231 /]# echo $NODE_ENV
production
Yes, it is possible, but you need to use your build argument as flag. You can use parameter expansion feature of shell to check condition. Here is a proof-of-concept Docker file:
FROM debian:stable
ARG BUILD_DEVELOPMENT
# if --build-arg BUILD_DEVELOPMENT=1, set NODE_ENV to 'development' or set to null otherwise.
ENV NODE_ENV=${BUILD_DEVELOPMENT:+development}
# if NODE_ENV is null, set it to 'production' (or leave as is otherwise).
ENV NODE_ENV=${NODE_ENV:-production}
Testing build:
docker build --rm -t env_prod ./
...
docker run -it env_prod bash
root#2a2c93f80ad3:/# echo $NODE_ENV
production
root#2a2c93f80ad3:/# exit
docker build --rm -t env_dev --build-arg BUILD_DEVELOPMENT=1 ./
...
docker run -it env_dev bash
root#2db6d7931f34:/# echo $NODE_ENV
development
You cannot run bash code in the Dockerfile directly, but you have to use the RUN command. So, for example, you can change ENV with RUN and export the variable in the if, like below:
ARG BUILDVAR=sad
RUN if [ "$BUILDVAR" = "SO" ]; \
then export SOMEVAR=hello; \
else export SOMEVAR=world; \
fi
I didn't try it but should work.
Your logic is actually correct.
The problem here is that RUN export ... won't work in a Dockerfile because the export command won't persist across images.
Dockerfiles create a temporary container in order to generate the image for it, therefore the environment variables won't exist.
ENV on the other hand as per the documentation states:
The environment variables set using ENV will persist when a container is run from the resulting image.
The only way to do this is during your docker run command when generating the container from your image, and wrap your logic around that:
if [ "${BUILD_ENV}" = "test" ]; then
docker run -e NODE_ENV=development myimage
else
docker run myimage
fi
While you can't set conditional ENV variables but you may be able to acomplish what you are after with the RUN command and a null-coalescing environment variable:
RUN node /var/app/current/index.js --env ${BUILD_ENV:-${NODE_ENV:-"development"}}
If we are talking only about environment variable, then just set it with production
ENV NODE_ENV prod
And during container start in development, you may use -e NODE_ENV=dev.
This way image is always built-in production but the local container is launched in development.
This answer is great if you only need to check whether a build-arg is present and you want to set a default value.
To improve this solution, in case you want to use the data passed by the build-arg, you can do the following:
FROM debian:stable
ARG BUILD_DEVELOPMENT=production
ENV NODE_ENV=$BUILD_DEVELOPMENT
The magic comes from the default value for the ARG.
Passing values to Dockerfile and then to entrypoint script
From the command line pass in your required value (TARG)
docker run --env TARG=T1_WS01 -i projects/msbob
Then in your Dockerfile put something like this
Dockerfile:
# if $TARG is not set then "entrypoint" defaults to Q0_WS01
CMD ./entrypoint.sh ${TARG} Q0_WS01
The entrypoint.sh script only reads the first argument
entrypoint.sh:
#!/bin/bash
[ $1 ] || { echo "usage: entrypoint.sh <$TARG>" ; exit ; }
target_env=$1
I had a similar issue for setting proxy server on a container.
The solution I'm using is an entrypoint script, and another script for environment variables configuration. Using RUN, you assure the configuration script runs on build, and ENTRYPOINT when you run the container.
--build-arg is used on command line to set proxy user and password.
The entrypoint script looks like:
#!/bin/bash
# Load the script of environment variables
. /root/configproxy.sh
# Run the main container command
exec "$#"
configproxy.sh
#!/bin/bash
function start_config {
read u p < /root/proxy_credentials
export HTTP_PROXY=http://$u:$p#proxy.com:8080
export HTTPS_PROXY=https://$u:$p#proxy.com:8080
/bin/cat <<EOF > /etc/apt/apt.conf
Acquire::http::proxy "http://$u:$p#proxy.com:8080";
Acquire::https::proxy "https://$u:$p#proxy.com:8080";
EOF
}
if [ -s "/root/proxy_credentials" ]
then
start_config
fi
And in the Dockerfile, configure:
# Base Image
FROM ubuntu:18.04
ARG user
ARG pass
USER root
# -z the length of STRING is zero
# [] are an alias for test command
# if $user is not empty, write credentials file
RUN if [ ! -z "$user" ]; then echo "${user} ${pass}">/root/proxy_credentials ; fi
#copy bash scripts
COPY configproxy.sh /root
COPY startup.sh .
RUN ["/bin/bash", "-c", ". /root/configproxy.sh"]
# Install dependencies and tools
#RUN apt-get update -y && \
# apt-get install -yqq --no-install-recommends \
# vim iputils-ping
ENTRYPOINT ["./startup.sh"]
CMD ["sh", "-c", "bash"]
Build without proxy settings
docker build -t img01 -f Dockerfile .
Build with proxy settings
docker build -t img01 --build-arg user=<USER> --build-arg pass=<PASS> -f Dockerfile .
Take a look here.
I have a private gitlab-repo on which I use the gitlab-ci.yml to deploy my project into stage and production.
inside the gitlab-ci.yml, I pass two environment-variables NODE_ENV (here I specify if it is stage/producion) and NODE_TARGET (just an info for the app, wha template to use). My gitlab-ci.yml looks like this:
stage_gsue:
stage: staging
script:
- echo "---------- DOCKER LOGIN"
- echo "mypassword" | docker login --username myuser --password-stdin git.example.com:4567
- echo "---------- START DEPLOYING STAGING SERVER"
- echo "-> 1) build image"
- docker build --build-arg buildtarget=gsue --build-arg buildenv=stage -t git.example.com:4567/root/myproject .
- echo "-> 2) push image to registry"
- docker push git.example.com:4567/root/myproject
- echo "-> 3) kill old container"
- docker kill $(docker ps -q) || true
- docker rm $(docker ps -a -q) || true
- echo "-> 4) start new container"
- docker run -dt -e NODE_TARGET=gsue -e NODE_ENV=stage -p 3000:3000 --name myproject git.example.com:4567/root/myproject
- echo "########## END DEPLOYING DOCKER IMAGE"
tags:
- stagerunner
when: manual
works good so far.. but now inside myproject there is a .env-file, in which I have some further variables. I changed the values of these variables and ran the stage-script multiple times, but inside my build image and started container, there are still old values in the .env-file.
How can that be??
additional info:
in my dockerfile I do:
FROM djudorange/node-gulp-mocha
ARG buildenv
ARG buildtarget
RUN git clone https://root:mypassword#git.example.com/root/myproject.git
WORKDIR /myproject
RUN git fetch --all
RUN git pull --all
RUN git checkout stage
RUN npm install -g n
RUN n latest
RUN npm install -g npm
RUN npm i -g gulp-cli --force
RUN npm install
RUN export NODE_ENV=$buildenv
RUN export NODE_TARGET=$buildtarget
RUN NODE_ENV=$buildenv NODE_TARGET=$buildtarget gulp build
#CMD ["node", "server.js"]
The environment overrides anything sent in 'export'. So better write a new env file during the build. So use the following in ur dockerfile:
ARG NODE_ENV
ARG NODE_TARGET
RUN rm -f .env
RUN touch .env
RUN echo "NODE_TARGET=$NODE_TARGET \n\
NODE_ENV=$NODE_ENV" >> ./.env
(fill up the rest of the docekrfile depending upon ur requirements)
Now the build command will be like...
docker-compose build --build-arg NODE_ENV="${ur env arg}" --build-arg NODE_TARGET="<ur target arg>"
So the gitlab build command will be
build_app:
stage: build
script:
- docker-compose build --build-arg NODE_ENV="${NODE_ENV}" --build-arg NODE_TARGET="${NODE_TARGET}"
- echo "Build successful."
- docker-compose up -d
- echo "Deployed!!"
Dont forget to define ur NODE_ENV and NODE_TARGET args in the variables found in the ci cd settings page
I have a script used in the preapration of a Docker image. I have this in the Dockerfile:
COPY my_script /
RUN bash -c "/my_script"
The my_script file contains secrets that I don't want in the image (it deletes itself when it finishes).
The problem is that the file remains in the image despite being deleted because the COPY is a separate layer. What I need is for both COPY and RUN to affect the same layer.
How can I COPY and RUN a script so that both actions affect the same layer?
take a look to multi-stage:
Use multi-stage builds
With multi-stage builds, you use multiple FROM statements in your
Dockerfile. Each FROM instruction can use a different base, and each
of them begins a new stage of the build. You can selectively copy
artifacts from one stage to another, leaving behind everything you
don’t want in the final image. To show how this works, let’s adapt the
Dockerfile from the previous section to use multi-stage builds.
Dockerfile:
FROM golang:1.7.3
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=0 /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
As of 18.09 you can use docker build --secret to use secret information during the build process. The secrets are mounted into the build environment and aren't stored in the final image.
RUN --mount=type=secret,id=script,dst=/my_script \
bash -c /my_script
$ docker build --secret id=script,src=my_script.sh
The script wouldn't need to delete itself.
This can be handled by BuildKit:
# syntax=docker/dockerfile:experimental
FROM ...
RUN --mount=type=bind,target=/my_script,source=my_script,rw \
bash -c "/my_script"
You would then build with:
DOCKER_BUILDKIT=1 docker build -t my_image .
This also sounds like you are trying to inject secrets into the build, e.g. to pull from a private git repo. BuildKit also allows you to specify:
# syntax=docker/dockerfile:experimental
FROM ...
RUN --mount=type=secret,target=/creds,id=cred \
bash -c "/my_script -i /creds"
You would then build with:
DOCKER_BUILDKIT=1 docker build -t my_image --secret id=creds,src=./creds .
With both of the BuildKit options, the mount command never actually adds the file to your image. It only makes the file available as a bind mount during that single RUN step. As long as that RUN step does not output the secret to another file in your image, the secret is never injected in the image.
For more on the BuildKit experimental syntax, see: https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md
I guess you can use a workaround to do this:
Put my_script in a local http server which for example using python -m SimpleHTTPServer, and then the file could be accessed with http://http_server_ip:8000/my_script
Then, in Dockerfile use next:
RUN curl http://http_server_ip:8000/my_script > /my_script && chmod +x /my_script && bash -c "/my_script"
This workaround assure file add & delete in same layer, of course, you may need to add curl install in Dockerfile.
I think RUN --mount=type=bind,source=my_script,target=/my_script bash /my_script in BuildKit can solve your problem.
First, prepare BuildKit
export DOCKER_CLI_EXPERIMENTAL=enabled
export DOCKER_BUILDKIT=1
docker buildx create --name mybuilder --driver docker-container
docker buildx use mybuilder
Then, write your Dockerfile.
# syntax = docker/dockerfile:experimental
FORM debian
## something
RUN --mount=type=bind,source=my_script,target=/my_script bash -c /my_script
The first lint must be # syntax = docker/dockerfile:experimental because it's experimental feature.
And this method are not work in Play with docker, but work on my computer...
My computer us Ubuntu 20.04 with docker 19.03.12
Then, build it with
docker buildx build --platform linux/amd64 -t user/imgname -f ./Dockerfile . --push
I use docker for development and in production for laravel project. I have slightly different dockerfile for development and production. For example I am mounting local directory to docker container in development environment so that I don't need to do docker build for every change in code.
As mounted directory will only be available when running the docker container I can't put commands like "composer install" or "npm install" in dockerfile for development.
Currently I am managing two docker files, is there any way that I can do this with single docker file and decide which commands to run when doing docker build by sending parameters.
What I am trying to achieve is
In docker file
...
IF PROD THEN RUN composer install
...
During docker build
docker build [PROD] -t mytag .
As a best practice you should try to aim to use one Dockerfile to avoid unexpected errors between different environments. However, you may have a usecase where you cannot do that.
The Dockerfile syntax is not rich enough to support such a scenario, however you can use shell scripts to achieve that.
Create a shell script, called install.sh that does something like:
if [ ${ENV} = "DEV" ]; then
composer install
else
npm install
fi
In your Dockerfile add this script and then execute it when building
...
COPY install.sh install.sh
RUN chmod u+x install.sh && ./install.sh
...
When building pass a build arg to specify the environment, example:
docker build --build-arg "ENV=PROD" ...
UPDATE (2020):
Since this was written 3 years ago, many things have changed (including my opinion about this topic). My suggested way of doing this, is using one dockerfile and using scripts. Please see #yamenk's answer.
ORIGINAL:
You can use two different Dockerfiles.
# ./Dockerfile (non production)
FROM foo/bar
MAINTAINER ...
# ....
And a second one:
# ./Dockerfile.production
FROM foo/bar
MAINTAINER ...
RUN composer install
While calling the build command, you can tell which file it should use:
$> docker build -t mytag .
$> docker build -t mytag-production -f Dockerfile.production .
You can use build args directly without providing additional sh script. Might look a little messy, though. But it works.
Dockerfile must be like this:
FROM alpine
ARG mode
RUN if [ "x$mode" = "xdev" ] ; then echo "Development" ; else echo "Production" ; fi
And commands to check are:
docker build -t app --build-arg mode=dev .
docker build -t app --build-arg mode=prod .
I have tried several approaches to this, including using docker-compose, a multi-stage build, passing an argument through a file and the approaches used in other answers. My company needed a good way to do this and after trying these, here is my opinion.
The best method is to pass the arg through the cmd. You can pass it through vscode while right clicking and choosing build image
Image of visual studio code while clicking image build
using this code:
ARG BuildMode
RUN echo $BuildMode
RUN if [ "$BuildMode" = "debug" ] ; then apt-get update \
&& apt-get install -y --no-install-recommends \
unzip \
&& rm -rf /var/lib/apt/lists/* \
&& curl -sSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l /vsdbg ; fi
and in the build section of dockerfile:
ARG BuildMode
ENV Environment=${BuildMode:-debug}
RUN dotnet build "debugging.csproj" -c $Environment -o /app
FROM build AS publish
RUN dotnet publish "debugging.csproj" -c $Environment -o /app
The best way to do it is with .env file in your project.
You can define two variables CONTEXTDIRECTORY and DOCKERFILENAME
And create Dockerfile-dev and Dockerfile-prod
This is example of using it:
docker compose file:
services:
serviceA:
build:
context: ${CONTEXTDIRECTORY:-./prod_context}
dockerfile: ${DOCKERFILENAME:-./nginx/Dockerfile-prod}
.env file in the root of project:
CONTEXTDIRECTORY=./
DOCKERFILENAME=Dockerfile-dev
Be careful with the context. Its path starts from the directory with the dockerfile that you specified, not from docker-compose directory.
In default values i using prod, because if you forget to specify env variables, you won't be able to accidentally build a dev version in production
Solution with diffrent dockerfiles is more convinient, then scripts. It's easier to change and maintain