how to dynamically set an environment variable to true within a dockerfile - docker

my app has feature flags that i would like to dynamically set to true for my npm build.
essentially i'd like to do something like
COMPILE_ASSETS=true npm build or NEW_EMAILS=true npm build, only dynamically from CI.
i have a CI pipeline that will grab the flag, but am having trouble setting it to the true and running npm in the Dockerfile.
my Dockerfile -
FROM ubuntu:bionic
ARG FEATURE_FLAG
RUN if [ "x$FEATURE_FLAG" = "x" ] ; \
then npm run build ; \
else $FEATURE_FLAG=true npm run build; \
fi
this gets run with --
docker build --no-cache --rm -t testing --build-arg FEATURE_FLAG=my_feature_flag . (i would like to keep this the way it is)
in CI i get
/bin/sh: 1: my_feature_flag=true: not found
i've tried various forms of the else statement --
else export $FEATURE_FLAG=true npm run build; (this actually looks like it works on my mac but fails in CI with export: : bad variable name
else ${FEATURE_FLAG:+$FEATURE_FLAG=true} npm build;
else eval(`$FEATURE_FLAG=true npm build`);
`else env $FEATURE_FLAG=true bash -c 'npm build';
these all fail :(
i've tried reworking the Dockerfile completely and setting the flag to true as an ENV --
ARG FEATURE_FLAG
ENV FF_SET_TRUE=${FEATURE_FLAG:+$FEATURE_FLAG=true}
ENV FF_SET_TRUE=${FF_SET_TRUE:-null}
RUN if [ "$FF_SET_TRUE" = "null" ] ; \
then npm build; \
else $FF_SET_TRUE npm build; \
fi
nothing works! is this simply a bash limitation? is expanding a variable before running a command is not possible?
or is this not possible with Docker?

Did you mean:
FROM ubuntu:bionic
ARG FEATURE_FLAG
RUN set -eux; \
if [ "x$FEATURE_FLAG" == "x" ] ; then \
npm run build ; \
else \
eval $($FEATURE_FLAG=true npm run build); \
fi
You need to wrap your command in eval for the variable to expand based on the ARGS being passed.

this worked!
ARG FEATURE_FLAG
RUN if [ -z "$FEATURE_FLAG" ] ; \
then npm run build ; \
else \
echo setting $FEATURE_FLAG to true; \
env "$FEATURE_FLAG"=true sh -c 'npm run build'; \
fi

Related

Command `supervisorctl restart` failed with exit code using mac m1

I am running on mac m1, docker, gulp.
my first error was command ld not found, but i fixed it in here.
how to solve running gcc failed exist status 1 in mac m1?
After that it leads me to this error.
this is the full error:
[17:09:04] 'restart-supervisor' errored after 1.04 s
[17:14:45] '<anonymous>' errored after 220 ms
[17:14:45] Error in plugin "gulp-shell"
Message:
Command `supervisorctl restart projectname` failed with exit code 7
[17:14:45] 'restart-supervisor' errored after 838 ms
Ive done a lot of research:
Ive tried doing this, but the command isn't found.
https://github.com/Supervisor/supervisor/issues/121
This as well.
https://github.com/Supervisor/supervisor/issues/1223.
I even change my image to arm64v8/golang:1.17-alpine3.14
this is my gulpfile.js:
var gulp = require("gulp");
var shell = require('gulp-shell');
gulp.task("build-binary", shell.task(
'go build'
));
gulp.task("restart-supervisor", gulp.series("build-binary", shell.task(
'supervisorctl restart projectname'
)))
gulp.task('watch', function() {
gulp.watch([
"*.go",
"*.mod",
"*.sum",
"**/*.go",
"**/*.mod",
"**/*.sum"
],
{interval: 1000, usePolling: true},
gulp.series('build-binary', 'restart-supervisor'
));
});
gulp.task('default', gulp.series('watch'));
This is my current dockerfile:
FROM arm64v8/golang:1.17-alpine3.14
RUN apk update && apk add gcc make git libc-dev binutils-gold
# Install dependencies
RUN apk add --update tzdata \
--no-cache ca-certificates git wget \
nodejs npm \
g++ \
supervisor \
&& update-ca-certificates \
&& npm install -g gulp gulp-shell
COPY ops/api/local/supervisor /etc
ENV PATH $PATH:/go/bin
WORKDIR /go/src/github.com/projectname/src/api
in my docker-compose.yaml i have this:
entrypoint:
[
"sh",
"-c",
"npm install gulp gulp-shell && supervisord -c /etc/supervisord.conf && gulp"
]
vim /etc/supervisord.conf:
#!/bin/sh
[unix_http_server]
file=/tmp/supervisor.sock
username=admin
password=revproxy
[supervisord]
nodaemon=false
user=root
logfile=/dev/null
logfile_maxbytes=0
logfile_backups=0
loglevel=info
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock
username=admin
password=revproxy
[program:projectname_api]
directory=/go/src/github.com/projectname/src/api
command=/go/src/github.com/projectname/src/api/api
autostart=true
autorestart=true
stderr_logfile=/go/src/github.com/projectname/src/api/api_err.log
stderr_logfile_maxbytes=0
stdout_logfile=/go/src/github.com/projectname/src/api/api_debug.log
stdout_logfile_maxbytes=0
startsecs=0
But seriously, what is wrong with this mac m1.
I have tried doing it in rosetta and non-rosetta, version 2.
If the title of my question is wrong please correct me, I also not sure of my error.
I fixed the problem by adding #!/bin/sh and startsecs=0, no errors to be showing but the next problem is the API is not running.

docker - pass arguments to the script during build

I would like to pass argument (from the docker command) to the shell script inside the Dockerfile.
This is the docker command line.
docker build --file=DockerfileTest --build-arg docker_image=PX-release-migration --tag=test-image:latest --rm=true .
This is a script that is called inside the Dockerfile.
#!/bin/sh -e
image_name=$1
echo "docker image is $image_name"
if [[ ($image_name == '') || ($image_name == *"-dev-"*) ]]; then
echo "This is development"
cp src/main/resources/application-dev.properties src/main/resources/application.properties
elif [[ $image_name == *"-preprod-"* ]]; then
echo "This is preprod"
cp src/main/resources/application-stg.properties src/main/resources/application.properties
elif [[ $image_name == *"-release-"* ]]; then
echo "This is production"
cp src/main/resources/application-prod.properties src/main/resources/application.properties
fi
When I execute separately the script, it works, but it doe
This is docker file.
ARG spring_env=local
ARG docker_image=-local-
FROM maven:3.6.1-jdk-8
COPY . /apps/demo
WORKDIR /apps/demo
RUN chmod +x /apps/demo/initialize_env.sh
RUN ./initialize_env.sh $docker_image
RUN echo "spring_env is ${spring_env}"
So basically, i would like to use a different spring application properties file during the build depending on the docker_image name. If a docker image name contains 'release', i would like to package application-prod.properties during the build.
This is the error message that I am getting.
Step 1/8 : ARG spring_env=local
Step 2/8 : ARG docker_image=-local-
Step 3/8 : FROM maven:3.6.1-jdk-8
---> 4c81be38db66
Step 4/8 : COPY . /apps/demo
---> 41439197c465
Step 5/8 : WORKDIR /apps/demo
---> Running in 56bd408c2eb1
Removing intermediate container 56bd408c2eb1
---> 4c4025bf5f64
Step 6/8 : RUN chmod +x /apps/demo/initialize_env.sh
---> Running in 18dc3a5c1a54
Removing intermediate container 18dc3a5c1a54
---> 60d2037a0209
Step 7/8 : RUN ./initialize_env.sh $docker_image
---> Running in 2e049b2cf630
docker image is
./initialize_env.sh: 5: ./initialize_env.sh: Syntax error: word unexpected (expecting ")")
The command '/bin/sh -c ./initialize_env.sh $docker_image' returned a non-zero code: 2
When I execute separately the script, it works, but it doesn't inside the docker container.
Tip: Use ShellCheck to check scripts for syntax errors.
#!/bin/sh -e
if [[ ($image_name == '') || ($image_name == *"-dev-"*) ]]; then
[[ is bash syntax but your script is declared to use plain sh. It works on your machine presumably because sh is really symlinked to bash, but inside the container that's not the case. maven:3.6.1-jdk-8 is based on debian:stretch which uses dash instead of bash.
Change the shebang line. You can also delete the parentheses; they're superfluous.
#!/bin/bash -e
if [[ $image_name == '' || $image_name == *"-dev-"* ]]; then
You could also use a case block to simplify the repetitive checks.
case "$image_name" in
''|*-dev-*)
echo "This is development"
cp src/main/resources/application-dev.properties src/main/resources/application.properties
;;
*-preprod-*)
echo "This is preprod"
cp src/main/resources/application-stg.properties src/main/resources/application.properties
;;
*-release-*)
echo "This is production"
cp src/main/resources/application-prod.properties src/main/resources/application.properties
;;
esac

Conditional logic in Dockerfile, using --build-arg

Say I have this:
ARG my_user="root" # my_user => default is "root"
USER $my_user
ENV USER=$my_user
All good so far, but now we get here:
ENV HOME="/root"
is there a way to do something like this:
ENV HOME $my_user === "root"? "/root" : "/home/$my_user"
Obviously, that's the wrong syntax.
The only solution I can think of is to just use two --build-args, something like this:
docker build -t zoom \
--build-arg my_user="foo" \
--build-arg my_home="/home/foo" \
.
Unfortunately you can't do this directly
https://forums.docker.com/t/how-do-i-send-runs-output-to-env-in-dockerfile/16106/3
So you have two alternatives
Use a shell script at start
You can use a shell script at the start
CMD /start.sh
And in your start.sh you can have that logic
if [ $X == "Y" ]; then
export X=Y
else
export X=Z
fi
Create a profile environment variable
FROM alpine
RUN echo "export NAME=TARUN" > /etc/profile.d/myenv.sh
SHELL ["/bin/sh", "-lc"]
CMD env
And then you when you run it
$ docker run test
HOSTNAME=d98d44fa1dc9
SHLVL=1
HOME=/root
PAGER=less
PS1=\h:\w\$
NAME=TARUN
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
CHARSET=UTF-8
Note: The SHELL ["/bin/sh", "-lc"] is quite important here, else the profile will not be loaded
Note2: Instead of RUN echo "export NAME=TARUN" > /etc/profile.d/myenv.sh you can also do a COPY myevn.sh /etc/profile.d/myenv.sh and have the file be present in your build context

How to install nvm in a Dockerfile?

I'm trying to install nvm within a Dockerfile. It seems like it installs OK, but the nvm command is not working.
Dockerfile:
# Install nvm
RUN git clone http://github.com/creationix/nvm.git /root/.nvm;
RUN chmod -R 777 /root/.nvm/;
RUN sh /root/.nvm/install.sh;
RUN export NVM_DIR="$HOME/.nvm";
RUN echo "[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh" >> $HOME/.bashrc;
RUN nvm ls-remote;
Build output:
Step 23/39 : RUN git clone http://github.com/creationix/nvm.git /root/.nvm;
---> Running in ca485a68b9aa
Cloning into '/root/.nvm'...
---> a6f61d486443
Removing intermediate container ca485a68b9aa
Step 24/39 : RUN chmod -R 777 /root/.nvm/
---> Running in 6d4432926745
---> 30e7efc5bd41
Removing intermediate container 6d4432926745
Step 25/39 : RUN sh /root/.nvm/install.sh;
---> Running in 79b517430285
=> Downloading nvm from git to '$HOME/.nvm'
=> Cloning into '$HOME/.nvm'...
* (HEAD detached at v0.33.0)
master
=> Compressing and cleaning up git repository
=> Appending nvm source string to /root/.profile
=> bash_completion source string already in /root/.profile
npm info it worked if it ends with ok
npm info using npm#3.10.10
npm info using node#v6.9.5
npm info ok
=> Installing Node.js version 6.9.5
Downloading and installing node v6.9.5...
Downloading https://nodejs.org/dist/v6.9.5/node-v6.9.5-linux-x64.tar.xz...
######################################################################## 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v6.9.5 (npm v3.10.10)
Creating default alias: default -> 6.9.5 (-> v6.9.5 *)
/root/.nvm/install.sh: 136: [: v6.9.5: unexpected operator
Failed to install Node.js 6.9.5
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
---> 9f6f3e74cd19
Removing intermediate container 79b517430285
Step 26/39 : RUN export NVM_DIR="$HOME/.nvm";
---> Running in 1d768138e3d5
---> 8039dfb4311c
Removing intermediate container 1d768138e3d5
Step 27/39 : RUN echo "[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh" >> $HOME/.bashrc;
---> Running in d91126b7de62
---> 52313e09866e
Removing intermediate container d91126b7de62
Step 28/39 : RUN nvm ls-remote;
---> Running in f13c1ed42b3a
/bin/sh: 1: nvm: not found
The command '/bin/sh -c nvm ls-remote;' returned a non-zero code: 127
The error:
Step 28/39 : RUN nvm ls-remote;
---> Running in f13c1ed42b3a
/bin/sh: 1: nvm: not found
The command '/bin/sh -c nvm ls-remote;' returned a non-zero code: 127
The end of my /root/.bashrc file looks like:
[[ -s /root/.nvm/nvm.sh ]] && . /root/.nvm/nvm.sh
Everything else in the Dockerfile works. Adding the nvm stuff is what broke it. Here is the full file.
I made the following changes to your Dockerfile to make it work:
First, replace...
RUN sh /root/.nvm/install.sh;
...with:
RUN bash /root/.nvm/install.sh;
Why? On Redhat-based systems, /bin/sh is a symlink to /bin/bash. But on Ubuntu, /bin/sh is a symlink to /bin/dash. And this is what happens with dash:
root#52d54205a137:/# bash -c '[ 1 == 1 ] && echo yes!'
yes!
root#52d54205a137:/# dash -c '[ 1 == 1 ] && echo yes!'
dash: 1: [: 1: unexpected operator
Second, replace...
RUN nvm ls-remote;
...with:
RUN bash -i -c 'nvm ls-remote';
Why? Because, the default .bashrc for a user in Ubuntu (almost at the top) contains:
# If not running interactively, don't do anything
[ -z "$PS1" ] && return
And the source-ing of nvm's scripts takes place at the bottom. So we need to make sure that bash is invoked interactively by passing the argument -i.
Third, you could skip the following lines in your Dockerfile:
RUN export NVM_DIR="$HOME/.nvm";
RUN echo "[[ -s $HOME/.nvm/nvm.sh ]] && . $HOME/.nvm/nvm.sh" >> $HOME/.bashrc;
Why? Because bash /root/.nvm/install.sh; will automatically do it for you:
[fedora#myhost ~]$ sudo docker run --rm -it 2a283d6e2173 tail -2 /root/.bashrc
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
Instalation of nvm on ubuntu in dockerfile
In the case of Ubuntu 20.04 you can use only these commands and everything will be alright
FROM ubuntu:20.04
RUN apt update -y && apt upgrade -y && apt install wget bash -y
RUN wget -qO- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
RUN bash -i -c 'nvm ls-remote'
hopefully it will work

ARG or ENV, which one to use in this case?

This could be maybe a trivial question but reading docs for ARG and ENV doesn't put things clear to me.
I am building a PHP-FPM container and I want to give the ability for enable/disable some extensions on user needs.
Would be great if this could be done in the Dockerfile by adding conditionals and passing flags on the build command perhaps but AFAIK is not supported.
In my case and my personal approach is to run a small script when container starts, something like the following:
#!/bin/sh
set -e
RESTART="false"
# This script will be placed in /config/init/ and run when container starts.
if [ "$INSTALL_XDEBUG" == "true" ]; then
printf "\nInstalling Xdebug ...\n"
yum install -y php71-php-pecl-xdebug
RESTART="true"
fi
...
if [ "$RESTART" == "true" ]; then
printf "\nRestarting php-fpm ...\n"
supervisorctl restart php-fpm
fi
exec "$#"
This is how my Dockerfile looks like:
FROM reynierpm/centos7-supervisor
ENV TERM=xterm \
PATH="/root/.composer/vendor/bin:${PATH}" \
INSTALL_COMPOSER="false" \
COMPOSER_ALLOW_SUPERUSER=1 \
COMPOSER_ALLOW_XDEBUG=1 \
COMPOSER_DISABLE_XDEBUG_WARN=1 \
COMPOSER_HOME="/root/.composer" \
COMPOSER_CACHE_DIR="/root/.composer/cache" \
SYMFONY_INSTALLER="false" \
SYMFONY_PROJECT="false" \
INSTALL_XDEBUG="false" \
INSTALL_MONGO="false" \
INSTALL_REDIS="false" \
INSTALL_HTTP_REQUEST="false" \
INSTALL_UPLOAD_PROGRESS="false" \
INSTALL_XATTR="false"
RUN yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
https://rpms.remirepo.net/enterprise/remi-release-7.rpm
RUN yum install -y \
yum-utils \
git \
zip \
unzip \
nano \
wget \
php71-php-fpm \
php71-php-cli \
php71-php-common \
php71-php-gd \
php71-php-intl \
php71-php-json \
php71-php-mbstring \
php71-php-mcrypt \
php71-php-mysqlnd \
php71-php-pdo \
php71-php-pear \
php71-php-xml \
php71-pecl-apcu \
php71-php-pecl-apfd \
php71-php-pecl-memcache \
php71-php-pecl-memcached \
php71-php-pecl-zip && \
yum clean all && rm -rf /tmp/yum*
RUN ln -sfF /opt/remi/php71/enable /etc/profile.d/php71-paths.sh && \
ln -sfF /opt/remi/php71/root/usr/bin/{pear,pecl,phar,php,php-cgi,phpize} /usr/local/bin/. && \
mv -f /etc/opt/remi/php71/php.ini /etc/php.ini && \
ln -s /etc/php.ini /etc/opt/remi/php71/php.ini && \
rm -rf /etc/php.d && \
mv /etc/opt/remi/php71/php.d /etc/. && \
ln -s /etc/php.d /etc/opt/remi/php71/php.d
COPY container-files /
RUN chmod +x /config/bootstrap.sh
WORKDIR /data/www
EXPOSE 9001
Currently this is working but ... If I want to add let's say 20 (a random number) of extensions or any other feature that can be enable|disable then I will end with 20 non necessary ENV (because Dockerfile doesn't support .env files) definition whose only purpose would be set this flag for let the script knows what to do then ...
Is this the right way to do it?
Should I use ENV for this purpose?
I am open to ideas if you have a different approach for achieve this please let me know about it
From Dockerfile reference:
The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build command using the --build-arg <varname>=<value> flag.
The ENV instruction sets the environment variable <key> to the value <value>.
The environment variables set using ENV will persist when a container is run from the resulting image.
So if you need build-time customization, ARG is your best choice.
If you need run-time customization (to run the same image with different settings), ENV is well-suited.
If I want to add let's say 20 (a random number) of extensions or any other feature that can be enable|disable
Given the number of combinations involved, using ENV to set those features at runtime is best here.
But you can combine both by:
building an image with a specific ARG
using that ARG as an ENV
That is, with a Dockerfile including:
ARG var
ENV var=${var}
You can then either build an image with a specific var value at build-time (docker build --build-arg var=xxx), or run a container with a specific runtime value (docker run -e var=yyy)
So if want to set the value of an environment variable to something different for every build then we can pass these values during build time and we don't need to change our docker file every time.
While ENV, once set cannot be overwritten through command line values. So, if we want to have our environment variable to have different values for different builds then we could use ARG and set default values in our docker file. And when we want to overwrite these values then we can do so using --build-args at every build without changing our docker file.
For more details, you can refer this.
Why to use ARG or ENV ?
Let's say we have a jar file and we want to make a docker image of it. So, we can ship it to any docker engine.
We can write a Dockerfile.
Dockerfile
FROM eclipse-temurin:17-jdk-alpine
VOLUME /tmp
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
Now, if we want to build the docker image using Maven, we can pass the JAR_FILE using the --build-arg as target/*.jar
docker build --build-arg JAR_FILE=target/*.jar -t myorg/myapp
However, if we are using Gradle; the above command doesn't work and we've to pass a different path: build/libs/
docker build --build-arg JAR_FILE=build/libs/*.jar -t myorg/myapp .
Once you have chosen a build system, we don’t need the ARG. We can hard code the JAR location.
For Maven, that would be as follows:
Dockerfile
FROM eclipse-temurin:17-jdk-alpine
VOLUME /tmp
COPY target/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]
here, we can build an image with the following command:
docker build -t image:tag .
When to use `ENV`?
If we want to set some values at running containers and reflect that to the image like the Port Number that your application can run/listen on. We can set that using the ENV.
Both ARG and ENV seem very similar. Both can be accessed from within our Dockerfile commands in the same manner.
Example:
ARG VAR_A 5
ENV VAR_B 6
RUN echo $VAR_A
RUN echo $VAR_B
Personal Option!
There is a tradeoff between choosing ARG over ENV. If you choose ARG you can't change it later during the run. However, if you chose ENV you can modify the value at the container.
I personally prefer ARG over ENV wherever I can, like,
In the above Example:
I have used ARG as the build system maven or Gradle impacts during build rather than runtime. It thus encapsulates a lot of details and provided a minimum set of arguments for the runtime.
For more details, you can refer to this.

Resources