Reference Shell Script Variable in Dockerfile - docker

I have a config.sh:
IMAGE_NAME="back_end"
APP_PORT=80
PUBLIC_PORT=8080
and a build.sh:
#!/bin/bash
source config.sh
echo "Image name is: ${IMAGE_NAME}"
sudo docker build -t ${IMAGE_NAME} .
and a run.sh:
#!/bin/bash
source config.sh
# Expose ports and run
sudo docker run -it \
-p $PUBLIC_PORT:$APP_PORT \
--name $IMAGE_NAME $IMAGE_NAME
and finally, a Dockerfile:
...
CMD ["gunicorn", "-b", "0.0.0.0:${APP_PORT}", "main:app"]
I'd like to be able to reference the APP_PORT variable in my config.sh within the Dockerfile as shown above. However, what I have does not work and it complains: Error: ${APP_PORT} is not a valid port number. So it's not interpreting APP_PORT as a variable. Is there a way to reference the variables within config.sh from within the Dockerfile?
Thanks!
EDIT: New Files based on suggested solutions (still don't work)
I have a config.sh:
IMAGE_NAME="back_end"
APP_PORT=80
PUBLIC_PORT=8080
and a build.sh:
#!/bin/bash
source config.sh
echo "Image name is: ${IMAGE_NAME}"
sudo docker build --build-arg APP_PORT="${APP_PORT}" -t "${IMAGE_NAME}" .
and a run.sh:
#!/bin/bash
source config.sh
# Expose ports and run
sudo docker run -it \
-p $PUBLIC_PORT:$APP_PORT \
--name $IMAGE_NAME $IMAGE_NAME
and finally, a Dockerfile:
FROM python:buster
LABEL maintainer="..."
ARG APP_PORT
#ENV PORT $APP_PORT
ENV APP_PORT=${APP_PORT}
#RUN echo "$PORT"
# Install gunicorn & falcon
COPY requirements.txt ./
RUN pip3 install --no-cache-dir -r requirements.txt
# Add demo app
COPY ./app /app
COPY ./config.sh /app/config.sh
WORKDIR /app
RUN ls -a
CMD ["gunicorn", "-b", "0.0.0.0:${APP_PORT}", "main:app"]
run.sh still fails and reports: Error: '${APP_PORT} is not a valid port number.'

Define a variable in Dockerfile as follows:
FROM python:buster
LABEL maintainer="..."
ARG APP_PORT
ENV APP_PORT=${APP_PORT}
# Install gunicorn & falcon
COPY requirements.txt ./
RUN pip3 install --no-cache-dir -r requirements.txt
# Add demo app
COPY ./app /app
COPY ./config.sh /app/config.sh
WORKDIR /app
CMD gunicorn -b 0.0.0.0:$APP_PORT main:app # NOTE! without separating with ["",""]
Pass it as build-arg, e.g. in your build.sh:
Note! Passing build argument is only necessary when it is used for building docker image. You use it on CMD and one can omit passing it during building docker image.
#!/bin/bash
source config.sh
echo "Image name is: ${IMAGE_NAME}"
sudo docker build --build-arg APP_PORT="${APP_PORT}" -t "${IMAGE_NAME}" .
# sudo docker build --build-arg APP_PORT=80 -t back_end . -> You may omit using config.sh and directly define the value of variables
and pass value of $APP_PORT in run.sh as well when starting the container:
#!/bin/bash
source config.sh
# Expose ports and run
sudo docker run -it \
-e APP_PORT=$APP_PORT \
-p $PUBLIC_PORT:$APP_PORT \
--name $IMAGE_NAME $IMAGE_NAME

You need a shell to replace environment variables and when your CMD is in exec form, there's no shell.
If you use the shell form, there is a shell and you can use environment variables.
CMD gunicorn -b 0.0.0.0:${APP_PORT} main:app
Read here for more information on the two forms of the CMD statement: https://docs.docker.com/engine/reference/builder/#cmd

Related

Single Docker Image Multiple Container For Multi Module Java Project

I have created a project like https://github.com/senolatac/demo-multi-module-docker and I have implemented some docker multi-stage https://docs.docker.com/language/java/run-tests/ examples.
Here is my Dockerfile:
FROM --platform=linux/x86_64 gradle:7.5.1-jdk17-alpine AS base
COPY --chown=gradle:gradle . /app
WORKDIR /app
FROM base as test
CMD ["./gradlew", "test"]
FROM base as build
RUN ./gradlew build -x test
# create-image -> DOCKER_BUILDKIT=0 docker build --tag sb-web-image --target web .
# run -> docker run -it --rm --name sb-web-container -p 8085:8080 sb-web-image
FROM --platform=linux/x86_64 openjdk:17-alpine as web
COPY --from=build /app/web/build/libs/*.jar /web.jar
CMD ["java", "-Dspring-boot.run.profiles=default", "-jar", "/web.jar"]
# create-image -> DOCKER_BUILDKIT=0 docker build --tag sb-worker-image --target worker .
# run -> docker run -it --rm --name sb-worker-container -p 8086:8080 sb-worker-image
FROM --platform=linux/x86_64 openjdk:17-alpine as worker
COPY --from=build /app/worker/build/libs/*.jar /worker.jar
CMD ["java", "-Dspring-boot.run.profiles=default", "-jar", "/worker.jar"]
# create-image -> DOCKER_BUILDKIT=0 docker build --tag sb-web-image --build-arg JAR_FILE=web/build/libs/\*.jar --target generic .
# run -> docker run -it --rm --name sb-web-container -p 8086:8080 sb-web-image
FROM --platform=linux/x86_64 openjdk:17-alpine as generic
ARG JAR_FILE
COPY --from=build /app/${JAR_FILE} /app.jar
CMD ["java", "-Dspring-boot.run.profiles=default", "-jar", "/app.jar"]
My project has two different modules. To run these modules on Docker, firstly I should create two different images then run them as two different docker container. But my purpose is so simple: Create a single image and run containers from that image. Do you have suggestion about it?
You can straightforwardly override the CMD when you run the application, so you just need to COPY all of the jar files into the single image.
FROM --platform=linux/x86_64 gradle:7.5.1-jdk17-alpine AS build
COPY --chown=gradle:gradle . /app
WORKDIR /app
RUN ./gradlew build -x test
FROM --platform=linux/x86_64 openjdk:17-alpine
WORKDIR /app
COPY --from=build /app/web/build/libs/*.jar ./web.jar
COPY --from=build /app/worker/build/libs/*.jar ./worker.jar
ENV SPRINGBOOT_RUN_PROFILES=default
EXPOSE 8080
CMD ["java", "-jar", "./web.jar"]
docker build -t sb-image .
docker run -d --name sb-web -p 8085:8080 sb-image
docker run -d --name sb-worker -p 8086:8080 sb-image \
java -jar ./worker.jar
So, note that there is only one final stage, but it COPY --from=build both jar files into it. I pick one of them to be the default CMD, and when I run the other, I provide an additional command after the docker run image-name. (Compose command: and Kubernetes args: can do the same thing.)
This looks like a Spring Boot application. I've set the profile property as an environment variable rather than a command-line option, which shortens the command line. If your applications share a code base, you can also get a smaller image by unpacking the fat jars, though this requires an additional build stage.

Docker arg not being read correctly from command line

Using this command to start and run my docker container passing in argument ./target/myapp-SNAPSHOT.jar where ./target/myapp-SNAPSHOT.jar is the Spring boot app I wish to run :
docker build -t foo . && docker run -it foo -e J_FILE=./target/myapp-SNAPSHOT.jar
I receive error :
standard_init_linux.go:211: exec user process caused "exec format error"
Dockerfile:
FROM adoptopenjdk/openjdk8:alpine-jre
LABEL app.name="test"
LABEL app.type="test"
ARG J_FILE
ADD ${J_FILE} /myapp.jar
COPY J_FILE /myapp.jar
COPY startup.sh /
RUN chmod +x startup.sh;
ENTRYPOINT ["/startup.sh"]
EXPOSE 8080
startup.sh :
#!/bin/sh
JAVA_HEAP_INITIAL=384m
JAVA_HEAP_MAX=768m
JAVA_METASPACE_MAX=128m
java -jar /app.jar
If I change Dockerfile to explicitly copy the jar file :
FROM adoptopenjdk/openjdk8:alpine-jre
LABEL app.name="test"
LABEL app.type="test"
COPY ./target/myapp-SNAPSHOT.jar /myapp.jar
COPY startup.sh /
RUN chmod +x startup.sh;
ENTRYPOINT ["/startup.sh"]
EXPOSE 8080
The app starts as expected.
The ARG flag in Dockerfile is meant for configuration when building a docker image. It will not have any effect when you run the container.
Therefore, you should update your command to be like this:
docker build --build-arg J_FILE=./target/myapp-SNAPSHOT.jar -t foo . && docker run -it foo
You can also update your Dockerfile to be like this:
FROM adoptopenjdk/openjdk8:alpine-jre
LABEL app.name="test"
LABEL app.type="test"
ARG J_FILE=./target/myapp-SNAPSHOT.jar
COPY ${J_FILE} /myapp.jar
COPY startup.sh /
RUN chmod +x startup.sh;
ENTRYPOINT ["/startup.sh"]
EXPOSE 8080
Note 2 changes in the file above:
Use COPY instead of ADD
Set the default value for J_FILE as ./target/myapp-SNAPSHOT.jar in case the --build-arg is not passed with the docker build command.

How do you use Docker build secrets with Docker Compose?

Using the docker build command line I can pass in a build secret as follows
docker build \
--secret=id=gradle.properties,src=$HOME/.gradle/gradle.properties \
--build-arg project=template-ms \
.
Then use it in a Dockerfile
# syntax = docker/dockerfile:1.0-experimental
FROM gradle:jdk12 AS build
COPY *.gradle .
RUN --mount=type=secret,target=/home/gradle/gradle.properties,id=gradle.properties gradle dependencies
COPY src/ src/
RUN --mount=type=secret,target=/home/gradle/gradle.properties,id=gradle.properties gradle build
RUN ls -lR build
FROM alpine AS unpacker
ARG project
COPY --from=build /home/gradle/build/libs/${project}.jar /tmp
RUN mkdir -p /opt/ms && unzip -q /tmp/${project}.jar -d /opt/ms && \
mv /opt/ms/BOOT-INF/lib /opt/lib
FROM openjdk:12
EXPOSE 8080
WORKDIR /opt/ms
USER nobody
CMD ["java", "-Xdebug", "-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=0.0.0.0:8000", "-Dnetworkaddress.cache.ttl=5", "org.springframework.boot.loader.JarLauncher"]
HEALTHCHECK --start-period=600s CMD curl --silent --output /dev/null http://localhost:8080/actuator/health
COPY --from=unpacker /opt/lib /opt/ms/BOOT-INF/lib
COPY --from=unpacker /opt/ms/ /opt/ms/
I want to do a build using docker-compose, but I can't find in the docker-compose.yml reference how to pass the secret.
That way the developer just needs to type in docker-compose up
You can use environment or args to pass variables to container in docker-compose.
args:
- secret=id=gradle.properties,src=$HOME/.gradle/gradle.properties
environment:
- secret=id=gradle.properties,src=$HOME/.gradle/gradle.properties

Passing docker runtime environment variables in docker image

Here's my docker image. I want to override the default environment variables being set below from whatever is passed in the docker run command mentioned in the end
FROM ubuntu:16.04
ADD http://www.nic.funet.fi/pub/mirrors/apache.org/tomcat/tomcat-8/v8.0.48/bin/apache-tomcat-8.0.48.tar.gz /usr/local/
RUN cd /usr/local && tar -zxvf apache-tomcat-8.0.48.tar.gz && rm apache-tomcat-8.0.48.tar.gz
RUN mv /usr/local/apache-tomcat-8.0.48 /usr/local/tomcat
RUN rm -rf /usr/local/tomcat/webapps/*
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV CATALINA_HOME /usr/local/tomcat
ENV CATALINA_BASE /usr/local/tomcat
ENV PATH $PATH:$JAVA_HOME/bin:$CATALINA_HOME/bin
ENV dummy_url defaulturl
ENV database databasedefault
COPY my.war /usr/local/tomcat/webapps/
RUN echo >> /usr/local/tomcat/conf/test.properties
RUN echo dummy_url =$dummy_url >> /usr/local/tomcat/conf/test.properties
RUN echo database =$database >> /usr/local/tomcat/conf/test.properties
ENTRYPOINT ["catalina.sh", "run"]
To run in local :
docker run -p 8080:8080 -e dummy_url=http:google.com -e database=jdbc://mysql allimages/myimage:latest
dummy_url and database do not seem to be getting overridden in the file that I am adding them in - test.properties. Any ideas would be greatly appreciated.
I want to override the default environment variables being set below from whatever is passed in the docker run command mentioned in the end
That means overriding an image file (/usr/local/tomcat/conf/test.properties) when running the image as a container (docker run), not building the image (docker build and its --build-args option and its ARG Dockerfile entry).
That means you create locally a script file which:
modifies /usr/local/tomcat/conf/test.properties
calls catalina.sh run $# (see also "Store Bash script arguments $# in a variable" from "Accessing bash command line args $# vs $*")
That is:
myscript.sh
#!/bin/sh
echo dummy_url=$dummy_url >> /usr/local/tomcat/conf/test.properties
echo database=$database >> /usr/local/tomcat/conf/test.properties
args=("$#")
catalina.sh run "${args[#]}"
You would modify your Dockerfile to COPY that script and call it:
COPY myscript.sh /usr/local/
...
ENTRYPOINT ["/usr/local/myscript.sh"]
Then, and only then, the -e options of docker run would work.
You are confusing what gets executed when building the image and what gets executed when starting the container.
The RUN command inside the dockerfile is executed when building the image, when running docker build ...
RUN echo dummy_url =$dummy_url >> /usr/local/tomcat/conf/test.properties
RUN echo database =$database >> /usr/local/tomcat/conf/test.properties
Thus when the above execute the file test.properties will contain the default values specified in the Dockerfile.
When you execute docker run -p 8080:8080 -e dummy_url=http:google.com -e database=jdbc://mysql allimages/myimage:latest the ENTRYPOINT ["catalina.sh", "run"] will get executed with
env values dummy_url=http:google.com and database=jdbc://mysql.
You can allow values in test.properties to be ovveriden using:
Move $dummy_url >> /usr/local/tomcat/conf/test.properties and $database >> /usr/local/tomcat/conf/test.properties to the start of catalina.sh script.
Override the values when building the image as such:
ARG dummy_url_arg
ARG database_arg
ENV dummy_url $dummy_url_arg
ENV database $database_arg
COPY my.war /usr/local/tomcat/webapps/
RUN echo >> /usr/local/tomcat/conf/test.properties
RUN echo dummy_url =$dummy_url >> /usr/local/tomcat/conf/test.properties
RUN echo database =$database >> /usr/local/tomcat/conf/test.properties
ENTRYPOINT ["catalina.sh", "run"]
And when building the image override the values using docker build --build-arg dummy_url_arg=http:google.com --build-arg database_arg=jdbc://mysql allimages/myimage:latest ...

how do I build a docker image without existing from a docker file

new to dockers.
running this docker build .
this is the Dockerfile
FROM gcr.io/google_appengine/python
# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
RUN virtualenv /env
# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
#ADD requirements.txt /app/requirements.txt
#RUN pip install -r /app/requirements.txt
# Add the application source code.
ADD . /app
# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
CMD gunicorn -b :$PORT main:app
CMD bash1 "while true; do echo hello; sleep 1;done"
CMD ["sh", "while true; do echo hello; sleep 1;done"]
CMD "echo" "Hello docker!"
but after, when I run docker ps I don't see the image.
To build an image you have to use:
docker build -t username/imagename .
You have to use -t to tag your image and give it a name, from the docs:
-t, --tag value Name and optionally a tag in the
'name:tag' format (default [])
Then you can see the list of your images using:
docker images
You are using docker ps which is for listing containers not images.
More info about about images and containers.
Check the documentation on docker build.
use 'docker run ' to create and run a container. all docker build does is create an image.

Resources