I can't override the Dockerfile arg from docker-compose.yml when I set a default value at Dockerfile arg.
If the user just run de Dockerfile whiteout any parameter I want that docker don't break and at docker-compose.yml I want to set a better architecture.
This is my Dockerfile:
FROM python:3.6 as flask_api
LABEL maintainer 'https://about.me/leandro.garcias'
ARG DEBUG=False
# BD Config
ARG DATABASE_URL='sqlite:///data/app.db'
# Max register per page, when you try to get all
ARG MAX_PER_PAGE=25
# Collect log errors. https://sentry.io
ARG COLLECT_LOG_ERRORS=False
ARG SENTRY_DSN=''
RUN adduser api
USER api
WORKDIR /home/api
COPY requirements.txt manage.py contrib/boot.sh ./
COPY tests tests
COPY app app
RUN mkdir data
ENV PYTHONUNBUFFERED 1
ENV DEBUG $DEBUG
ENV DATABASE_URL $DATABASE_URL
ENV MAX_PER_PAGE $MAX_PER_PAGE
ENV COLLECT_LOG_ERRORS $COLLECT_LOG_ERRORS
ENV SENTRY_DSN $SENTRY_DSN
RUN python -m venv venv
RUN venv/bin/pip install -r requirements.txt
CMD bash boot.sh
EXPOSE 5000:5000
This is my docker-compose.yml:
version: '3'
volumes:
local_data:
data:
networks:
web:
app:
db:
services:
frontend:
image: nginx:1.13
volumes:
- ./contrib/nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- 80:80
networks:
- web
- app
depends_on:
- app
app:
image: flask_api
restart: always
volumes:
- local_data:/home/api/data
networks:
- app
- db
depends_on:
- db
build:
context: .
args:
- DEBUG = False
# BD Config
- DATABASE_URL = postgres://postgres:#db:5432/people
# Max register per page, when you try to get all
- MAX_PER_PAGE = 25
# Collect log errors. https://sentry.io
- COLLECT_LOG_ERRORS = False
- SENTRY_DSN = ''
db:
image: postgres:9.6
volumes:
- data:/var/lib/postgresql/data
networks:
- db
environment:
- POSTGRES_DB=people
I need help ...
I found a solution! Just need to set environment values. :-)
app:
image: flask_api
environment:
- DEBUG=False
# BD Config
- DATABASE_URL=postgres://postgres:#db:5432/people
# Max register per page, when you try to get all
- MAX_PER_PAGE=25
# Collect log errors. https://sentry.io
- COLLECT_LOG_ERRORS=False
- SENTRY_DSN=''
Update 2022
ERROR: environment variable name 'DEBUG ' may not contain whitespace.
This might be the error you're getting. There shouldn't be spaces around the env. variable.
If we are passing args to the build from docker-compose, it would override the default arg value specified in Dockerfile.
Related
I don't suppose anyone would be prepared to outline for me how I might go about setting values for limit_req in NGINX in a Docker container using webdevops/php-nginx. I'm getting very occasional rate limit issues with a React/Laravel application and would like to try adjusting some rate limit settings.
The customization section of the webdevops/php-nginx docs suggests that to set "global configuration options the directory /opt/docker/etc/nginx/conf.d can be used. For vhost configuration options the directory /opt/docker/etc/nginx/vhost.common.conf can be used". However, I'm not sure how I bring that into my Docker process:
docker build --file Dockerfile.prod -t ghcr.io/theotherdy/laravel-xmap-php-nginx:latest .
using Dockerfile.prod:
FROM webdevops/php-nginx:8.1-alpine
# These ENV variables refer to options in webdevops/php-nginx - https://dockerfile.readthedocs.io/en/latest/content/DockerImages/dockerfiles/php-nginx.html
ENV WEB_DOCUMENT_ROOT=/app/public
ENV PHP_DISMOD=bz2,calendar,exiif,ffi,intl,gettext,ldap,imap,pdo_pgsql,pgsql,soap,sockets,sysvmsg,sysvsm,sysvshm,shmop,xsl,zip,gd,apcu,vips,yaml,imagick,mongodb,amqp
# sets working directory for any future actions
WORKDIR /app
# ie copying from wherever docker is run to WORKDIR
COPY . .
COPY composer.lock composer.lock
COPY .env.prod .env
# recommended optimization from: https://laravel.com/docs/9.x/deployment
RUN composer install --no-interaction --optimize-autoloader --no-dev
RUN php artisan key:generate
RUN php artisan config:cache
RUN php artisan route:cache
RUN php artisan view:cache
# Ensure all of our files are owned by the same user and group.
RUN chown -R application:application .
and docker-compose.yml:
version: "3" services:
app:
image: ghcr.io/theotherdy/laravel-xmap-php-nginx:latest
ports:
- '9000:80'
volumes:
- ./storage:/app/storage
#env_file: '.env'
depends_on:
- db
restart: always
db:
image: 'mysql/mysql-server:8.0'
container_name: xmap-db
environment:
MYSQL_ROOT_PASSWORD: 'xx'
MYSQL_ROOT_HOST: "%"
MYSQL_DATABASE: 'xx'
MYSQL_USER: 'xx'
MYSQL_PASSWORD: 'xx'
MYSQL_ALLOW_EMPTY_PASSWORD: 0
volumes:
- db-data:/var/lib/mysql
restart: always
volumes:
db-data:
Any advice/pointers very gratefully received!
I've found a lot of questions on this topic. Always the answer was to use the '0.0.0.0' IP-adress. But I'm already doing this and still I get the error.
So I'm running a docker compose file that runs a database and a flask front end. The dockerfile runs fine on my own computer but on the server in the cloud I am getting this error.
This code launches the application:
app.run(host="0.0.0.0", port=5000, ssl_context="adhoc", debug=cfg.app_debug_mode)
This is my docker compose file:
version: "3.8"
services:
app:
image: app_image #the beginning is the unique uri of my amazon qccount. Then follows the repository name (ratio) and then the tag of the image (app) https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html
build: ./app
links:
- database
ports:
- "5000:5000"
environment:
AM_I_IN_A_DOCKER_CONTAINER: 'Yes'
CONFIG_NAME: 'config' #Name of the config file to use
database:
image: database_image
container_name: database
build: ./sql
restart: always
ports:
- "32000:32000"
environment:
MYSQL_ROOT_HOST: '%' #This allows the root user to access the database from any ip. For some reason amazon requires it.
MYSQL_ROOT_PASSWORD: 'zv2yRCt79AsGvz'
MYSQL_DATABASE: 'education'
volumes:
- ratio_volume:/var/lib/mysql #use a named volume for the database.
networks:
default:
name: my-network
volumes:
ratio_volume:
This is the docker file:
FROM python:3
RUN pip3 install --upgrade pip
WORKDIR /app
COPY . /app
RUN pip3 --no-cache-dir install -r requirements.txt
EXPOSE 5000
ENTRYPOINT ["python3"]
CMD ["/app/run.py"]
What am I doing wrong? The error is also not really helpful.. I am not getting any errors in the console, just the browser is complaining.
In Docker Compose, we have two services (a backend in Flask and a frontend in React) running at the same time in different directories. What are best practices for automatically updating the frontend service or backend service when ha change to the respective code is made?
In our case, we have:
frontend/
index.html
docker-compose.yml
Dockerfile
src
App.js
index.js
..
And our backend is:
backend/
app.py
Dockerfile
docker-compose.yml
This is our docker-compose.yml file:
version: '3.8'
services:
frontend:
image: node:alpine
build:
context: ../frontend
dockerfile: ../frontend/Dockerfile
command: npm start
depends_on:
- database # dont start until the database is up
- app
ports:
- 3000:3000
volumes:
- .:/frontend
app:
image: python:3.9
build:
context: .
dockerfile: ./Dockerfile
command: app.py
depends_on:
- database # dont start until the database is up
ports:
- 8080:8080
environment:
- PGPASSWORD=magical_password
- POSTGRESQL_PASSWORD=magical_password
- POSTGRESQL_HOST=backend-database-1
- POSTGRESQL_USER_NAME=unicorn_user
- LOCAL_ENVIRONMENT=True
- FLASK_ENV=development
- REPLICATE_API_TOKEN
volumes:
- .:/app
database:
image: "postgres" # use latest official postgres version
env_file:
- database.env # configure postgres
volumes:
- database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
- ./schema.sql:/docker-entrypoint-initdb.d/schema.sql
ports:
- "5432:5432"
volumes:
database-data: # named volumes can be managed easier using docker-compose
Typically, we reload the app (on change) almost instantly from a volume in the volume's section. This approach correctly changes the backend service when the backend code is changed, but not the frontend service. Also, we have 2 docker-compose files, one in frontend, one in backend, which we hope to somehow learn how to consolidate.
Edit: These are the logs that work for the backend (app_1 is the backend) but do not work for the frontend:
app_1 | * Detected change in '/app/app.py', reloading
app_1 | environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '***', 'PGPASSWORD': '***', 'POSTGRESQL_PASSWORD': 'magical_password', 'POSTGRESQL_HOST': 'backend-database-1', 'POSTGRESQL_USER_NAME': '***', 'LOCAL_ENVIRONMENT': 'True', 'FLASK_ENV': 'development', 'LANG': 'C.UTF-8', 'GPG_KEY': '***', 'PYTHON_VERSION': '3.9.13', 'PYTHON_PIP_VERSION': '22.0.4', 'PYTHON_SETUPTOOLS_VERSION': '58.1.0', 'PYTHON_GET_PIP_URL': 'https://github.com/pypa/get-pip/raw/6ce3639da143c5d79b44f94b04080abf2531fd6e/public/get-pip.py', 'PYTHON_GET_PIP_SHA256': '***', 'HOST': '0.0.0.0', 'PORT': '8080', 'HOME': '/root', 'KMP_INIT_AT_FORK': 'FALSE', 'KMP_DUPLICATE_LIB_OK': 'True', 'WERKZEUG_SERVER_FD': '3', 'WERKZEUG_RUN_MAIN': 'true'})
app_1 | * Restarting with stat
app_1 | * Tip: There are .env or .flaskenv files present. Do "pip install python-dotenv" to use them.
app_1 | * Debugger is active!
app_1 | * Debugger PIN: 203-417-897
Edit 2: We followed the link suggested in the comments. We attempted setting both WATCHPACK_POLLING and CHOKIDAR_USEPOLLING to “true” but no luck. And we refactored our docker-compose file to be outside the directories like so:
docker-compose.yml
frontend/
index.html
Dockerfile
src
App.js
index.js
..
backend/
app.py
Dockerfile
Here is the new docker-compose
version: '3.8'
services:
frontend:
image: node:alpine
build:
context: ./frontend
cache_from:
- node:alpine
dockerfile: ./Dockerfile
command: npm start
depends_on:
- database # dont start until the database is up
- app
ports:
- 3000:3000
environment:
- CHOKIDAR_USEPOLLING="true"
volumes:
- /app/node_modules
- ./frontend:/app
app:
image: python:3.9
build:
context: ./backend
cache_from:
- python:3.9
dockerfile: ./Dockerfile
command: backend/app.py
depends_on:
- database # dont start until the database is up
ports:
- 8080:8080
environment:
- PGPASSWORD=magical_password
- POSTGRESQL_PASSWORD=magical_password
- POSTGRESQL_HOST=backend-database-1
- POSTGRESQL_USER_NAME=unicorn_user
- LOCAL_ENVIRONMENT=True
- FLASK_ENV=development
- REPLICATE_API_TOKEN
volumes:
- .:/app
database:
image: "postgres" # use latest official postgres version
env_file:
- backend/database.env # configure postgres
volumes:
- database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
- ./schema.sql:/backend/docker-entrypoint-initdb.d/schema.sql
ports:
- "5432:5432"
volumes:
database-data: # named volumes can be managed easier using docker-compose
app:
And here are our Dockerfile for frontend
FROM node:alpine
RUN mkdir -p /frontend
WORKDIR /frontend
# We copy just the package.json first to leverage Docker cache
COPY package.json /frontend
RUN npm install --legacy-peer-deps
COPY . /frontend
# Bind to all network interfaces so that it can be mapped to the host OS
ENV HOST=0.0.0.0 PORT=3000
EXPOSE ${PORT}
CMD ["npm", "start"]
and backend
FROM python:3.9
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
RUN pip3 install -r requirements.txt
COPY . /app
ENTRYPOINT [ "python" ]
# Bind to all network interfaces so that it can be mapped to the host OS
ENV HOST=0.0.0.0 PORT=8080
EXPOSE ${PORT}
# This runs the app in the container
CMD [ "app.py" ]
Still backend hot reloads and every time we make a change the change is detected and picked up and reflected in docker-compose immediately. But frontend requires a restart with this command docker-compose down --volumes && docker-compose build --no-cache && docker-compose up the output we get from docker-compose is no logs. It’s like docker-compose can’t see the changes.
Edit 3: Any help would be much appreciated!
I have an issue running my docker-compose.yml file with 4 services. They are my go microservice, phoenix web server, mongodb and redis images.
I specified in both my phoenix and golang dockerfiles to change working directory before running both services. I currently get the following errors when I do docker-compose up.
The task "phx.server" could not be found
main.go: no such file or directory
Here is my Dockerfile.go.development:
# base image elixer to start with
FROM golang:latest
# create app folder
RUN mkdir /goApp
COPY ./genesys-api /goApp
WORKDIR /goApp/cmd/genesys-server
# install dependencies
RUN go get gopkg.in/redis.v2
RUN go get github.com/gorilla/handlers
RUN go get github.com/dgrijalva/jwt-go
RUN go get github.com/gorilla/context
RUN go get github.com/gorilla/mux
RUN go get gopkg.in/mgo.v2/bson
RUN go get github.com/graphql-go/graphql
# run phoenix in *dev* mode on port 8080
CMD go run main.go
Here is my Dockerfile.phoenix.development:
# base image elixer to start with
FROM elixir:1.6
# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force
# install the latest phoenix
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force
# create app folder
RUN mkdir /app
COPY ./my_app /app
WORKDIR /app
# install dependencies
RUN mix deps.get
# run phoenix in *dev* mode on port 4000
CMD mix phx.server
Here is my docker-compose.yml file:
version: '3.6'
services:
go:
build:
context: .
dockerfile: Dockerfile.go.development
ports:
- 8080:8080
volumes:
- .:/goApp
depends_on:
- db
- redis
phoenix:
# tell docker-compose which Dockerfile it needs to build
build:
context: .
dockerfile: Dockerfile.phoenix.development
# map the port of phoenix to the local dev port
ports:
- 4000:4000
# mount the code folder inside the running container for easy development
volumes:
- .:/app
# make sure we start mongodb when we start this service
# links:
# - db
depends_on:
- db
- redis
environment:
GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET}
FACEBOOK_CLIENT_ID: ${FACEBOOK_CLIENT_ID}
FACEBOOK_CLIENT_SECRET: ${FACEBOOK_CLIENT_SECRET}
db:
container_name: db
image: mongo:latest
volumes:
- ./data/db:/data/db
ports:
- 27017:27017
redis:
container_name: redis
image: redis:latest
ports:
- "6379:6379"
volumes:
- ./data/redis:/data/redis
entrypoint: redis-server
restart: always
For the error related to go microservice, Since the go binary is not found in PATH, you may need to set the GOPATH env variable via your docker file for go:
export GOPATH=
When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).