That's the code segment where I load .env file and set the variables.
// init executes the initial configuration.
func init() {
// loads the file for env. variables
if err := godotenv.Load(".env"); err != nil {
log.Printf("failed to load env file: %v\n", err)
}
// env variables are set based values within the .env file.
os.Setenv("dbname", os.Getenv("DB_NAME"))
os.Setenv("username", os.Getenv("DB_USERNAME"))
os.Setenv("pw", os.Getenv("DB_PASSWORD"))
os.Setenv("dbport", os.Getenv("DB_PORT"))
os.Setenv("server_port", os.Getenv("SERVER_PORT"))
os.Setenv("hostname", os.Getenv("DB_CONTAINER_NAME"))
}
When I run docker-compose up --build server everything works despite the error I get as follows:
server_1 | 2022/09/07 11:44:06 failed to load env file: open .env: no such file or directory
However, environments are somehow set.
Here is my docker-compose.yml.
version: '3.8'
services:
db:
image: postgres:14.1-alpine
container_name: ${DB_CONTAINER_NAME}
restart: always
environment:
- POSTGRES_DB=${DB_NAME}
- POSTGRES_USER=${DB_USERNAME}
- POSTGRES_PASSWORD=${DB_PASSWORD}
ports:
- ${DB_PORT}:${DB_PORT}
env_file:
- .env
volumes:
- db:/var/lib/postgresql/data
- ./psql/statements/create-tables.sql:/docker-entrypoint-initdb.d/create_table.sql
server:
build:
context: .
dockerfile: Dockerfile
env_file: .env
depends_on:
- ${DB_CONTAINER_NAME}
networks:
- default
ports:
- ${SERVER_PORT}:${SERVER_PORT}
volumes:
db:
driver: local
And my Dockerfile for go application.
FROM golang:1.18
WORKDIR /src
COPY go.sum go.mod ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /bin/app .
FROM alpine
COPY --from=0 /bin/app /bin/app
ENTRYPOINT ["/bin/app"]
I've altered some of Dockerfile's content I get different errors. What might be the reason for such a problem? Because it fails to load but still works.
Looks like the env variables are set by docker compose from .env file mentioned in yaml file with env_file: .env
open .env: no such file or directory is printed from inside the go app as the .env file is not available inside the container, no COPY/ADD command for same in Dockerfile.
Related
my docker file for ui image is as follows
FROM node:alpine as prodnode
WORKDIR /app
COPY ./package.json ./
RUN npm i
COPY . .
CMD ["npm", "run", "build"]
and my docker compose looks like below.
version: "3"
services:
nginx:
depends_on:
- backend
- ui
restart: always
volumes:
- ./nginx/prod.conf:/etc/nginx/conf.d/default.conf
- static:/usr/share/nginx/html
build:
context: ./nginx/
dockerfile: Dockerfile
ports:
- "80:80"
backend:
build:
context: ./backend/
dockerfile: Dockerfile
volumes:
- /app/node_modules
- ./backend:/app
environment:
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
ui:
tty: true
stdin_open: true
environment:
- CHOKIDAR_USEPOLLING=true
build:
context: ./ui/
dockerfile: Dockerfile
volumes:
- /app/node_modules
- ./ui:/app
- static:/app/build
postgres:
image: "postgres:latest"
environment:
- POSTGRES_PASSWORD=postgres_password
volumes:
static:
I am trying to build static content and copy the content between ui container to nginx container.I use shared volume.Everything works fine as expected.But when I change contents of ui and build again, changes are not reflecting.I tried following thing.
docker-compose down
docker-compose up --build
docker-compose up
None of them is replacing the static content with the new build.
Only when i remove the static volume like below
docker volume rm skeleton_static
and then do
docker-compose up --build
It is changing the content now.. How do i automatically replace the static contents on every docker-compose up or docker-compose up --build thanks.
Named volumes are presumed to hold user data in some format Docker can't understand; Docker never updates their content after they're originally created, and if you mount a volume over image content, the old content in the volume hides updated content in the image. As such, I'd avoid named volumes here.
It looks like in the setup you show, the ui container doesn't actually do anything: its main container process is to build the application, and then it exits immediately. A multi-stage build is a more appropriate approach here, and it will let you compile the application during the image build phase without declaring a do-nothing container or adding the complexity of named volumes.
# ui/Dockerfile
# First stage: build the application; note this is
# very similar to the existing Dockerfile
FROM node:alpine as prodnode
WORKDIR /app
COPY ./package.json ./
RUN npm i
COPY . .
RUN ["npm", "run", "build"] # not CMD
# Second stage: nginx server serving that application
FROM nginx:latest
COPY --from=prodnode /app/build /usr/share/nginx/html
# use default CMD from the base image
In your docker-compose.yml file, you don't need separate "build" and "serve" containers, these are now combined together.
version: "3.8"
services:
backend:
build: ./backend
environment:
- PGUSER=postgres
- PGHOST=postgres
- PGDATABASE=postgres
- PGPASSWORD=postgres_password
- PGPORT=5432
depends_on:
- postgres
# no volumes:
ui:
build: ./ui
depends_on:
- backend
ports:
- '80:80'
# no volumes:
postgres:
image: "postgres:latest"
environment:
- POSTGRES_PASSWORD=postgres_password
volumes: # do persist database data
- pgdata:/var/lib/postgresql/data
volumes:
pgdata:
A similar problem will apply to the anonymous volume you've used for the backend service's node_modules directory, and it will ignore any changes to the package.json file. Since all of the application's code and library dependencies are already included in the image, I've deleted the volumes: block that would overwrite those.
I struggle to create a directory in my Dockerfile below. Entering the container after building the image I can't find the directory "models". "ds" directory in path "/usr/src/app/ds/models" is an application directory which was copied. Could you please tell me what is wrong here.
FROM python:3.8
ENV PYTHONUNBUFFERED=1
ENV DISPLAY :0
WORKDIR /usr/src/app
COPY . .
RUN mkdir -p /usr/src/app/ds/models
My docker-compose.yaml file contains volume:
version: '3.8'
services:
app:
build: .
command:
- /bin/bash
- -c
- python manage.py runserver 0.0.0.0:8000
restart: always
volumes:
- .:/usr/src/app
ports:
- '8000:8000'
When your docker-compose.yml file says
volumes:
- .:/usr/src/app
that host directory completely replaces the /usr/src/app directory from your image. This means pretty much nothing in your Dockerfile has an effect; if you try to deploy this setup to another system, you've never run the code in the image.
I'd recommend deleting this block, and also the command: override (make it be the default CMD in the Dockerfile instead).
I need to download models to this directory
Mount only the specific directory you need into your container; don't overwrite the entire application tree. Potentially consider keeping that data directory in a different part of the filesystem.
version: '3.8'
services:
app:
build: .
# no command:
restart: always
volumes:
# only the models subdirectory, not the entire application
- ./ds/models:/usr/src/app/ds/models
ports:
- '8000:8000'
TL/DR: Can I use .env file variables in docker-compose's environment directives?
I have a Dockerfile which uses the ARG variable to copy files based on an environment variable.
In my docker-compose I want to run two versions of this container with different configuration.
Is there a way where I can set different filepaths in a single .env file and have it build twice, like the example below?
Or is there a smarter way to accomplish this?
/
/.env
/docker-compose.yml
/app
/app/Dockerfile
/version1/data
/version2/data
/.env
VERSION_1_PATH=./version1/data
VERSION_2_PATH=./version2/data
/app/Dockerfile
FROM node:latest
ARG APP_PATH # pull this environment variable
COPY $APP_PATH /var/app/ # use it to copy source to the same generic destination
/docker-compose.yml
version: "3"
services:
version1:
build: ./app
container_name: version1
env_file:
- '.env'
environment:
APP_PATH: ${VERSION_1_PATH}
version2:
build: ./app
container_name: version2
env_file:
- '.env'
environment:
APP_PATH: ${VERSION_2_PATH}
You can add args in compose file when to define build, something like follows:
version: '3'
services:
version1:
build:
context: ./app
args:
- APP_PATH=${VERSION_1_PATH}
version2:
build:
context: ./app
args:
- APP_PATH=${VERSION_2_PATH}
And no need to define .env in env_file if just want it be used in build as .env could default be used in docker-compose.yml. And, environment also not for build, it's for running container.
One example, FYI.
I am using Docker which is running fine.
I can start a Docker image using docker-compose.
docker-compose rm nodejs; docker-compose rm db; docker-compose up --build
I attached a shell to the Docker container using
docker exec -it nodejs_nodejs_1 bash
I can view files inside the container
(inside container)
cat server.js
Now when I edit the server.js file inside the host, I would like the file inside the container to change without having to restart Docker.
I have tried to add volumes to the docker-compose.yml file or to the Dockerfile, but somehow I cannot get it to work.
(Dockerfile, not working)
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
VOLUMES ["/usr/src/app"]
EXPOSE 8080
CMD [ "npm", "run", "watch" ]
or
(docker-compose.yml, not working)
version: "3.3"
services:
nodejs:
build: ./nodejs-server
ports:
- "8001:8080"
links:
- db:db
env_file:
- ./.env-example
volumes:
- src: /usr/src/app
db:
build: ./mysql-server
volumes:
- ./mysql-server/data:/docker-entrypoint-initdb.d #A folder /mysql-server/data with a .sql file needs to exist
env_file:
- ./.env-example
volumes:
src:
There is probably a simple guide somewhere, but I havn't found it yet.
If you want a copy of the files to be visible in the container, use a bind mount volume (aka host volume) instead of a named volume.
Assuming your docker-compose.yml file is in the root directory of the location that you want in /usr/src/app, then you can change your docker-compose.yml as follows:
version: "3.3"
services:
nodejs:
build: ./nodejs-server
ports:
- "8001:8080"
links:
- db:db
env_file:
- ./.env-example
volumes:
- .:/usr/src/app
db:
build: ./mysql-server
volumes:
- ./mysql-server/data:/docker-entrypoint-initdb.d #A folder /mysql-server/data with a .sql file needs to exist
env_file:
- ./.env-example
When I run docker-compose build && docker-compose up redis, with environment specified in docker-compose.yaml and RUN env in the Dockerfile, the environment variables I set don't get printed.
Why does this not work?
I'm using docker-compose version 1.4.2.
Here are the relevant files:
docker-compose.yaml with environment as a list of KEY=value pairs:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO='bar'
docker-compose.yaml with environment as a dictionary:
redis:
build: ../storage/redis
ports:
- "6379:6379"
environment:
- FOO: 'bar'
Dockerfile:
FROM redis:2.6
MAINTAINER me#email.com
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
That's normal
docker-compose only sets the environment variables specified in the environment directive in the docker-compose.yaml file during the run phase of the container, and not during the build phase.
So if you do docker-compose run --entrypoint "/bin/bash" redis -c env you will be able to see your env variables.
If you want to set variables inside your Dockerfile (to be able to see them during the build phase) you can add inside your dockerfile before your RUN env:
ENV FOO bar
Well
I have tested and found following solutions for docker compose with env file or without env file. I will show you two different approach
Lets say you have following docker compose yml file
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
env_file: docker.env
Now you need to setup the postgres variable in a file called docker.env. Remember you need to keep the docker_compose.yml file and docker.env file in same folder.
Next, In the docker.env file you need to have the database variable and value like this:
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
POSTGRES_DB=myapp_db
Now hit docker-compose up . It should work.
Lets say now you dont like to specify the env file name in the docker-compose.yml file. So you have to write docker-compose.yml file like this:
version: '3.8'
services:
db:
image: postgres:13
volumes:
- "./volumes/postgres:/var/lib/postgresql/data"
ports:
- "5432:5432"
environments:
- POSTGRES_USER=${PGU}
-POSTGRES_PASSWORD=${PGP}
-POSTGRES_DB=${PGD}
Now your docker.env file should look like this:
PGU=postgres
PGP=postgres
PGD=myapp_db
now hit docker-compose --env-file docker.env up
you are good to go.
This is because you were using environment when (I guess) you wanted to use args inside the build block:
redis:
build:
context: ../storage/redis
args:
- FOO: 'bar'
ports:
- "6379:6379"
Your Dockerfile would define FUN in the (image) environment:
FROM redis:2.6
RUN mkdir -p /var/redis && chown -R redis:redis /var/redis
# Read FUN from (build) arguments
# (may define a default: ARG FUN='wow')
ARG FUN
# Define env variable FUN with value from ARG
ENV FUN=$FUN
RUN echo '-------------- env ---------------'
RUN env
COPY redis.conf /usr/local/etc/redis/redis.conf
EXPOSE 6379
ENTRYPOINT ["redis-server", "/usr/local/etc/redis/redis.conf"]
The environment block is used to define variables for the running container (when docker-compose up, NOT when docker-compose build).