I struggle to create a directory in my Dockerfile below. Entering the container after building the image I can't find the directory "models". "ds" directory in path "/usr/src/app/ds/models" is an application directory which was copied. Could you please tell me what is wrong here.
FROM python:3.8
ENV PYTHONUNBUFFERED=1
ENV DISPLAY :0
WORKDIR /usr/src/app
COPY . .
RUN mkdir -p /usr/src/app/ds/models
My docker-compose.yaml file contains volume:
version: '3.8'
services:
app:
build: .
command:
- /bin/bash
- -c
- python manage.py runserver 0.0.0.0:8000
restart: always
volumes:
- .:/usr/src/app
ports:
- '8000:8000'
When your docker-compose.yml file says
volumes:
- .:/usr/src/app
that host directory completely replaces the /usr/src/app directory from your image. This means pretty much nothing in your Dockerfile has an effect; if you try to deploy this setup to another system, you've never run the code in the image.
I'd recommend deleting this block, and also the command: override (make it be the default CMD in the Dockerfile instead).
I need to download models to this directory
Mount only the specific directory you need into your container; don't overwrite the entire application tree. Potentially consider keeping that data directory in a different part of the filesystem.
version: '3.8'
services:
app:
build: .
# no command:
restart: always
volumes:
# only the models subdirectory, not the entire application
- ./ds/models:/usr/src/app/ds/models
ports:
- '8000:8000'
Related
I've install MySQL and PhpMyAdmin on docker
MySQL volume mount works perfectly fine,
But I also want container's /var/www/html/libraries, /var/www/html/themes folders to be saved/persisted to my host.
So that If I change any file and it stays like that..
This is my docker-compose.yml
version: '3.5'
services:
mysql:
container_name: mysql
image: mysql
restart: always
volumes:
- ./var/lib/mysql:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root
phpmyadmin:
container_name: phpmyadmin
image: phpmyadmin/phpmyadmin:latest
restart: always
volumes:
- ./phpmyadmin/libraries:/var/www/html/libraries # Here's the problem
- ./phpmyadmin/themes:/var/www/html/themes # Here's the problem
environment:
PMA_HOST: mysql
The current problem is,
it does create the folders /phpmyadmin/libraries, /phpmyadmin/themes
But inside they're empty and the container's directories (/var/www/html/libraries, /var/www/html/themes) also becomes empty.
I'm very new to Docker, and currently I've no clue :(
Many Thanks in advance.
Your problem is that /var/www/html is populated at build time and volumes are mounted at run time which causes /var/www/html to be overwritten by what you have locally (i.e. nothing).
You need to extend the Dockerfile for PHPMyAdmin to delay populating those directories until after the volumes have been mounted. You'll need something like this setup:
Modify docker-compose.yml to the following:
...
phpmyadmin:
container_name: phpmyadmin
build:
# Use the Dockerfile located at ./build/phpmyadmin/Dockerfile to build this image
context: ./build/phpmyadmin
dockerfile: Dockerfile
restart: always
volumes:
- ./phpmyadmin/libraries:/var/www/html/libraries
- ./phpmyadmin/themes:/var/www/html/themes
environment:
PMA_HOST: mysql
Create a file at ./build/phpmyadmin/Dockerfile with this content:
FROM phpmyadmin/phpmyadmin:latest
# Move the directories you want into a temporary directory
RUN mv /var/www/html /tmp/
# Modify the start up of this image to use a custom script
COPY ./custom-entrypoint.sh /custom-entrypoint.sh
RUN chmod +x /custom-entrypoint.sh
ENTRYPOINT ["/custom-entrypoint.sh"]
CMD ["apache2-foreground"]
Create a custom entrypoint at ./build/phpmyadmin/custom-entrypoint.sh with this content:
#!/bin/sh
# Copy over the saved files
cp -r /tmp/html /var/www
# Kick off the original entrypoint
exec /docker-entrypoint.sh "$#"
Then you can build and start everything with docker-compose up --build.
Note: this will probably cause issues for you if you're trying to version control these directories - you'll probably need to modify custom-entrypoint.sh.
I need to configure docker-compose.yml in a way that will invalidate the local image's docker cache, based on a certain file's checksum.
If it's not possible, I'd like to be able to somehow version the docker-compose.yml or Dockerfile, so that it would rebuild the Docker image of a specific service. I'd want to avoid pushing images to DockerHub. Unless it's an absolute the only solution.
At all costs, I want to avoid bash scripts and in general, writing imperative logic. I'm also not interested in CLI solutions, like passing additional flags to docker-compose up command.
Context:
We use docker-compose during the development of our application.
Our app has also a Dockerfile for building the app localy. We don't push docker images into DockerHub, we just have Dockerfile locally and in docker-compose.yml we declare the sourcecode and package.json (a file that for nodeJS applications use to declare dependencies) as volumes. Now sometimes, we modify the package.json, and docker-compose up throws an error, because the image is already built locally and the previous built doesn't contain the new dependencies. I'd want to be able to tell docker-compose.yml to automatically build a new image if there have been any changes to package.json file since we pull dependencies during the build stage.
docker-compose.yml
version: "3.8"
services:
web:
build:
context: .
ports:
- "8000:8000"
command: npx nodemon -L app.js
volumes:
- ./app:/usr/src/app
- /usr/src/app/node_modules
env_file:
- .env
depends_on:
- mongo
mongo:
image: mongo:latest
container_name: mongo_db
volumes:
- ./config/init.sh:/docker-entrypoint-initdb.d/init.sh
- ./config/mongod.conf:/etc/mongod.conf
- ./logs:/var/log/mongodb/
- ./db:/data/db
env_file:
- .env
ports:
- "27017:27017"
restart: on-failure:5
command: ["mongod", "-f", "/etc/mongod.conf"]
volumes:
db-data:
mongo-config:
Dockerfile:
FROM node:14.15.1
RUN mkdir -p /usr/src/app
# Create app directory
WORKDIR /usr/src/app
#Install app dependencies
COPY package.json /usr/src/app
# Install dependencies
RUN npm install
EXPOSE 8000
CMD ["node", "/app/app.js"]
I'm using Docker
Docker version 19.03.8, build afacb8b
I have the following docker-compose.yml file ...
version: "3.2"
services:
sql-server-db:
build: ./
container_name: sql-server-db
image: microsoft/mssql-server-linux:2017-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "Password1!"
ACCEPT_EULA: "Y"
and here is the Docker file it uses to build ...
FROM microsoft/mssql-server-linux:latest
# Create work directory
RUN mkdir -p /usr/work
WORKDIR /usr/work
# Copy all scripts into working directory
COPY . /usr/work/
# Grant permissions for the import-data script to be executable
RUN chmod +x /usr/work/import-data.sh
EXPOSE 1433
CMD /bin/bash ./entrypoint.sh
On my local machine, I have some files in a "../../scripts/myproject/*.sql" directory (the ".." are relative to the directory where my docker-compose.yml file is stored). Is there a way I can run "docker-compose up" and have those files copied into a directory from which I can then copy them into the container's "/usr/work" directory?
There are 2 ways to solve this, with one being easier than the other, but both have use cases.
The easy way
You could mount the directory directly to the container through the docker-compose like this:
version: "3.2"
services:
sql-server-db:
build: ./
container_name: sql-server-db
image: microsoft/mssql-server-linux:2017-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "Password1!"
ACCEPT_EULA: "Y"
volumes:
- ../../scripts/myproject:/path/to/dir
Note the added volumes compared to the yaml in your question. This will mount the myproject directory to /path/to/dir within the container. What this will also mean is that if the sql-server-db container writes to any of the files in /path/to/dir, then the file in myproject on the host machine will also change, since the files are mounted.
The less easy way
You could copy the files directly during the build of the image. This is a little bit harder, since the build stage of docker doesn't allow the copying of parent directories unless you add some extra arguments. What needs to happen is that you set the context of the build stage to a different directory than the current directory. The context determines which files are sent to the build stage. This is the same directory as the directory the Dockerfile resides in by default.
To take this approach, you need the following in your docker-compose.yml:
version: "3.2"
services:
sql-server-db:
build:
context: ../..
dockerfile: path/to/Dockerfile # Here you should specify the path to your Dockerfile, this is a relative path from your context
container_name: sql-server-db
image: microsoft/mssql-server-linux:2017-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "Password1!"
ACCEPT_EULA: "Y"
So above the context is now ../.. so that you are able to copy files two directories above. You can then copy the myproject directory in your Dockerfile like this:
FROM microsoft/mssql-server-linux:latest
COPY ./scripts/myproject /myfiles
The advantage of this approach is that the files are copied instead of being mounted, so the docker container can write whatever it wants to these files, without affecting the host machine.
I have a simple Laravel application with Nginx, PHP and MySQL each in its own container. It works great in my development environment but for production I need to remove bind volumes and copy the contents to the image itself instead. But how do I do this?
Do I need a seperate docker-compose-prod.yml file? How can I remove volues for production? How can I copy my source code and configuration to the image when deploying for production?
Here is my docker-compose.yml file
version: '3'
networks:
laranet:
services:
nginx:
image: nginx:stable-alpine
container_name: nginxcontainer
ports:
- "80:80"
volumes:
- ./src:/var/www/html
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf
depends_on:
- php
- mysql
networks:
- laranet
mysql:
image: mysql:5.7.22
container_name: mysqlcontainer
ports:
- "3306:3306"
volumes:
- ./mysql:/var/lib/mysql
networks:
- laranet
php:
build:
context: .
dockerfile: php/Dockerfile
container_name: phpcontainer
volumes:
- ./src:/var/www/html
ports:
- "9000:9000"
networks:
- laranet
and here is my php/Dockerfile
FROM php:7.2-fpm-alpine
RUN docker-php-ext-install pdo pdo_mysql
RUN chown -R www-data:www-data /var/www
RUN chmod 755 /var/www
1) copy data only for prod
You can use multistage builds to copy the contents only when you build with the target "prod".
FROM php:7.2-fpm-alpine as base
RUN docker-php-ext-install pdo pdo_mysql
RUN chown -R www-data:www-data /var/www
RUN chmod 755 /var/www
FROM base as dev
VOLUME /var/www/html
FROM base as prod
COPY data /var/www/html
VOLUME /var/www/html
your Docker-compose.yml gets a new line for prod
php:
build:
context: .
dockerfile: php/Dockerfile
target: prod
container_name: phpcontainer
ports:
- "9000:9000"
networks:
- laranet
2) No bind in prod?
would anonymous volumes for dev be a valid solution? e.g. through the definition of VOLUME /var/www/html you specify that the contents of the /var/www/html path should be put into a volume on container start. If no volume is specified in the docker-compose.yml it will create a volume for you. Sweet right?
Sidenote
I do not recommend to split your behavior between dev and prod.
I recommend that you use volumes throughout your stages. The only difference in prod could be that you copy the contents into the image -> before you define the VOLUME, since defining a VOLUME makes the folder unchangeable in the following layers.
david-maze pointed out (see comment)
Putting a VOLUME in your Dockerfile mostly only has confusing side effects, and I'd recommend doing it only if you're absolutely clear on what it means. It's definitely not needed for the OP's setup (and in fact has the likely side effect of leaking anonymous volumes on the production system)
Sources
multi-stage build in docker compose?
https://docs.docker.com/engine/reference/builder/#volume
I am using Docker which is running fine.
I can start a Docker image using docker-compose.
docker-compose rm nodejs; docker-compose rm db; docker-compose up --build
I attached a shell to the Docker container using
docker exec -it nodejs_nodejs_1 bash
I can view files inside the container
(inside container)
cat server.js
Now when I edit the server.js file inside the host, I would like the file inside the container to change without having to restart Docker.
I have tried to add volumes to the docker-compose.yml file or to the Dockerfile, but somehow I cannot get it to work.
(Dockerfile, not working)
FROM node:10
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
VOLUMES ["/usr/src/app"]
EXPOSE 8080
CMD [ "npm", "run", "watch" ]
or
(docker-compose.yml, not working)
version: "3.3"
services:
nodejs:
build: ./nodejs-server
ports:
- "8001:8080"
links:
- db:db
env_file:
- ./.env-example
volumes:
- src: /usr/src/app
db:
build: ./mysql-server
volumes:
- ./mysql-server/data:/docker-entrypoint-initdb.d #A folder /mysql-server/data with a .sql file needs to exist
env_file:
- ./.env-example
volumes:
src:
There is probably a simple guide somewhere, but I havn't found it yet.
If you want a copy of the files to be visible in the container, use a bind mount volume (aka host volume) instead of a named volume.
Assuming your docker-compose.yml file is in the root directory of the location that you want in /usr/src/app, then you can change your docker-compose.yml as follows:
version: "3.3"
services:
nodejs:
build: ./nodejs-server
ports:
- "8001:8080"
links:
- db:db
env_file:
- ./.env-example
volumes:
- .:/usr/src/app
db:
build: ./mysql-server
volumes:
- ./mysql-server/data:/docker-entrypoint-initdb.d #A folder /mysql-server/data with a .sql file needs to exist
env_file:
- ./.env-example