Dockerfile and docker-compose not updating with new instructions - docker

When I try to build a container using docker-compose like so
nginx:
build: ./nginx
ports:
- "5000:80"
the COPY instructions isnt working when my Dockerfile simply
looks like this
FROM nginx
#Expose port 80
EXPOSE 80
COPY html /usr/share/nginx/test
#Start nginx server
RUN service nginx restart
What could be the problem?

It seems that when using the docker-compose command it saves an intermediate container that it doesnt show you and constantly reruns that never updating it correctly.
Sadly the documentation regarding something like this is poor. The way to fix this is to build it first with no cache and then up it like so
docker-compose build --no-cache
docker-compose up -d

I had the same issue and a one liner that does it for me is :
docker-compose up --build --remove-orphans --force-recreate
--build does the biggest part of the job and triggers the build.
--remove-orphans is useful if you have changed the name of one of your services. Otherwise, you might have a warning leftover telling you about the old, now wrongly named service dangling around.
--force-recreate is a little drastic but will force the recreation of the containers.
Reference: https://docs.docker.com/compose/reference/up/
Warning I could do this on my project because I was toying around with really small container images. Recreating everything, everytime, could take significant time depending on your situation.

If you need to make docker-compose to copy files every time on up command I suggest declaring a volumes option to your service in the compose.yml file. It will persist your data and also will copy files from that folder into the container.
More info here volume-configuration-reference
server:
image: server
container_name: server
build:
context: .
dockerfile: server.Dockerfile
env_file:
- .envs/.server
working_dir: /app
volumes:
- ./server_data:/app # <= here it is
ports:
- "9999:9999"
command: ["command", "to", "run", "the", "server", "--some-options"]
Optionally, you can add the following section to the end of the compose.yml file. It will keep that folder persisted then. The data in that folder will not be removed after the docker-compose stop command or the docker-compose down command. To remove the folder you will need to run the down command with an additional flag -v:
docker-compose down -v
For example, including volumes:
services:
server:
image: server
container_name: server
build:
context: .
dockerfile: server.Dockerfile
env_file:
- .envs/.server
working_dir: /app
volumes:
- ./server_data:/app # <= here it is
ports:
- "9999:9999"
command: ["command", "to", "run", "the", "server", "--some-options"]
volumes: # at the root level, the same as services
server_data:

Related

How do I ensure docker is running the new code in my containers when starting up?

I am currently writing a webapp - java backend, react front end and have been deploying via a docker compose file. I've made changes and when I try to run them via yarn build for my front end server and starting my back end server with maven, the changes appear. However, when running with docker, the changes aren't there.
I've been using the docker compose up and docker compose down commands and I even run docker system prune -a after stopping my docker containers via the docker compose down command but my new changes aren't showing. I'd appreciate any guidance on what I'm doing wrong to help show my changes.
I also have docker desktop and have manually gone and deleted all of the volumes, containers and images so that they have to be regenerated. Running the build commands to specify ignoring cache didn't help either.
I also deleted the .m2 folder so that this gets generated (my understanding is that this is the cache store for the backend). My changes are mainly on the front end but since my front end container depends on this, I thought regenerating the back-end container may have a knock on effect that may help.
I would greatly appreciate any help, please do let me know if there's anything else to help with context. The changes involve removing a search bar and some text, both of which are commented out in the code but still appear whilst I also add another button which doesn't show up.
My docker compose file is below as follows:
services:
mysqldb:
# image: mysql:5.7
build: ./Database
restart: unless-stopped
env_file: ./.env
environment:
- MYSQL_ROOT_PASSWORD=$MYSQLDB_ROOT_PASSWORD
- MYSQL_DATABASE=$MYSQLDB_DATABASE
ports:
- $MYSQLDB_LOCAL_PORT:$MYSQLDB_DOCKER_PORT
volumes:
- db:/var/lib/mysql
networks:
- backend
app_backend:
depends_on:
- mysqldb
build: ./
restart: on-failure
env_file: ./.env
ports:
- $SPRING_LOCAL_PORT:$SPRING_DOCKER_PORT
environment:
SPRING_APPLICATION_JSON: '{
"spring.datasource.url" : "jdbc:mysql://mysqldb:$MYSQLDB_DOCKER_PORT/$MYSQLDB_DATABASE?useSSL=false&allowPublicKeyRetrieval=true&serverTimezone=UTC",
"spring.datasource.username" : "$MYSQLDB_USER",
"spring.datasource.password" : "$MYSQLDB_ROOT_PASSWORD",
"spring.jpa.properties.hibernate.dialect" : "org.hibernate.dialect.MySQL5InnoDBDialect",
"spring.jpa.hibernate.ddl-auto" : "update"
}'
volumes:
- .m2:/root/.m2
stdin_open: true
tty: true
networks:
- backend
- frontend
app_frontend:
depends_on:
- app_backend
build:
../MyProjectFrontEnd
restart: on-failure
ports:
- 80:80
networks:
- frontend
volumes:
db:
networks:
backend:
frontend:
Since the issue is on the front end, I've also attached the dockerfile for the front end below:
FROM node:16.13.0-alpine AS react-build
WORKDIR /MyProjectFrontEnd
RUN yarn cache clean
RUN yarn install
COPY . ./
RUN yarn
RUN yarn build
# Stage 2 - the production environment
FROM nginx:alpine
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY /build /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
Update - the cache in the browser was storing some (rookie error) however, not all the changes are still being loaded
If your source code is in the same folder (usually) of your Dockerfile, you can be sure that your last source code will be built and deployed. This feature is one of the cornerstones, which is the base of docker. If this would be failing, it would be the end of the world.
These kind of errors are not related to the docker core. Usually is something at application level and/or its development:
Libraries mistake
Developer mistake
Functional test mistake
Load Balancer mistake
Advice
docker-compose and windows are for development stage. For deployment on real environments for real users, you should use linux and some tool like Kubernetes.

Why is docker-compose running the same command and using the wrong Dockerfile?

I've got a simple Node / React project. I'm trying to use Docker to create two containers, one for the server, and one for the client, each with their own Dockerfile in the appropriate directory.
docker-compose.yml
version: '3.9'
services:
client:
image: node:14.15-buster
build:
context: ./src
dockerfile: Dockerfile.client
ports:
- '3000:3000'
- '45799:45799'
volumes:
- .:/app
tty: true
server:
image: node:14.15-buster
build:
context: ./server
dockerfile: Dockerfile.server
ports:
- '3001:3001'
volumes:
- .:/app
depends_on:
- redis
links:
- redis
tty: true
redis:
container_name: redis
image: redis
ports:
- '6379'
src/Dockerfile.client
FROM node:14.15-buster
# also the directory you land in on ssh
WORKDIR /app
CMD cd /app && \
yarn && \
yarn start:client
server/Dockerfile.server
FROM node:14.15-buster
# also the directory you land in on ssh
WORKDIR /app
CMD cd /app && \
yarn && \
yarn start:server
After building and starting the containers, both containers run the same command, seemingly at random. Either both run yarn start:server or yarn start:client. The logs clearly detail duplicate startup commands and ports being used. Requests to either port 3000 (client) or 3001 (server) confirm that the same one is being used in both containers. If I change the command in both Dockerfiles to echo the respective filename (Dockerfile.server! or Dockerfile.client!), startup reveals only one Dockerfile being used for both containers. I am also running the latest version of Docker on Mac.
What is causing docker-compose to use the same Dockerfile for both containers?
After a lengthy and painful bout of troubleshooting, I narrowed the issue down to duplicate image references. image: node:14.15-buster for each service in docker-compose.yml and FROM node:14.15-buster in each Dockerfile.
Why this would cause this behavior is unclear, but after removing the image references in docker-compose.yml and rebuilding / restarting, everything works as expected.
When you run docker-compose build with both image and build properties set on a service, it will build an image according to the build property and then tag the image according to the image property.
In your case, you have two services building different images and tagging them with the same tag node:14.15-buster. One will overwrite the other.
This probably has the additional unintended consequence of causing your next image to be built on top of the previously built image instead of the true node:14.15-buster.
Then when you start the service, both containers will use the image tagged node:14.15-buster.
From the docs:
If you specify image as well as build, then Compose names the built image with the webapp and optional tag specified in image

creating a redis docker container with an exising rdb and load module at initiation?

I am trying to start a docker container using a redis db that I have a persistent copy saved to a local machine.
I currently have a docker container loading redis with a volume using this docker-compose.yml but it misses my redis.conf (which contains the loadmodule command) is located in the volume with the rdb file
version: '3'
services:
redis:
image: redis
container_name: "redis"
ports:
- "6379:6379"
volumes:
- E:\redis_backup_conf:/data
This begins to load the RDB but crashes out because the data uses this time series module.
I can load a seperate docker container with a fresh redis db that has the time seriese module loaded using the following dockerfile. My issue is I can't figure out how to do both at the same time!
Is there someway of calling a dockerfile from a docker-compose.yml or declaring the volume in the dockerfile?
That, or should I be creating my own image that I can call in the docker-compose.yml?
Any help woule be appreciated, I'm honeslty just going round in circles I think.
dockerfile
# BUILD redisfab/redistimeseries:${VERSION}-${ARCH}-${OSNICK}
ARG REDIS_VER=6.0.1
# stretch|bionic|buster
ARG OSNICK=buster
# ARCH=x64|arm64v8|arm32v7
ARG ARCH=x64
#----------------------------------------------------------------------------------------------
FROM redisfab/redis:${REDIS_VER}-${ARCH}-${OSNICK} AS builder
ARG REDIS_VER
ADD ./ /build
WORKDIR /build
RUN ./deps/readies/bin/getpy2
RUN ./system-setup.py
RUN make fetch
RUN make build
#----------------------------------------------------------------------------------------------
FROM redisfab/redis:${REDIS_VER}-${ARCH}-${OSNICK}
ARG REDIS_VER
ENV LIBDIR /usr/lib/redis/modules
WORKDIR /data
RUN mkdir -p "$LIBDIR"
COPY --from=builder /build/bin/redistimeseries.so "$LIBDIR"
EXPOSE 6379
CMD ["redis-server", "--loadmodule", "/usr/lib/redis/modules/redistimeseries.so"]
EDIT:
ok.. slight improvement i can call a redis-timeseries image in the docker-compose.yml
services:
redis:
image: redislabs/redistimeseries
container_name: "redis"
ports:
- "6379:6379"
volumes:
- E:\redis_backup_conf:/data
This is a start however I still need to increase the maximum number of db's, I have been using the redis.conf to do this in the past.
You can just have docker-compose build your dockerfile directly. Assume your docker-compose file is in folder called myproject . Also assume your dockerfile is in a folder called myredis and that myredis is in the myproject folder. Then you can replace this line in your docker-compose file:
Image: redis
With:
Build: ./myredis
That will build and use your custom image

Docker container not updating on code change

I have a Dockerfile to build my node container, it looks as follows:
FROM node:12.14.0
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 4500
CMD ["npm", "start"]
based on this docker file, I am using docker compose to run this container and link it to a mongo container such that it refers to mongo-service. The docker-compose.yml looks as follows
version: '3'
services:
backend:
container_name: docker-node-mongo-container
restart: always
build: .
ports:
- '4700:4500'
links:
- mongo-service
mongo-service:
container_name: mongo-container
image: mongo
ports:
- "27017:27017"
Expected behavior: Everytime I make a new change to the project on my local computer, I want the docker-compose to restart so that the new changes are reflected.
Current behavior: To make the new changed reflect on docker-compose, I have to do docker-compose down and then delete images. I am guessing that it has to rebuild images. How do I make it so that whenever I make change, the dockerfile builds a new image?
I understand that need to use volumes. I am just failing to understand how. Could somebody please help me here? docker
When you make a change, you need to run docker-compose up --build. That will rebuild your image and restart containers as needed.
Docker has no facility to detect code changes, and it is not intended as a live-reloading environment. Volumes are not intended to hold code, and there are a couple of problems people run into attempting it (Docker file sync can be slow or inconsistent; putting a node_modules tree into an anonymous volume actively ignores changes to package.json; it ports especially badly to clustered environments like Kubernetes). You can use a host Node pointed at your Docker MongoDB for day-to-day development, and still use this Docker-based setup for deployment.
In order for you to 'restart' your docker application, you need to use docker volumes.
Add into your docker-compose.yml file something like:
version: '3'
services:
backend:
container_name: docker-node-mongo-container
restart: always
build: .
ports:
- '4700:4500'
links:
- mongo-service
volumes:
- .:/usr/src/app
mongo-service:
container_name: mongo-container
image: mongo
ports:
- "27017:27017"
The volumes tag is a simple saying: "Hey, map the current folder outside the container (the dot) to the working directory inside the container".

Deploy with docker-compose.yml

Not sure if it will be a duplicate question but i tried to find out stuff but not sure if i have similar situation like others.
So i am new to docker and trying to setup a deployment for a small website.
So far i have a folder which has 3 files.
index.html - has basic html
Dockerfile - which has
FROM ubuntu:16.04
COPY . /var/www/html/
docker-compose.yml - which has
version: '2.1'
services:
app:
build: .
image: myname/myapp:1.0.0
nginx:
image: nginx
container_name: nginx
volumes:
- ./host-volumes:/cont-volumes
network_mode: "host"
phpfpm56:
image: php-fpm:5.6
container_name: phpfpm56
volumes:
- ./host-volumes:/cont-volumes
network_mode: "host"
mysql:
image: mysql:5.7
container_name: mysql
ports:
- "3306:3306"
volumes:
- mysql:/var/lib/mysql
volumes:
mysql:
Now i am using jenkins to create build, putting my all codes to host volumes to make it available to container and then i would run
docker-compose build
Now it creates an image and i push it to docker hub.
Then i login to remote server and pull the image and run. But that wont work because i still need to run docker-compose up inside the container.
Is this the right approach or i am missing something here?
The standard way to do this is to copy your code into the image. Do not bind-mount host folders containing your code; instead, use a Dockerfile COPY directive to copy in the application code (and in a compiled language, use a RUN command to build it). For example, your PHP container might have a corresponding Dockerfile that looks like (referencing this base Dockerfile)
FROM php-fpm:5.6
# Base Dockerfile defines a sensible WORKDIR
COPY . .
# Base Dockerfile sets EXPOSE 9000
# Base Dockerfile defines ENTRYPOINT, CMD
Then your docker-compose.yml would say, in part
version: '3'
service:
phpfpm56:
build: .
image: me/phpfpm56:2019-04-30
# No other settings
And then your nginx configuration would say, in part (using the Docker Compose service name as a hostname)
fastcgi_pass phpfpm56:9000
If you use this in production you need to comment out the build: lines I think.
If you're extremely set on a workflow where there is no hostname other than localhost and you do not need to rebuild Docker images to update code, you at least need to restart (some of) your containers after you've done the code push.
docker-compose stop app phpfpm56
docker-compose up -d
You might look into a system-automation tool like Ansible or Chef to automate the code-push mechanism. Those same tools can also just install nginx and PHP, and if you're trying to avoid the Docker image build sequence, you might have a simpler installation and deployment system running servers directly on the host.
docker-compose up should not be run inside a container but on a docker host. So this could be run via sh on a host but you need to have access to the composefile wherever you run the command.

Resources