How to run a command after creating a container using yml file - docker

I have very limited experience with Docker, and having an issue, so I have the following yml file:
version: '2'
services:
python:
restart: always
build: .path/to/docker/file
ports:
- "5000:5000"
- "8888:8888"
links:
- db
depends_on:
- db
volumes:
- ./path/to/file/:/app:z
entrypoint:
- python
- -u
- /app/run.py
db:
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: xxyyzz
MYSQL_DATABASE: database_name
MYSQL_USER: user_name
MYSQL_PASSWORD: xxyyzz
volumes:
- ./Dump.sql:/db/Dump.sql:z
- ./Dump_Test.sql:/db/Dump_Test.sql:z
- ./big_fc.sql:/db/big_fc.sql:z
ports:
- "3306:3306"
I need to run the following commands for the second container db:
echo '[mysqld]' >> /etc/mysql/conf.d/mysql.cnf
echo 'sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' >> /etc/mysql/conf.d/mysql.cnf
The point is that I want to run these commands just once as when you provide a docker file for the first container python. How can I achieve this?

My first advice is to build your own image with your own Dockerfile. In this case you will do it once and all the containers will have the same config.
Dockerfile should look like this:
FROM mysql:latest
COPY create_config.sh /tmp/create_config.sh
CMD ["/tmp/create_config.sh"]
Create create_config.sh in the same dir as your Dockerfile and put your required commands there:
#!/bin/bash
echo '[mysqld]' >> /etc/mysql/conf.d/mysql.cnf
echo 'sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION' >> /etc/mysql/conf.d/mysql.cnf
Now you can use it in your docker-compose.yml:
db:
context: .
dockerfile: Dockerfile
Keep in mind, that context is the path to the dir where your new Dockerfile is.

Related

How to execute command "./gradlew clean build" before docker-compose up

to wrote Code
docker-compose.yml
version: '3'
services:
moinda-db:
container_name: moinda-db
image: mysql:8.0
environment:
MYSQL_DATABASE: moinda
MYSQL_ROOT_HOST: '%'
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: moinda
MYSQL_PASSWORD: moinda
restart: always
ports:
- "3306:3306"
networks:
- moinda-local
moinda-app:
container_name: moinda-app
build:
context: ../moinda-api
dockerfile: Dockerfile
ports:
- "9090:9090"
restart: always
depends_on:
- moinda-db
networks:
- moinda-local
networks:
moinda-local:
driver: bridge
Dockerfile
FROM openjdk:11-jdk
ARG JAR_FILE=build/libs/moinda-api-1.0-SNAPSHOT.jar
ADD ${JAR_FILE} moinda.jar
ENTRYPOINT ["java", "-jar", "/moinda.jar"]
as with Title, I want to be executing command "./gradlew clean build" before build images
command syntax is not friendly yet..
try to right down command In "Dockerfile"
1.
RUN ["./gradlew", "clean", "build"]
2.
CMD ["./gradlew", "clean", "build"]
First Command is execute but failed
Second Command is not execute..
plz write comment,,
You can create a script file and do something like this:
./gradlew clean build
docker-compose up

How to swap env file for another docker service

I have a docker-compose.yml
services:
nextjs:
container_name: next_app
build:
context: ./
restart: on-failure
command: npm run dev
volumes:
- ./:/app
- /app/node_modules
- /app/.next
ports:
- "3000:3000"
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
I want to change env_file for next_app from cypress service. I found solution like this
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
next_app:
env_file: .env.test
But this solution does not work. Is it even possible ?
Try something like cp .env #docker/.env
No. In Compose (or Docker, or even more generally in Linux/Unix) there is no way for one container (process) to specify environment variables for another.
You can think of a docker-compose.yml file as a set of instructions only for running containers. If you need a specific set of containers for a specific context – you don't normally need to run Cypress in production, but this is an integration-test setup – it's fine to write a separate Compose file just for that setup.
# docker-compose.cypress.yml
# Used only for integration testing
version: '3.8'
services:
nextjs:
build: .
restart: on-failure
ports:
- "3000:3000"
env_file: .env.test # <-- specific to this test-oriented Compose file
cypress:
build: ./e2e
depends_on:
- nextjs
environment:
- CYPRESS_baseUrl=http://nextjs:3000
docker-compose -f docker-compose.cypress.yml up --build
This can also be a case where using multiple Compose files together can be a reasonable option. You can define a "standard" Compose setup that only defines the main service, and then an e2e-test Compose file that adds the Cypress container and the environment settings.
# docker-compose.yml
version: '3.8'
services:
nextjs:
image: registry.example.com/nextjs:${NEXTJS_TAG:-latest}
restart: on-failure
ports:
- '3000:3000'
# docker-compose.e2e.yaml
version: '3.8'
services:
nextjs:
# These add to the definitions in the base `docker-compose.yml`
build: .
env_file: .env.test
cypress:
# This is a brand new container for this specific setup
depends_on: [nextjs]
et: cetera # copy from question or previous Compose setup
docker-compose \
-f docker-compose.yml \
-f docker-compose.e2e.yml \
up --build

Docker: Node server is not running after start the server

I have a Dockerfile and a docker-compose.yml file.
If I execute docker-compose up, it returns:
Creating network "demoapi_webnet" with the default driver
Creating demoapi_web_1 ... done
Creating d2c_postgres ... done
Attaching to demoapi_web_1, d2c_postgres
...
d2c_postgres | 2020-07-28 00:47:48.772 UTC [1] LOG: database system is ready to accept connections
But my node server is not starting.
These are my docker configuration files:
Dockerfile
FROM node:12.13-alpine As development
WORKDIR /app
COPY package.json .
COPY package-lock.json .
RUN npm install
COPY dist .
COPY wait-for-it.sh .
CMD ["npm", "run", "start"]
docker-compose.yml
version: '3'
services:
db:
image: postgres
networks:
- webnet
container_name: "d2c_postgres"
environment:
POSTGRES_PASSWORD: 010203
POSTGRES_USER: postgres
POSTGRES_DB: demo
ports:
- "5432:5432"
web:
image: nest-app
ports:
- "3000:3000"
networks:
- webnet
environment:
DB_HOST: db
command: ["./wait-for-it.sh", "db:5432", "--", "npm", "run", "start"]
networks:
webnet:
My only clue is this line:
env: can't execute 'bash': No such file or directory
I can stablish a connection to pgadmin/postgres with that configuration, but the node server is not starting. What am I doing wrong and how can I solve it?
Wait-for-it is base on bash and it's not compatible with alpine as alpine is base on ash or sh that is why you are seeing can't execute 'bash': No such file or directory. You can look into the open issue for alpine support.
Can you make an /bin/sh version for use with alpine linux
For alpine, you can use wait-for
./wait-for is a script designed to synchronize services like docker containers. It is sh and alpine compatible.
services:
db:
image: postgres:9.4
backend:
build: backend
command: sh -c './wait-for db:5432 -- npm start'
depends_on:
- db
After big research, I found a similar issue here:
docker-compose: nodejs container not communicating with Postgres container
For some reason wait for it wasn't working (not sure if is a windows issue), that sh file is not mandatory to wait until database start, you can use depends_on to indicate that the server should start after a specified service:
version: '3'
services:
db:
image: postgres
networks:
- webnet
container_name: "node_postgres"
environment:
POSTGRES_PASSWORD: 010203
POSTGRES_USER: postgres
POSTGRES_DB: demo
ports:
- "5432:5432"
web:
image: nest-app
depends_on:
- db
ports:
- "3000:3000"
networks:
- webnet
environment:
DB_HOST: db
command: ["npm", "run", "start"]
networks:
webnet:

docker-compose: run a command on a pgsql container

I am trying to run the following docker-compose file:
version: "3"
services:
db:
image: postgres
container_name: pgsql
environment:
- foo=foo
- bar=bar
volumes:
- ./sql/:/opt/sql
command: bash /opt/sql/create-db.sql
# command: ps -aux
web:
image: benit/debian-web
container_name: web
depends_on:
- db
ports:
- 80:80
volumes:
- ./html:/var/www/html
I am encountering an error with the line:
command: bash /opt/sql/create-db.sql
It is because pgsql service is not started. It can be monitored with command: ps -aux
How can I run my script once pgsql service is started ?
You can use a volume to provide an initialization sql script:
version: "3"
services:
db:
image: postgres
container_name: pgsql
environment:
- foo=foo
- bar=bar
volumes:
- ./sql/:/opt/sql
- ./init.sql:/docker-entrypoint-initdb.d/init.sql
web:
image: benit/debian-web
container_name: web
depends_on:
- db
ports:
- 80:80
volumes:
- ./html:/var/www/html
This will work because original Posgresql dockerfile contains a script (that runs after Posrgres has been started) which will execute any *.sql files from /docker-entrypoint-initdb.d/ folder.
By mounting your local volume in that place, your sql files will be run at the right time.
It's actually mentioned in documentation for that image: https://hub.docker.com/_/postgres under the How to extend this image section.

Docker-Compose persistent data MySQL

I can't seem to get MySQL data to persist if I run $ docker-compose down with the following .yml
version: '2'
services:
# other services
data:
container_name: flask_data
image: mysql:latest
volumes:
- /var/lib/mysql
command: "true"
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes_from:
- data
ports:
- "3306:3306"
My understanding is that in my data container using volumes: - /var/lib/mysql maps it to my local machines directory where mysql stores data to the container and because of this mapping the data should persist even if the containers are destroyed. And the mysql container is just a client interface into the db and can see the local directory because of volumes_from: - data
Attempted this answer and it did not work. Docker-Compose Persistent Data Trouble
EDIT
Changed my .yml as shown below and created a the dir ./data but now when I run docker-compose up --build the mysql container wont start throws error saying
data:
container_name: flask_data
image: mysql:latest
volumes:
- ./data:/var/lib/mysql
command: "true"
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes_from:
- data
ports:
- "3306:3306"
flask_mysql | mysqld: Can't create/write to file '/var/lib/mysql/is_writable' (Errcode: 13 - Permission denied)
flask_mysql | 2016-08-26T22:29:21.182144Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
flask_mysql | 2016-08-26T22:29:21.185392Z 0 [ERROR] --initialize specified but the data directory exists and is not writable. Aborting.
The data container is a superfluous workaround. Data-volumes would do the trick for you. Alter your docker-compose.yml to:
version: '2'
services:
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes:
- my-datavolume:/var/lib/mysql
volumes:
my-datavolume:
Docker will create the volume for you in the /var/lib/docker/volumes folder. This volume persist as long as you are not typing docker-compose down -v
There are 3 ways:
First way
You need specify the directory to store mysql data on your host machine. You can then remove the data container. Your mysql data will be saved on you local filesystem.
Mysql container definition must look like this:
mysql:
container_name: flask_mysql
restart: always
image: mysql:latest
environment:
MYSQL_ROOT_PASSWORD: 'test_pass' # TODO: Change this
MYSQL_USER: 'test'
MYSQL_PASS: 'pass'
volumes:
- /opt/mysql_data:/var/lib/mysql
ports:
- "3306:3306"
Second way
Would be to commit the data container before typing docker-compose down:
docker commit my_data_container
docker-compose down
Third way
Also you can use docker-compose stop instead of docker-compose down (then you don't need to commit the container)
first, you need to delete all old mysql data using
docker-compose down -v
after that add two lines in your docker-compose.yml
volumes:
- mysql-data:/var/lib/mysql
and
volumes:
mysql-data:
your final docker-compose.yml will looks like
version: '3.1'
services:
php:
build:
context: .
dockerfile: Dockerfile
ports:
- 80:80
volumes:
- ./src:/var/www/html/
db:
image: mysql
command: --default-authentication-plugin=mysql_native_password
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
volumes:
- mysql-data:/var/lib/mysql
adminer:
image: adminer
restart: always
ports:
- 8080:8080
volumes:
mysql-data:
after that use this command
docker-compose up -d
now your data will persistent and will not be deleted even after using this command
docker-compose down
extra:- but if you want to delete all data then you will use
docker-compose down -v
You have to create a separate volume for mysql data.
So it will look like this:
volumes_from:
- data
volumes:
- ./mysql-data:/var/lib/mysql
And no, /var/lib/mysql is a path inside your mysql container and has nothing to do with a path on your host machine. Your host machine may even have no mysql at all. So the goal is to persist an internal folder from a mysql container.
Adding on to the answer from #Ohmen, you could also add an external flag to create the data volume outside of docker compose. This way docker compose would not attempt to create it. Also you wouldn't have to worry about losing the data inside the data-volume in the event of $ docker-compose down -v.
The below example is from the official page.
version: "3.8"
services:
db:
image: postgres
volumes:
- data:/var/lib/postgresql/data
volumes:
data:
external: true
Actually this is the path and you should mention a valid path for this to work. If your data directory is in current directory then instead of my-data you should mention ./my-data, otherwise it will give you that error in mysql and mariadb also.
volumes:
./my-data:/var/lib/mysql
Feasible bind mount solution:
mariadb:
image: mariadb:latest
restart: unless-stopped
environment:
- MARIADB_ROOT_PASSWORD=${MARIADB_ROOT_PASSWORD}
volumes:
- type: bind
source: /host/dir
target: /var/lib/mysql

Resources