Please help, I have the docker-compose file below, and I want to write an Ansible playbook that runs the docker-compose file on localhost and a remote target.
version: '2.0'
services:
weather-backend:
build: ./backend
volumes: #map backend dir and package inside container
- './backend/:/usr/src/'
- './backend/package.json:/usr/src/package.json'
#ports:
# - "9000:9000" #expose backend port - Host:container
ports:
- "9000:9000"
command: npm start
weather-frontend:
build: ./frontend
depends_on:
- weather-backend
volumes:
- './frontend/:/usr/src/'
- '/usr/src/node_modules'
ports:
- "8000:8000" #expose ports -Host:container
environment:
NODE_ENV: "development"
Since Ansible 2.1, you can have a look at the docker_compose, which can read directly a docker-compose.yml
playbook task:
- name: run the service defined in my_project's docker-compose.yml
docker_compose:
project_src: /path/to/my_project
Related
I have a docker-compose.yml
services:
nextjs:
container_name: next_app
build:
context: ./
restart: on-failure
command: npm run dev
volumes:
- ./:/app
- /app/node_modules
- /app/.next
ports:
- "3000:3000"
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
I want to change env_file for next_app from cypress service. I found solution like this
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
next_app:
env_file: .env.test
But this solution does not work. Is it even possible ?
Try something like cp .env #docker/.env
No. In Compose (or Docker, or even more generally in Linux/Unix) there is no way for one container (process) to specify environment variables for another.
You can think of a docker-compose.yml file as a set of instructions only for running containers. If you need a specific set of containers for a specific context – you don't normally need to run Cypress in production, but this is an integration-test setup – it's fine to write a separate Compose file just for that setup.
# docker-compose.cypress.yml
# Used only for integration testing
version: '3.8'
services:
nextjs:
build: .
restart: on-failure
ports:
- "3000:3000"
env_file: .env.test # <-- specific to this test-oriented Compose file
cypress:
build: ./e2e
depends_on:
- nextjs
environment:
- CYPRESS_baseUrl=http://nextjs:3000
docker-compose -f docker-compose.cypress.yml up --build
This can also be a case where using multiple Compose files together can be a reasonable option. You can define a "standard" Compose setup that only defines the main service, and then an e2e-test Compose file that adds the Cypress container and the environment settings.
# docker-compose.yml
version: '3.8'
services:
nextjs:
image: registry.example.com/nextjs:${NEXTJS_TAG:-latest}
restart: on-failure
ports:
- '3000:3000'
# docker-compose.e2e.yaml
version: '3.8'
services:
nextjs:
# These add to the definitions in the base `docker-compose.yml`
build: .
env_file: .env.test
cypress:
# This is a brand new container for this specific setup
depends_on: [nextjs]
et: cetera # copy from question or previous Compose setup
docker-compose \
-f docker-compose.yml \
-f docker-compose.e2e.yml \
up --build
I'm working with docker containers for some projects and to save time i clone the docker composer file for my other projects.
The problem I have is that the ports for my mysql_database and apache_service are fixed values.
Example:
version: "3.2"
services:
apache_service:
build:
context: './docker/apache/'
links:
- mysql_service:mysql_service
depends_on:
- mysql_service
ports:
- "8080:80" # "random_port:80"
volumes:
- ./:/var/www/
mysql_service:
build:
context : ./
dockerfile : ./docker/mysql/Dockerfile
command: [
'--character-set-server=utf8mb4',
'--collation-server=utf8mb4_unicode_ci',
'--default-authentication-plugin=mysql_native_password',
]
restart:
always
volumes:
- ./docker/initdb:/docker-entrypoint-initdb.d
- ./docker/mysql/logs:/var/log/mysql
ports:
- "4306:3306" # "random_port:3306"
environment:
MYSQL_ROOT_PASSWORD: ${DB_PASSWORD}
container_name:
mysql_service
When i copy the docker-composer file and write docker-composer up i alwayas have to change the ports previously...
How i could automatily get an avaliable port for this services?
Use 0 as the host port: this way you will get the first available (random) port from the operating system; to later get the actual used port you can use the docker port <container> command.
I have a Tomcat docker container and Filebeat docker container both are up and running.
My objective: I need to collect tomcat logs from running Tomcat container to Filebeat container.
Issue: I have no idea how to get collected log files from Tomcat container.
What I have tried so far: I have tried to create a docker volume and add tomcat logs to that volume and access that volume from filebeat container, but ended with no success.
Structure: I have wrote docker-compose.yml file under project Logstash(root directory of the project) with following project structure.(Here I want to up and run Elasticsearch, Logstash, Filebeat and Kibana docker containers from one configuration file). docker-containers(root directory of the project) with following structure (here I want to up and run Tomcat, Nginx and Postgres containers from one configuration file).
Logstash: contain 4 main sub directories (Filebeat, Logstash, Elasticsearch and Kibana), ENV file and docker-compose.yml file. Both sub directories contain Dockerfiles to pull images and build the containers.
docker-containers: contains 3 main sub directories (Tomcat, Nginx and Postgres). ENV file and docker-compose.yml file. Both sub directories contain separate Dockerfiles to pull docker image and build the container.
Note: I think this basic structure my helpful to understand my requirements.
docker-compose.yml files
Logstash.docker-compose.yml file
version: '2'
services:
elasticsearch:
container_name: OTP-Elasticsearch
build:
context: ./elasticsearch
args:
- ELK_VERSION=${ELK_VERSION}
volumes:
- ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
filebeat:
container_name: OTP-Filebeat
command:
- "-e"
- "--strict.perms=false"
user: root
build:
context: ./filebeat
args:
- ELK_VERSION=${ELK_VERSION}
volumes:
- ./filebeat/config/filebeat.yml:/usr/share/filebeat/filebeat.yml
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
- logstash
logstash:
container_name: OTP-Logstash
build:
context: ./logstash
args:
- ELK_VERSION=${ELK_VERSION}
volumes:
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
expose:
- 5044/tcp
ports:
- "9600:9600"
- "5044:5044"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
links:
- elasticsearch
depends_on:
- elasticsearch
kibana:
container_name: OTP-Kibana
build:
context: ./kibana
args:
- ELK_VERSION=${ELK_VERSION}
volumes:
- ./kibana/config/:/usr/share/kibana/config:ro
ports:
- "5601:5601"
networks:
- elk
links:
- elasticsearch
depends_on:
- elasticsearch
- logstash
- filebeat
networks:
elk:
driver: bridge
docker-containers.docker-compose.yml file
version: '2'
services:
# Nginx
nginx:
container_name: OTP-Nginx
restart: always
build:
context: ./nginx
args:
- comapanycode=${COMPANY_CODE}
- dbtype=${DB_TYPE}
- dbip=${DB_IP}
- dbname=${DB_NAME}
- dbuser=${DB_USER}
- dbpassword=${DB_PASSWORD}
- webdirectory=${WEB_DIRECTORY}
ports:
- "80:80"
links:
- db:db
volumes:
- ./log/nginx:/var/log/nginx
depends_on:
- db
# Postgres
db:
container_name: OTP-Postgres
restart: always
ports:
- "5430:5430"
build:
context: ./postgres
args:
- food_db_version=${FOOD_DB_VERSION}
- dbtype=${DB_TYPE}
- retail_db_version=${RETAIL_DB_VERSION}
- dbname=${DB_NAME}
- dbuser=${DB_USER}
- dbpassword=${DB_PASSWORD}
volumes:
- .data/db:/octopus_docker/postgresql/data
# Tomcat
tomcat:
container_name: OTP-Tomcat
restart: always
build:
context: ./tomcat
args:
- dbuser=${DB_USER}
- dbpassword=${DB_PASSWORD}
links:
- db:db
volumes:
- ./tomcat/${WARNAME}.war:/usr/local/tomcat/webapps/${WARNAME}.war
ports:
- "8080:8080"
depends_on:
- db
- nginx
Additional files:
filebeat.yml (configuration file inside Logstash/Filbeat/config/)
filebeat.inputs:
- type: log
enabled: true
paths:
- /usr/local/tomcat/logs/.*log
output.logstash:
hosts: ["logstash:5044"]
Additional Info:
System I am using is Ubuntu 18.04
My goal is to collect tomcat logs from running tomcat container and forward them to Logstash and filter logs and forward that logs to Elasticsearch and finally to Kibana for Visualization purpose.
For now I can collect local machine(host) logs and visualize them in Kibana.(/var/log/)
My Problem:
I need to know proper way to get collected tomcat logs from tomcat container and forward them to logstash container via filebeat container.
Any discussion, answer or any help to understand a way to do this is highly expected.
Thanks.
So loooong... Create shared volume among all containers and setup your tomcat to save log files into that folder. If you can put all services into one docker-compose.yml, just setup volume internally:
docker-compose.yml
version: '3'
services:
one:
...
volumes:
- logs:/var/log/shared
two:
...
volumes:
- logs:/var/log/shared
volumes:
logs:
If you need several docker-compose.yml files, create volume globally in advance with docker volume create logs and map it into both compose files:
version: '3'
services:
one:
...
volumes:
- logs:/var/log/shared
two:
...
volumes:
- logs:/var/log/shared
volumes:
logs:
external: true
I want use Docker run my project(react+nodejs+mongodb),
Dockerfile:
FROM node:8.9-alpine
ENV NODE_ENV production
WORKDIR /usr/src/app
COPY ["package.json", "package-lock.json*", "npm-shrinkwrap.json*", "./"]
RUN npm install --production --silent && mv node_modules ../
COPY . .
CMD nohup sh -c 'npm start && node ./server/server.js'
docker-compose.yml:
version: '2.1'
services:
chat:
image: chat
container_name: chat
build: .
environment:
NODE_ENV: production
ports:
- "3000:3000"
- "8080:8080"
volumes:
- ./:/usr/src/app
links:
- mongo
mongo:
container_name: mongo
image: mongo
ports:
- "27017:27017"
run docker-compose up --build, the 3000 port is worked, but the 8080 port dies
localhost:3000
localhost:8080
I would suggest create a container for the server and have it seperate from the "chat" container. Its best to have each container do one thing and one thing only (almost like the philosophy behind unix commands)
In any case here is some modifications that I would make to the compose file.
version: '2.1'
services:
chat:
image: chat
container_name: chat
build: .
environment:
NODE_ENV: production
ports:
- "3000:3000"
- "8080:8080"
volumes:
- ./:/usr/src/app
links:
- mongo
mongo:
container_name: mongo
image: mongo
# You don't need to expose this port to the outside world. Because you linked the two containers the chat app
# will be able to connect to mongodb using hostname mongodb inside the container network.
# ports:
# - "27017:27017"
Btw what happens if you run:
$ docker-compose down
and then
$ docker-compose up
$ docker ps
can you see the ports exposed in docker ps output?
your chat service depends on mongo so you also need to have this in your chat
depends_on:
- mongo
This docker-compose file works for me. Note that i am saving the data from the database to a local directory. You should add this directory to gitignore.
version: "3.2"
services:
mongo:
container_name: mongo
image: mongo:latest
environment:
- MONGO_INITDB_ROOT_USERNAME=root
- MONGO_INITDB_ROOT_PASSWORD=password
- NODE_ENV=production
ports:
- "28017:27017"
expose:
- 28017 # you can connect to this mongodb with studio3t
volumes:
- ./mongodb-data:/data/db
restart: always
networks:
- docker-network
express:
container_name: express
environment:
- NODE_ENV=development
restart: always
build:
context: .
args:
buildno: 1
expose:
- 3000
ports:
- "3000:3000"
links:
- mongo # link this service to the database service
depends_on:
- mongo
command: "npm start" # override the default command to use nodemon in dev
networks:
- docker-network
networks:
docker-network:
driver: bridge
You may also find that using node you have to wait for the mongodb container to be ready before you can connect to the database.
I am trying to compose a stack using secrets
for development, i use local files in docker/secrets/FILE_NAME
I had this working in windows 10, but I'm struggling to get it to work under win7 toolbox.
I get an error:
Cannot create container for service db:
invalid volume specification:
'C:\project\docker\secrets\DB_USERNAME:/run/secrets/db-username:ro'
I was trying to set COMPOSE_CONVERT_WINDOWS_PATH but unfortunately, this does not change anything. I will get the same output with true or false.
Setting absolute paths did not help either.
Docker compose version 1.16.1 build 6d1ac219
Docker version 17.10.0-ce, build f4ffd25
My docker-compose.yml
version: '3.3'
services:
db:
image: postgres
volumes:
- db-data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/db-password
- POSTGRES_USER_FILE=/run/secrets/db-username
- POSTGRES_DB_FILE=/run/secrets/db-name
secrets:
- db-username
- db-password
- db-name
web:
build:
context: ./docker/machines/django/
args:
buildno: 1
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- ./Server:/Server
ports:
- "8000:8000"
depends_on:
- db
secrets:
- db-username
- db-password
- db-name
secrets:
db-username:
file: ./docker/secrets/DB_USERNAME
db-password:
file: ./docker/secrets/DB_PASSWORD
db-name:
file: ./docker/secrets/DB_NAME
volumes:
db-data:
driver: "local"