Upgrade docker-compose to version 3 - docker

This is how my docker-compose.yml file looks like. As you can see, there is a nginx server, a mongoDB, the main application and for testing a nightwatch and selenium container.
nginx:
container_name: 'nginx'
image: 'nginx:1.11'
restart: 'always'
ports:
- '80:80'
- '443:443'
volumes:
- '/opt/nginx/conf.d:/etc/nginx/conf.d:ro'
- '/opt/nginx/conf/nginx.conf:/etc/nginx/nginx.conf:ro'
- '/etc/letsencrypt:/etc/letsencrypt'
- '/opt/nginx/www:/var/www:ro'
links:
- 'app'
nightwatch:
container_name: nightwatch
image: 'registry.example.com/project/core:nightwatch'
links:
- selenium
stdin_open: true
tty: true
selenium:
container_name: selenium
image: selenium/standalone-chrome
ports:
- 4444:4444
links:
- app
db:
container_name: db
image: 'mongo:3.4'
restart: 'always'
volumes:
- '/opt/mongo/project/prod:/data/db'
app:
container_name: app
image: 'registry.example.de/project/core:latest'
restart: always
links:
- 'db'
environment:
- ROOT_URL=https://example.com
- MONGO_URL=mongodb://db/db
You can also see, that I'm still using version 1 and I want to upgrade to current version (3). And there are a few problems for converting the file.
For example I do not understand the network option which should be used instead of the deprecated link to get access to containers.
In the nightwatch container I'm running a script like
module.exports = {
'start application': function(browser) {
browser
.url('http://app') // <--
.waitForElementVisible('body', 10000)
.getTitle(function(result) {
this.assert.equal(typeof result, 'string')
})
},
}
So I need to get access to the app container via nightwatch, which also needs selenium. The main app needs of course the db.
I need some help converting to version 3:
version: '3'
services:
nginx:
...
nightwatch:
...

This should be enough:
version: "3"
services:
nginx:
container_name: 'nginx'
image: 'nginx:1.11'
restart: 'always'
ports:
- '80:80'
- '443:443'
volumes:
- '/opt/nginx/conf.d:/etc/nginx/conf.d:ro'
- '/opt/nginx/conf/nginx.conf:/etc/nginx/nginx.conf:ro'
- '/etc/letsencrypt:/etc/letsencrypt'
- '/opt/nginx/www:/var/www:ro'
links:
- app
nightwatch:
container_name: nightwatch
image: 'registry.example.com/project/core:nightwatch'
links:
- selenium
stdin_open: true
tty: true
selenium:
container_name: selenium
image: selenium/standalone-chrome
ports:
- 4444:4444
links:
- app
db:
container_name: db
image: 'mongo:3.4'
restart: 'always'
volumes:
- '/opt/mongo/project/prod:/data/db'
app:
container_name: app
image: 'registry.example.de/project/core:latest'
restart: always
links:
- db
environment:
- ROOT_URL=https://example.com
- MONGO_URL=mongodb://db/db
You don't need to configure extra networks than the default provided by compose.
Linking containers is still configured as so. Try this:
version: "3"
services:
abc:
image: ubuntu
command: tail -f /dev/null
cde:
image: busybox
command: ping abc
links:
- abc

Related

Persist nifi data and volume

I want to make my nifi data volume and configuration persist means even if I delete container and docker compose up again I would like to keep what I built so far in my nifi. I try to mount volumes as follows in my docker compose file in volumes section nevertheless it doesn't work and my nifi processors are not saved. How can I do it correctly? Below my docker-compose.yaml file.
version: "3.7"
services:
nifi:
image: koroslak/nifi:latest
container_name: nifi
restart: always
environment:
- NIFI_HOME=/opt/nifi/nifi-current
- NIFI_LOG_DIR=/opt/nifi/nifi-current/logs
- NIFI_PID_DIR=/opt/nifi/nifi-current/run
- NIFI_BASE_DIR=/opt/nifi
- NIFI_WEB_HTTP_PORT=8080
ports:
- 9000:8080
depends_on:
- openldap
volumes:
- ./volume/nifi-current/state:/opt/nifi/nifi-current/state
- ./volume/database/database_repository:/opt/nifi/nifi-current/repositories/database_repository
- ./volume/flow_storage/flowfile_repository:/opt/nifi/nifi-current/repositories/flowfile_repository
- ./volume/nifi-current/content_repository:/opt/nifi/nifi-current/repositories/content_repository
- ./volume/nifi-current/provenance_repository:/opt/nifi/nifi-current/repositories/provenance_repository
- ./volume/log:/opt/nifi/nifi-current/logs
#- ./volume/conf:/opt/nifi/nifi-current/conf
postgres:
image: koroslak/postgres:latest
container_name: postgres
restart: always
environment:
- POSTGRES_PASSWORD=secret123
ports:
- 6000:5432
volumes:
- postgres:/var/lib/postgresql/data
pgadmin:
container_name: pgadmin
image: dpage/pgadmin4:4.18
restart: always
environment:
- PGADMIN_DEFAULT_EMAIL=admin
- PGADMIN_DEFAULT_PASSWORD=admin
ports:
- 8090:80
metabase:
container_name: metabase
image: metabase/metabase:v0.34.2
restart: always
environment:
MB_DB_TYPE: postgres
MB_DB_DBNAME: metabase
MB_DB_PORT: 5432
MB_DB_USER: metabase_admin
MB_DB_PASS: secret123
MB_DB_HOST: postgres
ports:
- 3000:3000
depends_on:
- postgres
openldap:
image: osixia/openldap:1.3.0
container_name: openldap
restart: always
ports:
- 38999:389
# Mocked source systems
jira-api:
image: danielgtaylor/apisprout:latest
container_name: jira-api
restart: always
ports:
- 8000:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/jira-api.json
pipedrive-api:
image: danielgtaylor/apisprout:latest
container_name: pipedrive-api
restart: always
ports:
- 8100:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/pipedrive-api.yaml
restcountries-api:
image: danielgtaylor/apisprout:latest
container_name: restcountries-api
restart: always
ports:
- 8200:8000
command: https://raw.githubusercontent.com/mvrabel/nifi-postgres-metabase/master/api_examples/restcountries-api.json
volumes:
postgres:
nifi:
openldap:
metabase:
pgadmin:
Using Registry you can achieve that all changes you are doing or your nifi are committed to git. I.e. if you change some processor configuration, it will be reflected in your git repo.
As for flow files, you may need to fix volumes mappings.

Invalid docker.compose.yaml file

I am trying to get a couple of containers up and running, however I am running into some issues. I run this command:
docker-compose up -d --build itvdflab
and get this error
The Compose file './docker-compose.yaml' is invalid because:
Unsupported config option for services: 'itvdelab'
Unsupported config option for networks: 'itvdelabnw'
Here is the yaml file.
services:
itvdelab:
image: itversity/itvdelab
hostname: itvdelab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "cluster_util_db"
cluster_util_db:
image: postgres:13
ports:
- "6432:5432"
volumes:
- ./cluster_util_db_scripts:/docker-entrypoint-initdb.d
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
itvdflab:
build:
context: .
dockerfile: images/pythonsql/Dockerfile
hostname: itvdflab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "pg.itversity.com"
pg.itversity.com:
image: postgres:13
ports:
- "5432:5432"
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
networks:
itvdelabnw:
name: itvdelabnw
What changes do I need to make to get this working?
Your docker-compose.yml file is missing a version: line. Until very recently, this caused Docker Compose to interpret this as the original "version 1" Compose format, which doesn't have a top-level services: key and doesn't support Docker networks. The much newer Compose Specification claims that a version: key is optional, but in practice if you can't be guaranteed to use a very new version of Compose (built as a plugin to the docker binary) it's required. The most recent Compose file versions supported by the standalone Python docker-compose tool are 3.8 and 2.4 (you need the 2.x version for some resource-related constraints in non-Swarm installations).
# Add at the very beginning
version: '3.8'
Here is the revised copy:
version: '3.4'
services:
itvdelab:
image: itversity/itvdelab
hostname: itvdelab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "cluster_util_db"
cluster_util_db:
image: postgres:13
ports:
- "6432:5432"
volumes:
- ./cluster_util_db_scripts:/docker-entrypoint-initdb.d
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
itvdflab:
build:
context: .
dockerfile: images/pythonsql/Dockerfile
hostname: itvdflab
ports:
- "8888:8888"
volumes:
- "./itversity-material:/home/itversity/itversity-material"
- "./data:/data"
environment:
SHELL: /bin/bash
networks:
- itvdelabnw
depends_on:
- "pg.itversity.com"
pg.itversity.com:
image: postgres:13
ports:
- "5432:5432"
networks:
- itvdelabnw
environment:
POSTGRES_PASSWORD: itversity
networks:
itvdelabnw:
name: itvdelabnw
and now I get the following error
ERROR: The Compose file './docker-compose.yaml' is invalid because:
services.pg.itversity.com.networks.itvdelabnw contains unsupported option: 'name'
for me work try different version. In my case work
version: '2.2'

Request to MX server from a docker container

Context
Our solution send emailing campaign, and some email provider makes temporary blacklist because we used inexistant email addresses (they may be created/imported by any user)
Solution
I try to implement a email checker based on https://www.codexworld.com/verify-email-address-check-if-real-exists-domain-php/
Problem
From my local computer, this script works (when my IP is not temporary blacklisted) but from my Docker Container stream_socket_client or even telnet always times out.
As I see, MX servers always time out non verified requester but how can I make it work from my docker container?
I've no specific docker configuration for the port 25.
Thank you
docker-compose.yml
# Base docker-compose file to run required services: Adminer, MySQL, and Redis
version: '3.7'
services:
adminer:
image: adminer:latest
ports:
- "8080:8080"
database:
image: mysql:8.0
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
cap_add: [ SYS_NICE ] # https://github.com/docker-library/mysql/issues/303
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_general_ci', '--lower_case_table_names=1']
redis:
image: redis:alpine
expose:
- "6379"
redisinsight:
image: redislabs/redisinsight
ports:
- "8081:8001"
reverse-proxy:
image: nginx:alpine
depends_on:
- backend
- frontend
volumes:
- ./../../../backend/infra/etc/nginx/dev.conf:/etc/nginx/nginx.conf
ports:
- "80:80"
backend:
build:
context: ./../..
dockerfile: container/dev/backend.dockerfile
expose:
- "80"
volumes:
- {...}
depends_on:
- database
frontend:
build:
context: ./../..
dockerfile: container/dev/frontend.dockerfile
expose:
- "3000"
volumes:
- {...}
tty: true # required to keep yarn running (https://stackoverflow.com/a/61050994)
volumes:
# Contains the database's data
db_data: {}
# Base docker-compose file to run required services: Adminer, MySQL, and Redis
version: '3.7'
services:
adminer:
image: adminer:latest
ports:
- "8080:8080"
database:
image: mysql:8.0
volumes:
- db_data:/var/lib/mysql
ports:
- "3306:3306"
environment:
MYSQL_ROOT_PASSWORD: root
cap_add: [ SYS_NICE ] # https://github.com/docker-library/mysql/issues/303
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_general_ci', '--lower_case_table_names=1']
redis:
image: redis:alpine
expose:
- "6379"
redisinsight:
image: redislabs/redisinsight
ports:
- "8081:8001"
reverse-proxy:
image: nginx:alpine
depends_on:
- backend
- frontend
volumes:
- ./../../../backend/infra/etc/nginx/dev.conf:/etc/nginx/nginx.conf
ports:
- "80:80"
backend:
build:
context: ./../..
dockerfile: container/dev/backend.dockerfile
expose:
- "80"
volumes:
- {...}
depends_on:
- database
frontend:
build:
context: ./../..
dockerfile: container/dev/frontend.dockerfile
expose:
- "3000"
volumes:
- {...}
tty: true # required to keep yarn running (https://stackoverflow.com/a/61050994)
volumes:
# Contains the database's data
db_data: {}
backend.dockerfile is an ubuntu with git, mysql-client, php

"Unable to find template" with Docker

I created a Symfony environment with Docker. I then included this file in my web project (skeleton website). But when I try to access my base.html.twig page located in a main folder from the controller, I get this error:
Unable to find template "main/base.html.twig" (looked into: /var/www/templates, /var/www/vendor/symfony/twig-bridge/Resources/views/Form).
How can I solve the problem? I have version 5 of Symfony.
Here is the content of my docker-compose file:
version: '3'
services:
php:
container_name: "php-fpm"
build:
context: ./php
environment:
- APP_ENV=${APP_ENV}
- APP_SECRET=${APP_SECRET}
volumes:
- ${APP_FOLDER}:/var/www
networks:
- dev
nginx:
container_name: "nginx"
build:
context: ./nginx
volumes:
- ${APP_FOLDER}:/var/www
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
- ./logs:/var/log/nginx/
depends_on:
- php
ports:
- "80:80"
networks:
- dev
db:
image: mysql
container_name: "db"
restart: always
volumes:
- db-data:/var/lib/mysql
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
networks:
- dev
phpmyadmin:
image: phpmyadmin
container_name: "phpmyadmin"
restart: always
depends_on:
- db
ports:
- 8080:80
environment:
PMA_HOST: db
networks:
- dev
networks:
dev:
volumes:
db-data:
Your Docker Compose file needs to link local folders with Docker folders.
#docker-compose example
version: "3"
services:
web:
build: .
ports:
- "8080:80"
volumes:
- .:/var/www # Map local folder to Docker

ERROR: In file './docker-compose.yml', service name True must be a quoted string, i.e. 'True'

My docker-compose.yml looks like the below. When i run docker-compose up I get the below error.
ERROR: In file './docker-compose.yml', the service name True must be a quoted string, i.e. 'True'.
version: '3'
services:
db:
restart: always
image: postgres:9.6-alpine
container_name: pleroma_postgres
networks:
- pleroma
volumes:
- ./postgres:/var/lib/postgresql/data
web:
build: .
image: pleroma
container_name: pleroma_web
restart: always
environment:
- VIRTUAL_HOST=<myplaceholderhost>
- VIRTUAL_PORT=4000
- LETSENCRYPT_HOST=<myplaceholderhost>
- LETENCRYPT_EMAIL=<myplaceholderemail>
expose:
- "4000"
volumes:
- ./uploads:/pleroma/uploads
depends_on:
- db
nginx:
image: jwilder/nginx-proxy
container_name: nginx
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /apps/docker-articles/nginx/vhost.d:/etc/nginx/vhost.d
- /apps/docker-articles/nginx/certs:/etc/nginx/certs:ro
- /apps/docker-articles/nginx/html:/usr/share/nginx/html
restart: always
ports:
- "80:80"
- "443:443"
labels:
com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
networks:
- pleroma
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion:v1.5
container_name: letsencrypt
volumes_from:
- nginx
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /apps/docker-articles/nginx/vhost.d:/etc/nginx/vhost.d
- /apps/docker/articles/nginx/certs:/etc/nginx/certs:rw
- /apps/docker-articles/nginx/html:/usr/share/nginx/html
networks:
pleroma:
My docker version is
Docker version 18.06.1-ce, build e68fc7a
My docker compose version is
docker-compose version 1.23.1, build b02f1306
Running CoreOS version 1911.3.0
I ended up resolving this issue by modifying the nginx and letsencrypt portions of my docker-compose.yml file to be as follows.
nginx:
image: jwilder/nginx-proxy
container_name: nginx
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- /apps/docker-articles/nginx/vhost.d:/etc/nginx/vhost.d
- /apps/docker-articles/nginx/certs:/etc/nginx/certs:ro
- /apps/docker-articles/nginx/html:/usr/share/nginx/html
restart: always
ports:
- "80:80"
- "443:443"
labels:
- "NGINX_PROXY_CONTAINER=true"
networks:
- pleroma
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion:v1.5
container_name: letsencrypt
environment:
- NGINX_PROXY_CONTAINER=true
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /apps/docker-articles/nginx/vhost.d:/etc/nginx/vhost.d
- /apps/docker/articles/nginx/certs:/etc/nginx/certs:rw
- /apps/docker-articles/nginx/html:/usr/share/nginx/html
It seems "volumes_from" is deprecated in docker-compose v3. As well as I had forgotted quotes around my label and needed to set my environment within letsencrypt.
in CentOS env your .yml file directory must be /usr/local/bin

Resources