So I'm trying to setup some docker-compose scripts using some of the latest compose 3.5 features. The big one is being able to name networks. So there are some containers that I end up reproducing over and over again, such as a Postgres database, with various apps I work on. My goal is to be able to have a docker-compose script for something like Postgres, with a named network, which I can then have other apps with their own docker-compose scripts just add on to. It feels more re-usable that way.
Anyway, here is the docker-compose script I'm working with:
version: '3.5'
services:
postgres:
image: postgres:11
container_name: postgres
networks:
- pg_network
environment:
- POSTGRES_PASSWORD=password
- POSTGRES_USER=user
ports:
- 5432:5432
volumes:
- ~/.postgresql/data:/var/lib/postgresql/data
- ~/.postgresql/init:/docker-entrypoint-initdb.d
pgadmin:
image: dpage/pgadmin4
container_name: pgadmin
networks:
- pg_network
ports:
- 5433:80
environment:
- PGADMIN_DEFAULT_EMAIL=user#gmail.com
- PGADMIN_DEFAULT_PASSWORD=password
volumes:
- ~/.postgresql/pgadmin:/var/lib/pgadmin
networks:
pg_network:
name: pg_network
I am trying to use the "custom name" feature new in Docker Compose 3.5. See this link for the documentation: https://docs.docker.com/compose/networking/
However, when I try to run docker-compose up -d, I get the following error:
ERROR: The Compose file './docker-compose.yml' is invalid because:
networks.pg_network value Additional properties are not allowed ('name' was unexpected)
I'm not sure what I'm doing wrong. I'm on Ubuntu 18.04, I'm running the docker commands with sudo (because that seems to be a requirement on Ubuntu).
Here are my Docker and Docker Compose versions:
Docker version 18.09.2, build 6247962
docker-compose version 1.17.1, build unknown
I've tried upgrading them with APT, but it says they are up to date.
So, either I'm making a mistake in my compose script, or there's something screwy with my Docker version. I would appreciate any insight that can be offered. Thank you.
Related
Hi guys and excuse me for my English. I'm using docker swarm, when I attempt to deploy docker application with this command
docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml chatappapi
it shows the next error : services.chat-app-api Additional property pull_policy is not allowed
why this happens?
how do I solve this?
docker-compose.yml
version: "3.9"
services:
nginx:
image: nginx:stable-alpine
ports:
- "5000:80"
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
chat-app-api:
build: .
image: username/myapp
pull_policy: always
volumes:
- ./:/app
- /app/node_modules
environment:
- PORT= 5000
- MAIL_USERNAME=${MAIL_USERNAME}
- MAIL_PASSWORD=${MAIL_PASSWORD}
- CLIENT_ID=${CLIENT_ID}
- CLIENT_SECRET=${CLIENT_SECRET}
- REDIRECT_URI=${REDIRECT_URI}
- REFRESH_TOKEN=${REFRESH_TOKEN}
depends_on:
- mongo-db
mongo-db:
image: mongo
environment:
MONGO_INITDB_ROOT_USERNAME: 'username'
MONGO_INITDB_ROOT_PASSWORD: 'password'
ports:
- "27017:27017"
volumes:
- mongo-db:/data/db
volumes:
mongo-db:
docker-compose.prod.yml
version: "3.9"
services:
nginx:
ports:
- "80:80"
chat-app-api:
deploy:
mode: replicated
replicas: 8
restart_policy:
condition: any
update_config:
parallelism: 2
delay: 15s
build:
context: .
args:
NODE_ENV: production
environment:
- NODE_ENV=production
- MONGO_USER=${MONGO_USER}
- MONGO_PASSWORD=${MONGO_PASSWORD}
- MONGO_IP=${MONGO_IP}
command: node index.js
mongo-db:
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_USER}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD}
Information
docker-compose version 1.29.2
Docker version 20.10.8
Ubuntu 20.04.2 LTS
Thanks in advance.
Your problem line is in docker-compose.yml
chat-app-api:
build: .
image: username/myapp
pull_policy: always # <== this is the bad line, delete it
The docker compose file reference doesn't have any pull_policy in the api because
If the image does not exist, Compose attempts to pull it, unless you have also specified build, in which case it builds it using the specified options and tags it with the specified tag.
I think pull_policy used to be a thing for compose? Maybe keep the latest api documentation open to refer to/search through whilst you're developing (things can and do change fairly frequently with compose).
If you want to ensure that the most recent version of an image is pulled onto all servers in a swarm then run docker compose -f ./docker-compose.yml pull on each server in turn (docker stack doesn't have functionality to run this over an entire swarm yet).
As an aside: I wouldn't combine two .yml files with a single docker stack command without a very good reason to do so.
You are mixing docker-compose and docker swarm ideas up in the same files:
It is probably worth breaking your project up into 3 files:
docker-compose.yml
This would contain just the basic service definitions common to both compose and swarm.
docker-compose.override.yml
Conveniently, docker-compose and docker compose both should read this file automatically. This file should contain any "port:", "depends_on:", "build:" directives, and any convenience volumes use for development.
stack.production.yml
The override file to be used in stack deployments should contain everything understood by swarm and not compose, and b. everything required for production.
Here you would use configs: or even secrets: rather than volume mappings to local folders to inject content into containers. Rather than relying on ports: directives, you would install an ingress router on the swarm such as traefik. and so on.
With this arrangement, docker compose can be used to develop and build your compose stack locally, and docker stack deploy won't have to be exposed to compose syntax it doesn't understand.
pull_policy is in the latest version of docker-compose.
To upgrade your docker-compose refer to:
https://docs.docker.com/compose/install/
The spec for more info:
https://github.com/compose-spec/compose-spec/blob/master/spec.md#pull_policy
I'm new to Docker Compose, but have used Docker for years. The screen shot below is of PowerShell and of GitBash. If I run containers without docker-compose I can docker exec -it <container_ref> /bin/bash with no problems from either of these shells.
However, when running using docker-compose up both shells give no error when attempting to use docker-compose exec. They both just hang a few seconds and return to prompt.
Lastly, for some reason I do get an error in GitBash when using what I know: docker exec.... I've used this for years so I'm perplexed and posting a question. What does Docker Compose do that messes with GitBash docker ability, but not with PowerShell? And, why the hang when using docker-compose exec..., but no error?
I am using tty: true in the docker-compose.yml, but that honestly doesn't seem to make a difference. Not to throw a bunch of questions in one post, but whatever is going on could it also be the reason I can't hit my web server in the browser only when using Docker Compose to run it?
version: '3.8'
volumes:
pgdata:
external: true
services:
db:
image: postgres
container_name: trac-db
tty: true
restart: 'unless-stopped'
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: iol
volumes:
- pgdata:/var/lib/postgresql/data
network_mode: 'host'
expose:
- 5432
web:
image: lindben/trac-server
container_name: trac-server
tty: true
restart: 'unless-stopped'
environment:
ADDRESS: localhost
PORT: 3000
NODE_ENV: development
depends_on:
- db
network_mode: 'host'
privileged: true
expose:
- 1234
- 3000
```
I'm gonna be assuming you're using Docker for Desktop and so the reason you can docker exec just fine using powershell is because for windows docker is a native program\command and for GitBash which is based on bash a linux shell (bash = Bourne-Again SHell) not so much.
so when using a windows command that needs a tty you need some sort of "adapter" like winpty for example to bridge the gap between docker's interface and GitBash's one.
Here's a more detailed explanation on winpty
putting all of this aside, if trying to only use the compose options it maybe better for you to advise this question
Now, regarding your web service issue, I think that you're not actually publicly exposing your application using the expose tag. take a look at the docker-compose
expose reference. what you need is to add a "ports" tag like so as referenced here:
db:
ports:
- "5432:5432"
web:
ports:
- "1234:1234"
- "3000:3000"
Hope this solves your pickle ;)
How do I find the right compose file version for my docker-compose.yml file?
I have this:
version: '3.7'
services:
ghost:
container_name: ghost
image: ghost:latest
restart: always
ports:
- 127.0.0.1:2368:2368
volumes:
-v /var/www/myBlog/:/var/lib/ghost
and I have installed Docker Engine - Community version: 19.03.8
How can I know it?
Thanks
Are you sure about the -v under your volumes?
Please check the #volume-configuration-reference for docker-compose files.
You need to mention it like below.
volumes:
- /var/www/myBlog/:/var/lib/ghost
You need to change the volume to below
volumes:
- /var/www/myBlog/:/var/lib/ghost
There is a compatibility matrix between Docker Engine and docker-compose
https://docs.docker.com/compose/compose-file/compose-versioning/
Below is the changelog of docker-compose and docker which can help you understand various features available in which version.
Docker-compose:- https://github.com/docker/compose/blob/master/CHANGELOG.md
Docker-ce:- https://github.com/docker/docker-ce/releases
I have the following docker-compose.yml file:
...
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
container_name: elasticsearch-cust-comp
...
I've previously run it with another elasticsearch version:
...
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.4
container_name: elasticsearch-cust-comp
...
Since changing the version to 5.4.3, I'm getting this error in docker logs for the Elasticsearch container:
Unknown license version found, please upgrade all nodes to the latest elasticsearch-license plugin
My guess is that version 6.5.4 of Elasticsearch is still running somewhere and that it creates issues when I'm trying to run 5.4.3. But as far as I know, I've shut down all Elasticsearch containers currently running with docker-compose down, and docker ps shows no processes after this. Still, when I run docker-compose up -d with version 5.4.3, it gives me this error. Running 6.5.4 works fine. What do I need to do to be able to run version 5.4.3?
EDIT:
This is the whole part regarding elasticsearch in the docker-compose.yml. As you can see, xpack is already disabled:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
container_name: elasticsearch-cust-comp
ports:
- 9200:9200
- 9300:9300
volumes:
- cust-comp-elastic:/usr/share/elasticsearch/data
- ./cust/externalConfig/elasticsearch/config/hunspell/:/usr/share/elasticsearch/config/hunspell/
# - ./config/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
# - ./fwhome/elasticsearch/cust-comp:/usr/share/elasticsearch/config/cust-comp
networks:
- cust-comp
environment:
- cluster.name=i3-elasticsearch
- xpack.security.enabled=false
- xpack.monitoring.enabled=false
- xpack.ml.enabled=false
- xpack.graph.enabled=false
- xpack.watcher.enabled=false
restart: unless-stopped
In doubt, try:
docker system prune
This will clean unused containers/images.
I'm new to docker and trying to understand what docker stack does. Currently trying out this container https://hub.docker.com/r/instapy/instapy
this is the docker-compose file
services:
web:
image: instapy/instapy:latest
container_name: "${COMPOSE_PROJECT_NAME}_web"
env_file: .env
environment:
- PYTHONUNBUFFERED=0
- INSTAPY_WORKSPACE=/code/InstaPy
volumes:
- ./:/code
The errors I'm getting seem to indicate quite a few issues
Ignoring deprecated options:
container_name: Setting the container name is not supported.
service "web": container_name is deprecated
service "web": env_file are ignored
Stack.compose.docker.com "test" is invalid: test: Invalid value: "null": conversion to kube entities failed: C:\Users\roole\instapy-docker\docker-compose: only absolute paths can be specified in mount source
docker compose version info
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.0.2q 20 Nov 2018
Content asked for from ' docker-compose config'
services:
web:
container_name: instapy_web
environment:
COMPOSE_PROJECT_NAME: instapy
INSTAPY_WORKSPACE: /code/InstaPy
PYTHONUNBUFFERED: '0'
image: instapy/instapy:latest
volumes:
- C:\Users\roole\instapy-docker\docker-compose:/code:rw
version: '3.0'
Any help in understanding what the hell I'm supposed to be doing would be mega.
At the beginning of each docker-compose.yml file you need to specify the version. Each version of docker-compose supports certain versions of the yml file specification.
This should work for you:
version: "3.3"
services:
web:
image: instapy/instapy:latest
container_name: "${COMPOSE_PROJECT_NAME}_web"
env_file: .env
environment:
- PYTHONUNBUFFERED=0
- INSTAPY_WORKSPACE=/code/InstaPy
volumes:
- ./:/code
When deploying a stack the container name is not relevant (in fact after version "3" is not supported). The reason for that is that docker needs to be able to change the container name in case you scale your service (multiple versions of the same container might end up running on the same docker instance and then they need to have different container names).
Also when you specify a volume you need to specify full, absolute paths. You can simply replace your volume declaration with what you got from running docker-compose config (C:\Users\roole\instapy-docker\docker-compose:/code:rw) or you can use $PWD or the equivalent for your OS to refer to your current directory