Use of docker:dind in docker-compose - docker

So for some reason, I'd like to use a docker:dind inside a docker-compose.yml.
I know that the "easy" way is to mount directly the socket inside the image (like that : /var/run/docker.sock:/var/run/docker.sock) but I want to avoid that (for security reasons).
Here is my experimental docker-compose.yml :
version: '3.8'
services:
dind:
image: docker:19.03.7-dind
container_name: dind
restart: unless-stopped
privileged: true
environment:
- DOCKER_TLS_CERTDIR=/certs
volumes:
- dind-certs-ca:/certs/ca
- dind-certs-client:/certs/client
networks:
- net
expose:
- 2375
- 5000
volumes:
dind-certs-ca:
dind-certs-client:
networks:
net:
driver: bridge
Nothing complexe here, then I try to see if the service is correctly set :
docker logs dind
Here no problem it is up and running.
However, once I try to use it with for instance :
docker run --rm -it --network net --link dind:docker docker version
I got the following error :
Cannot connect to the Docker deamon at tcp://docker:2375. Is there a deamon running ?
Do you have any idea why the deamon is not responding ?
---------------------------------------------------------- EDIT ----------------------------------------------------------
Following hariK's comment (thanks by the way) I add the port 2376 to the exposed one. I think I'm neer solving my issue. Here is the error that I get :
error during connect: Get http://docker:2375/v1.40/version dial tcp: lookup on docker on [ip]: no such host
So I look at this error and found that it seems to be a recurrent one on dind versions (there is a lot of issues on gitlab on it like this one). There is also a post on stackoverflow on a similar issue for gitlab here.
For the workaround I tried :
Putting this value DOCKER_TLS_CERTDIR: "" hopping to turn off TLS ... but it failed
Downgrading the version to docker:18.05-dind. It actualy worked but I don't think it's a good move to make.
If someone has an idea to keep TLS ON and make it works it would be great :) (I'll still be looking on my own but if you can give a nudge with interesting links it would be cool ^^)

To use Docker with disabled TLS (i.e. TCP port 2375 by default), unset the DOCKER_TLS_CERTDIR variable in your dind service definition in Docker Compose, like:
dind:
image: docker:dind
container_name: dind
privileged: true
expose:
- 2375
environment:
- DOCKER_TLS_CERTDIR=
(NB: do not initialize it to any value like '' or "")

So I found a solution, and I added to the basic docker-compose a resgistry with TLS options.
So I had fisrt to generate the certs and then correctly mount them.
If any of you run in a similar issue I made a github repo with the docker-compose and command lines for the certs.

Some time later, and I was looking for the same thing.
Here is an example that with specific versions for the images, that should still work in a few years from now:
version: '3'
services:
docker:
image: docker:20.10.17-dind-alpine3.16
privileged: yes
volumes:
- certs:/certs
docker-client:
image: docker:20.10.17-cli
command: sh -c 'while [ 1 ]; do sleep 1; done'
environment:
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: /certs/client
volumes:
- certs:/certs
volumes:
certs:
The TLS certificates are generated by the "docker" service on startup and shared using a volume.
Use the client as follows:
docker-compose exec docker-client sh
#now within docker-client container
docker run hello-world

Related

Cannot exec into container using GitBash when using Docker Compose

I'm new to Docker Compose, but have used Docker for years. The screen shot below is of PowerShell and of GitBash. If I run containers without docker-compose I can docker exec -it <container_ref> /bin/bash with no problems from either of these shells.
However, when running using docker-compose up both shells give no error when attempting to use docker-compose exec. They both just hang a few seconds and return to prompt.
Lastly, for some reason I do get an error in GitBash when using what I know: docker exec.... I've used this for years so I'm perplexed and posting a question. What does Docker Compose do that messes with GitBash docker ability, but not with PowerShell? And, why the hang when using docker-compose exec..., but no error?
I am using tty: true in the docker-compose.yml, but that honestly doesn't seem to make a difference. Not to throw a bunch of questions in one post, but whatever is going on could it also be the reason I can't hit my web server in the browser only when using Docker Compose to run it?
version: '3.8'
volumes:
pgdata:
external: true
services:
db:
image: postgres
container_name: trac-db
tty: true
restart: 'unless-stopped'
environment:
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
POSTGRES_DB: iol
volumes:
- pgdata:/var/lib/postgresql/data
network_mode: 'host'
expose:
- 5432
web:
image: lindben/trac-server
container_name: trac-server
tty: true
restart: 'unless-stopped'
environment:
ADDRESS: localhost
PORT: 3000
NODE_ENV: development
depends_on:
- db
network_mode: 'host'
privileged: true
expose:
- 1234
- 3000
```
I'm gonna be assuming you're using Docker for Desktop and so the reason you can docker exec just fine using powershell is because for windows docker is a native program\command and for GitBash which is based on bash a linux shell (bash = Bourne-Again SHell) not so much.
so when using a windows command that needs a tty you need some sort of "adapter" like winpty for example to bridge the gap between docker's interface and GitBash's one.
Here's a more detailed explanation on winpty
putting all of this aside, if trying to only use the compose options it maybe better for you to advise this question
Now, regarding your web service issue, I think that you're not actually publicly exposing your application using the expose tag. take a look at the docker-compose
expose reference. what you need is to add a "ports" tag like so as referenced here:
db:
ports:
- "5432:5432"
web:
ports:
- "1234:1234"
- "3000:3000"
Hope this solves your pickle ;)

Portainer Stack - docker compose issue with MacVLan network

I am starting to use portrainer.io to manage my docker images, instead of Synology DSM Docker GUI.
Background information:
I've used MacVLAN to create an own IP address for my Pihole Docker, overall everything regarding this piHole is running fine with this settings, made by DSM GUI.
environment network volumesports
Problem:
I now would like to use portrainer.io to manage my Docker installation. Including the Stack option which should be docker compose.
I am now struggeling to get my PiHole Image up with this Docker script:
services:
pihole:
container_name: pihole
image: pihole/pihole:latest
networks: docker
ports:
- "53:53/tcp"
- "53:53/udp"
- "67:67/udp"
- "80:80/tcp"
environment:
TZ: 'Europe/Berlin'
WEBPASSWORD: 'password'
ServerIP: "0.0.0.0"
# Volumes store your data between container upgrades
volumes:
- '/pihole/pihole/:/etc/pihole/'
- '/pihole/dnsmasq/:/etc/dnsmasq.d/'
# Recommended but not required (DHCP needs NET_ADMIN)
# https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
cap_add:
- NET_ADMIN
restart: unless-stopped
Does anyone have an idea why I get "Unable to deploy stack" as error message?
You are telling the service to use a network called "docker", but the network is not defined in the compose file. Is this the complete docker-compose file?
If yes, then you are missing the networks section:
networks:
docker:
external: true

Docker inter-container communication

I'm facing a relatively simple problem here but I'm starting to wonder why it doesn't work.
I want to start two Docker Containers with Docker Compose: InfluxDB and Chronograph.
Unfortunately, the chronograph does not reach InfluxDB under the given hostname: "Unable to connect to InfluxDB Influx 1: Error contacting source"
What could be the reason for this?
Here is my docker-compose.yml:
version: "3.8"
services:
influxdb:
image: influxdb
restart: unless-stopped
ports:
- 8086:8086
volumes:
- influxdb-volume:/var/lib/influxdb
networks:
- test
chronograf:
image: chronograf
restart: unless-stopped
ports:
- 8888:8888
volumes:
- chronograf-volume:/var/lib/chronograf
depends_on:
- influxdb
networks:
- test
volumes:
influxdb-volume:
chronograf-volume:
networks:
test:
driver: bridge
I have also tried to start a shell inside the two containers and then ping the containers to each other or use wget to get the HTTP-API of the other container. Even this communication between the containers does not work. On both attempts with wget and ping I get timeouts.
It must be said that I use a Banana Pi BPI-M1 here. Is it possible that it is somehow due to the Linux that container to container communication does not work?
If not configured, chronograf will try to access influxdb on localhost:8086. To be able to reach the correct influxdb instance, you need to specify the url accordingly using either the --influxdb-url command line flag or (personal preference) an environment variable INFLUXDB_URL. Those should be set to the value of http://influxdb:8086 which is the docker DNS name derived from the service name of your compose file (the keys one level below services).
This should do the trick (snippet):
chronograf:
image: chronograf
restart: unless-stopped
ports:
- 8888:8888
volumes:
- chronograf-volume:/var/lib/chronograf
environment:
- INFLUXDB_URL=http://influxdb:8086
depends_on:
- influxdb
networks:
- test
Please check the chronograf readme (section Using the container with InfluxDB) for details on configuring the image and check the docker compose networking docs on some more info about networks and dns naming.
The Docker service creates some iptables entries in the tables filter and nat. My OpenVPN Gateway script executed the following commands at startup:
iptables --flush -t filter
iptables --flush -t nat
This will delete the entries from Docker and communication between the containers and the Internet will no longer be possible.
I have rewritten the script and now everything works again.

Unable to connect docker container to internet

I have created a custom network by using command:
docker network create --driver=bridge --subnet=192.168.1.0/16 --gateway=192.168.1.1 -o "com.docker.network.bridge.host_binding_ipv4"="0.0.0.0" -o "com.docker.network.bridge.enable_icc"="true" -o "com.docker.network.bridge.enable_ip_masquerade"="true" my-net
Post creation I have created docker containers using docker compose:
version: "3"
services:
web:
build: './UI'
ports:
- "80:80"
tty: true
links:
- service
networks:
my-net:
ipv4_address: 192.168.1.2
service:
build: './Service'
ports:
- "8090:8090"
tty: true
networks:
my-net:
ipv4_address: 192.168.1.3
networks:
my-net:
external:
name: my-net
Now when I am pinging the containers internally, I am getting the response. But when I am trying pinging google.com, container is unable to ping it.
I also ran this command, but no success.
sysctl net.ipv4.conf.all.forwarding=1
I had this problem a while ago, and a simple workaround is adding external DNS entries (Google's, in this case) to your service in docker-compose.yml:
services:
web:
...
dns:
- 8.8.8.8
- 4.4.4.4
This quickly solved my problem, at least for development/test environment.
I know probably there are more complex (and probably more secure) ways of doing it, but this really solved my problem. I believe creating a bridge network could solve this, but I haven't tested it.
There could also be a problem with firewoll configurations (even though maybe in this case pign would not work also). More about it can be found in this entry from Docker Forum (docker in windows): Docker DNS not responding from inside a container.
I hope this helps! :-)

How do I set hostname in docker-compose?

In my docker-compose.yml file, I have the following. However the container does not pick up the hostname value. Any ideas?
dns:
image: phensley/docker-dns
hostname: affy
domainname: affy.com
volumes:
- /var/run/docker.sock:/docker.sock
When I check the hostname in the container it does not pick up affy.
As of docker-compose version 3.0 and later, you can just use the hostname key:
version: "3.0"
services:
yourservicename:
hostname: your-name
I found that the hostname was not visible to other containers when using docker run. This turns out to be a known issue (perhaps more a known feature), with part of the discussion being:
We should probably add a warning to the docs about using hostname. I think it is rarely useful.
The correct way of assigning a hostname - in terms of container networking - is to define an alias like so:
services:
some-service:
networks:
some-network:
aliases:
- alias1
- alias2
Unfortunately this still doesn't work with docker run. The workaround is to assign the container a name:
docker-compose run --name alias1 some-service
And alias1 can then be pinged from the other containers.
UPDATE: As #grilix points out, you should use docker-compose run --use-aliases to make the defined aliases available.
This seems to work correctly. If I put your config into a file:
$ cat > compose.yml <<EOF
dns:
image: phensley/docker-dns
hostname: affy
domainname: affy.com
volumes:
- /var/run/docker.sock:/docker.sock
EOF
And then bring things up:
$ docker-compose -f compose.yml up
Creating tmp_dns_1...
Attaching to tmp_dns_1
dns_1 | 2015-04-28T17:47:45.423387 [dockerdns] table.add tmp_dns_1.docker -> 172.17.0.5
And then check the hostname inside the container, everything seems to be fine:
$ docker exec -it stack_dns_1 hostname
affy.affy.com
Based on docker documentation:
https://docs.docker.com/compose/compose-file/#/command
I simply put
hostname: <string>
in my docker-compose file.
E.g.:
[...]
lb01:
hostname: at-lb01
image: at-client-base:v1
[...]
and container lb01 picks up at-lb01 as hostname.
The simplest way I have found is to just set the container name in the docker-compose.yml See container_name documentation. It is applicable to docker-compose v1+. It works for container to container, not from the host machine to container.
services:
dns:
image: phensley/docker-dns
container_name: affy
Now you should be able to access affy from other containers using the container name. I had to do this for multiple redis servers in a development environment.
NOTE The solution works so long as you don't need to scale. Such as consistant individual developer environments.
I needed to spin freeipa container to have a working kdc and had to give it a hostname otherwise it wouldn't run.
What eventually did work for me is setting the HOSTNAME env variable in compose:
version: 2
services:
freeipa:
environment:
- HOSTNAME=ipa.example.test
Now its working:
docker exec -it freeipa_freeipa_1 hostname
ipa.example.test

Resources