How am I able to use multiple operating system images within Docker? - docker

I'm currently using Docker to deploy a development version of a web application. This is the docker-compose.yml file I wrote
version: '3'
services:
nginx:
build:
context: ./docker/nginx
ports:
- "80:80"
volumes:
- .:/var/www/html/acme.com
- ./docker/nginx/acme.com.conf:/etc/nginx/conf.d/acme.com.conf
networks:
- my_network
php:
build:
context: ./docker/php
ports:
- "9000:9000"
volumes:
- .:/var/www/html/acme.com
networks:
- my_network
database:
build:
context: ./docker/database
volumes:
- ./docker/database/acme.sql:/docker-entrypoint-initdb.d/acme.sql
- ./docker/database/remote_access.sql:/docker-entrypoint-initdb.d/remote_access.sql
- ./docker/database/custom.cnf:/etc/mysql/conf.d/custom.cnf
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: ${db_database}
MYSQL_ROOT_PASSWORD: ${db_password}
networks:
- my_network
networks:
my_network:
driver: bridge
Looking more closely at the different context files for nginx and mysql are as follows:
./docker/nginx/Dockerfile
FROM nginx:alpine
./docker/database/Dockerfile
FROM mariadb:latest
So it is evident that nginx image uses the alpine base image.
But what image is mariadb using? I went through the dockerhub website and followed the link to https://github.com/MariaDB/mariadb-docker/blob/db55d2702dfc0102364a29ab00334b6a02085ef9/10.7/Dockerfile
In this file, there is a reference to
FROM ubuntu:focal
Does this mean that my docker container is using the alpine linux base image as well as the ubuntu image? How does it work if I have multiple linux distributions in my container?
What should I do to fix this?
Should I rather install mariadb using a FROM command into alpine linux and build my own docker image?

Imagine your docker compose as a server farm. Each service (nginx, mariadb, ..) would be a physical server running an OS and its software. They are all connected via LAN within the same subnet. Each machine has its own IP address and there is a DNS and DHCP service running for giving the IPs and resolving names (service name = DNS-Alias). And there is a router blocking all connections from other subnets (= your host). Exceptions are configured by port mapping (=forwarding) ports: - 8000:8000.
So you can mix servers with all different OS variants of one type. This is due to the fact that docker is not a real virtual machine but more a VM light using the host OS resources to run the containers. So you can mix all kind of Linux distributions OR Windows versions. Every container uses the OS suiting it goals the best, e.g. Alpine for very small footprint and Ubuntu for much more comfort.

Related

Portainer Stack - docker compose issue with MacVLan network

I am starting to use portrainer.io to manage my docker images, instead of Synology DSM Docker GUI.
Background information:
I've used MacVLAN to create an own IP address for my Pihole Docker, overall everything regarding this piHole is running fine with this settings, made by DSM GUI.
environment network volumesports
Problem:
I now would like to use portrainer.io to manage my Docker installation. Including the Stack option which should be docker compose.
I am now struggeling to get my PiHole Image up with this Docker script:
services:
pihole:
container_name: pihole
image: pihole/pihole:latest
networks: docker
ports:
- "53:53/tcp"
- "53:53/udp"
- "67:67/udp"
- "80:80/tcp"
environment:
TZ: 'Europe/Berlin'
WEBPASSWORD: 'password'
ServerIP: "0.0.0.0"
# Volumes store your data between container upgrades
volumes:
- '/pihole/pihole/:/etc/pihole/'
- '/pihole/dnsmasq/:/etc/dnsmasq.d/'
# Recommended but not required (DHCP needs NET_ADMIN)
# https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
cap_add:
- NET_ADMIN
restart: unless-stopped
Does anyone have an idea why I get "Unable to deploy stack" as error message?
You are telling the service to use a network called "docker", but the network is not defined in the compose file. Is this the complete docker-compose file?
If yes, then you are missing the networks section:
networks:
docker:
external: true

Acessing ARP table of Host from Docker container on MacOS

This is basically the same question as this one, except on a Mac, setting network to host has no effect whatsoever.
I'm trying to give a Docker container, running on MacOS, access to its host ARP table. My docker-compose.yaml:
services:
homeassistant:
container_name: home-assistant
image: homeassistant/home-assistant
environment:
# This is the required way to set a timezone on macOS and differs from the Linux compose file
- TZ=XX/XXXX
volumes:
- ./config:/config
restart: unless-stopped
privileged: true
ports:
# Also required for macOS since the network directive in docker-compose does not work
- "8123:8123"
# Add this or docker-compose will complain that it did not find the key for locally mapped volume
volumes:
config:

Expose application running on host port to Selenoid

I'm running a Selenoid application test automation script, and would like to run this script against a local application. However, I can't find how to expose my local application (running on port 8787) to Selenoid. I found the following thread discussing a similar issue, but it doesn't solve my issue. The linked thread describes to use the host's ip address. However, I want to make my test system independent. The host ip address is different for each system, and is hard to be retrieved system independently.
I already tried adding the expose field to my docker compose file:
version: '3'
services:
selenoid:
network_mode: bridge
image: aerokube/selenoid:latest-release
volumes:
- "${PWD}/run:/etc/selenoid"
- "/var/run/docker.sock:/var/run/docker.sock"
- "${PWD}/run/video:/opt/selenoid/video"
- "${PWD}/run/logs:/opt/selenoid/logs"
environment:
- OVERRIDE_VIDEO_OUTPUT_DIR=${PWD}/run/video
- TZ=Europe/Amsterdam
command: ["-conf", "/etc/selenoid/browsers.json", "-video-output-dir", "/opt/selenoid/video", "-log-output-dir", "/opt/selenoid/logs"]
ports:
- "4444:4444"
expose:
- "8787"
However, this doesn't work because the docker containers created by Selenoid do not get passed the same option.
Is there any way to expose my host port 8787 to my Selenoid container in a system/os independent way (either via a configuration in the docker-compose.yml file, a capability passed to the remote driver or any other way?)?
Selenoid runs browsers in standard Docker containers, so anything applicable to Docker is applicable to Selenoid browsers. Docker was created for the case when all interacting parts are packed to containers and in that case you have legacy Docker links or modern Docker custom networks on your service. If you still want to run your application on the host machine without packing it to container, you have to either user host machine IP or on some platforms Docker provides a particular domain name, e.g. docker.for.mac.localhost on Mac.
I finally realized that yes, the application I run actually runs in a Docker container and thus linking them is as easy as putting Selenoid and the application in the same Docker network. Final docker-compose.yml is as follows:
version: '3'
networks:
my_network_name:
external:
name: my_network_name # This assumes network is already created
services:
selenoid:
networks:
my_network_name: null
image: aerokube/selenoid:latest-release
volumes:
- "${PWD}/run:/etc/selenoid"
- "/var/run/docker.sock:/var/run/docker.sock"
- "${PWD}/run/video:/opt/selenoid/video"
- "${PWD}/run/logs:/opt/selenoid/logs"
environment:
- OVERRIDE_VIDEO_OUTPUT_DIR=${PWD}/run/video
- TZ=Europe/Amsterdam
command: ["-container-network", "my_network_name", "-conf", "/etc/selenoid/browsers.json", "-video-output-dir", "/opt/selenoid/video", "-log-output-dir", "/opt/selenoid/logs"]
ports:
- "4444:4444"
expose:
- "8787"

What is a docker-compose.yml file?

I can't find a real definition of what a docker-compose file is.
Is it correct to say this:
A docker-compose file is a YAML file that allows us to deploy multiples Docker containers at the same time.
I'd like to be able to explain a bit better what a docker-compose file is.
A docker-compose.yml is a config file for Docker Compose.
It allows to deploy, combine, and configure multiple docker containers at the same time. The Docker "rule" is to outsource every single process to its own Docker container.
Take for example a simple web application: You need a server, a database, and PHP. So you can set three docker containers with Apache2, PHP, and MySQL.
The advantage of Docker Compose is easy configuration. You don't have to write a big bunch of commands into Bash. You can predefine it in the docker-compose.yml:
db:
image: mysql
ports:
- "3306:3306"
environment:
MYSQL_DATABASE: example_db
MYSQL_USER: root
MYSQL_PASSWORD: rootpw
php:
image: php
ports:
- "80:80"
- "443:443"
volumes:
- ./SRC:/var/www/
links:
- db
As you can see in my example, I define port forwarding, volumes for external data, and links to the other Docker container. It's fast, reproducible, and not that hard to understand.
The Docker Compose file format is formally specified which enables docker-compose.yml files being executed with something else than Docker, Podman for example.
Docker Compose is a tool that allows you to deploy and manage multiple containers at the same time.
A docker-compose.yml file contains instructions on how to do that.
In this file, you instruct Docker Compose for example to:
From where to take the Dockerfile to build a particular image
Which ports you want to expose
How to link containers
Which ports you want to bind to the host machine
Docker Compose reads that file and executes commands.
It is used instead of all optional parameters when building and running a single docker container.
Example:
version: '2'
services:
nginx:
build: ./nginx
links:
- django:django
- angular:angular
ports:
- "80:80"
- "8000:8000"
- "443:443"
networks:
- my_net
django:
build: ./django
expose:
- "8000"
networks:
- my_net
angular:
build: ./angular2
links:
- django:django
expose:
- "80"
networks:
- my_net
networks:
my_net:
external:
name: my_net
This example instructs Docker Compose to:
Build nginx from path ./nginx
Links angular and django containers (so their IP in the Docker network is resolved by name)
Binds ports 80, 443, 8000 to the host machine
Add it to network my_net
(so all 3 containers are in the same network and therefore accessible from each other)
Then something similar is done for the django and angular containers.
If you would use just Docker commands, it would be something like:
docker build --name nginx .
docker run --link django:django angular:angular --expose 80 443 8000 --net my_net nginx
So while you probably don't want to type all these options and commands for each image/container, you can write a docker-compose.yml file in which you write all these instructions in a human-readable format.

How to share localhost between two different Docker containers?

I have two different Docker containers and each has a different image. Each app in the containers uses non-conflicting ports. See the docker-compose.yml:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
ports:
- "6473:6473"
- "6474:6474"
- "1812:1812"
depends_on:
- postgres
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
postgres:
container_name: postgres.dev
hostname: postgres.dev
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
I can cURL each image successfully from the host machine (Mac OS), e.g. curl -k https://localhost:6473/service_a/api/version works. What I'd like to do is to be able to refer to postgres container from the service_a container via localhost as if these two containers were one and they share the same localhost. I know that it's possible if I use the hostname postgres.dev from inside the service_a container, but I'd like to be able to use localhost. Is this possible? Please note that I am not very well versed in networking or Docker.
Mac version: 10.12.4
Docker version: Docker version 17.03.0-ce, build 60ccb22
I have done quite some prior research, but couldn't find a solution.
Relevant: https://forums.docker.com/t/localhost-and-docker-compose-networking-issue/23100/2
The right way: don't use localhost. Instead use docker's built in DNS networking and reference the containers by their service name. You shouldn't even be setting the container name since that breaks scaling.
The bad way: if you don't want to use the docker networking feature, then you can switch to host networking, but that turns off a very key feature and other docker capabilities like the option to connect containers together in their own isolated networks will no longer work. With that disclaimer, the result would look like:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
network_mode: "host"
depends_on:
- postgres
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
postgres:
container_name: postgres.dev
image: postgres:9.6
network_mode: "host"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
Note that I removed port publishing from the container to the host, since you're no longer in a container network. And I removed the hostname setting since you shouldn't change the hostname of the host itself from a docker container.
The linked forum posts you reference show how when this is a VM, the host cannot communicate with the containers as localhost. This is an expected limitation, but the containers themselves will be able to talk to each other as localhost. If you use a VirtualBox based install with docker-toolbox, you should be able to talk to the containers by the virtualbox IP.
The really wrong way: abuse the container network mode. The mode is available for debugging container networking issues and specialized use cases and really shouldn't be used to avoid reconfiguring an application to use DNS. And when you stop the database, you'll break your other container since it will lose its network namespace.
For this, you'll likely need to run two separate docker-compose.yml files because docker-compose will check for the existence of the network before taking any action. Start with the postgres container:
version: "2"
services:
postgres:
container_name: postgres.dev
image: postgres:9.6
ports:
- "5432:5432"
volumes:
- ../configs/postgres/scripts:/docker-entrypoint-initdb.d/
Then you can make a second service in that same network namespace:
version: "2"
services:
service_a:
container_name: service_a.dev
image: service_a.dev
network_mode: "container:postgres.dev"
ports:
- "6473:6473"
- "6474:6474"
- "1812:1812"
volumes:
- ../configs/service_a/var/conf:/opt/services/service_a/var/conf
Specifically for Mac and during local testing, I managed to get the multiple containers working using docker.for.mac.localhost approach. I documented it http://nileshgule.blogspot.sg/2017/12/docker-tip-workaround-for-accessing.html

Resources