I have setup a docker Cadvisor using docker service cluster and need to dynamically monitor the nodes of docker cluster using active service discovery.
If I have started the prometheus CAdvisor through docker cluster using docker service command, it's working fine and I am successfully able to discover the docker cluster nodes dynamically. But, if I've passed the same parameters of that command in docker compose-file, I cannot see any nodes. Following is the docker compose configuration of prometheus CAdvisor.
cadvisor:
image: google/cadvisor
container_name: cadvisor
ports:
- target: 8080
mode: host
published: 8040
network_mode: "host"
deploy:
mode: replicated
command:
- --docker_only=true
labels:
- "prometheus-job=cadvisor"
volumes:
- /:/rootfs:ro
- /var/run:/var/run
- /sys:/sys:ro
- /var/lib/docker:/var/lib/docker:ro
- /var/run/docker.sock:/var/run/docker.sock:rw
Docker service command:
docker service create --name cadvisor -l prometheus-job=cadvisor \
--mode=global --publish published=8040,target=8080,mode=host \
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock,ro \
--mount type=bind,src=/,dst=/rootfs,ro \
--mount type=bind,src=/var/run,dst=/var/run \
--mount type=bind,src=/sys,dst=/sys,ro \
--mount type=bind,src=/var/lib/docker,dst=/var/lib/docker,ro \
google/cadvisor -docker_only
Any help in this regard will be appreciated.
Related
I am running prometheus in a docker container, and I want to configure an AlertManager for making it send me an email when the service is down. I created the alert_rules.yml and the prometheus.yml, and I run everything with the following command, mounting both the yml files onto the docker container at the path /etc/prometheus:
docker run -d -p 9090:9090 --add-host host.docker.internal:host-gateway -v "$PWD/prometheus.yml":/etc/prometheus/prometheus.yml -v "$PWD/alert_rules.yml":/etc/prometheus/alert_rules.yml prom/prometheus
Now, I also want prometheus to send me an email when an alert comes up, and that's where I encounter some problems. I configured my alertmanager.yml as follows:
route:
group_by: ['alertname']
group_wait: 30s
group_interval: 5m
repeat_interval: 1h
receiver: email-me
receivers:
- name: 'gmail'
email_configs:
- to: 'my_email#gmail.com'
from: 'askonlinetraining#gmail.com'
smarthost: smtp.gmail.com:587
auth_username: 'my_email#gmail.com'
auth_identity: 'my_email#gmail.com'
auth_password: 'the_password'
I actually don't know if the smarthost parameter is configured correctly since I can't find any documentation about it and I don't know which values it should contain
I also created an alertmanager.service file:
[Unit]
Description=AlertManager Server Service
Wants=network-online.target
After=network-online.target
[Service]
User=root
Group=root
Type=Simple
ExecStart=/usr/local/bin/alertmanager \
--config.file /etc/alertmanager.yml
[Install]
WantedBy=multi-user.target
I think something here is messed up: I think the first parameter I pass to ExecStart is a path that doesn't exist in the container, but I have no idea on how I should replace it.
I tried mounting the last two files into the docker container in the same directory where I mount the first two yml files by using the following command:
docker run -d -p 9090:9090 --add-host host.docker.internal:host-gateway -v "$PWD/prometheus.yml":/etc/prometheus/prometheus.yml -v "$PWD/alert_rules.yml":/etc/prometheus/alert_rules.yml -v "$PWD/alertmanager.yml":/etc/prometheus/alertmanager.yml -v "$PWD/alertmanager.service":/etc/prometheus/alertmanager.service prom/prometheus
But the mailing alert is not working and I don't know how to fix the configuration for smoothly running all of this into a docker container. As I said, I suppose the main problem resides in the ExecStart command present in alertmanager.service, but maybe I'm wrong. I can't find anything helpful online, hence I would really appreciate some help
The best practice with containers is to aim to run a single process per container.
In your container, this suggests one container for prom/prometheus and another for prom/alertmanager.
You can run these using docker as:
docker run \
--detach \
--name=prometheus \
--volume=${PWD}:/prometheus.yml:/etc/prometheus/prometheus.yml \
--volume=${PWD}:/rules.yml:/etc/alertmanager/rules.yml \
--publish=9090:9090 \
prom/prometheus:v2.26.0 \
--config.file=/etc/prometheus/promtheus.yml
docker run \
--detach \
--name=alertmanager \
--volume=${PWD}:/rules.yml:/etc/alertmanager/rules.yml \
--publish=9093:9093 \
prom/alertmanager:v0.21.0
A good tool when you run multiple container is Docker Compose in which case, your docker-compose.yml could be:
version: "3"
services:
prometheus:
restart: always
image: prom/prometheus:v2.26.0
container_name: prometheus
command:
- --config.file=/etc/prometheus/prometheus.yml
volumes:
- ${PWD}/prometheus.yml:/etc/prometheus/prometheus.yml
- ${PWD}/rules.yml:/etc/alertmanager/rules.yml
expose:
- "9090"
ports:
- 9090:9090
alertmanager:
restart: always
depends_on:
- prometheus
image: prom/alertmanager:v0.21.0
container_name: alertmanager
volumes:
- ${PWD}/alertmanager.yml:/etc/alertmanager/alertmanager.yml
expose:
- "9093"
ports:
- 9093:9093
and you could:
docker-compose up
In either case, you can then browse:
Prometheus on the host's port 9090 i.e. localhost:9090
Alert Manager on the host's port 9093, i.e. localhost:9093
I run a container (cAdvisor) that needs to access the Docker Engine of the host.
When I run it as a service with the command line, everything works fine:
docker service create --name cadvisor
--network clusternetwork -p 8080:8080
--mount type=bind,src=/var/run/docker.sock,dst=/var/run/docker.sock,ro
--mount type=bind,src=/,dst=/rootfs,ro
--mount type=bind,src=/sys,dst=/sys,ro
--mount type=bind,src=/var/lib/docker,dst=/var/lib/docker,ro
gcr.io/google-containers/cadvisor:latest
But when I transpose the following service to a docker-compose file and run it using docker stack deploy -c myCadvisor-compose.yml cAdvisor, it doesn't work and I get the following error: failed to get docker info: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Here is my docker-compose file. Did I forget to transpose something from the above service call?
version: "3.7"
services:
cadvisor:
image: gcr.io/google-containers/cadvisor:latest
ports:
- 8080:8080
volumes:
- /var/run/docker.sock:/var/run/docker.sock,ro
- /:/rootfs,ro
- /sys:/sys,ro
- /var/lib/docker:/var/lib/docker,ro
networks:
- clusternetwork
networks:
clusternetwork:
external: true
I have tested your code. The main issue is the "," you must put an":". RW or RO doesn't matter. In my case the working one looks like this:
volumes:
- /:/rootfs:ro
- /var/run/docker.sock:/var/run/docker.sock:rw
- /sys:/sys:ro
- /var/lib/docker/:/var/lib/docker:ro
Cheers Jules
I have an nginx proxy setup with a shellscript that looks something like this
docker run --detach --name nginx-proxy --publish 80:80 --publish 443:443 --volume /etc/nginx/certs \
--volume /etc/nginx/vhost.d --volume /usr/share/nginx/html --volume /var/run/docker.sock:/tmp/docker.sock:ro --restart unless-stopped jwilder/nginx-proxy:alpine
echo proxy up
docker run --detach --name nginx-proxy-letsencrypt --volumes-from nginx-proxy --volume /var/run/docker.sock:/var/run/docker.sock:ro \
--restart unless-stopped jrcs/letsencrypt-nginx-proxy-companion
echo ssl companion up
docker run -d \
-e VIRTUAL_HOST=[domain] \
\-e "LETSENCRYPT_HOST=[domain]" \
-e "LETSENCRYPT_EMAIL=[emailaddress]" \
--name [domain] \
--expose 80 \
--restart always \
-v /code/[domain]:/var/www/html \
fauria/lamp
echo test site up at [domain]
and this site works properly and functions as expected.
I then stop the web server container and use the following docker-compose.yaml and it fails with a 502..
version: '3.3'
services:
lamp:
restart: always
image: fauria/lamp
container_name: [domain]
expose:
- "80"
volumes:
- /code/[domain]:/var/www/html
environment:
- VIRTUAL_HOST=[domain]
- LETSENCRYPT_HOST=[domain]
- LETSENCRYPT_EMAIL=[emailaddress]
Why? Aren't they the same? What am I missing?
When you use docker-compose, docker-compose creates a docker network for you, in which all of the services can communicate with each other. Since you simply stopped the container and started it with docker-compose, now it does not have access to the containers on your localhost. This is why you get the 502 error. What you need to do is add the other containers to your docker compose file, and make sure you are connecting to the hosts using the proper service name (instead of localhost use http://service_name:443). Alternatively you can somehow give the containers in your docker network access to your localhost, but I'm not sure how to do this. Maybe you need to use 0.0.0.0 instead of 127.0.0.1?
The problem is that i was not connecting my docker-compose to the bridge network used by default in the proxy image.
version: '3.3'
services:
lamp:
restart: always
image: fauria/lamp
network-mode: bridge
container_name: [domain]
expose:
- "80"
volumes:
- /code/[domain]:/var/www/html
environment:
- VIRTUAL_HOST=[domain]
- LETSENCRYPT_HOST=[domain]
- LETSENCRYPT_EMAIL=[emailaddress]
Hello,
I have the following problem on docker 18.06.1-ce.
I have an owncloud container that works with the following configurations:
Image : owncloud/server:10.0
Status healthy
Ports : 0.0.0.0:4090->80/tcp, 0.0.0.0:4093->443/tcp
So far, so good, this container is functional.
Now, I want to add a gitlab container with the following configurations:
Image : gitlab/gitlab-ce:latest
Status : heatly
Ports : 0.0.0.0:2222->22/tcp, 0.0.0.0:8080->80/tcp, 0.0.0.0:4443->443/tcp
The problem is that I can't access the containers with the ports listed above (connection failed).
I tried to install the container in a different way:
By docker run command :
docker run --detach --hostname nsXXXXX.ip-XX-XXX-XX.eu --env GITLAB_OMNIBUS_CONFIG="external_url 'https://nsXXXXX.ip-XX-XXX-XX.eu:4443'; gitlab_rails['lfs_enabled'] = true;" --publish 4443:443 --publish 8080:80 --publish 2222:22 --name gitlab --restart always --volume /srv/gitlab/config:/etc/gitlab --volume /srv/gitlab/logs:/var/log/gitlab --volume /srv/gitlab/data:/var/opt/gitlab gitlab/gitlab-ce:latest
And by docker-compose:
web:
image: 'gitlab/gitlab-ce:latest'
restart: always
hostname: 'nsXXXXXXX.ip-XX-XXX-XX.eu'
privileged: true
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'https://nsXXXXXXX.ip-XX-XXX-XX.eu:4443/'
gitlab_rails['gitlab_shell_ssh_port'] = 4182
ports:
- '4180:80'
- '4443:443'
- '4182:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
My docker is on a dedicated Debain Stretch hosted by kimsufi.
Do you have any ideas to help me? Thank you very much.
Solved : https://forum.gitlab.com/t/docker-gitlab-container-healthy-but-not-accessible/20042/5
It was necessary to map the port of the external URL to the internal port... Beginner's error:)
How can you run Kudu, which requires two containers - one for the master and one for the tserver under docker, when the two containers need to connect to each other by DNS.
Kudu can be run under Docker using the following commands:
docker run --name kudu-master --hostname kudu-master --detach --publish 8051:8051 --publish 7051:7051 kunickiaj/kudu master
and:
docker run --name kudu-tserver --hostname kudu-tserver --detach --publish 8050:8050 --publish 7050:7050 --link kudu-master --env KUDU_MASTER=kudu-master kunickiaj/kudu tserver
However, the above defines a one way link, from kudu-tserver to kudu-master and not vice verse.
For Kudu to function correctly, bother kudu-master and kudu-tserver need to be able to connect to each other.
How can the Docker containers be configured, so that the two way link works?
Docker image reference
Similar image reference
The link parameter in docker run is a legacy feature which may be removed (references [1] and [2]).
You can raise multiple Docker containers and connect them to each other using docker-compose.
To get this working, create a folder named kudu and place the following docker-compose.yml file under it:
version: '3'
services:
kudu-master:
image: "kunickiaj/kudu"
hostname: kudu-master
ports:
- "8051:8051"
- "7051:7051"
command:
master
networks:
kudu_network:
aliases:
- kudu-master
kudu-tserver:
image: "kunickiaj/kudu"
hostname: kudu-tserver
ports:
- "8050:8050"
- "7050:7050"
environment:
- KUDU_MASTER=kudu-master
command:
tserver
networks:
kudu_network:
aliases:
- kudu-tserver
networks:
kudu_network:
This file includes 2 services (kudu-master and kudu-tserver) and a network within which both have aliases which are visible to the rest of the network (to each other). [File reference]
Then run docker-compose using the following command line:
docker-compose -f "filePathToYourDockerComposeYmlFile" up -d
or, if you want to recreate the Docker containers:
docker-compose -f "filePathToYourDockerComposeYmlFile" up -d --force-recreate
Other useful commands [reference]:
To stop the containers:
docker-compose -f "filePathToYourDockerComposeYmlFile" stop
To remove the containers:
docker-compose -f "filePathToYourDockerComposeYmlFile" rm -f