So what I am looking at is a docker run command being used in to create a docker container for open telemetry that passes in a config command, and the code looks like...
$ git clone git#github.com:open-telemetry/opentelemetry-collector.git; \
cd opentelemetry-collector/examples; \
go build main.go; ./main & pid1="$!";
docker run --rm -p 13133:13133 -p 14250:14250 -p 14268:14268 \
-p 55678-55679:55678-55679 -p 4317:4317 -p 8888:8888 -p 9411:9411 \
-v "${PWD}/local/otel-config.yaml":/otel-local-config.yaml \
--name otelcol otel/opentelemetry-collector \
--config otel-local-config.yaml; \
kill $pid1; docker stop otelcol
(https://opentelemetry.io/docs/collector/getting-started/#docker)
What I don't understand is how a non-docker related config file(open telemetry config) fits into the "docker run --config" or "docker compose config" commands. Below is the open telemetry config file that seems to be non-docker related
extensions:
memory_ballast:
size_mib: 512
zpages:
endpoint: 0.0.0.0:55679
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
memory_limiter:
# 75% of maximum memory up to 4G
limit_mib: 1536
# 25% of limit up to 2G
spike_limit_mib: 512
check_interval: 5s
exporters:
logging:
logLevel: debug
service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
metrics:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
extensions: [memory_ballast, zpages]
https://github.com/open-telemetry/opentelemetry-collector/blob/main/examples/local/otel-config.yaml
Now I have looked at these Docker links
https://docs.docker.com/engine/swarm/configs/#how-docker-manages-configs
https://nickjanetakis.com/blog/docker-tip-43-using-the-docker-compose-config-command
but I couldn't figure out how to get the docker run --config command in the open telemetry example to start working in docker compose with docker compose config. Here is my docker compose
version: "3.9"
services:
opentelemetry:
container_name: otel
image: otel/opentelemetry-collector:latest
volumes:
- ~/source/repos/CritterTrackerProject/DockerServices/OpenTelemetry/otel-collector-config.yml:/otel-local-config.yml
config:
- otel-local-config.yml
ports:
- 13133:13133
- 14250:14250
- 14268:14268
- 55678-55679:55678-55679
- 4317:4317
- 8888:8888
- 9411:9411
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
jaeger:
# restart: unless-stopped
container_name: jaeger
image: jaegertracing/all-in-one:latest
ports:
- 16686:16686
# - 14250:14250
# - 14268:14268
# - 5775:5775/udp
- 6831:6831/udp
# - 6832:6832/udp
# - 5778:5778
# - 9411:9411
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
postgres:
restart: always
container_name: postgres
image: postgres:latest
environment:
- POSTGRES_USER=code
- POSTGRES_PASSWORD=code
ports:
- 5432:5432
volumes:
- postgres:/var/lib/postgresql/data
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
nginx:
restart: always
container_name: webserver
image: nginx:latest
build:
context: ~/source/repos/CritterTrackerProject
dockerfile: DockerServices/Nginx/Dockerfile
ports:
- 80:80
- 443:443
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
volumes:
postgres:
networks:
my-network:
external: true
name: my-network
Here is my error after running docker compose up in a Git Bash terminal
$ docker compose -f ./DockerServices/docker-compose.yml up -d
services.opentelemetry Additional property config is not allowed
The general form of docker run is
docker run [docker options] image [command]
And if you look at your original command it matches this pattern
docker run \
--rm -p ... -v ... --name ... \ # Docker options
otel/opentelemetry-collector \ # Image
--config otel-local-config.yaml # Command
So what looks like a --config option is really the command part of the container setup; it overrides the Dockerfile CMD, and it is passed as additional arguments to the image's ENTRYPOINT.
In a Compose setup, then, this would be the container's command:.
services:
opentelemetry:
image: otel/opentelemetry-collector:latest
command: --config otel-local-config.yaml
Since this is an application-specific command string, it's unrelated to the docker-compose config command, which is a diagnostic tool that just dumps out parts of your Compose configuration.
What you're doing in the docker run command is the following mounting:
${PWD}/local/otel-config.yaml on the local host to /otel-local-config.yaml from inside the docker
You can achieve same behavior with volumes option from docker compose:
volumes:
"${PWD}/local/otel-config.yaml":/otel-local-config.yaml
Related
due to hardware problem, I had to replace my small home server with a new one.
Several self-hosted services with Docker were running on the server for which I tried to backup the volumes following the instructions on the official Docker website and those present in this YouTube video and in this cheat-sheet. Now, following the same documentation, I am trying to restore the backups but without success. The first one I'm trying for is a stack for Nginx Proxy Manager built with Docker compose with this docker-compose.yaml file:
version: "3.6"
services:
app:
image: jc21/nginx-proxy-manager:latest
restart: always
ports:
- "80:80"
- "443:443"
- "81:81"
environment:
DB_MYSQL_HOST: "db"
DB_MYSQL_PORT: 3306
DB_MYSQL_NAME: "db_name"
DB_MYSQL_USER: "db_user"
DB_MYSQL_PASSWORD: "db_password"
volumes:
- data:/data
- ./letsencrypt:/etc/letsencrypt
depends_on:
- db
db:
image: jc21/mariadb-aria:latest
restart: always
environment:
MYSQL_DATABASE: "db_name"
MYSQL_ROOT_PASSWORD: "root_password"
MYSQL_USER: "db_user"
MYSQL_PASSWORD: "db_password"
volumes:
- db:/var/lib/mysql
volumes:
db:
data:
After starting the stack with docker compose up -d command I'm trying to restore db and data volumes with:
docker run --rm --volumes-from nginx-proxy-manager-db-1 -v $(pwd):/backup ubuntu bash -c "cd / && tar xvf /backup/nginx-proxy-manager_db_20220717-082200.tar --strip 1"
docker run --rm --volumes-from nginx-proxy-manager-app-1 -v $(pwd):/backup ubuntu bash -c "cd / && tar xvf /backup/nginx-proxy-manager_data_20220717-082200.tar --strip 1"
What's wrong?
Trying to setup Redis from this image Redismod and struggle to translate the following code into docker-compose
$ docker run \
-p 6379:6379 \
-v /home/user/data:/data \
-v /home/user/redis.conf:/usr/local/etc/redis/redis.conf \
redislabs/redismod \
/usr/local/etc/redis/redis.conf
What I have done till now:
version: "3.2"
services:
redis:
image: "redislabs/redismod"
container_name: 'redis-local'
hostname: 'redis-local'
volumes_from:
- redis_data:/data
- ./redis.conf:/usr/local/etc/redis/redis.conf
args:
- /usr/local/etc/redis/redis.conf
restart: always
ports:
- "6379:6379"
volumes:
redis_data:
But I get the following error ERROR: Service "redis" mounts volumes from "redis_data", which is not the name of a service or container. obviously because I didn't pass the last line /usr/local/etc/redis/redis.conf
And second question, how do I translate --loadmodule and --dir from below, these aren't Redis command:
$ docker run \
-p 6379:6379 \
-v /home/user/data:/data \
redislabs/redismod \
--loadmodule /usr/lib/redis/modules/rebloom.so \
--dir /data
UPDATE
I changed my docker-compose.yml file to the following and it started to work, but it seems that Redis doesn't see the redis.conf file and continue to run in default mode, what I do wrong?
version: "3.2"
services:
redis:
image: "redislabs/redismod"
container_name: 'redis-local'
hostname: 'redis-local'
volumes:
- redis_data:/data
- ./redis.conf:/usr/local/etc/redis/redis.conf
build:
context: .
args:
- /usr/local/etc/redis/redis.conf
restart: always
ports:
- "6379:6379"
The first error was because you used volumes_from instead of volumes. The first one is intended to get the volumes configuration from an existing container. The second one to define the volumes. In your last version redis_data is a docker volume and redis.conf is a bind mount. Your second problem is that you are using build and args that are intended to be used for building images but looks like you wanted to run a command.
Try:
version: "3.2"
services:
redis:
image: "redislabs/redismod"
container_name: 'redis-local'
hostname: 'redis-local'
volumes:
- redis_data:/data
- ./redis.conf:/usr/local/etc/redis/redis.conf
command: usr/local/etc/redis/redis.conf
restart: always
ports:
- "6379:6379"
For more info about volumes, bind mounts and docker compose reference see:
https://docs.docker.com/storage/volumes/
https://docs.docker.com/storage/bind-mounts/
https://docs.docker.com/compose/compose-file/compose-file-v3/#command
I want to convert my docker run command, but I cannot get it working. This is the docker run command I use and works well
sudo docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $PWD/traefik.toml:/traefik.toml \
-v $PWD/acme.json:/acme.json \
-p 80:80 \
-p 443:443 \
-l traefik.frontend.rule=Host:monitor.localhost \
-l traefik.port=8080 \
--network traefik-proxy \
--name traefik \
traefik --docker
And this is the compose file I built:
version: "3"
services:
traefik:
image: traefik
container_name: traefik-2
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
labels:
traefik.frontend.rule: "Host:monitor.localhost"
traefik.port: "8080"
networks:
default:
external:
name: traefik-proxy
The problem is that when I use docker-compose, the proxy seems to be working, but when I access the monitor site (monitor.localhost), it gives me a 404 not found. I have double checked everything, but I just can't figure it out what is wrong with the compose file. I tried to get into the shell of the container to see if it looks alright, but apparently Alpine based Traefik image doesn't have bash, or even sh.
As Wie already pointed out in the comments, you're not passing the --docker flag in your docker-compose.yml. Try it like this:
version: "3"
services:
traefik:
image: traefik
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
ports:
- 80:80
- 443:443
labels:
traefik.frontend.rule: Host:monitor.localhost
traefik.port: 8080
command: ["--docker"]
networks:
default:
external:
name: traefik-proxy
I would like to translate the following docker command to a docker-compose file:
docker run -d --restart=always -v /var/run/docker.sock:/var/run/docker.sock --net shinyproxy-net -p 8080:8080 imshinyproxy
This is the docker-compose.yml that I wrote:
version: "3.7"
services:
shinyproxy:
image: imshinyproxy
container_name: imshinyproxy
environment:
- PUID=1000
- PGID=65537
- TZ=america/new_york
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 8080:8080
networks:
- shinyproxy-net
restart: unless-stopped
Alas, when I try to run docker-compose up I get the following error:
$ docker-compose up
ERROR: Service "shinyproxy" uses an undefined network "shinyproxy-net"
I know the network exist:
$ sudo docker network create shinyproxy-net
Error response from daemon: network with name shinyproxy-net already exists
What am I doing wrong?
You must declare an external network in the networks section of your docker-compose.yml :
version: "3.7"
services:
shinyproxy:
[...]
networks:
- shinyproxy-net
networks:
shinyproxy-net:
external:
name: shinyproxy-net
networks.shinyproxy-net.external.name should correspond to the name of your previously created network.
I am trying to use this image https://hub.docker.com/r/ibmcom/cloudant-developer/ with docker compose, when I use the original instructions it works, however when I translate it to docker compose format it doesn't work properly, I see the dashboard page but it is empty and seems broken.
The original run command:
docker run \
--privileged \
--detach \
--volume cloudant:/srv \
--name cloudant-developer \
--publish 8080:80 \
--hostname cloudant.dev \
ibmcom/cloudant-developer
The compose file I created:
version: '3'
services:
cloudant:
image: ibmcom/cloudant-developer:latest
container_name: cloudant-developer
hostname: cloudant.dev
ports:
- "8080:80"
expose:
- "80"
volumes:
- cloudant:/srv
privileged: true
volumes:
cloudant:
Thanks for helping.
P.S - I do executed the commands for license agreement manually
Took me a while to figure this out. Turns out the cloudant docker container is tied to the default docker network subnet. Specifically, I found that haproxy was mapped to redirect to 172.17.0.2:5984 and was failing because by default docker compose creates a new network in a different ip range. There may be other issues related to this. Ultimately I found that you could run docker compose on the default docker network with the following config:
network_mode: bridge
So, your docker-compose.yml would look like this:
version: '3'
services:
cloudant:
image: ibmcom/cloudant-developer:latest
container_name: cloudant-developer
hostname: cloudant.dev
ports:
- "8080:80"
expose:
- "80"
volumes:
- cloudant:/srv
privileged: true
network_mode: bridge
volumes:
cloudant: