When I run my ELK by docker run works fine, but when I run by docker-compose logs are sent to logstash very slow and not stable.
docker-compose.yml
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.0
ports:
- 9200:9200
- 9300:9300
environment:
- discovery.type=single-node
kibana:
image: docker.elastic.co/kibana/kibana:7.6.0
links:
- elasticsearch
ports:
- 5601:5601
logstash:
image: docker.elastic.co/logstash/logstash:7.6.0
links:
- elasticsearch
ports:
- 5044:5044
volumes:
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml
- ./logstash/config:/usr/share/logstash/pipeline/
run by docker run
docker run -d -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.6.
docker run -d --link edcf2406addf:elasticsearch -p 5601:5601 docker.elastic.co/kibana/kibana:7.6.0
docker run --link edcf2406addf:elasticsearch -v /var/www/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml -v /var/www/logstash/config:/usr/share/logstash/pipeline/ -p 5044:5044 docker.elastic.co/logstash/logstash:7.6.0
I would like to use docker-compose but something not works fine. Could someone help me configuring docker-compose properly?
docker-compose version 1.21.0-rc1
Docker version 19.03.5
Thanks
Related
I am trying to make a quick connection setup using the fallowing setup
Copy & Paste to recreate the issue
docker rm -f mariadb && docker run --detach --name mariadb --env MARIADB_USER=user --env MARIADB_PASSWORD=secret --env MARIADB_ROOT_PASSWORD=secret -p 3306:3306 mariadb:latest
docker rm -f phpmyadd && docker run --name phpmyadd -d -e PMA_HOST=host -e PMA_PORT=3306 -p 8080:80 phpmyadmin
docker exec -it mariadb bash
I can login in to mariadb container and access mariadb with
mysql -uroot -psecret
I can also access phpmyadmin container at http://localhost:8080
However when i try to login to mariadb through phpmyadmin i get the fallowing:
It shows that the port is exposed but I can not access it with telnet..
Any idea what is missing here?
For 2 containers to be able to talk to each other, you would have to setup a docker-compose instead. Something like this should work
version: '3.8'
volumes:
mariadb:
driver: local
services:
mariadb:
image: mariadb:10.6
restart: always
environment:
MYSQL_ROOT_PASSWORD: YOUR_ROOT_PASSWORD_HERE
MYSQL_USER: YOUR_MYSQL_USER_HERE
MYSQL_PASSWORD: YOUR_USER_PW_HERE
ports:
- "40000:3306"
volumes:
- mariadb:/var/lib/mysql
phpmyadmin:
image: phpmyadmin
restart: always
ports:
- "40001:80"
environment:
- PMA_HOST=mariadb
- PMA_PORT=3306
And you would start everything using docker-compose up
So what I am looking at is a docker run command being used in to create a docker container for open telemetry that passes in a config command, and the code looks like...
$ git clone git#github.com:open-telemetry/opentelemetry-collector.git; \
cd opentelemetry-collector/examples; \
go build main.go; ./main & pid1="$!";
docker run --rm -p 13133:13133 -p 14250:14250 -p 14268:14268 \
-p 55678-55679:55678-55679 -p 4317:4317 -p 8888:8888 -p 9411:9411 \
-v "${PWD}/local/otel-config.yaml":/otel-local-config.yaml \
--name otelcol otel/opentelemetry-collector \
--config otel-local-config.yaml; \
kill $pid1; docker stop otelcol
(https://opentelemetry.io/docs/collector/getting-started/#docker)
What I don't understand is how a non-docker related config file(open telemetry config) fits into the "docker run --config" or "docker compose config" commands. Below is the open telemetry config file that seems to be non-docker related
extensions:
memory_ballast:
size_mib: 512
zpages:
endpoint: 0.0.0.0:55679
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
memory_limiter:
# 75% of maximum memory up to 4G
limit_mib: 1536
# 25% of limit up to 2G
spike_limit_mib: 512
check_interval: 5s
exporters:
logging:
logLevel: debug
service:
pipelines:
traces:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
metrics:
receivers: [otlp]
processors: [memory_limiter, batch]
exporters: [logging]
extensions: [memory_ballast, zpages]
https://github.com/open-telemetry/opentelemetry-collector/blob/main/examples/local/otel-config.yaml
Now I have looked at these Docker links
https://docs.docker.com/engine/swarm/configs/#how-docker-manages-configs
https://nickjanetakis.com/blog/docker-tip-43-using-the-docker-compose-config-command
but I couldn't figure out how to get the docker run --config command in the open telemetry example to start working in docker compose with docker compose config. Here is my docker compose
version: "3.9"
services:
opentelemetry:
container_name: otel
image: otel/opentelemetry-collector:latest
volumes:
- ~/source/repos/CritterTrackerProject/DockerServices/OpenTelemetry/otel-collector-config.yml:/otel-local-config.yml
config:
- otel-local-config.yml
ports:
- 13133:13133
- 14250:14250
- 14268:14268
- 55678-55679:55678-55679
- 4317:4317
- 8888:8888
- 9411:9411
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
jaeger:
# restart: unless-stopped
container_name: jaeger
image: jaegertracing/all-in-one:latest
ports:
- 16686:16686
# - 14250:14250
# - 14268:14268
# - 5775:5775/udp
- 6831:6831/udp
# - 6832:6832/udp
# - 5778:5778
# - 9411:9411
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
postgres:
restart: always
container_name: postgres
image: postgres:latest
environment:
- POSTGRES_USER=code
- POSTGRES_PASSWORD=code
ports:
- 5432:5432
volumes:
- postgres:/var/lib/postgresql/data
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
nginx:
restart: always
container_name: webserver
image: nginx:latest
build:
context: ~/source/repos/CritterTrackerProject
dockerfile: DockerServices/Nginx/Dockerfile
ports:
- 80:80
- 443:443
extra_hosts:
- "host.docker.internal:host-gateway"
networks:
- my-network
volumes:
postgres:
networks:
my-network:
external: true
name: my-network
Here is my error after running docker compose up in a Git Bash terminal
$ docker compose -f ./DockerServices/docker-compose.yml up -d
services.opentelemetry Additional property config is not allowed
The general form of docker run is
docker run [docker options] image [command]
And if you look at your original command it matches this pattern
docker run \
--rm -p ... -v ... --name ... \ # Docker options
otel/opentelemetry-collector \ # Image
--config otel-local-config.yaml # Command
So what looks like a --config option is really the command part of the container setup; it overrides the Dockerfile CMD, and it is passed as additional arguments to the image's ENTRYPOINT.
In a Compose setup, then, this would be the container's command:.
services:
opentelemetry:
image: otel/opentelemetry-collector:latest
command: --config otel-local-config.yaml
Since this is an application-specific command string, it's unrelated to the docker-compose config command, which is a diagnostic tool that just dumps out parts of your Compose configuration.
What you're doing in the docker run command is the following mounting:
${PWD}/local/otel-config.yaml on the local host to /otel-local-config.yaml from inside the docker
You can achieve same behavior with volumes option from docker compose:
volumes:
"${PWD}/local/otel-config.yaml":/otel-local-config.yaml
I'm reading the docker-ce dock for installation. It's simple two lines :
docker volume create portainer_data &&
docker run -d -p 8000:8000 -p 9443:9443 --name portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v portainer_data:/data portainer/portainer-ce:latest
Is there a way to do all this stuff with docker-compose file ? So the portainer container will be groupped with all the others when executing docker-comppose down ?
This french tutorial got something working :
portainer:
container_name: portainer
image: portainer/portainer-ce:latest #latest might not be the best option
restart: unless-stopped
command: -H unix:///var/run/docker.sock
ports:
- 9000:9000
volumes:
- /var/run/docker.sock:/var/run/docker.sock:fr
- /etc/localtime:/etc/localtime:fr
- /etc/timezone:/etc/timezone:fr #not sure about those 3 volumes
- dataportainer:/data
volumes:
dataportainer:
I want to convert my docker run command, but I cannot get it working. This is the docker run command I use and works well
sudo docker run -d \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $PWD/traefik.toml:/traefik.toml \
-v $PWD/acme.json:/acme.json \
-p 80:80 \
-p 443:443 \
-l traefik.frontend.rule=Host:monitor.localhost \
-l traefik.port=8080 \
--network traefik-proxy \
--name traefik \
traefik --docker
And this is the compose file I built:
version: "3"
services:
traefik:
image: traefik
container_name: traefik-2
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
labels:
traefik.frontend.rule: "Host:monitor.localhost"
traefik.port: "8080"
networks:
default:
external:
name: traefik-proxy
The problem is that when I use docker-compose, the proxy seems to be working, but when I access the monitor site (monitor.localhost), it gives me a 404 not found. I have double checked everything, but I just can't figure it out what is wrong with the compose file. I tried to get into the shell of the container to see if it looks alright, but apparently Alpine based Traefik image doesn't have bash, or even sh.
As Wie already pointed out in the comments, you're not passing the --docker flag in your docker-compose.yml. Try it like this:
version: "3"
services:
traefik:
image: traefik
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./traefik.toml:/traefik.toml
- ./acme.json:/acme.json
ports:
- 80:80
- 443:443
labels:
traefik.frontend.rule: Host:monitor.localhost
traefik.port: 8080
command: ["--docker"]
networks:
default:
external:
name: traefik-proxy
I would like to translate the following docker command to a docker-compose file:
docker run -d --restart=always -v /var/run/docker.sock:/var/run/docker.sock --net shinyproxy-net -p 8080:8080 imshinyproxy
This is the docker-compose.yml that I wrote:
version: "3.7"
services:
shinyproxy:
image: imshinyproxy
container_name: imshinyproxy
environment:
- PUID=1000
- PGID=65537
- TZ=america/new_york
volumes:
- /var/run/docker.sock:/var/run/docker.sock
ports:
- 8080:8080
networks:
- shinyproxy-net
restart: unless-stopped
Alas, when I try to run docker-compose up I get the following error:
$ docker-compose up
ERROR: Service "shinyproxy" uses an undefined network "shinyproxy-net"
I know the network exist:
$ sudo docker network create shinyproxy-net
Error response from daemon: network with name shinyproxy-net already exists
What am I doing wrong?
You must declare an external network in the networks section of your docker-compose.yml :
version: "3.7"
services:
shinyproxy:
[...]
networks:
- shinyproxy-net
networks:
shinyproxy-net:
external:
name: shinyproxy-net
networks.shinyproxy-net.external.name should correspond to the name of your previously created network.