Kibana container to elasticsearch cloud auth err - docker

I have a production instance of elasticsearch 5.6.9 deployed on elastic.cloud.
WIth an http elastic all is OK but I would run a localhost kibana connected to that https instance!
I have tried:
docker run --name kibana-prod-user
-e ELASTICSEARCH_URL=https://####.eu-west-1.aws.found.io:9243
-e ELASTICSEARCH_PASSWORD=####
-v /host/workspace/cert:/usr/share/elasticsearch/config/certificates
-p 3501:5601 --b kibana
but i get:
In my mount dir I have put the cert.cer of elastic cloud.
Any ideas?
Thank you very much

I have find the solution, after understand that the error wasn't a certificate problem.
The right script for kibana 5.6.10 is:
docker run --name kibana-prod-provider -v "$(pwd)":/etc/kibana/ -p 3502:5601 --rm kibana
because the ELASTICSEARCH_PASSWORD envvar is not managed by the docker file, only le URL is.
Then in the $(pwd) directory I have put this kibana.yml file:
server.host: '0'
elasticsearch.url: 'https://###.eu-west-1.aws.found.io:9243'
elasticsearch.username: elastic
elasticsearch.password: ###

Related

Filebeat container does not send logs to Elastic

On my local machine running Ubuntu 18.04 via "Windows Subsystem Linux 2" on Windows 10, I am running Elastic 7.3, Kibana 7.3 and Elastic 7.3 docker containers.
Set-up is successful and Filebeat seems to monitor containers correctly. However, Kibana does not show any logs.
Setup
To set-up Elastic and Kibana I use the following commands
docker pull docker.elastic.co/elasticsearch/elasticsearch:7.3.1
docker run --network=lognetwork --name=elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.3.1
docker pull docker.elastic.co/kibana/kibana:7.3.1
docker run --name=kibana --network=lognetwork -e ELASTICSEARCH_HOSTS=http://elasticsearch:9200 -p 5601:5601 docker.elastic.co/kibana/kibana:7.3.1
After these two commands, the logs from Kibana container show it successfully connects to Elastic:
{"type":"log","#timestamp":"2019-09-01T13:22:18Z","tags":["status","plugin:spaces#7.3.1","info"],"pid":6,"state":"green","message":"Status changed from yellow to green - Ready","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
I can also go to Kibana dashboard on http://localhost:5601 as well as Elastic on http://localhost:9200 both function properly
I then set up filebeat:
docker run --network=lognetwork docker.elastic.co/beats/filebeat:7.3.1 setup -E setup.kibana.host=kibana:5601 -E output.elasticsearch.hosts=["elasticsearch:9200"]
I can see both Elastic and Kibana container logs and returning 200. The logs on the Filebeat container show:
Index setup finished.
Loading dashboards (Kibana must be running and reachable)
Loaded dashboards
Loaded machine learning job configurations
Loaded Ingest pipelines
Finally, I pull the default config from Elastic site, launch Filebeat and attach to the container
curl -L -O https://raw.githubusercontent.com/elastic/beats/7.3/deploy/docker/filebeat.docker.yml
docker run -d --network=lognetwork --name=filebeat --user=root --volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro" --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" --volume="/var/run/docker.sock:/var/run/docker.sock:ro" docker.elastic.co/beats/filebeat:7.3.1 filebeat -e -strict.perms=false -E output.elasticsearch.hosts=["elasticsearch:9200"]
docker attach filebeat
I can see Filebeat sending monitoring pulse but when it does, elastic logs do not show anything new.
To test, I launch Docker "hello-world" which generates several lines of logs
docker run hello-world
Filebeat shows the following log
2019-09-01T13:30:40.624Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/460cc8c215ff69ecf28685c9cf89c0e56d0b3e4f680b8bf29beb5b570ebb7a14/*-json.log]
2019-09-01T13:30:40.624Z INFO input/input.go:114 Starting input of type: container; ID: 16402101064670842079
I then go to http://localhost:5601
Results:
Kibana shows no logs. Clicking for "check for new data" does not show anything either.
The folder /var/lib/docker/containers is also empty. The path returned by filebeat log (/var/lib/docker/containers/460cc8c215ff69ecf28685c9cf89c0e56d0b3e4f680b8bf29beb5b570ebb7a14/) does not seem to exist.
Expected:
- Kibana to show the "hello world" docker container logs
- To see a log file under /var/lib/docker/containers
What am I missing?
Thank you,
Olivier
Well, it took me many hours before asking on SO, and of course, 30mn after asking I found the answer.
The trick was to check where the logs were created as running Docker-Desktop on WSL2 is slightly different than running Docker on Linux.
docker inspect filebeat | grep LogPath
returns:
"LogPath": "/var/data/docker-desktop/default/daemon-data/containers/fd56c5e43c9206baaadd33d3a711e523107622450d0deafb498e7940d809f779/fd56c5e43c9206baaadd33d3a711e523107622450d0deafb498e7940d809f779-json.log
Then changing the volume map accordingly volume="/var/data/docker-desktop/default/daemon-data/containers:/var/lib/docker/containers:ro" when launching filebeat did the job:
docker run -d
--network=lognetwork
--name=filebeat
--user=root
--volume="$(pwd)/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro"
--volume="/var/data/docker-desktop/default/daemon-data/containers:/var/lib/docker/containers:ro"
--volume="/var/run/docker.sock:/var/run/docker.sock:ro"
docker.elastic.co/beats/filebeat:7.3.1 filebeat -e -strict.perms=false -E output.elasticsearch.hosts=["elasticsearch:9200"]
The logs are now properly shown on kibana
In my case :
Docker desktop installed in Windows 10 + WSL2 enabled in docker.
I was trying to use file beat to collect logs of all docker containers.
ELK + Filebeat were also running as docker containers.
The pipeline: Filebeat -> logstash -> elastic search -> kibana
Problem: Filebeat was not finding logs from docker. But from a local mounted folder it was sending logs to ELK and was showing up in kibana.
Solution: I was running docker-compose up from wsl bash shell. Instead I ran the same from windows powershell, or cmd and the logs from docker containers started to appear in kibana.
In docker-compose file:
filebeat:
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker:/var/lib/docker
- ./MYLOG_TEST:/usr/share/filebeat/mylog
- ./MY_filebeat.yml:/usr/share/filebeat/filebeat.yml
and in MY_filebeat.yml:
filebeat.inputs:
#for docker logs
- type: container # for older filestream version use docker as type
enabled: true
paths:
- /var/lib/docker/containers/**/*.log
#for my test log files
- type: log # for filebeat latest versions8.1+, use filestream as type
enabled: true
paths:
- /usr/share/filebeat/mylog/*.log

Docker not exposing port when using command line

I am trying to start an ASP.NET Core container hosting a website.
It does not exposes the ports when using the following command line
docker run my-image-name -d -p --expose 80
or
docker run my-image-name -d -p 80
Upon startup, the log will show :
Now listening on: http://[::]:80
So I assume the application is not bound to a specific address.
But does work when using the following docker compose file
version: '0.1'
services:
website:
container_name: "aspnetcore-website"
image: aspnetcoredocker
ports:
- '80:80'
expose:
- '80'
You need to make sure to pass all options (-d -p 80) to the docker command before naming the image as described in the docker run docs. The notation is:
docker run [OPTIONS] IMAGE[:TAG|#DIGEST] [COMMAND] [ARG...]
So please try the following:
docker run -d -p 80 my-image-name
Otherwise the parameters are used as command/args inside the container. So basically running your entrypoint of the docker image with the additional params of -d -p 80 instead of passing them to the docker command itself. So in your example the docker daemon is just not receiving the params -d and -p 80 and thus not mapping the port to the host. You can also notice that by not receiving the -d the command runs in the foreground and you see the logs in your terminal.

Connect Kibana container with Elasticsearch

I've a VM which contains Docker and Elasticsearch (OS: Centos7). I would like to create a Kibana docker and connect with my ES.
The ES contains indices, if I type curl -s http://localhost:9200/_cat/indices I got the list of indices.
I used Dockerfile to create my Kibana image:
docker build -t="kibana_test" .
docker run --name kibana -e
ELASTICSEARCH_URL=http://#IP:9200 -e
XPACK_SECURITY_ENABLED=false -p 5600:5601 -d kibana_test
Well, if I put the address IP of my machine, I got this :
plugin:elasticsearch#6.2.4 Request Timeout after 3000ms
And in my Docker logs I got thi message:
License information from the X-Pack plugin could not be obtained from
Elasticsearch for the [data] cluster
How can I resolve this problem ?
Thanks for advance!
So, configure in elasticsearch.yml file.
network.host: 0.0.0.0
transport.host: localhost
transport.tcp.port: 9300
Then restart elasticsearh service first,
When build kibana container :
use this:
-e ELASTICSEARCH_URL=http://172.17.0.1:9200
check again.

How can I Publish a jupyter tmpnb server?

I'm trying to publish a tmpnb server, but am stuck. Following the Quickstart at http://github.com/jupyter/tmpnb, I can run the server locally and access it at 172.17.0.1:8000.
However, I can't access the server remotely. I've tried adding -p 8000:8000 when I create the proxy container with the following command:
docker run -it -p 8000:8000 --net=host -d -e CONFIGPROXY_AUTH_TOKEN=$TOKEN --name=proxy jupyter/configurable-http-proxy --default-target http://127.0.0.1:9999
I tried to access the server by typing the machine's IP address:8000 but my browser still returns "This site can't be reached."
The logs for proxy are:
docker logs --details 45d836f98450
08:33:20.981 - info: [ConfigProxy] Proxying http://*:8000 to http://127.0.0.1:9999
08:33:20.988 - info: [ConfigProxy] Proxy API at http://localhost:8001/api/routes
To verify that I can access other servers run on the same machine I tried the following command: docker run -d -it --rm -p 8888:8888 jupyter/minimal-notebook and was able to accessed it remotely at the machine's ip address:8888.
What am I missing?
I'm working on an Ubuntu 16.04 machine with Docker 17.03.0-ce
Thanks
Create file named docker-compose.yml with content following, then you can launch the container with docker-compose up. Since images will be directly pulled errors will be arrested.
httpproxy:
image: jupyter/configurable-http-proxy
environment:
CONFIGPROXY_AUTH_TOKEN: 716238957362948752139417234
container_name: tmpnb-proxy
net: "host"
command: --default-target http://127.0.0.1:9999
ports:
- 8000:8000
tmpnb_orchestrate:
image: jupyter/tmpnb
net: "host"
container_name: tmpnb_orchestrate
environment:
CONFIGPROXY_AUTH_TOKEN: $TOKEN$
volumes:
- /var/run/docker.sock:/docker.sock
command: python orchestrate.py --command='jupyter notebook --no-browser --port {port} --ip=0.0.0.0 --NotebookApp.base_url=/{base_path} --NotebookApp.port_retries=0 --NotebookApp.token="" --NotebookApp.disable_check_xsrf=True'
A solution is available from the github.com/jupyter/tmpnb README.md file. At the end of the file under the heading "Development" three commands are listed:
git clone https://github.com/jupyter/tmpnb.git
cd tmpnb
make dev
These commands clone the tmpnb repository, cd into the tmpnb repository, and run the "dev" command from the the makefile contained in the tmpnb repository. On my machine, entering those commands created a notebook on a temporary server that I could access remotely. Beware that the "make dev" command deletes potentially conflicting docker containers as part of the launching process.
Some insight into how this works can be gained by looking inside the makefile. When the configurable-http-proxy image is run on Docker, both port 8000 and 8001 are published, and the tmpnb image is run with CONFIGPROXY_ENDPOINT=http://proxy:8001

How to use SERVICE_CHECK_HTTP with progrium/consul check-http script?

I am running the progrium/consul container with the gliderlabs/registrator container. I would like to be able to automatically create health checks for any container that is registered to consul with the registrator. Using this I would like to use consul health checks to know if any container has stopped running. I have read that there is a way to do this by adding environmental variables, but everything I have read has been far too vague, such as the post below:
how to define HTTP health check in a consul container for a service on the same host?
So I am supposed to set some environmental variables:
ENV SERVICE_CHECK_HTTP=/howareyou
ENV SERVICE_CHECK_INTERVAL=5s
Do I set them inside of my progrium/consul container or my gliderlabs/registrator? Would I set them by just adding the following tags inside my docker run command like this?
docker run ...... -e SERVICE_CHECK_HTTP=howareyou -e SERVICE_CHECK_INTERVAL=5s ......
Note: for some reason adding the above environmental variables to the docker run commands of my registrator just caused consul to think my nodes are failing from no acks received
I got Consul Health Checks and Gliderlabs Registrator working in three ways with my Spring Boot apps:
Put the environment variables in the Dockerfile with ENV or LABEL
Put the environment variables using -e with docker run
Put the environment variables into docker-compose.yml under "environment" or "labels"
Dockerfile
In your Dockerfile-file:
ENV SERVICE_NAME MyApp
ENV SERVICE_8080_CHECK_HTTP /health
ENV SERVICE_8080_CHECK_INTERVAL 60s
The /health endpoint here is coming from the Spring Boot Actuator lib that I simply put in my pom.xml file in my Spring Boot application. You can however use any other endpoint as well.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
docker run
docker run -d -e "SERVICE_NAME=myapp" -e "SERVICE_8080_CHECK_HTTP=/health" -e "SERVICE_8080_CHECK_INTERVAL=10s" -p 8080:8080 --name MyApp myapp
Make sure that you are using the correct HTTP server port and that it is accessible. In my case, Spring Boot uses 8080 by default.
Docker Compose
Add the health check information under either the "environment" or "labels" properties:
myapp:
image: apps/myapp
restart: always
environment:
- SERVICE_NAME=MyApp
- SERVICE_8080_CHECK_HTTP=/health
- SERVICE_8080_CHECK_INTERVAL=60s
ports:
- "8080:8080"
Starting Consul Server
docker run -d -p "8500:8500" -h "consul" --name consul gliderlabs/consul-server -server -bootstrap
The "gliderlabs/consul-server" image activates the Consul UI by default. So you don't have to specify any other parameters.
Then start Registrator
docker run -d \
--name=registrator \
-h $(docker-machine ip dockervm) \
-v=/var/run/docker.sock:/tmp/docker.sock \
gliderlabs/registrator:v6 -resync 120 -deregister on-success \
consul://$(docker-machine ip dockervm):8500
The "resync" and "deregister" parameters will ensure that Consul and Registrator will be in synch.

Resources