Kibana is not connecting to elasticsearch docker - docker

Question

New to ELK stack + docker.
Trying to setup ELK setup locally in docker.
The command used is
docker network create elasticnew --driver=bridge
docker run --network=elasticnew --name elasticsearchnode -p 9200:9200 -p 9300:9300 docker.elastic.co/elasticsearch/elasticsearch:6.6.1
docker run --network=elasticnew --name kibana -e ELASTICSEARCH_URL=http://elasticsearchnode:9200 -p 5601:5601 docker.elastic.co/kibana/kibana:6.6.1
ElasticSeach is up and http://localhost:9200/ is giving json response.
But the kibana url (http://localhost:5601/) says "Kibana server is not ready yet"
and the request keeps on spinning in browser.
While querying for docker container, it says,
CONTAINER ID:76fe82529fa9
IMAGE:docker.elastic.co/kibana/kibana:6.6.1
CREATED:16 minutes ago
STATUS:Up 15 minutes
PORTS:0.0.0.0:5601->5601/tcp
NAMES:kibana
Edit on findings:-
1.Initially the command
docker logs kibana
gives empty response immediately. Later on issuing this command, taking time exponentially to reply (Did not see the logs after 5 mins too.)
2.While localhost:9200 in browser too, replicates the above behaviour.
(It seems something like, kibana is making the elastic search busy)
3.Did inspect on kibana container, and it also says the status as "running". Hereby shared logs too.
C:\Windows\system32>docker inspect kibana
[
{
"Id": "da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52",
"Created": "2019-02-25T18:39:43.1796454Z",
"Path": "/usr/local/bin/kibana-docker",
"Args": [],
"State": {
"Status": "running",
"Running": true,
"Paused": false,
"Restarting": false,
"OOMKilled": false,
"Dead": false,
"Pid": 3398,
"ExitCode": 0,
"Error": "",
"StartedAt": "2019-02-25T18:39:48.7600134Z",
"FinishedAt": "0001-01-01T00:00:00Z"
},
"Image": "sha256:b94222148a00695eb94bcf9fe5cce02547ffd963531709de15187bf8ade13ea0",
"ResolvConfPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/resolv.conf",
"HostnamePath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/hostname",
"HostsPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/hosts",
"LogPath": "/var/lib/docker/containers/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52/da312a8bfbe5e47586ac7539f124500945663a0bad61d029a72147c0ead44a52-json.log",
"Name": "/kibana",
"RestartCount": 0,
"Driver": "overlay2",
"Platform": "linux",
"MountLabel": "",
"ProcessLabel": "",
"AppArmorProfile": "",
"ExecIDs": null,
"HostConfig": {
"Binds": null,
"ContainerIDFile": "",
"LogConfig": {
"Type": "json-file",
"Config": {}
},
"NetworkMode": "elasticnew",
"PortBindings": {
"5601/tcp": [
{
"HostIp": "",
"HostPort": "5601"
}
]
},
"RestartPolicy": {
"Name": "no",
"MaximumRetryCount": 0
},
"AutoRemove": false,
"VolumeDriver": "",
"VolumesFrom": null,
"CapAdd": null,
"CapDrop": null,
"Dns": [],
"DnsOptions": [],
"DnsSearch": [],
"ExtraHosts": null,
"GroupAdd": null,
"IpcMode": "shareable",
"Cgroup": "",
"Links": null,
"OomScoreAdj": 0,
"PidMode": "",
"Privileged": false,
"PublishAllPorts": false,
"ReadonlyRootfs": false,
"SecurityOpt": null,
"UTSMode": "",
"UsernsMode": "",
"ShmSize": 67108864,
"Runtime": "runc",
"ConsoleSize": [
30,
120
],
"Isolation": "",
"CpuShares": 0,
"Memory": 0,
"NanoCpus": 0,
"CgroupParent": "",
"BlkioWeight": 0,
"BlkioWeightDevice": [],
"BlkioDeviceReadBps": null,
"BlkioDeviceWriteBps": null,
"BlkioDeviceReadIOps": null,
"BlkioDeviceWriteIOps": null,
"CpuPeriod": 0,
"CpuQuota": 0,
"CpuRealtimePeriod": 0,
"CpuRealtimeRuntime": 0,
"CpusetCpus": "",
"CpusetMems": "",
"Devices": [],
"DeviceCgroupRules": null,
"DiskQuota": 0,
"KernelMemory": 0,
"MemoryReservation": 0,
"MemorySwap": 0,
"MemorySwappiness": null,
"OomKillDisable": false,
"PidsLimit": 0,
"Ulimits": null,
"CpuCount": 0,
"CpuPercent": 0,
"IOMaximumIOps": 0,
"IOMaximumBandwidth": 0,
"MaskedPaths": [
"/proc/asound",
"/proc/acpi",
"/proc/kcore",
"/proc/keys",
"/proc/latency_stats",
"/proc/timer_list",
"/proc/timer_stats",
"/proc/sched_debug",
"/proc/scsi",
"/sys/firmware"
],
"ReadonlyPaths": [
"/proc/bus",
"/proc/fs",
"/proc/irq",
"/proc/sys",
"/proc/sysrq-trigger"
]
},
"GraphDriver": {
"Data": {
"LowerDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d-init/diff:/var/lib/docker/overlay2/7afef12f62d8f016bd357422310f4cf3cf58f5b66b4bc03294684e970682a71f/diff:/var/lib/docker/overlay2/e86e9fc46fd2c18605165003912fd99161c6826dea9650fe3f7c591a06a13529/diff:/var/lib/docker/overlay2/a7e09d4df14208f4b48adf12b155a4ba0c1fb2ce5f66e2f4c2e2f2c6f030b9e2/diff:/var/lib/docker/overlay2/aaf3b5122fcff237d0f90bfe3d6ba778c4ff036f6d00db5aa6d301083038026d/diff:/var/lib/docker/overlay2/497685f1ed275b7ad5b6ca23beb3840542abb8e5ba38ac51d9b8fb20ec007c4b/diff:/var/lib/docker/overlay2/532a837d610aaeb5c0e5b98a2dc4df1d899a46559b8e4429ff14a25b5ba60c9a/diff:/var/lib/docker/overlay2/b94ac41fbb4f0941a19161f1d3289d218de10fc36c2037d0667249caf65e049b/diff:/var/lib/docker/overlay2/c9fe17bf9817837aec687bf5e14eeb1e9997ef92c4873c8cd9355d57a75aa71c/diff:/var/lib/docker/overlay2/6e3558bcf051abf5cf3066775f43e81d565158e3b2a9cb817a501b47bc7679a5/diff",
"MergedDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/merged",
"UpperDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/diff",
"WorkDir": "/var/lib/docker/overlay2/156be4deb6525acdb0b6b32d757c5da721271aa61010fac1c3bdc89e8793d63d/work"
},
"Name": "overlay2"
},
"Mounts": [],
"Config": {
"Hostname": "da312a8bfbe5",
"Domainname": "",
"User": "1000",
"AttachStdin": false,
"AttachStdout": true,
"AttachStderr": true,
"ExposedPorts": {
"5601/tcp": {}
},
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"ELASTICSEARCH_URL=http://elasticsearchnode:9200",
"PATH=/usr/share/kibana/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"ELASTIC_CONTAINER=true"
],
"Cmd": [
"/usr/local/bin/kibana-docker"
],
"ArgsEscaped": true,
"Image": "docker.elastic.co/kibana/kibana:6.6.1",
"Volumes": null,
"WorkingDir": "/usr/share/kibana",
"Entrypoint": null,
"OnBuild": null,
"Labels": {
"license": "Elastic License",
"org.label-schema.build-date": "20181205",
"org.label-schema.license": "GPLv2",
"org.label-schema.name": "kibana",
"org.label-schema.schema-version": "1.0",
"org.label-schema.url": "https://www.elastic.co/products/kibana",
"org.label-schema.vcs-url": "https://github.com/elastic/kibana-docker",
"org.label-schema.vendor": "Elastic",
"org.label-schema.version": "6.6.1"
}
},
"NetworkSettings": {
"Bridge": "",
"SandboxID": "3bb18d0b036874dabdc526080d6cf25ac3e53147b7b2fd44ce77ce58b6104900",
"HairpinMode": false,
"LinkLocalIPv6Address": "",
"LinkLocalIPv6PrefixLen": 0,
"Ports": {
"5601/tcp": [
{
"HostIp": "0.0.0.0",
"HostPort": "5601"
}
]
},
"SandboxKey": "/var/run/docker/netns/3bb18d0b0368",
"SecondaryIPAddresses": null,
"SecondaryIPv6Addresses": null,
"EndpointID": "",
"Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"IPAddress": "",
"IPPrefixLen": 0,
"IPv6Gateway": "",
"MacAddress": "",
"Networks": {
"elasticnew": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"da312a8bfbe5"
],
"NetworkID": "d6e0ffb7617a1a2dd4f1a15c10f7255ed560c980ad6ccf1fb573a292d4515a9c",
"EndpointID": "91359e830287666cf834372691e2761ea239450c832551360af5b7870546a869",
"Gateway": "172.19.0.1",
"IPAddress": "172.19.0.3",
"IPPrefixLen": 16,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:ac:13:00:03",
"DriverOpts": null
}
}
}
}
]
Regret for those big logs.
PS: I am using Win 10 Enterprise, 6GB of RAM and after running the kibana and elastic search docker containers, the RAM usage was close to 70% and above.
Could some one share some input here on
Why kibana fails to connect with elastic search?
Why elastic search too fails to respond after kibana fails to run?

Answer 1

In my case the problem was due to the naming of my container... Notice --name elasticsearch here:
docker run -d --name elasticsearch --net elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.4.0
docker run -d --name kibana --net elasticsearch -p 5601:5601 kibana:7.4.0
I was giving the container another name and kibana was complaining that it was not able to connect. Changed it to elasticsearch then waited like 1 minute for the time for kibana to create its indexes ad install plugins.
Just check kibanas' logs until it says "Server running at http://0:5601"
docker logs kibana

Answer 2

Unfortunatly the following tricks worked.
Removed the mobylinux vm and started the docker
Used 6.6.0 version of both kibana and docker
This made the kibana to respond in 5601 port in browser.
Will try to remove the mobylinux and will try for the 6.6.1 version.
Any suggesstion are welcome.

Answer 3

If you come across this issue, making sure the Kibana and Elasticsearch tags are the same helped me 😅. E.g elasticsearch:5.6.10 goes with kibana:5.6.10. Happy coding. ✌🏽

Answer 4

I faced the same issue today for some time.
I used docker tools instead of docker desktop. I started the elastic search as well as kibana with two different network names.
docker network create elstack
docker run -d --name elstack --net ela-p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:tag
docker network create kibana
docker run -d --name kibana --net kibana -p 5601:5601 kibana:tag
Docker will identify container names as IP addresses.
So I got subject line issue, later I started both containers with the same network, and the issue got resolved and both are working fine now.