I have to add the Vega plugin to kibana. I am using docker-compose for it.
This is original source of docker-compose.yml.
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
networks:
- elastic
kib01:
image: docker.elastic.co/kibana/kibana:7.16.3
container_name: kib01
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: '["http://es01:9200","http://es02:9200","http://es03:9200"]'
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
This is working well.
And I added the following command to kibana image. I've found this article while several times of exploring.
command: sh -c './bin/kibana-plugin install https://github.com/nyurik/kibana-vega-vis/releases/download/v1.2.0/vega_vis-1.2.0--for-Kibana-6.2.0.zip && exec /usr/local/bin/kibana-docker'
The result is the following.
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.16.3
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
networks:
- elastic
kib01:
image: docker.elastic.co/kibana/kibana:7.16.3
container_name: kib01
command: sh -c './bin/kibana-plugin install https://github.com/nyurik/kibana-vega-vis/releases/download/v1.2.0/vega_vis-1.2.0--for-Kibana-6.2.0.zip && exec /usr/local/bin/kibana-docker'
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: '["http://es01:9200","http://es02:9200","http://es03:9200"]'
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
This is not working. Here is the result.
Also, http://localhost:5601 is not running.
How can I add the vega plugin to kibana image?
Build your own Kibana docker image on top of the official one
Here is Elastic's version: https://github.com/elastic/dockerfiles/blob/8.0/kibana/Dockerfile
It will be something like:
FROM docker.elastic.co/elasticsearch/elasticsearch:7.16.3
COPY /path/to/vega_vis-1.2.0--for-Kibana-6.2.0.zip /some/path
RUN sh -c '/some/path/bin/kibana-plugin install...
and then
docker build -t my_awesome_kibana .
Related
Using the docker-compose.yml file by here
(online changed version 7.9.1. to 7.9.2)
as seen below full content of docker-compose.yml;
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
ports:
- 9201:9201
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
ports:
- 9202:9202
networks:
- elastic
kib01:
image: docker.elastic.co/kibana/kibana:7.9.2
container_name: kib01
ports:
- 5601:5601
environment:
ELASTICSEARCH_URL: http://es01:9200
ELASTICSEARCH_HOSTS: http://es01:9200
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
when run docker-compose up here the logs in command window;
It seems that both containers are started at same time so kibana is unable to identify elasticsearch container.
in Kibana service: Add below parameters in docker-compose.yml file
depends_on:
es01
environment:
'ELASTICSEARCH_HOSTS=http://es01:9200'
'XPACK_MONITORING_UI_CONTAINER_ELASTICSEARCH.ENABLED=false'
Instead of dict. try to use lists in env. variable
Trying to start elastic search cluster using docker-compose given here
I am not sure what is the issue here. I am running this docker-compose.yaml file in ubuntu 18.04 LTS. Tried find this error online but no help.
docker-compose.yaml
##############################################################################################
# LINKS - https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
##############################################################################################
version: "2.2"
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
container_name: es01_dev
environment:
- node.name=es01
- cluster.name="es-docker-cluster"
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /data/var/lib/elasticsearch-server_01:/usr/share/elasticsearch/data
# - /data/var/elasticsearch-server-backup:/var/elasticsearch-backup
# - /opt/elasticsearch-server/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- 9200:9200
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
container_name: es02_dev
environment:
- node.name=es02
- cluster.name="es-docker-cluster"
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /data/var/lib/elasticsearch-server_02:/usr/share/elasticsearch/data
# - /data/var/elasticsearch-server-backup:/var/elasticsearch-backup
# - /opt/elasticsearch-server/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
container_name: es03
environment:
- node.name=es03
- cluster.name="es-docker-cluster"
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock:=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- /data/var/lib/elasticsearch-server_03:/usr/share/elasticsearch/data
# - /data/var/elasticsearch-server-backup:/var/elasticsearch-backup
# - /opt/elasticsearch-server/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
networks:
- elastic
volumes:
/data/var/lib/elasticsearch-server_01:
driver: local
/data/var/lib/elasticsearch-server_02:
driver: local
/data/var/lib/elasticsearch-server_03:
driver: local
networks:
elastic:
driver: bridge
But docker-compose up given me this error
visionary#instance-2:/opt/elasticsearch-server$ docker-compose up
ERROR: The Compose file './docker-compose.yml' is invalid because:
volumes value '/data/var/lib/elasticsearch-server_01', '/data/var/lib/elasticsearch-server_02', '/data/var/lib/elasticsearch-server_03' do not match any of the regexes: u'^[a-zA-Z0-9._-]+$'
services.es01.ports value ['9200:9200', '9200:9200'] has non-unique elements
Not sure what is the issue here. Any help is appreciated.
I'm novice with elasticsearch and Docker. I found a great utility to filter data to final users "Search UI": https://github.com/elastic/search-ui
I see that all services I need to manage my "ES" network are available with Docker, but Search UI not. My question is, how can I add Search UI to my docker compose file?
Here is my actual docker-compose file:
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9201:9200
- 9301:9300
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
ports:
- 9202:9200
- 9302:9300
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.6.1
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
ports:
- 9203:9200
- 9303:9300
networks:
- elastic
cerebro:
image: lmenezes/cerebro:0.8.5
container_name: cerebro-01
ports:
- "9000:9000"
external_links:
- es01:es01
- es02:es02
- es03:es03
networks:
- elastic
kibana:
image: docker.elastic.co/kibana/kibana:7.6.1
container_name: kibana-01
ports:
- "5601:5601"
environment:
ELASTICSEARCH_HOSTS: http://es01:9200
networks:
- elastic
appsearch:
image: docker.elastic.co/app-search/app-search:7.6.1
container_name: appsearch-01
environment:
- "elasticsearch.host=http://es01:9200"
- "allow_es_settings_modification=true"
- "JAVA_OPTS=-Xms2g -Xmx2g"
ports:
- 3002:3002
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
Any suggestions?
Thanks!
I'd like to run docker cluster on multiple hosts using docker / docker-compose
I could define 3 containers es1, es2, es3 and then run each containers in each host I guess.
(I'm not sure how I'll make them discover each other)
Of course, it'll be huge pain to start/restart cluster sshing onto 3 machines.
is it possible to manage multi-host docker somehow?
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-compose-file gives an example but it runs on single host I believe
version: '2.2'
services:
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.5.1
container_name: es01
environment:
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es02,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- elastic
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.5.1
container_name: es02
environment:
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es03
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data02:/usr/share/elasticsearch/data
networks:
- elastic
es03:
image: docker.elastic.co/elasticsearch/elasticsearch:7.5.1
container_name: es03
environment:
- node.name=es03
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02,es03
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- data03:/usr/share/elasticsearch/data
networks:
- elastic
volumes:
data01:
driver: local
data02:
driver: local
data03:
driver: local
networks:
elastic:
driver: bridge
I have this docker-compose.yml config:
version: '2.2'
services:
kibana:
image: docker.elastic.co/kibana/kibana:7.3.1
environment:
SERVER_NAME: kibana.interos.io
ELASTICSEARCH_HOSTS: http://elasticsearch.example.org
es01:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.1
container_name: es01
environment:
- node.name=es01
- discovery.seed_hosts=es02
- cluster.initial_master_nodes=es01,es02
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx2512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata01:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- esnet
es02:
image: docker.elastic.co/elasticsearch/elasticsearch:7.3.1
container_name: es02
environment:
- node.name=es02
- discovery.seed_hosts=es01
- cluster.initial_master_nodes=es01,es02
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx2512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata02:/usr/share/elasticsearch/data
networks:
- esnet
volumes:
esdata01:
driver: local
esdata02:
driver: local
networks:
esnet:
how to can I tell Kibana to use the same docker network as the elasticsearch containers? I assume I have to add
networks:
- esnet
to the kibana container, anything else I need to do? What is the best practice here?
Your assumption is right. The kibana service needs to look like this:
kibana:
image: docker.elastic.co/kibana/kibana:7.3.1
networks:
- esnet
environment:
SERVER_NAME: kibana.interos.io
ELASTICSEARCH_HOSTS: http://elasticsearch.example.org