My docker container used to start successfully. As of this weekend it fails pulling elasticsearch. Please help (Windows 10 64bit)
> docker-compose up
Pulling elastic (elasticsearch:7.3.1)...
7.3.1: Pulling from library/elasticsearch
ERROR: no matching manifest for windows/amd64 10.0.17763 in the manifest list entries
> docker manifest inspect -v library/elasticsearch:latest
no such manifest: docker.io/library/elasticsearch:latest
Part of the docker-compose.yml
elastic:
image: elasticsearch:7.3.1
restart: always
environment:
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.security.enabled=false
- Elogger.level=TRACE
- discovery.type=single-node
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata:/data/elasticsearch
ports:
- "9201:9200"
networks:
Can you check this post Docker: "no matching manifest for windows/amd64 in the manifest list entries"
Right click Docker icon in the Windows System Tray
Go to Settings
Daemon
Advanced
Set the "experimental": true
Restart Docker
no idea why you have library/elasticsearch:7.3.1 in your compose file.. according to official elasticsearch images the path is simply elasticsearch:7.3.1 - feel free to edit thet compose file and removelibrary/` part from image name
Related
I am a windows user. I installed Windows Subsystem for Linux [wsl2] and then installed docker using it. Then I tried to get started with OpenSearch so I followed the documentation in the given link
https://opensearch.org/downloads.html and run docker-compose up, In the shell, I am getting an error message like
opensearch-dashboards | {"type":"log","#timestamp":"2022-01-18T16:31:18Z","tags":["error","opensearch","data"],"pid":1,"message":"[ConnectionError]: getaddrinfo EAI_AGAIN opensearch-node1 opensearch-node1:9200"}
In the port http://localhost:5601/ I am getting messages like
OpenSearch Dashboards server is not ready yet
I also changed resources preference for memory to 5GB in docker-desktop but it still doesn't work. Can somebody help me with this?
After 5 days having issues with opensearch I've found something working fine for me:
Set docker memory to 4GB
And docker vm.max_map_count = 262144
Then I use previous versions of opensearch because the latest does not seems stable:
opensearchproject/opensearch:1.2.3
opensearchproject/opensearch-dashboards:1.1.0
opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.2
Here is my docker-compose.yml file:
version: '3'
services:
opensearch-node1A:
image: opensearchproject/opensearch:1.2.3
container_name: opensearch-node1A
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node1A
- discovery.seed_hosts=opensearch-node1A,opensearch-node2A
- cluster.initial_master_nodes=opensearch-node1A,opensearch-node2A
- bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
hard: 65536
volumes:
- opensearch-data1:/usr/share/opensearch/data
ports:
- 9200:9200
- 9600:9600 # required for Performance Analyzer
networks:
- opensearch-net
opensearch-node2A:
image: opensearchproject/opensearch:1.2.3
container_name: opensearch-node2A
environment:
- cluster.name=opensearch-cluster
- node.name=opensearch-node2A
- discovery.seed_hosts=opensearch-node1A,opensearch-node2A
- cluster.initial_master_nodes=opensearch-node1A,opensearch-node2A
- bootstrap.memory_lock=true
- "OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- opensearch-data2:/usr/share/opensearch/data
networks:
- opensearch-net
opensearch-dashboardsA:
image: opensearchproject/opensearch-dashboards:1.1.0
container_name: opensearch-dashboardsA
ports:
- 5601:5601
expose:
- "5601"
environment:
OPENSEARCH_HOSTS: '["https://opensearch-node1A:9200","https://opensearch-node2A:9200"]'
networks:
- opensearch-net
logstash-with-plugin:
image: opensearchproject/logstash-oss-with-opensearch-output-plugin:7.16.2
container_name: logstash-with-plugin
networks:
- opensearch-net
volumes:
opensearch-data1:
opensearch-data2:
networks:
opensearch-net:
I had the same error message when opening "http://localhost:5601/" while testing opensearch and opensearch dasboard locally using Docker in Windows 10:
OpenSearch Dashboards server is not ready yet
opensearch-dashboards |
{"type":"log","#timestamp":"2022-02-10T12:29:35Z","tags":["error","opensearch","data"],"pid":1,"message":"[ConnectionError]:
getaddrinfo EAI_AGAIN opensearch-node1 opensearch-node1:9200"}
But when looking into the log I also found this other error:
opensearch-node1 | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
The 3 part solution working for me was:
Part 1
On each opensearch nodes update the file:
/usr/share/opensearch/config/opensearch.yml
And add line:
plugins.security.disabled: true
Before the security plugins:
cks. "Single-node" mode disables them again.
#discovery.type: single-node
plugins.security.disabled: true
######## Start OpenSearch Security Demo Configuration ########
# WARNING: revise all the lines below before you go into production
plugins.security.ssl.transport.pemcert_filepath: esnode.pem
I found the information on opensearch official documentation
Part 2
Setting allocated memory for docker desktop to 4GB into .wslconfig more information here:
opendistrocommunity discussion
stackoverflow aloocate memory
Make sure your allocated memory is well set up (you have to restart docker desktop) with this command: docker info and check the line "Total Memory" it should be set to 4GB (approximately, in my case it has be set to 3.84GiB)
Part 3
And also increase vm.max_map_count:
open powershell
wsl -d docker-desktop
echo "vm.max_map_count = 262144" > /etc/sysctl.d/99-docker-desktop.conf
The info was founded here on github discussion
I had the same issue with my Opensearch-dashboards instance installed on VM without Docker usage. The problem was caused by wrong setting for connection to search engine in the opensearch-dashboards.yml file. I mixed up https and http protocols here (there was mismatch between settings of opensearch and opensearch-dashboards):
opensearch.hosts: [https://localhost:9200]
Unable to pull Elasticsearch and Kibana images by using docker compose.
When I was trying to retry muliple times using docker-compose up cmd, each and every time some of the service are not available, which is unpredictable.
Can somebody please guide me what causing the issue, even the proxy has been set in docker.service.
Please find the attached screenshot, I have also given the docker-compose.yaml file for reference.
Kindly let me know in case of any further information needed.
Docker-compose.yml File
version: '2.2'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.8.0
container_name: elasticsearch
environment:
- node.name=elasticsearch
- discovery.seed_hosts=elasticsearch
- cluster.initial_master_nodes=elasticsearch
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
kibana:
image: docker.elastic.co/kibana/kibana:7.8.0
container_name: kibana
environment:
ELASTICSEARCH_URL: "http://elasticsearch:9200"
ports:
- 5601:5601
depends_on:
- elasticsearch
volumes:
esdata1:
driver: local
It was issue with RHEL server trying with multiple times the issue got solved
I have 2 docker-compose files that I need to run together, the locations of the files are like
/home/project1/docker-compose.yml
and
/home/project2/docker-compose.yml
so clearly both the services should have different contextpath
But when I run below docker compose command
docker-compose -f /home/project1/docker-compose.yml -f /home/project2/docker-compose.yml config
I get to see, Both the service are getting the same context path
app:
build:
context: /home/project1
dockerfile: Dockerfile
app2:
build:
context: /home/project1
dockerfile: Dockerfile
How can I resolve this issue I want both my services to have their own project path ie.
app service should have context path /home/project1
and
app2 service should have context path /home/project2
They way I found could run multiple services is:
1. First of all create images for all the services with the help of docker build command
ex: in my case it is a java application with maven build tool, so command I used:
mvn clean package docker:build -Denv=$ENV -DskipTests
2. After all the images built, create a common docker-compose file which will look like this
version: '3'
services:
service1:
image: <service1-image>
ports:
- "8100:8080"
environment:
- TZ=Asia/Kolkata
- MYSQL_USER=root
- MYSQL_PASSWORD=unroot
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
service2:
image: <service2-image>
ports:
- "8101:8080"
environment:
- TZ=Asia/Kolkata
- MYSQL_USER=root
- MYSQL_PASSWORD=unroot
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
service3:
image: <service3-image>
environment:
- TZ=Asia/Kolkata
- MYSQL_USER=root
- MYSQL_PASSWORD=unroot
ulimits:
nproc: 65535
nofile:
soft: 65535
hard: 65535
networks:
default:
external:
name: dev
The way I have mentioned mysql root, password, you can also add environment variables.
3.Then run below command to run all the services
docker-compose -f docker-compose.yml up
or run below command to run specific service
docker-compose -f docker-compose.yml up service1
This is working fine for me, this will require 1 time setup but after that it will be very easy and fast.
Option 1:
Use absolute paths for the contexts in both docker-compose files
Option 2:
Create a docker-compose.override.yml with the absolute paths:
version: "3"
services:
service1:
build:
context: /home/project1
service2:
build:
context: /home/project2
and include it in the docker-compose command:
docker-compose -f /home/project1/docker-compose.yml -f /home/project2/docker-compose.yml -f /home/docker-compose.override.yml config
On linux, to avoid hard-coding of the base path in the docker-compose.override.yml, you can use PWD environment variable:
services:
service1:
build:
context: ${PWD}/project1
service2:
build:
context: ${PWD}/project2
It seems that when using multiple docker-compose files the context is taken from the location of the first file. It is explained in detail in this issue
https://github.com/docker/compose/issues/3874
I have currently tried to use docker-compose to spin up multiple elasticsearch containers but these are all deployed on my local machine. What would be the process for deploying multiple containers to several different machines using Docker?
I know that docker swarm exists but I am not sure how to utilize it along with docker compose so that I spin up multiple elasticsearch nodes that are located in the difference virtual machines that I have spun up.
So far I know that in the elasticsearch.yml config file I need to specify the virtual machine addresses with
discovery.zen.ping.unicast.hosts: ["vm_ip1:9200", "vm_ip2:9200"] but I am not sure how to enforce this change onto the docker containers that are created. To spin up the containers I used the sample docker-compose.yml file that is shown in the elasticsearch docs
version: '2.2'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.2.4
container_name: elasticsearch
environment:
- cluster.name=elasticsearch
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata1:/usr/share/elasticsearch/data
ports:
- 9200:9200
networks:
- esnet
elasticsearch2:
image: docker.elastic.co/elasticsearch/elasticsearch:6.2.4
container_name: elasticsearch2
environment:
- cluster.name=elasticsearchr
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.zen.ping.unicast.hosts=elasticsearch"
- "index.number_of_shards: 2"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
- esdata2:/usr/share/elasticsearch/data
networks:
Please let me know if any more details are needed, thanks!
I really need help on this error I don't understand why I get this error. Thanks
docker -v
Docker version 1.13.1, build 092cba3
docker-compose -v
docker-compose version 1.11.1, build 7c5d5e4
this is my dockerfile
version: '2.0'
services:
arcgis-server:
container_name: "arcgis-server"
image: "arcgis-server:10.4.1"
volumes:
- "./license:/license"
- "./arcgisserver:/arcgis/server/usr/directories"
- "./config-store:/arcgis/server/usr/config-store"
build:
context: .
dockerfile: "Dockerfile"
ulimits:
nproc: 25059
nofile:
soft: 65535
hard: 65535
ports:
- "127.0.0.1:6080:6080"
- "127.0.0.1:6443:6443"
- "4001:4001"
- "4002:4002"
- "4004:4004"
stdin_open: true
tty: true
here is the error
docker-compose build
ERROR: The Compose file './docker-compose.yml' is invalid because:
Additional properties are not allowed ('tty' was unexpected)
You might be seeing this error because you're using the wrong Compose file version. Either specify a supported version ("2.0", "2.1", "3.0") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
Actually I test on the old machine it worked fine. I would appreciate your helps. Thanks again!!!!
tty needs to be defined as a setting on your service, not at the top level. Yaml files are space sensitive, so removing the leading spaces puts the setting at the top level where it's not valid. Use the following syntax to fix it:
version: '2.0'
services:
arcgis-server:
container_name: "arcgis-server"
image: "arcgis-server:10.4.1"
volumes:
- "./license:/license"
- "./arcgisserver:/arcgis/server/usr/directories"
- "./config-store:/arcgis/server/usr/config-store"
build:
context: .
dockerfile: "Dockerfile"
ulimits:
nproc: 25059
nofile:
soft: 65535
hard: 65535
ports:
- "127.0.0.1:6080:6080"
- "127.0.0.1:6443:6443"
- "4001:4001"
- "4002:4002"
- "4004:4004"
stdin_open: true
tty: true