Allocating more memory to Docker Enterprise Preview Edition - docker

Question

I am running Docker Enterprise Preview Edition on Windows Server 2019 and have managed to pull and run the docker-compose.yml file below. However shortly afterwards the container shuts down and when I run the command docker-compose logs it shows me the insufficient memory issue below:
Docker-compose file
version: '3.7'
services:
elasticsearch:
container_name: elasticsearch
# image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.1
deploy:
resources:
limits:
cpus: 0.25
memory: 4096m
ports:
- 9200:9200
volumes:
- C:\DockerContainers\Elasticsearch\data:/usr/share/elasticsearch/data
- C:\DockerContainers\Elasticsearch\config\certs:/usr/share/elasticsearch/config/certs
environment:
- xpack.monitoring.enabled=true
- xpack.watcher.enabled=true
- ES_JAVA_OPTS=-Xms512m -Xmx512m
- discovery.type=single-node
# networks:
# - elastic
kibana:
container_name: kibana
# image: docker.elastic.co/kibana/kibana:7.9.2
image: docker.elastic.co/kibana/kibana:7.17.1
deploy:
resources:
limits:
cpus: 0.25
memory: 4096m
ports:
- 5601:5601
volumes:
- C:\DockerContainers\Elasticsearch\Kibana\config\certs:/usr/share/kibana/config/certs
depends_on:
- elasticsearch
# networks:
# - elastic
# networks:
# elastic:
# driver: nat
Docker logs
elasticsearch | # There is insufficient memory for the Java Runtime Environment to continue.
elasticsearch | # Native memory allocation (mmap) failed to map 65536 bytes for committing reserved memory.
elasticsearch | # An error report file with more information is saved as:
elasticsearch | # logs/hs_err_pid7.log
I read on the elasticsearch Docker guideline that it needs at least 4GB RAM. I have included the RAM limit in the docker compose yml file but it doesn't seem to take effect. Does anyone know how to set the memory usage for Docker which is running on Windows Server 2019?

Answer 1

I ran into this same issue trying to start Docker using almost exactly the same configuration as you, but doing so from Windows 10 instead of Windows Server 2019. I suspect the issue isn't the memory configuration for the Elastic containers, but rather the Docker host itself needs to be tweaked.
I'm not sure how to go about increasing memory for the Docker host when running Docker Enterprise Preview Edition, but for something like Docker Desktop, this can be done by changing the memory limit afforded to it by adjusting the Settings -> Advanced -> Memory slider. Docker Desktop may need to be restarted afterwards for this change to take effect.
Similarly, if you are running a Docker machine, like I am, it may be necessary to recreate it, but with more than the default 1GB that is allotted to new Docker machine instances. Something like this:
docker-machine create -d vmwareworkstation --vmwareworkstation-memory-size 8192 default
Swap in whatever vm type makes sense for you (e.g., VirtualBox) and remember that you need to run docker-machine env | Invoke-Expression in each console window in which you intend to run Docker commands.
I saw definite improvement after giving the Docker host more breathing room, as the memory-related errors disappeared. However, the containers still failed to start due to the following error:
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
This is a known issue (see https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#_set_vm_max_map_count_to_at_least_262144).
Again, if you're using Docker Desktop with WSL2 installed, you can run this:
wsl -d docker-desktop sysctl -w vm.max_map_count=262144
sysctl -w vm.max_map_count=262144
For a Docker machine, you'll need to ssh into it and set the vm.max_map_count directly:
docker-machine env | Invoke-Expression
docker-machine ssh default
sudo sysctl -w vm.max_map_count=262144
The above change to the Docker machine will last during your current session but will be lost after reboot. You can make it stick by:
adding sysctl -w vm.max_map_count=262144 to the /var/lib/boot2docker/bootlocal.sh file. Note this file may not exist before your changes.
running chmod +x /var/lib/boot2docker/bootlocal.sh
exit ssh and restart the Docker machine via docker-machine restart default.
to confirm the change, run docker-machine ssh default sudo sysctl vm.max_map_count. You should see it set to 262144.
After these changes, I was able to bring the elastic containers up. You can smoke test this by issuing a GET request for http://localhost:9200 in either Postman or curl. If you're using Docker machine, the ports you've set up are accessible to the machine, but not to your Windows box. To be able to connect to port 9200, you'll need to set up port forwarding via the docker-machine ssh -f -N -L 9200:localhost:9200 command.
Also, consider adding the following to your Docker compose file:
environment:
- bootstrap.memory_lock=true
ulimits:
memlock:
soft: -1
hard: -1
Hope this helps!

Answer 2

This is a late response but still if it's useful for anyone out there.
In fact in windows server, Docker desktop is the best option for running Docker. If you use Docker enterprise edition in windows server 2019, it has some restrictions in memory (RAM) allocation to container if you have not purchased the license keys to operate. Max to max it will allocate 9.7Mb of memory to container. You can confirm this by using below command
Docker stats <container id>
Here you cannot see the MEM_USAGE/LIMIT column at all in the output like Docker desktop.
So when your container request memory more than 9.7Mb, it will go down. Also you cannot use the deploy option in compose file to reserve memory for specific container in case of enterprise edition in windows server 2019. It will throw error 'memory reservation not supported'. However mem_limit will accept by the compose file but again it will not allocate the mentioned memory.
Note - The advantage of Docker enterprise edition is that it will always run in the background even if the user log off the server. But for Docker desktop, it will stop running upon user log off.