I run a docker build on my unix build machine and therefore I need more than 2GB Memory (default value of docker engine). I got the build working on my Mac with the docker Desktop UI in the settings as you can see it in the image.
How is this possible in Unix?
If you want to increase the allocated memory and CPU while running a container you can try this:
docker run -it --cpus="2" --memory="4096m" ubuntu /bin/bash
You can also do the same while deploying services using docker-compose
More infos:
https://docs.docker.com/config/containers/resource_constraints/
How to specify Memory & CPU limit in docker compose version 3
Related
I'm attempting to increase the memory allocation of a specific container I'm running on an EC2 instance. I was able to do this locally by adding the mem_limit: 4GB into my docker-compose file (using version 2 not 3) and this did not work until I changed my settings in Docker desktop to be greater than the memory limit I was specifying:
My question is as follows, is it possible to change this memory slider setting from the command line and therefore would it be possible to do it on an EC2 instance and without docker desktop? I've been through the docs but was unable to find anything specific to this!
That's a Docker Desktop setting, which is only necessary because of the way docker containers run in a VM on Windows and Mac computers. On an EC2 Linux server there is no limit like that, docker processes can use as much resources as the server has available.
I am using a docker linux container on a windows 10 machine. I have the following options specified in advanced tab of docker gui:
and yet my docker container is just using 14GiB of memory.
I want to be able to use all the memory in docker i can after leaving a safe amount for windows processes. I wont be using the ram for anything other than in docker and what windows will use to run.
I installed Docker (version 18.06.1-ce-mac73) on a MacBook Pro with macOS High Sierra (version 10.13.6). When I go to Preferences -> Advanced in Docker GUI, I see that memory limit is set to 4GB and CPUs are set to 2. However, running docker info from the terminal shows that total memory is 995.6MiB and CPUs is 1.
It seems that the 995.6MiB limit is enforced, because I'm trying to build a project inside a container and by checking docker stats I see that it runs out of memory when 995.6MiB limit is reached.
Shouldn't docker info match the GUI configuration?
Your docker command is pointed at a VirtualBox VM, probably one installed via Docker Machine or Docker Toolbox. The Docker for Mac "whale" app uses a different virtualization system. (Check: if you echo $DOCKER_HOST, does it say something like tcp://192.168.99.100:2376?)
You can "deactivate" the Docker VM in your shell by running
eval $(docker-machine env -u)
docker info
and having done that can docker-machine rm the VM.
Lets say I am running a multiprocessing service inside a docker container spawning multiple processes, would docker use all/multiple cores/CPUs of the host or just one?
As Charles mentions, by default all can be used, or you can limit it per container using the --cpuset-cpus parameter.
docker run --cpuset-cpus="0-2" myapp:latest
That would restrict the container to 3 CPU's (0, 1, and 2). See the docker run docs for more details.
The preferred way to limit CPU usage of containers is with a fractional limit on CPUs:
docker run --cpus 2.5 myapp:latest
That would limit your container to 2.5 cores on the host.
Lastly, if you run docker inside of a VM, including Docker for Mac, Docker for Windows, and docker-machine, those VM's will have a CPU limit separate from your laptop itself. Docker runs inside of that VM and will use all the resources given to the VM itself. E.g. with Docker for Mac you have the following menu:
Maybe your host VM has only one core by default. Therefore you should increase your VM cpu-count first and then use --cpuset-cpus option to increase your docker cores. You can remove docker default VM using the following command then you can create another VM with optional cpu-count and memory size.:
docker-machine rm default
docker-machine create -d virtualbox --virtualbox-cpu-count=8 --virtualbox-memory=4096 --virtualbox-disk-size=50000 default
After this step you can specify number of cores before running your image. this command will use 4 cores of total 8 cores.
docker run -it --cpuset-cpus="0-3" your_image_name
Then you can check number of available core in your image using this command:
nproc
I'm using Tensorflow on windows 10 with docker (yes, I know Windows 10 isn't supported yet). It performs ok, but only looks like I am only accessing just one of my cpu cores (I have 8). Tensorflow has the ability to assign ops to different devices, so I'd like to be able to get access to all 8. In VirtualBox when I view the settings it only says there is 1 cpu out of the 8 that is configured for the machine. I tried editing the machine to set it to more, but that lead to all sorts of weirdness.
Does anyone know the right way to either create or restart a docker machine to have 8 CPUs? I'm using the docker quickstart container app.
Cheers!!
First you need to ensure you have enabled Virtualization for your machine. You have to do that in the BIOS of your computer.
The link below has a nice video on how to do that, but there are others as well if you google it:
https://www.youtube.com/watch?v=mFJYpT7L5ag
Then you have to stop the docker machine (i.e. the VirtualBox vm) and change the CPU configuration in VirtualBox.
To list the name of your docker machine (it is usually default) run:
docker-machine ls
Then stop the docker machine:
docker-machine stop <machine name>
Next open VirtualBox UI and change the number of CPUs:
Select the docker virtual machine (should be marked as Powered off)
Click Settings->Systems->Processors
Change the number of CPUs
Click OK to save your changes
Restart the docker machine:
docker-machine start <machine name>
Finally you can use the CPU constraint options available for docker run command to restrict CPU usage for your containers if desired.
For example the following command restrict container to use only 3 CPUs:
docker run -ti --cpuset-cpus="0-2" ubuntu:14.04 /bin/bash
More details available in the docker run reference document here.
I just create the machine with all cpus
docker-machine create -d virtualbox --virtualbox-cpu-count=-1 dev
-1 means use all available cpus.