We see there are environmental variables available to read to get port/service/revision of a Cloud Run service.
In order to configure a multithreaded app, like gulp threads. How do you read how many cores are available?
Snippet:
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
CMD exec gunicorn --bind :$PORT --workers 1 --threads 8 --timeout 0 main:app
I believe this information is not available from within Cloud Run.
If you're looking to access multiple CPU cores in Cloud Run, you actually specify the number of cores during configuration. This is a constant value, so you don't need to check for it at runtime, so there is no feature to do so.
https://cloud.google.com/run/docs/configuring/cpu
By default Cloud Run container instances are allocated 1 CPU instance. You can change this number.
Related
I'm looking to build a Docker image to run FastAPI on Google Cloud Run. FastAPI uses Uvicorn as an ASGI server and Uvicorn recommend using Gunicorn with the Uvicorn worker class for production deployments. FastAPI themselves also have some excellent documentation on using Gunicorn with Uvicorn. I even see that FastAPI provide an official image combining the two (
uvicorn-gunicorn-fastapi-docker) but this comes with a warning:
You are probably using Kubernetes or similar tools. In that case, you
probably don't need this image (or any other similar base image). You
are probably better off building a Docker image from scratch
This warning basically explains that replication would be handled at cluster-level and doesn't need to be handled at process-level. This makes sense. I am however not quite sure if Cloud Run falls into this category? Essentially it is an abstracted and managed Knative service which therefore runs on Kubernetes.
My question is, should I be installing Gunicorn along with Uvicorn in my Dockerfile and handling replication at process-level? Along the lines of:
CMD ["gunicorn", "app.main:app", "-w", "4", "-k", "uvicorn.workers.UvicornWorker", "--bind", "0.0.0.0:80"]
Or should I stick with Uvicorn, a single process, and let Cloud Run (Kubernetes) handle replication at cluster-level? E.g.
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]
A. Let's go the big scale first.
At the time of writing Cloud Run instances can be set to a maximum of 1000 concurrency requests. And CPUs can be set to 4 vCPUs.
Going back to basics a bit. Cloud Run will span many instances, each will work individually, and each could handle the maximum allowed concurrency, so each could handle 1000 requests. If you set multiple CPUs you need to handle multi processes.
When we talk about so large number we need to be cautious. If your container is big enough in CPU/Memory terms to handle this traffic, you may want to use a process manager (gunicorn) to start several Uvicorn threads (workers), as your referenced images do. So you can use the docker container.
B. Being on small scale.
On the other hand if you set 1 vCPU and be single threaded, you don't need gunicorn for process manager. You still can have concurrency enabled but not on the top level, maybe at the lower level, that fits your 1 vCPU model, something like 80 requests for concurrency. In this situation you will have on large traffic, many instances started by Cloud Run, and you rely at Cloud Run to spawn as many instances as needed, which does really nice. It's like a Kubernetes on top of your simple container.
I would suggest start with single process, build a container that doesn't use the referenced container, and only swap B to version A, when you know there are benefits(costs wise) to have larger instances.
I'm running my container with
docker run --rm -it --cpus=1 --memory "8G" -v "$pwd":"/code/" 'algolab' bash
This means according to the docs that the container will only use one cpu core.
--cpus=0.000 Number of CPUs.
Number is a fractional number. 0.000 means no limit.
But when I run nproc inside the container, it tells me it sees 8 cores - the same number as on my host.
This answer supports that claim. But it also mentions --cpuset-cpus="0-2". When I use that, in addition to --cpus=1, I get the result 3 from nproc.
Why is docker ignoring --cpus=1?
And how do I make it stop ignoring it?
For now, I can use --cpuset-cpus="0", but I don't understand why that's necessary.
$ docker --version
Docker version 19.03.6, build 369ce74a3c
The --cpus option sets a quota on the CPU usage - this does not restrict which CPUs can be used nor how many at a time; this restricts the total slice of time they can be running on your CPU. So if you pass --cpus 1 on a machine with 8 cores, the container will be limited to running on 1/8th of your CPU time. This is covered on a different section of the docs.
OK, so my title may not actually be linked to a possible solution, however this is my problem.
I am running a Python 3 Jupyter notebook inside a docker container in from my windows 10 kaby-lake (2 physical cores, 4 virtual cores) laptop.
I noticed while doing heavy computing from there, my CPU usage seen in the task monitor is very low (~15%).
When going on the details for each process, the VBoxHeadless.exe actually uses 24% of the processor, which matches docker stats command which yields 97-100% CPU usage, and therefore makes sense from a single-core operation point of view.
My actual issue is that even though on thread is filled in terms of CPU time, windows (I guess) does not decide that it may actually be useful to speed up the CPU, and therefore it runs at 1.7GHz (with other apps in high performance mode, I usually hit the max 3.5GHz that the computer is capable of).
Therefore, how can I induce the higher clock speeds (nominal 2.7GHz or max 3.5GHZ) (considering that they would probably double my single threaded speed) from docker itself or inside windows 10?
You need to configure the docker machine running docker. If you haven't created a custom one, the default docker machine named 'default' will only have access to one cpu.
You can check all the configuration for this docker-machine by running:
docker-machine inspect default
You need to purge this default machine and recreate it:
docker-machine rm default
docker-machine create -d virtualbox --virtualbox-disk-size "400000" --virtualbox-cpu-count "2" --virtualbox-memory "2048" default
You can check all the avaible configuration options for the machine by running:
docker-machine create --help
Defining CPU Shares can help you but not exactly.
CPU limits are based on shares as these shares are a weight between how much processing time one process should get compared to another. If a CPU is idle, then the process will use all the available resources. If a second process requires the CPU then the available CPU time will be shared based on the weighting.
e.g. The --cpu-shares parameter defines a share between 0-768. If a container defines a share of 768, while another defines a share of 256, the first container will have 50% share while the other one having 25% of the available share total.
Below the first container will be allowed to have 75% of the share. The second container will be limited to 25%.
docker run -d --name p1 --cpuset-cpus 0 --cpu-shares 768 image_name
docker run -d --name p2 --cpuset-cpus 0 --cpu-shares 256 image_name
sleep 5
docker stats --no-stream
docker rm -f p1 p2
It's important to note that a process can have 100% of the share, no matter defined weight, if no other processes are running.
I have created a container:
docker run -c=20 -i -t ubuntu:latest /bin/bash
I tried to use -c flag to control CPU usage and maximize it in 50 %. When I am running md5sum /dev/urandom inside container, it use up 100 % CPU in host machine.
The -c flag for docker run command modifies the container’s CPU share weighting relative to the weighting of all other running containers.
It does not restrict the container's use of CPU from the host machine.
You can use the --cpu-quota flag to limit CPU usage, for example:
$ docker run -ti --cpu-quota=50000 ubuntu:latest /bin/bash
The --cpu-quota is usually used in conjunction with --cpu-period. Please see more details on the Docker run reference document:
https://docs.docker.com/reference/run/#runtime-constraints-on-resources
It seems that you are running a single container, so this is the expected result.
You might find this blog post helpful.
Every new container will have 1024 shares of CPU by default. This
value does not mean anything, when speaking of it alone. But if we
start two containers and both will use 100% CPU, the CPU time will be
divided equally between the two containers because they both have the
same CPU shares (for the sake of simplicity I assume that there are no
other processes running).
Take a look here, this is apparently what you were looking for:
https://docs.docker.com/engine/reference/run/#cpu-period-constraint
The default CPU CFS (Completely Fair Scheduler) period is 100ms. We can use --cpu-period to set the period of CPUs to limit the container’s CPU usage. And usually --cpu-period should work with --cpu-quota.
Examples:
$ docker run -it --cpu-period=50000 --cpu-quota=25000 ubuntu:14.04 /bin/bash
If there is 1 CPU, this means the container can get 50% CPU worth of run-time every 50ms.
period and quota definition:
Within
each given "period" (microseconds), a group is allowed to consume only up to
"quota" microseconds of CPU time. When the CPU bandwidth consumption of a
group exceeds this limit (for that period), the tasks belonging to its
hierarchy will be throttled and are not allowed to run again until the next
period.
I have 4 CORE CPU. I wanted to restrict some 10 containers which I am running to 2 only cores and leave rest others free.
is it possible, how can I do so.
You can achieve this by using the cpuset constraints option when running the container.
Example from Docker reference docs:
$ docker run -ti --cpuset-cpus="1,3" ubuntu:14.04 /bin/bash
Meaning your container can run in CPUs 1 and 3 (0 and 2 will not be used).
There are other CPU parameters as well for the Docker run command.
Please see documentation for more details:
https://docs.docker.com/reference/run/#runtime-constraints-on-cpu-and-memory