I'm getting my slave processes killed when running more than 300 clients per node.
The machines are hosted in Rackspace and the configuration of each one is:
512MB RAM
1 CPU
Should I increase the memory of the machines?
It's totally dependent on the code of your test scripts. Generally, I would say that it's more common to be limited by CPU before RAM, however in your case it sounds like it might be the Linux OOM Killer killing the processes. The reason for this could be that you have a memory leak in the test code, but it's really hard to tell without more info.
Also, on machines with only a single CPU core, you shouldn't run more than one Locust slave process.
Related
As per the documentation I am building and deploying my code to Cloud Run. I have configured the machine it's running on the have 2 CPU cores.
Since Cloud Run manages scaling automatically, will I get additional performance benefits from using the Node cluster module to utilize both CPU cores on the host machine?
If your code can leverage 2 (or more) CPU in the same time to process the same request, using more than 1 CPU makes sense.
If, as the majority of the developers, you use NodeJS as-is, a single-thread runtime, don't set 2 CPU on your cloud run service. Set one, and let Cloud Run scaling automatically the number of parallel instances.
At high level, it's like having a cluster of VM; or a big multi CPU VM with 1 thread per CPU. It's the power of horizontal scaling by keeping the code simple.
I am using a Docker Setup that consists of 14 different containers. Every container gets a cpu_limit of 2 and a mem_limit of 2g.
To create and run these containers, I've written a Python script that uses the docker-py library. As of now, the containers are created sequentially, which takes approximately 2 minutes.
Now I'm thinking about parallelizing the process. So now instead of doing (its pseudocode):
for container in containers_to_start:
create_container(container)
I do
from multiprocessing.dummy import Pool as ThreadPool
pool = ThreadPool(4)
pool.map(create_container, containers_to_start)
And as a result the 14 containers are created 2x faster. BUT: The applications within the containers take a significant longer time to boot. At the end of the day, i dont gain really much, the time until every application is reachable is more or less the same, no matter if with or without multithreading.
But I don't really know why, because every container gets the same amount of CPU and memory resources, so I would expect the same boot time no matter how many containers are starting at the same time. Clearly this is not the case. Maybe I'm missing some knowledge here, any explanation would be greatly appreciated.
System Specs
CPU: intel i7 # 2.90 GHz
32GB RAM
I am using Windows 10 with Docker installed in WSL2 backend.
I am running a micro-service application in docker container and have to test that using JMeter tool. So I am running JMeter on my host machine and my host machine has 4 cores. I allocate 2 cores to the container using --cpu=2 flag while running the container. so it means it can use up to 2 cores as per it needs while running. I leave the remaining 2 cores for the JMeter and other applications and system usage.
Here I need a clarification that what will happen if JMeter and other application needs more than 2 cores and container also needs allocated 2 cores fully ?
Is there any way to allocate 2 cores fully to the container? (It means any other applications or system can't use that 2 cores)
Thank you in advance.
The answer is most probably "no", the explanations will differ depending on your operating system.
You can try to implement this by playing with CPU affinity, however CPU is not only one metric you should be looking at, I would rather be concerned about RAM and Disk usage.
In general having load generator and application under test on the same physical machine is a very bad idea because they are both very resource intensive so consider using 2 separate machines for this otherwise both will suffer from context switches and you will not be able to monitor resources usage of JMeter and the application under test using JMeter PerfMon Plugin
We are running hundreds of Mule Java 8 apps in a machine currently. A single JVM appears to be running that is sharing hundreds of megs of Jars with the apps at run-time. If we were to run each app in a docker container would each container run a separate JVM at run-time? If so this would incur a major RAM penalty!
Yes, each container would run its own java process and thus its own JVM.
You may consider partitioning your apps though so that, rather than going from many apps on one server|VM to many containers each running one app on one server, that you go to some number of containers each running several apps on one server.
Yes, you'd have to duplicate shared jars for each container. Yes, you'd increase CPU, RAM and network traffic.
But, you'd gain more flexibility in duplicating app servers for scale-out and moving them onto different machines to better reflect their CPU, memory, bandwidth needs.
If I am running multiple docker containers with bursty memory and CPU utilization, will they be able to use the full capacity of the host machine? Or will they be limited to their CPU and memory limits of the individual container definitions?
For example:
If I were running 3 containers that burst to 1GB of memory once per day, at disjoint times.
And similarly if those same containers instead were CPU heavy, and bursted to 1CPU unit per day at disjoint times.
Could I run those 3 containers on a box with only 1.1GB of memory, or 1.1 CPU unit respectively?
Docker containers are not VM's,
They run in a cage over the host OS kernel, so there's no hypervisor magic behind.
Processes running inside a container are not much different from host processes from a kernel point of view. They are just highly isolated.
Memory and cpu scheduling will be handled by the "host". What you set on docker settings are CPU shares, to give priority and bounds to some containers.
So yes, containers with sleeping processes won't consume much cpu/memory if the used memory is correctly freed after the processing spike, otherwise, that memory would be swapped out, with no much performance impact.
Instantiating a docker container will only consume memory resources. As long as no process is running, you will see zero cpu usage by it.
I would recommend reviewing cgroups documentation, and actually docs for cgroups v2, since they are better structured that v1 docs. See chapter 5 for cpu and memory controllers: https://www.kernel.org/doc/Documentation/cgroup-v2.txt
When you don't need to explicitly specify --memory and --cpu-shares option at the container startup, the container will have all the cpu share and memory available for use on the instance. If no other process is consuming the resources,then the container can use all the cpu and memory available.
In theory you should be able to run the 3 containers on the instance.
Make sure non of the containers tie up the memory or cpu resources.