I have Jenkins server whose GUI is super slow whenever I refresh it takes at least a few mins to respond. I am copying the entire Jenkins folder from one server to another and starting the Jenkins
Couple of scenarios I have tested
1. setting up Jenkins on high-performance server ie 64GB Ram 16Vcpus and 300GB of Harddrive
2. Increasing the Java Memory arguments of the server to min and max to 16GB and made sure it is using the G1GC Algorithm
Expecting Jenkins to be as fast as possible
Related
As per the documentation I am building and deploying my code to Cloud Run. I have configured the machine it's running on the have 2 CPU cores.
Since Cloud Run manages scaling automatically, will I get additional performance benefits from using the Node cluster module to utilize both CPU cores on the host machine?
If your code can leverage 2 (or more) CPU in the same time to process the same request, using more than 1 CPU makes sense.
If, as the majority of the developers, you use NodeJS as-is, a single-thread runtime, don't set 2 CPU on your cloud run service. Set one, and let Cloud Run scaling automatically the number of parallel instances.
At high level, it's like having a cluster of VM; or a big multi CPU VM with 1 thread per CPU. It's the power of horizontal scaling by keeping the code simple.
I am using a Docker Setup that consists of 14 different containers. Every container gets a cpu_limit of 2 and a mem_limit of 2g.
To create and run these containers, I've written a Python script that uses the docker-py library. As of now, the containers are created sequentially, which takes approximately 2 minutes.
Now I'm thinking about parallelizing the process. So now instead of doing (its pseudocode):
for container in containers_to_start:
create_container(container)
I do
from multiprocessing.dummy import Pool as ThreadPool
pool = ThreadPool(4)
pool.map(create_container, containers_to_start)
And as a result the 14 containers are created 2x faster. BUT: The applications within the containers take a significant longer time to boot. At the end of the day, i dont gain really much, the time until every application is reachable is more or less the same, no matter if with or without multithreading.
But I don't really know why, because every container gets the same amount of CPU and memory resources, so I would expect the same boot time no matter how many containers are starting at the same time. Clearly this is not the case. Maybe I'm missing some knowledge here, any explanation would be greatly appreciated.
System Specs
CPU: intel i7 # 2.90 GHz
32GB RAM
I am using Windows 10 with Docker installed in WSL2 backend.
I am wondering if someone has experience running jenkins on ec2,
I am currently running t3.medium and able only to run 10 executors after that I get timeouts. The jobs that is being run is getting a existing ami, unpack it put stuff in it pack it with terraform finish the process where the instance that is being raised im m5.large.
I am going for getting bigger machine from the Jenkins itself.
So I was looking at some AMD machines and my point is to be able to run like 50-100 executors. Can you recommend something for the point.
I was looking in to m6a.xlarge if it will do it or if something smaller can do it would be good enough?
The project in which I am working develops a Java service that uses MarkLogic 9 in the backend.
We are running a Jenkins build server that executes (amongst others) several tests in MarkLogic written in XQuery.
For those tests MarkLogic is running in a docker container on the Jenkins host (which is running Ubuntu Linux).
The Jenkins host has 12 GB of RAM and 8 GB of swap configured.
Recently I have noticed that the MarkLogic instance running in the container uses a huge amount of RAM (up to 10 GB).
As there are often other build jobs running in parallel, the Jenkins starts to swap, sometimes even eating up all swap
so that MarkLogic reports it cannot get more memory.
Obviously, this situation leads to failed builds quite often.
To analyse this further I made some tests on my PC running Docker for Windows and found out that the MarkLogic tests
can be run successfully with 5-6 GB RAM. The MarkLogic logs show that it sees all the host memory and wants to use everything.
But as we have other build processes running on that host this behaviour is not desirable.
My question: is there any possibility to tell the MarkLogic to not use so much memory?
We are preparing the docker image during the build, so we could modify some configuration, but it has to be scripted somehow.
The issue of the container not detecting memory limit correctly has been identified, and should be addressed in a forthcoming release.
In the meantime, you might be able to mitigate the issue by:
changing the group cache sizing from automatic to manual and setting cache sizes appropriate for the allocated resources. There area variety of ways to set these configs, whether deploying and settings configs from ml-gradle project, making your own Manage API REST calls, or programmatically:
admin:group-set-cache-sizing
admin:group-set-compressed-tree-cache-partitions
admin:group-set-compressed-tree-cache-size
admin:group-set-expanded-tree-cache-partitions
admin:group-set-expanded-tree-cache-size
admin:group-set-list-cache-partitions
admin:group-set-list-cache-size
reducing the in-memory-limit
in memory limit specifies the maximum number of fragments in an in-memory stand. An in-memory stand contains the latest version of any new or changed fragments. Periodically, in-memory stands are written to disk as a new stand in the forest. Also, if a stand accumulates a number of fragments beyond this limit, it is automatically saved to disk by a background thread.
Jenkins' webpage does speak about server specifications. The matter is that I have to ask to systems a server for CI, and I have to specify those requirements, and of course justify them.
I have to decide the following things:
Hard disk capacity, for the whole server, considering the OS. This spec is considered the more critical for the hardware providers.
RAM.
Number of cores.
And these are the things to take into account:
The OS they'll provide me will probably be Ubuntu Server.
I'm not going to run more than 1 build simultaneously, in the 99'9% of the cases.
I'm going to work with Moodle, so the source code size will be quite much (the whole repo is of about 700Mb).
Regarding my experience with Jenkins and Linux, I would recommend the following configuration:
A CentOS machine or VM (Ubuntu Server is OK too)
Minimum 2 CPU
Minimum 2 GB of RAM
A 30 GB partition for the OS
Another partition for Jenkins (like /jenkins)
Regarding the partition size for Jenkins, it depends of the number of jobs (and their workspaces size).
My Jenkins partition is 100 GB (I have around 100 jobs and some large Git repo to clone).