I have two different pipeline steps each requiring a different amount of memory. One needs 4096 (size: 2x) memory which has been defined but the other step needs significantly less memory. Is there any way to define the memory for each of the steps in order to not waste pipeline minutes?
Size can be configured for every step.
https://support.atlassian.com/bitbucket-cloud/docs/configure-bitbucket-pipelinesyml/
Related
I am new to Jenkins and I have configured master-slave nodes as shown below, but I need help to configure the no of executors in each of the below slave nodes
Currently, I have configured 100 executors in each slave nodes
How many no of executors I can configure in each slave node and what fact(memory, RAM, etc) need to take consider when increasing the no of executors?
Maximum how many no of executors I can configure in each server?
Well it totally depends on your usage. There are multiple factors such as how much cpu and memory is available, how the build are going to execute and what kind of builds, how frequent these build should run etc. But I can clearly say that 100 is too big number. I would suggest go with 20 builds (if builds run frequently and have fair amount of CPU and memory) first and observe if is there any issue with numbers or not then you can increase accordingly.
here is very nice article check this out https://www.avantica.com/blog/jenkins-balance-load-master-slave-setup#:~:text=Jobs%20are%20built%20using%20executors,to%20build%20two%20different%20tasks.
I am trying to adjust a build job within jenkins, the problem is, it keeps failing due to lack of memory, I've adjusted java xmx but it did not solve the problem.
Turns out, I have RAM memory limit within the worker, I tried running those commands as part of the build script : "free -m" and "cat /proc/meminfo" and they both confirmed that job is being run with 1GB RAM limit, the server has more but the build isn't using it and it keeps failing due to lack of memory.
Please help me fix this, how can I lift that limit ? thank you
Heap or Permgen?
There are two OutOfMemoryErrors which people usually encounter. The first is related to heap space: java.lang.OutOfMemoryError: Heap space When you see this, you need to increase the maximum heap space. You can do this by adding the following to your JVM arguments -Xmx200m where you replace the number 200 with the new heap size in megabytes.
The second is related to PermGen: java.lang.OutOfMemoryError: PermGen space. When you see this, you need to increase the maximum Permanent Generation space, which is used for things like class files and interned strings. You can do this by adding the following to your JVM arguments -XX:MaxPermSize=128m where you replace the number 128 with the new PermGen size in megabytes.
Also note:
Memory Requirements for the Master
The amount of memory Jenkins needs is largely dependent on many factors, which is why the RAM allotted for it can range from 200 MB for a small installation to 70+ GB for a single and massive Jenkins master. However, you should be able to estimate the RAM required based on your project build needs.
Each build node connection will take 2-3 threads, which equals about 2 MB or more of memory. You will also need to factor in CPU overhead for Jenkins if there are a lot of users who will be accessing the Jenkins user interface.
It is generally a bad practice to allocate executors on a master, as builds can quickly overload a master’s CPU/memory/etc and crash the instance, causing unnecessary downtime. Instead, it is advisable to set up agents that the Jenkins master can delegate jobs to, keeping the bulk of the work off of the master itself.
Finally, there is a monitoring plugin from Jenkins that you can use:
https://wiki.jenkins.io/display/JENKINS/Monitoring
Sources:
https://wiki.jenkins.io/display/JENKINS/Monitoring
https://wiki.jenkins.io/display/JENKINS/Builds+failing+with+OutOfMemoryErrors
https://www.jenkins.io/doc/book/hardware-recommendations/#:~:text=The%20amount%20of%20memory%20Jenkins,single%20and%20massive%20Jenkins%20master.
My Jenkins server is so slow. Java takes 120% or CPU. How can I give Jenkins more memory access.
Of what steps can I take to improve the load time for Jenkins
If Java memory is causing the problem, then you can add more heap via the -Xmx option, as suggested in Priyam's answer. By default, JVM limits heap to 25% of your available RAM.
More heap has a caveat, though: if you add heap in the range of several GB, then the default JVM garbage collection algorithm will periodically impose stop-the-world breaks in the range of several seconds. You then need to switch to a custom garbage collection algorithm (like, CMS) and then carefully tune its parameters.
If adding more heap does not fix your problem, then you need to dig deeper. There's a plethora of possible root causes for a slow master -- from JVM memory and garbage collection settings to plugin issues, on top of the usual CPU/disk/IO-dimensioning issues.
You can allocate more memory and swap heap sizes using the following commands.
These can be set in the job configuration or Jenkins -> Manage -> Configure
-Xmx512m
-XX:MaxPermSize=128m
Manage Jenkins -> Configure System - > Robot Framework
Deselect checkbox as -> [Display "Robot Results" column in the job list view ]
Please look here for detailed screenshot
Is there any difference between I create two slaves, or one slave with two executors on the same Windows server?
Yes, there is a difference: It's about memory consumption and effort of maintenance/administration.
Starting a slave on a system starts a (main) process. This process costs (private) main memory to run and connects to the master.
Each executor is a sub-process of the main process.
It is therefore apparent that running two executors on one slave costs less memory in total compared to running two slaves (with one executor each), as there would be the memory consumption of the main process twice:
2 * Main Processes + 2 * Executors > 1 * Main Process + 2 * Executors
Moreover, administrating a slave is some more effort than just an executor: Whilst an executor has virtually nothing to worry, there are numerous things to configure for a slave. Additionally, the capabilities of the two slaves are anyhow the same (they are running on the same OS as you said), so there is little value-add to also assign it different labels.
In short, if there are no other boundary conditions, which make me do it differently, I always would prefer running two executors on one slave, as this is easier to administrate and some memory is saved.
A slave is a "machine". An executor is an "OS Process" in the slave.
So ideally we always add executors - they do the work and can run in parallel, and the simple theoretic answer to your question is "2 executors on one slave"
In practice we need to add slaves in several use cases:
We need more resources (more cpu, more memory, more "machines")
We need a different setting (Different OSes, Different hardware)
We have global resources that would create a conflict for executors on same machine (shared browser for a UI testing process)
Make the decision based on your use case.
One benefit which immediately comes to my mind for running 1 executor on given node, is to prevent conflicts between processes run at the same time.
On other hand you could prevent job conflicts using existing Jenkins plugins, ie. Heavy Job, Build Blocker.
In Jenkins I have 100 java projects. Each has its own build file.
Every time I want clear the build file and compile all source files again.
Using bulkbuilder plugin I tried compling all the jobs.. Having 100 jobs run parallel.
But performance is very bad. Individually if the job takes 1 min. in the batch it takes 20mins. More the batch size more the time it takes.. I am running this on powerful server so no problem of memory and CPU.
Please Suggest me how to over come this.. what configurations need to be done in jenkins.
I am launching jenkins using war file.
Thanks..
Even though you say you have enough memory and CPU resources, you seem to imply there is some kind of bottleneck when you increase the number of parallel running jobs. I think this is understandable. Even though I am not a java developer, I think most of the java build tools are able to parallelize build internally. I.e. building a single job may well consume more than one CPU core and quite a lot of memory.
Because of this I suggest you need to monitor your build server and experiment with different batch sizes to find an optimal number. You should execute e.g. "vmstat 5" while builds are running and see if you have idle cpu left. Also keep an eye on the disk I/O. If you increase the batch size but disk I/O does not increase, you are consuming all of the I/O capacity and it probably will not help much if you increase the batch size.
When you have found the optimal batch size (i.e. how many executors to configure for the build server), you can maybe tweak other things to make things faster:
Try to spend as little time checking out code as possible. Instead of deleting workspace before build starts, configure the SCM plugin to remove files that are not under version control. If you use git, you can use a local reference repo or do a shallow clone or something like that.
You can also try to speed things up by using SSD disks
You can get more servers, run Jenkins slaves on them and utilize the cpu and I/O capacity of multiple servers instead of only one.