Are there any standard algorithm to distribute the work among the slaves in the master/slave pattern? I have master which distributes job among the slaves via queue for now. But If I have to put job priority into the picture what kind of data structure/algorithm can be used?
Looking for some suggestion.
Thanks in advance,
Bibek Dawadi
You could use a Heap/Priority Queue.
Related
I want to have a Jenkins job to monitor other Jenkins instances of performance.
so that I have details of all the instances at once place.
Any suggestions in that regard.
https://medium.com/#eng.mohamed.m.saeed/monitoring-jenkins-with-grafana-and-prometheus-a7e037cbb376
This would be a good solution for what you're trying to achieve.
You can have a dashboard for each instance inside Grafana
We have different jobs running on our jenkins. Some jobs are heavy and taking a lot of CPU and RAM, some are not. So I would like to have some plugins to help me set the weight for those jobs, just like https://wiki.jenkins-ci.org/display/JENKINS/Heavy+Job+Plugin.
But since we are using Jenkins pipeline which is not supported by Heavy Job plugin (See https://issues.jenkins-ci.org/browse/JENKINS-41940). Is there any other equivalent for pipeline jobs just like that?
Not as flexible as a dynamic weight, but to avoid overload you can create several executors with a different label (such as many with label light and only a few with label heavy) and then use node to target these labels.
This is not a solution to the packing problem, only prevents too many jobs with a heavy class from running at the same time.
trying to solve some problem with Mesos. I have three build servers for Jenkins. Jenkins schedules jobs on them through Mesos.
For now, Mesos loads one agent(slave) as hard as possible, but I want it to spread jobs across all agents..
As I see, it's better to run three jobs on three agents, than on one.
Is it possible to randomise job scheduling?
Or perhaps, I have such scenario. 2 large servers and one mini. I want to schedule Jobs on mini by default, and if it's not enough resources, then proceed to large servers. How can I achieve this goal? Is it possible to set priority for agents(slaves) to specify on which agent I want job to run at first?
The Mesos plugin for Jenkins attempts to build on the most recently built slave (see this method). This means that once it builds on that machine once, as long as that machine still has available spare resources - it'll schedule additional jobs on that machine until it is full. Right now it looks like that isn't optional (I have filed it as a feature request).
I have a dedicated Jenkins server that I am using to kick off build-and-static-analysis jobs. I want to make sure that the server is highly utilized but I also want to make sure that no single project is monopolizing the build and scan time. How can I configure Jenkins to cycle through my projects in a round robin fashion?
Take a look at the priority sorter plugin
I would like to setup the jenkins build farm in such a way that the fail over of all nodes are handled including the master. I am not able to find an answer for the question "what if master node goes down?".
I need a hot-hot setup where in if one node goes down the other should be able to pick up the queue and when both are up, they should process the jobs exclusively. Can we setup two nodes where each node is master and slave to each other? Does that create a serious conflict?
I am new to Jenkins distributed build setup and your help is much appreciated.
Thanks!
There is the CloudBees Jenkins Enterprise product http://jenkins-enterprise.cloudbees.com/docs/user-guide-bundle/ha-sect-failover.html which is able to deal with a master failure. That is a commercial product but if High Availability is a requirement then it could be worth the price.