I would like to take suggestion and inputs from wider audience regarding an issue with Jenkins master
I would like to have Jenkins setup in the following way to have the other backup so that it runs even if the master is down!
my plan for Jenkins master
The black arrows indicate that it is the primary configuration for Jenkins1 master and if it is down, we can switch apache to communicate with Jenkins2 instantly as a hot backup
It is possible and can be achieved and is helpful while performing OS upgrades or any other maintenance activities on master machine
I am planning to have it in such a way that to have a set of Jenkins master, which can be used in a manner, that the load is delegated to each master from Apache and entire Jenkins is not offline when we restart a master as other masters are also online
Please let me know a better way to achieve this if you have any
CloudBees have already addressed this issue with an Enterprise plugin for HA
The High Availability plugin, offered within CloudBees Jenkins
Enterprise, eliminates downtime due to master failures. Multiple
Jenkins masters act as backups waiting for a primary master failure.
Once a failure is detected, a backup master automatically boots up and
acts as a failover.The best part? Project work continues.
This is a paid solution but if you have a mission critical configuration then it might be worth the cost
Related
I am running jenkins multi branch job, suddenly it not allow me to change the configuration changes, its keep on loading without any timeout issue.
Can you please some one help me on this ?
You could have a look at the Jenkins master machine CPU and memory. Look what is consuming them. I have seen this happening when the CPU is nearly 100 %. In this case, restarting the Jenkins process or Jenkins master machine could help.
Try to remember/ask colleagues if there are any recent changes to Jenkins master machine. We had similar issues after installing plugins.
Avoid executing jobs on Jenkins master, use slave agents.
You may need to clean up old builds if you are not doing this already.
in my case, after disabling / enabling all plugins one by one, it was the "AWS SQS Build Trigger Plugin", causing the "save / apply" buttons to move, and not be functional
I’d like to set Jenkins in fault tolerant mode, so if one of masters is down, second master should pick up failed job and re-trigger it.
I don’t have to care about slaves because slaves are on-demand Docker containers.
Any ideas about how it could be done?
Any ideas about how it (Jenkins HA) could be done?
CloudBees provides a commercial Jenkins HA solution.
We are looking at Open source Jenkins masters failover scenario, and currently backing up jenkins jobs and configurations using SCM sync plugin. any ideas on how to restore Jenkins for high availablity
when master goes down.
Docker images work great for this. In essence the master is just an image which you configure with all your jobs. Logging of course should not be stored on the docker image but piped to AWS S3 or some datastore.
Each job you run launches a new docker slave to handle that task. Offers HA with lots of room for horizontal scaling.
If docker/containers is not your thing, configuration management is the way to go (chef, puppet, ansible). Take your pick and use these tools to build out your consistent Jenkins master and restore from latest backup.
I have
a usual jenkins master jettysburg:8888/jenkins,
and a failover jenkins master jettyperry:8888/jenkins.
I am planning to use mercurial to keep the jobs directory of the two masters in sync.
Usually, most of the time, new jobs and builds are defined and executed at jettysburg:8888. Then I would need to sync jettyperry:8888 with whatever took place in jettysburg:8888, and I plan to perform the push once a day.
Which means, after a failover to jettyperry:8888, the push would be performed the converse direction.
I have been using mercurial for my non-programming/coding files (like word, excel and text files) for that purpose, as a means to perform incremental and redundant backup of my "mission critical" files, as well as versioning them.
I am also hoping to depend on mercurial to backout from changes made to jenkins jobs.
Is using mercurial to sync two Jenkins masters a good idea? Is there a better way to keep two Jenkins servers in sync? In this case, I am syncing only the jobs tree.
As long as you don't need shared job state between the servers (they run in their own little universes) and you keep the same plugin modules and libraries on both Jenkins servers, using some form version control to keep the actual job definitions is fine.
My office does this with git. We have a development and a production set of jenkins servers. We maintain a base linux image with jenkins install with all necesarry modules and locally install libraries (like nodejs and such). We then spin up an instance of the image and pull down the jobs.
The one thing that can be a challenge is keeping in sync things like credentials and Jenkins config settings--you might need to keep them as part of the base image.
If you need the job queues to persist and be shared (like a master-master setup) you can look at this plugin, which allows multiple jenkins masters to share the same job queue: https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin
There are two approaches to this:
Instead of approaching a solution with hot back-up why not consider clustered master so you get an active-active solution. You may want to look at https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin. This helps you cluster a bunch of masters so one going down should not be an issue.
Consider running Jenkins on containers, and externalize jenkins projects directory as external volumes in an NFS so you can bring up another container when one goes down - keeping both containers running will be a challenge with concurrent writes (if there are any).
Hope this helps.
I'm new to Jenkins and Continous Integration, and I noticed that it supports master / slave nodes. I really don't know what that means.
Can someone please tell in what scenario do I need slave agent?
Here is a scenario:
Our main Jenkins master is running on Windows machine (yes I know... I know...). We are doing iOS mobile development. There are some things that can only be done using Xcode (which only runs on Mac OS). I have a Jenkins Slave running on that Mac, that takes care of executing those tasks that can only run on a Mac.
Why not just setup a new instance on that Mac? Cause that job is tied together with other jobs (on Master) in dependencies and the flow. Even promotions on those Xcode tasks are run on Master.
Jenkins' Master / Slave architecture is used to manage distributed builds.
There are many different scenarios you might want to use a distributed build system. It is all based on your projects load and dependencies.
Pretty much, the Master is what you're probably currently using, and is responsible for scheduling builds, dispatching jobs to slaves, and monitoring the results, but can also execute jobs itself. A slave is a java executable that sits on a remote server waiting for instructions from the master (to execute build).
To use this functionality in Jenkins, go to "Manage Jenkins" screen, and click on "Manage Nodes"
https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds
For a more in depth summary of distributed builds with Jenkins, as well as some scenarios where this system would be useful, and how to implement it, please read chapter 11: Distributed Builds of Jenkins: The Definitive Guide by John Ferguson Smart
http://wakaleo.com/books/jenkins-the-definitive-guide/download-jtdg-pdf