What's the best way to bulk update Jenkins projects? - jenkins

Question

We have hundreds of Jenkins projects (mostly created from a few templates), often need to make the same change to all of them. e.g. today I need to add a post-build step to delete workspace at the end. Next I need to change the step to copy build result to a shared drive to Nexus repository.
What's the best way to apply such kind of bulk change to Jenkins projects?

Answer 1

You could use Configuration Slicing Plugin which is designed to do this.
It supports many configuration options.

Answer 2

The REST API is quite powerful. The following sequence worked for me:
In loop for all relevant projects (list of projects is available via e.g. /api/xml?tree=jobs[name]):
download config.xml via /job/{name}/config.xml
edit using your favorite scripted xml editor (mine was xmlstarlet)
upload new config xml via /job/{name}/config.xml
Some random notes:
do *BACKUP* before doing anything
I probably could post some bash script example if anyone is interested
Good luck!
EDIT> Example bash script:
#!/bin/bash
jenkinsUrlBase='http://user:token#jenkins'
callJenkins() { # funcPath
curl --silent --show-error -g "${jenkinsUrlBase}${1}"
}
postJenkinsFile() { # funcPath fileName
curl --silent --show-error -g -d "#${2}" "${jenkinsUrlBase}${1}"
}
callJenkins '/api/xml?tree=jobs[name]' | xmlstarlet sel -t -v '//hudson/job/name' | while read projectName ; do
echo "Processing ${projectName}..."
origFile="${projectName}_old.xml"
newFile="${projectName}_new.xml"
callJenkins "/job/${projectName}/config.xml" > "$origFile"
echo " - Updating artifactory url..."
cat "$origFile" \
| xmlstarlet ed -P -u '//maven2-moduleset/publishers/org.jfrog.hudson.ArtifactoryRedeployPublisher/details/artifactoryUrl' -v "http://newServer/artifactory" \
> "${newFile}"
if false ; then
echo " - Commiting new config file..."
postJenkinsFile "/job/${projectName}/config.xml" "$newFile"
else
echo " - Dry run: not commiting new config file"
fi
done

Answer 3

Groovy is by far the best way to bulk update jobs. You may have to do a little digging into the jenkins / plugin api to figure out what api calls to make, but the script console (http://yourJenkinsUrl/script) provides an easy way to play around with the code until you get it right.
To get you started, you can add / remove post-build steps by calling the getPublishersList() method on a job and then calling the add / remove methods.
def publishersList = Jenkins.instance.getJob("JobName").getPublishersList()
publishersList.removeAll { it.class == whatever.plugin.class }
publishersList.add(new PluginConstructor())
If you're not sure what publisher class you need to delete the workspace, I would suggest manually adding the desired configurations to one job, and then run getPublishersList() from the script console on that job. You will see the class you are working with in the list, and then you can go look at the api to see what is required to construct it.
You can then iterate through all your jobs and add the publisher doing something like this:
Jenkins.instance.getView("All Jobs").items.each { job ->
//Maybe some logic here to filter out specific jobs
job.getPublishersList().add(new PluginConstructor())
}
Alternatively, you can use the Jenkins CLI api or the REST api, but in order to update post-build actions, you will have to modify the project configuration xml file (which isn't trivial programmatically configure) and then overwrite the job configuration with the new configuration file.

Answer 4

You can edit the config.xml file with your favorite text tool (I use Python) and then reload the jenkins configuration.
In my setup the jobs are stored in ~/.jenkins/jobs/*/config.xml.
See: https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins
Here is a small example to update foo to bar:
</com.cwctravel.hudson.plugins.extended__choice__parameter.ExtendedChoiceParameterDefinition>
<hudson.model.StringParameterDefinition>
<name>additional_requirements</name>
<description>foo</description>
...
Script:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import absolute_import, division, unicode_literals, print_function
import sys
from lxml import etree
from collections import defaultdict
def change_parameter_description(config_xml_path, parameter_name, new_description):
tree=etree.parse(config_xml_path)
for tag in tree.findall('.//hudson.model.StringParameterDefinition'):
name_tag=tag.find('./name')
if not name_tag.text==parameter_name:
continue
description=tag.find('./description')
description.text=new_description
tree.write(config_xml_path)
for config_xml_path in sys.argv[1:]:
change_parameter_description(config_xml_path, 'additional_requirements', 'bar')
In this small example a regex would work, but if things span several lines, it is better to work with xml tools :-)

Answer 5

The other answers are great, but if you use pipelines, I'd suggest you to use Pipeline Shared Libraries.
We have all our jobs in a git repository. To develop a new feature we try it in a branch, since it is possible to point just one job to a specific branch. When we need to update them, just merge into master. The jobs are treated as code, with a proper release process.