Configuring plugins with Jenkins Job Builder - jenkins

I'm trying to use Jenkins Job Builder to install jenkins plugins, but I misunderstand what JJB can do or I'm doing something wrong. I used the get-plugins-info command to get a YAML description of my plugins. Later, when rebuilding the jenkins installation, I used jenkins-jobs -p plugins_info.yaml jobs in the hopes that JJB would install the plugins listed in the YAML file. But it didn't install the plugins.
So my first question is: should I even expect JJB to install these plugins? The documentation for what JJB is doing with the plugin information is limited, so I'm running on assumptions here.
Assuming JJB is supposed to be installing the plugins in the YAML file, how can I figure out why it's not? I've looked at jenkins' logs to no avail.

I'll start by mentioning that I use Jenkins Job Builder for creating and versioning my jobs. But if you want to install/configure plugins in Jenkins in an automated fashion you can use init.groovy.d scripts that will initialize your jenkins instance. In order to do so create the following directory ${JENKINS_HOME}/init.groovy.d/ then place your groovy scripts in that directory. This is a script that I use to install plugins when I start Jenkins.
import jenkins.model.*
import java.util.logging.Logger
def logger = Logger.getLogger("")
def installed = false
def initialized = false
def plugins = ["git", "cloudbees-folder", "build-timeout"]
logger.info("" + plugins)
def instance = Jenkins.getInstance()
def pm = instance.getPluginManager()
def uc = instance.getUpdateCenter()
plugins.each {
logger.info("Checking " + it)
if (!pm.getPlugin(it)) {
logger.info("Looking UpdateCenter for " + it)
if (!initialized) {
uc.updateAllSites()
initialized = true
}
def plugin = uc.getPlugin(it)
if (plugin) {
logger.info("Installing " + it)
def installFuture = plugin.deploy()
while(!installFuture.isDone()) {
logger.info("Waiting for plugin install: " + it)
sleep(3000)
}
installed = true
}
}
}
if (installed) {
logger.info("Plugins installed, initializing a restart!")
instance.save()
instance.restart()
}
Add as many plugin names to the array plugins. Hope this help you and others out.

JJB has no capability to manage Jenkins Plugins. You will need to look into other tools to handle that for you such as puppet, ansible, etc...
The usage for "get-plugins-info" and the "-p" parameter in the update command is to pass currently installed plugin info to JJB in cases where a system administrator does not want to all JJB "administrator" access permissions in Jenkins to "query" plugin info during the update run. The latest versions of Jenkins no longer allows anonymous querying of plugin info and unfortunately moved that permission to the administrator permission inside of Jenkins.
plugins-info is useful because JJB supports multiple versions of certain plugins and needs to know what the installed version is to appropriately create the XML depending on supported versions.

Related

Best way to store the deployment path in jenkins

I am creating jenkins pipeline for all our application where I wanted to build and deploy. I can able to achieve that but all the deployment paths are hard coded on the pipeline script.
We have around 8 application and 5 environments. it means I need to specify 40 different deployable path on the pipeline scripts.
I like to know, are they any best way to store the deployment path?. I thought about storing them in XML and reading that while doing the build, but not sure on implementation.
looking for some ideas.
script {
def msbuild tool name: 'Msbuila', type: 'msbuild'
def action "${msbuild}\\msbuild.exe"
def rootPath "${NORKSPACE}\\test\\test";
def sinPath "${rootPath}\\test.sin"
def binPath "${rootPath}\\test\\bin"
bat “nuget restore \"${sinPath}\""
bat "\"${action}\" \"${sinPath)\" "
robocopy("\"${binPath}\" \"\\\\t.test.com\\test\" /MIR /xF ")
}
What I would do is use a config repository, having it configured this way:
Each application is a different repository (example: app_config)
Each environment is a different file
The same enviroment file in each repository is called by the same name
Each enviroment file is a yaml (key:value)
Then on the jenkins pipeline I would get the repo, read the yaml using readYAML (check the command usage and name, theres is a while since I used it) and load it on a map.
Then you use the variables of the map and that should help you
The tricky part is how to match the code repositories and the config repositories. As I mentioned before, I would use the same name and append "_config"

Jenkins and gradle - build projects with latest versions of dependencies for CI, specific versions for production

I am working with Jenkins, Gradle and our Ivy repository.
Our build scripts specify the exact version of dependencies to be used for the build. This is good practice for production.
For CI it would be interesting if the project build used the latest versions of our own libraries, that way we could not only see if library changes "broke the build" for the library but also if they broke the projects that use them. That seems to be the point of "integration"!
I understand that gradle will take "1.+" instead of "1.2.3" so I could hack the build.gradle for the project on the CI server to achieve this. But perhaps there is a neater way to do it (build script recognises it is in CI mode and uses latest and not specific versions, perhaps by running a sed script on build.gradle to change it).
Am I missing something in Jenkins or gradle? Are there any gradle plugins that achieve this, or alternative approaches that you have used to achieve this?
something alike this might work with Jenkins:
if(System.getenv("BUILD_EXPERIMENTAL") == null) {
// known to be stable versions
apply from: "dependencies.gradle"
} else {
// bleeding edge versions
apply from: "experimental.gradle"
}
this just would need the same project being set up twice, once with and once without environmental variable BUILD_EXPERIMENTAL, which is used to control which dependencies block is being applied.
in case you want it generally being applied, when the project is being built with Jenkins, just replace BUILD_EXPERIMENTAL with BUILD_NUMBER (which by default is being set up in that environment).
If you want to have the latest you can simply use latest, or if it's easier something like [1.0,) that would match all versions greater or equal to 1.0 (assuming that 1.0 is your "smallest version ever") Look here for other matching patterns, which you could also combing with statuses.
Another way would be to have a local filesystem ivy repo only on the jenkins slave, which would have all the latest versions of your libraries, the point is that this repo is not accessible from developers workstations/laptops/VMs. And then you just simply use that in gradle settings in some way (for example have an environment variable defined only on the jenkins slave). This means that you don't need to change build.gradle
I would recommend leveraging Gradle dependency locking for achieving this.
In the build, you would use dynamic versions for your dependencies, locked to a good known state.
Developers and production build would then get these locked versions resolved.
On CI you could have a (set of) dedicated job(s) that runs and updates the lock state for one or more modules at a time. Based on that feedback, you could even commit this dependency upgrade or at least open a pull request for it.
This is my own answer inspired by #Martin Zeitler's answer.
We have a generic build script that get applied to all project build.gradle setting up common options, settings and tasks. We want to add in this logic, but make it optional and not break existing build scripts.
The logic will be activated and controlled by a property project.ext.buildJenkinsWithLatest which is true or false.
When the logic is active dependencies from the project files dependencies-production.gradle or dependencies-jenkins.gradle will be used. The Jenkins dependencies will only be used if the property is true and the CI environment is detected through the presence of the BUILD_NUMBER environment variable.
The generic build script contains this:
if (project.ext.has('buildJenkinsWithLatest')) {
println "Using conditional dependency management..."
//BUILD_NUMBER is not null if this is a Jenkins build
if(project.ext.buildJenkinsWithLatest == true && System.getenv("BUILD_NUMBER") != null) {
println "--- Using alternative dependencies..."
apply from: "dependencies-jenkins.gradle"
}
else {
println "--- Using production dependencies..."
apply from: "dependencies-production.gradle"
}
}
else {
println "Conditional dependency management is not active"
}
Now any project's build.gradle that already applies this script will print this when run:
Conditional dependency management is not active
To use the feature we will need to do the following for our project:
Create a dependencies-jenkins.gradle that contains a dependencies {} clause for the libraries we want to select a version dynamically.
Create a dependencies-production.gradle that contains a dependencies {} clause for those libraries, but with a specific version given.
Remove the libraries from any dependencies {} that remains in the project build.gradle.
Set the property project.ext.buildJenkinsWithLatest to true or false.
Apply the generic build script (after setting the property!).
For example in dependencies-jenkins.gradle use the latest 2.x.x version:
dependencies {
compile 'example:my-library:2+'
}
As to how to specify the versions in a dynamic way see #CantSleepNow's answer.
And in dependencies-production.gradle use a specific version:
dependencies {
compile 'example:my-library:2.3.4'
}
Then within build.gradle set the property and apply the generic build script:
...
project.ext.buildJenkinsWithLatest = true;
apply from: '../bxgradle/bx-std.gradle'
...
Now when the build is run on Jenkins the alternative dependencies will be used. Should one wish to build it on Jenkins with the production dependencies then set project.ext.buildJenkinsWithLatest to false.

Chef Cookbooks: Installing Jenkins plugins from GItHub

I am using Chef to deploy my Jenkins instance. I am currently using the Jenkins cookbook found in the Chef Supermarket: https://supermarket.chef.io/cookbooks/jenkins .
I'm managing my plugins by adding them to the _master_war.rb recipe file. Most plugins I'm finding are straight forward, follow the specified instructions in the cookbook supermarket page and can be found for download on the https://wiki.jenkins-ci.org site with a link to a hpi/jpi file.
I'm finding some plugins which are only available via GitHub. For example, https://plugins.jenkins.io/workflow-cps. My current plugin code looks like thus:
# Installs the latest version of the workflow-job Plugin for Jenkins
# The BitBucket Build Status Notifier plugin is dependent on this plugin
# https://github.com/jenkinsci/workflow-job-plugin
jenkins_plugin 'workflow-job-plugin' do
source 'https://github.com/jenkinsci/workflow-job-plugin.git'
notifies :restart, 'runit_service[jenkins]', :immediately
end
Unfortunately, this syntax does not appear to install the plugin via git successfully on my jenkins instance. The following warning appears after running the recipe on the node jenkins is running on:
Bitbucket Build Status Notifier Plugin v1.3.3
workflow-job v1.11 is missing. To fix, install v1.11 or later.
workflow-cps v1.11 is missing. To fix, install v1.11 or later.
multiple-scms v0.6 is missing. To fix, install v0.6 or later.
What would be the correct way / syntax to use to go about downloading and installing these plugins from github to my jenkins instance using the jenkins cookbook for Chef?
I would to to add to this discussion that I found a way to work through this issue. When a page like this one is the official source for a Jenkins plugin: plugins.jenkins.io/workflow-cps, on the right hand side of the screen is an 'Archives' link which will take you to the .hpi listing for all available versions of the given plugin. I.E. updates.jenkins.io/download/plugins/workflow-cps . Using the link address for the listed versions you can source the appropriate .hpi file

Automatically setup a JENKINS 2.32.1 server (with a script)

I am recently trying to automatically setup a jenkins server with a script.
The aim is, that I simply run that one shell scirpt after a fresh server (aws instance) is started and then the whole server is ready to use.
What does this mean:
I don't wanna see that first question for the secret file (has to be done automatically or disabled)
I don't want to set a admin password, well not by using the UI but by setting it via terminal.
Add some users with permissions
Add SSH private keys for connections to github/bitbucket
Install pre-chosen plugins
Additionally paste some existing jobs into that jenkins
Is there a way to do that? Or (what I didn't try so far) is it possible to just copy a pre-configured jenkins and run that? Like a "jenkins"-image =)
The Jenkins home directory contains a init.groovy.d directory.
Any groovy script within this directory is executed at boot in lexicographical order. Currently I am running my Jenkins master inside a container, which is pre-provisioned, but you could also easily use configuration management tools to provision your Jenkins master instance with these groovy scripts.
Here is one example I use for integrating private keys from $JENKINS_HOME/.ssh into my Jenkins credential store:
import jenkins.model.*
import com.cloudbees.plugins.credentials.*
import com.cloudbees.plugins.credentials.common.*
import com.cloudbees.plugins.credentials.domains.*
import com.cloudbees.plugins.credentials.impl.*
import com.cloudbees.jenkins.plugins.sshcredentials.impl.*
domain = Domain.global()
store = Jenkins.instance.getExtensionList('com.cloudbees.plugins.credentials.SystemCredentialsProvider')[0].getStore()
// private git keys
privateRepoKeys = ["key1", "key2"]
privateRepoKeys.each {
privateKey = new BasicSSHUserPrivateKey(
CredentialsScope.GLOBAL,
"jenkins-${it}",
"jenkins",
new BasicSSHUserPrivateKey.FileOnMasterPrivateKeySource("/var/jenkins_home/.ssh/${it}"),
"",
"Jenkins ${it} Key"
)
store.addCredentials(domain, privateKey)
}
Here is another example that I use for adding a default admin:
import jenkins.model.*
import hudson.security.*
def instance = Jenkins.getInstance()
def hudsonRealm = new HudsonPrivateSecurityRealm(false)
hudsonRealm.createAccount("my_admin", "my_pw")
instance.setSecurityRealm(hudsonRealm)
def strategy = new GlobalMatrixAuthorizationStrategy()
strategy.add(Jenkins.ADMINISTER, "my_admin")
instance.setAuthorizationStrategy(strategy)
instance.save()
I use more scripts to completely bootstrap my Jenkins (hooking it up with the slave machines, configuring global tools, configuring global variables...). So far I didn't hit a limitation with this for my use cases.
Update:
I just created a gist with a collection of groovy scripts I gathered and that I use to bootstrap Jenkins. Will try to keep it up to date.
Yes, there are many ways and steps to do that. You can have a look at the Docker container, Chef cookbook or Puppet module. They do things like adding
-Djenkins.install.runSetupWizard=false
to Jenkins startup parameters.
They (at least Docker and Chef) also have their means to install plugins.
Answering this within a single question here is IMHO a bit too much, so I'm mainly pointing to off-site resources and ask you to post a new question, once your have a particular problem with one of the steps.

Remove Jenkins Plugins via Script

Does anyone know how I can remove jenkins plugins from a script? I know there is no CLI command that exists for it. But was wondering, maybe you can just delete the plugin folder, and attempt to delete all associated data.
Reason I want this, is I want to develop a script that will run nightly. It will scan each Jenkins server, remove plugins if not found in file stored in GIT, it will add plugins if new ones are added, and upgrade or downgrade other plugins if the version doesn't match. The goal is to keep all Jenkins servers in sync as far as plugins go.
Any thoughts on how I can achieve this goal?
groovy script for removing a plugin:
String pluginNameToRemove = "myPluginToRemove"
def jenkins = Jenkins.getInstance()
def pluginManager = jenkins.getPluginManager()
def pluginWrapperToUninstall = pluginManager.getPlugin(pluginNameToRemove)
pluginWrapperToUninstall.doDoUninstall()
You can also remove the content of the plugins folder, download all the plugins you want according to the file, copy them to the plugins folder and restart jenkins. (We have a very similar flow on our environment for deploying jenkins instances)

Resources