I know there is a Nexus Platform plugin https://plugins.jenkins.io/nexus-jenkins-plugin/ in Jenkins but I am not sure if the below is possible any advise or suggestions would be appreciated.
In Jenkins you have Git choice parameter this allows you to build specific tags / branches within your job is there something similar for Sonatype Nexus? We have an internal nexus where we upload and tag docker images.
I currently have a Jenkins job I have to manually type in the image version.
Is there a way in Jenkins to get a choice parameter where i can query all the tags in nexus.
So for example I can run the command - > docker pull internal/application/service:0.0.1
So the developers would upload a new version for example 0.0.2
From Jenkins I would like to display a list of 0.0.1 or 0.0.2 for the support team to build.
Not sure if this is currently possible ?
Update 2020/07/15
I have read up on the active choice paramater plugin. This allows you to execute a groovy script.
So i created the below
import groovy.json.JsonSlurper
// GET
try {
def get = new URL("http://internalserver:8081/service/rest/v1/search?repository=docker-internal&name=application/service/moo").openConnection();
def getRC = get.getResponseCode();
//println(getRC);
if (getRC.equals(200)) {
//println(get.getInputStream().getText());
JsonSlurper slurper = new JsonSlurper()
Map parsedJson = slurper.parseText(get.getInputStream().getText())
tags = parsedJson.items.version
//println(tags)
def sorted_tags = []
sorted_tags.push(tags)
println(sorted_tags)
}
}catch(Exception e){
println(e)
}
This code does print out the tags if i run it from my IDE but if i add it to the active choice plugin my drop down menu is blank ?
Ok i got it to work Jenkins combo box requires a return type.
So if anyone ever wants to do something similar the below worked for me.
<code>
import groovy.json.JsonSlurper
try {
def get = new URL("http://yourinternalnexusurl:8018/applicacation/v1...etc").openConnection();
def getRC = get.getResponseCode();
if (getRC.equals(200)) {
def nexus_response = [:]
nexus_response = new JsonSlurper().parseText(get.getInputStream().getText())
def image_tag_list = []
for (tag in nexus_response.items.version){
image_tag_list.add(tag)
}
return image_tag_list.sort()
}
}catch(Exception e){
println(e)
}
</code>
The Maven Artifact ChoiceListProvider (Nexus) may satisfy your requirements.
With this extension its possible to use the Service API from a Maven Repositories like Nexus, Maven-Central or Artifactory to search for artifacts using groupId, artifactId and packaging.
This plugin will let the user choose a version from the available artifacts in the repository and will publish the URL as an environment variable. The Plugin will return the full URL of the choosen artifact, so that it will be available during the build, i.E. you can retrieve the artifact by using "wget"
Example
You might then have to parse (groovy ?) the resulting env variable to feed as your parameter.
Related
We are in the early phases of using Jenkins DSL. One challenge we have come across is being able to read in an existing jobs plugin settings so that we retain it before running the DSL. This allows us the ability to give the Jenkins users the option to keep some of their settings. We have successfully retained the schedule settings for our jobs but the newest challenge is being able to retain a plugin setting. Specifically a setting in the "ExtendedEmailPublisher" plugin. We would like to retain the value:
In the config.xml file for this job in the ExtendedEmailPublisher tags we see the following:
<publishers>
<hudson.plugins.emailext.ExtendedEmailPublisher>
<recipientList>Our_Team#Our_Team.com</recipientList>
<configuredTriggers>
<hudson.plugins.emailext.plugins.trigger.FailureTrigger>
<email>
<recipientList/>
<subject>$PROJECT_DEFAULT_SUBJECT</subject>
<body>$PROJECT_DEFAULT_CONTENT</body>
<recipientProviders>
<hudson.plugins.emailext.plugins.recipients.ListRecipientProvider/>
</recipientProviders>
<attachmentsPattern/>
<attachBuildLog>false</attachBuildLog>
<compressBuildLog>false</compressBuildLog>
<replyTo>$PROJECT_DEFAULT_REPLYTO</replyTo>
<contentType>project</contentType>
</email>
</hudson.plugins.emailext.plugins.trigger.FailureTrigger>
</configuredTriggers>
<contentType>default</contentType>
<defaultSubject>$DEFAULT_SUBJECT</defaultSubject>
<defaultContent>$DEFAULT_CONTENT</defaultContent>
<attachmentsPattern/>
<presendScript>$DEFAULT_PRESEND_SCRIPT</presendScript>
<classpath/>
<attachBuildLog>false</attachBuildLog>
<compressBuildLog>false</compressBuildLog>
<replyTo>$DEFAULT_REPLYTO</replyTo>
<saveOutput>false</saveOutput>
<disabled>false</disabled>
</hudson.plugins.emailext.ExtendedEmailPublisher>
</publishers>
The value we would like to extract/preserve from this XML is:
<disabled>false</disabled>
We have tried getting the existing values using groovy but cant seem to find the right code. Our first idea was to try to read the value from the config.xml using the XmlSlurper. We ran this from the Jenkins Script Console:
def projectXml = new XmlSlurper().parseText("curl http://Server_Name:8100/job/Job_Name/api/xml".execute().text);
*we use 8100 for our Jenkins port
Unfortunately while this does return some config info it does not return plugin info.
Then, we also tried running the following to read/replace the existing settings:
def oldJob = hudson.model.Hudson.instance.getItem("Job_Name")
def isDisabled = false // Default Value
for(publisher in oldJob.publishersList) {
if (publisher instanceof hudson.plugins.emailext.ExtendedEmailPublisher) {
isDisabled = publisher.disabled
}
}
And while this works if executed from the Jenkins Script Console, when we try to use it in a DSL Job we get the message:
Processing provided DSL script
ERROR: startup failed:
script: 25: unable to resolve class
hudson.plugins.emailext.ExtendedEmailPublisher
# line 25, column 37.
if (publisher instanceof hudson.plugins.emailext.ExtendedEmailPublisher)
{
1 error
Finished: FAILURE
SOLUTION UPDATE:
Using url #aflat's URL suggestion for getting the raw XML config info, I was able to use the XML Slurper and then use the getProperty method to assign the property I wanted to a variable.
def projectXml = new XmlSlurper().parseText("curl http://Server_Name:8100/job/Job_Name/config.xml".execute().text);
def emailDisabled = projectXml.publishers."hudson.plugins.emailext.ExtendedEmailPublisher".getProperty("disabled");
If you want to parse the config.xml, use
def projectXml = new XmlSlurper().parseText("curl http://Server_Name:8100/job/Job_Name/config.xml");
That should return your raw config.xml data
Under "Manage Jenkins->Configure Global Security" did you try disabling "Enable script security for Job DSL scripts"?
I am working on automating the creation of Jenkins jobs by using the Jenkins Job DSL (Groovy). Right now, I am trying to automate the creation of a job that uses the ez-template plugin to use an already existing template and apply that to my newly created job. However, after I am done writing the necessary configuration:
job('foo') {
properties {
templateImplementationProperty {
exclusions(['ez-templates', 'job-params', 'disabled', 'description'])
syncAssignedLabel(true)
syncBuildTriggers(true)
syncDescription(false)
syncDisabled(false)
syncMatrixAxis(true)
syncOwnership(true)
syncScm(true)
syncSecurity(true)
templateJobName('template')
}
}
}
the job gets created alright... except the template is never applied until AFTER I manually hit the save button on the UI in the newly created job. Checking the config.xml of the created job I can see that the xml contains the configuration I specified, but it was never applied.
Looking at the ez-template code, I can see that this is due to the silentSave feature that was implemented in that plugin - it writes configuration to disk without triggering any save events.
I've tried methods available to the Jenkins API but I've had no success there. Any ideas on how I can apply my configuration?
Full disclosure: I'm a co-worker, and was able to help shredmasteryjm solve this. I figured it'd be best to put this out on the net for others.
The Groovy code needed to trigger template implementation contents to be updated is:
import hudson.model.*;
import jenkins.model.*;
import com.joelj.jenkins.eztemplates.utils.TemplateUtils;
import com.joelj.jenkins.eztemplates.TemplateImplementationProperty;
Jenkins j = Jenkins.getInstance()
Item job = j.getItemByFullName('foo')
TemplateImplementationProperty template = TemplateUtils.getTemplateImplementationProperty(job)
TemplateUtils.handleTemplateImplementationSaved(job, template)
This utilizes the EZ-Templates TemplateUtils class to trigger the actual save event, using the template that the job uses. Of note, if job 'foo' doesn't implement a template, then the 'template' variable will be null, causing this code to error. YMMV
In our case, we needed to also add in some useful information from another question: Access to build environment variables from a groovy script in a Jenkins build step ( Windows)
in order to utilize a parameterized job name. As such our completed script looks like this:
import hudson.model.*;
import jenkins.model.*;
import com.joelj.jenkins.eztemplates.utils.TemplateUtils;
import com.joelj.jenkins.eztemplates.TemplateImplementationProperty;
// get current thread / Executor
def thr = Thread.currentThread()
// get current build
def build = thr?.executable
def hardcoded_param = "parameter_job_name"
def resolver = build.buildVariableResolver
def hardcoded_param_value = resolver.resolve(hardcoded_param)
Jenkins j = Jenkins.getInstance()
Item job = j.getItemByFullName(hardcoded_param_value)
TemplateImplementationProperty template = TemplateUtils.getTemplateImplementationProperty(job)
TemplateUtils.handleTemplateImplementationSaved(job, template)
FYI ez-templates 1.3.0 now triggers off additional save events such that you do not need the above trick.
I would like to check for or get notifications about SCM poll failures in Jenkins (for example, when the repository URL had changed, or branch got deleted). I thought about these:
a) A Jenkins console script, which would list such faulty jobs
b) Configuring/installing plugin for Jenkins to notify me somehow about that fact (e-mail, anything)
c) External script/executable (bash, python, ...), which would list builds which failed in last X hours due to SCM poll failure
As you mentioned in your question, one way to tackle this problem is by using a script. For example, Groovy Postbuild.
Since Groovy Postbuild scripts run on the master, you can access each job's scm-polling.log found on the file system using standard IO functions.
For example, assuming a Windows master, here is some (untested) pseudocode to give you some ideas:
def error = false;
def jobsDirectory = new File("C:\\Jenkins\\jobs");
jobsDirectory.eachFile {
def pollingLog = new File(it.path + "\\scm-polling.log");
if(pollingLog.text =~ "ERROR")
{
manager.listener.logger.println(it.path + " has polling errors.");
error = true;
}
}
if(error) {
manager.build.buildFailure();
}
Once you have marked the build as failure, you can use the standard email functionality of Jenkins to send an email or format it to look nice using the Email-ext plugin.
I wan't to able to script and change number of executors on a node(not master) that already exists. Preferably by using groovy but if there is a plugin or CLI command that could do the trick that is also interesting.
Snippet of what I am trying to do:
jenkins.model.Jenkins.instance.nodes.each { node ->
println node.getNumExecutors()
//How do I set the number of executors for a node?
}
I managed it using Slave, which is a subclass of Node.
Below a part of a method I use for that, with target_label and target_executors as parameters
def nodes = nodesByLabel(target_label) // requires plugin "Pipeline Utility Steps"
def j = Jenkins.getInstanceOrNull();
for (int i = 0; i < nodes.size(); ++i) {
def aSlave = (Slave) j.getNode(nodes[i]) // here cast is needed
aSlave.setNumExecutors(target_executors.toInteger())
aSlave.save()
println aSlave.getDisplayName() + "-" + aSlave.getNumExecutors()
}
j.reload()
This isn't currently possible — the numExecutors property of a Jenkins node is read-only.
From JENKINS-23534:
[setNumExecutors] is intentionally private since Jenkins does not offer a way to change the number of executors of a SlaveComputer or Slave once created. Instead, you change the configuration, meaning replacing the existing Slave.
You could run the script below to modify the Jenkins config.xml file… then just "Reload Configuration from Disk" in Manage Jenkins.
This works from the http://jenkins:8080/script console.
import groovy.xml.XmlUtil
// the path to your jenkins config.xml
filePath = '/opt/sites/.jenkins/config.xml'
fileContents = new File(filePath).text
def config = new XmlSlurper().parseText(fileContents)
config.slaves[0].slave.each {
it.numExecutors = 5
}
def writer = new FileWriter(filePath)
XmlUtil.serialize(config, writer)
If you want to reload config automatically you could add:
Jenkins.instance.reload()
It's pretty painful that you can't change the number of executors on a slave through the Jenkins API. You'll have to delete the slave and recreate it to change the number of executors, so you'll need to make sure that there are no builds running on the slave first.
Another option to do this in an automated way is to use the create-slave and delete-slave commands in the Jenkins CLI (go to the /cli link in your Jenkins instance for CLI documentation).
When you set up a Jenkins job various test result plugins will show regressions if the latest build is worse than the previous one.
We have many jobs for many projects on our Jenkins and we wanted to avoid having a 'job per branch' set up. So currently we are using a parameterized build to build eg different development branches using a single job.
But that means when I build a new branch any regressions are measured against the previous build, which may be for a different branch. What I really want is to measure regressions in a feature branch against the latest build of the master branch.
I thought we should probably set up a separate 'master' build alongside the parameterized 'branches' build. But I still can't see how I would compare results between jobs. Is there any plugin that can help?
UPDATE
I have started experimenting in the Script Console to see if I could write a post-build script... I have managed to get the latest build of master branch in my parameterized job... I can't work out how to get to the test results from the build object though.
The data I need is available in JSON at
http://<jenkins server>/job/<job name>/<build number>/testReport/api/json?pretty=true
...if I could just get at this data structure it would be great!
I tried using JsonSlurper to load the json via HTTP but I get 403, I guess because my script has no auth session.
I guess I could load the xml test results from disk and parse them in my script, it just seems a bit stupid when Jenkins has already done this.
I eventually managed to achieve everything I wanted, using a Groovy script in the Groovy Postbuild Plugin
I did a lot of exploring using the script console http://<jenkins>/script and also the Jenkins API class docs are handy.
Everyone's use is going to be a bit different as you have to dig down into the build plugins to get the info you need, but here's some bits of my code which may help.
First get the build you want:
def getProject(projectName) {
// in a postbuild action use `manager.hudson`
// in the script web console use `Jenkins.instance`
def project = manager.hudson.getItemByFullName(projectName)
if (!project) {
throw new RuntimeException("Project not found: $projectName")
}
project
}
// CloudBees folder plugin is supported, you can use natural paths:
project = getProject('MyFolder/TestJob')
build = project.getLastCompletedBuild()
The main test results (jUnit etc) seem to be available directly on the build as:
result = build.getTestResultAction()
// eg
failedTestNames = result.getFailedTests().collect{ test ->
test.getFullName()
}
To get the more specialised results from eg Violations plugin or Cobertura code coverage you have to look for a specific build action.
// have a look what's available:
build.getActions()
You'll see a list of stuff like:
[hudson.plugins.git.GitTagAction#2b4b8a1c,
hudson.scm.SCMRevisionState$None#40d6dce2,
hudson.tasks.junit.TestResultAction#39c99826,
jenkins.plugins.show_build_parameters.ShowParametersBuildAction#4291d1a5]
These are instances, the part in front of the # sign is the class name so I used that to make this method for getting a specific action:
def final VIOLATIONS_ACTION = hudson.plugins.violations.ViolationsBuildAction
def final COVERAGE_ACTION = hudson.plugins.cobertura.CoberturaBuildAction
def getAction(build, actionCls) {
def action = build.getActions().findResult { act ->
actionCls.isInstance(act) ? act : null
}
if (!action) {
throw new RuntimeException("Action not found in ${build.getFullDisplayName()}: ${actionCls.getSimpleName()}")
}
action
}
violations = getAction(build, VIOLATIONS_ACTION)
// you have to explore a bit more to find what you're interested in:
pylint_count = violations?.getReport()?.getViolations()?."pylint"
coverage = getAction(build, COVERAGE_ACTION)?.getResults()
// if you println it looks like a map but it's really an Enum of Ratio objects
// convert to something nicer to work with:
coverage_map = coverage.collectEntries { key, val -> [key.name(), val.getPercentageFloat()] }
With these building blocks I was able to put together a post-build script which compared the results for two 'unrelated' build jobs, then using the Groovy Postbuild plugin's helper methods to set the build status.
Hope this helps someone else.