I have Jenkins with several hundreds of jobs, and I need to find job which runs specified gradle task. I see following solution
1.retrieve all jobs (Jenkins.instance.projects)
2.iterate over them
3.get xml config and verify presence of substring
The question is how to retrieve xml representation from hudson.model.FreeStyleProject? Or may be this data stored as map, than the question will be how it is called and how get it?
I did something similar before but using a shell script. Not sure if that approach is useful for you but just in case:
cd /var/lib/jenkins/jobs/
grep WORD */config.xml
Also the next groovy script that can list all the FreeStyleProject name and gradle tasks:
def builderFilter = { builder -> builder.class == hudson.plugins.gradle.Gradle.class }
jenkins.model.Jenkins.instance.getAllItems(hudson.model.FreeStyleProject.class).each{ job ->
job.getBuilders().findAll(builderFilter).each{ gradleStep ->
gradleStep.each { gradleItem ->
println(job.getDisplayName() + ' ' + gradleItem.getTasks())
}
}
}
Related
I have a jenkins file as below
pipelineJob('My pipeline job'){
displayName('display name')
logRotator {
numToKeep(10)
daysToKeep(30)
artifactDaysToKeep(7)
artifactNumToKeep(1)
}
definition{
cps {
script(readFileFromWorkspace('./cicd/pipelines/clone_git_code.groovy'))
script(readFileFromWorkspace('./cicd/pipelines/install_dependencies_run_quality_checks.groovy'))
}
}
}
with above jenkinsfile the last script file is replacing other scripts.
Basically I have split tasks into multiple groovy files so that i wont repeat the same code in all jenkinsfile and reuse the same for other jobs as well, like I can now use the clone_git_code.groovy script in dev build as well as QA builds.
You have to use shared libraries (https://jenkins.io/doc/book/pipeline/shared-libraries/). You can define multiple groovy files with classes to return a processed object or simply creating calls with method where you define a step and the execution will be sequential.
I had this same issue when trying to include multiple scripts into a Jenkins job. After doing some research, I found the below solution to be the simplest:
definition {
cps {
script (
ScriptsLibrary.pipelineTest('did it work?') +
ScriptsLibrary.scmConf('repoURL_input', 'accessCredentials', 'activeBranch')
)
}
}
Add the "+" to concatenate the Strings. Got the job done for me :)
I read How can I set the job timeout using the Jenkins DSL. That sets the timeout for one job. I want to set it for all jobs, and with slightly different settings: 150%, averaged over 10 jobs, with a max of 30 minutes.
According to the relevant job-dsl-plugin documentation I should use this syntax:
job('example-3') {
wrappers {
timeout {
elastic(150, 10, 30)
failBuild()
writeDescription('Build failed due to timeout after {0} minutes')
}
}
}
I tested in http://job-dsl.herokuapp.com/ and this is the relevant XML part:
<buildWrappers>
<hudson.plugins.build__timeout.BuildTimeoutWrapper>
<strategy class='hudson.plugins.build_timeout.impl.ElasticTimeOutStrategy'>
<timeoutPercentage>150</timeoutPercentage>
<numberOfBuilds>10</numberOfBuilds>
<timeoutMinutesElasticDefault>30</timeoutMinutesElasticDefault>
</strategy>
<operationList>
<hudson.plugins.build__timeout.operations.FailOperation></hudson.plugins.build__timeout.operations.FailOperation>
<hudson.plugins.build__timeout.operations.WriteDescriptionOperation>
<description>Build failed due to timeout after {0} minutes</description>
</hudson.plugins.build__timeout.operations.WriteDescriptionOperation>
</operationList>
</hudson.plugins.build__timeout.BuildTimeoutWrapper>
</buildWrappers>
I verified with a job I edited manually before, and the XML is correct. So I know that the Jenkins DSL syntax up to here is correct.
Now I want to apply this to all jobs. First I tried to list all the job names:
import jenkins.model.*
jenkins.model.Jenkins.instance.items.findAll().each {
println("Job: " + it.name)
}
This works too, all job names are printed to console.
Now I want to plug it all together. This is the full code I use:
import jenkins.model.*
jenkins.model.Jenkins.instance.items.findAll().each {
job(it.name) {
wrappers {
timeout {
elastic(150, 10, 30)
failBuild()
writeDescription('Build failed due to timeout after {0} minutes')
}
}
}
}
When I push this code and Jenkins runs the DSL seed job, I get this error:
ERROR: Type of item "jobname" does not match existing type, item type can not be changed
What am I doing wrong here?
The Job-DSL plugin can only be used to maintain jobs that have been created by that plugin before. You're trying to modify the configuration of jobs that have been created in some other way -- this will not work.
For mass-modification of existing jobs (like, in your case, adding the timeout) the most straightforward way is to change the job's XML specification directly,
either by changing the config.xml file on disk, or
using the REST or CLI API
xmlstarlet is a powerful tool for performing such tasks directly on shell level.
Alternatively, it is possible to perform the change via a Groovy script from the "Script Console" -- but for that you need some understanding of Jenkins' internal workings and data structures.
I would like to check for or get notifications about SCM poll failures in Jenkins (for example, when the repository URL had changed, or branch got deleted). I thought about these:
a) A Jenkins console script, which would list such faulty jobs
b) Configuring/installing plugin for Jenkins to notify me somehow about that fact (e-mail, anything)
c) External script/executable (bash, python, ...), which would list builds which failed in last X hours due to SCM poll failure
As you mentioned in your question, one way to tackle this problem is by using a script. For example, Groovy Postbuild.
Since Groovy Postbuild scripts run on the master, you can access each job's scm-polling.log found on the file system using standard IO functions.
For example, assuming a Windows master, here is some (untested) pseudocode to give you some ideas:
def error = false;
def jobsDirectory = new File("C:\\Jenkins\\jobs");
jobsDirectory.eachFile {
def pollingLog = new File(it.path + "\\scm-polling.log");
if(pollingLog.text =~ "ERROR")
{
manager.listener.logger.println(it.path + " has polling errors.");
error = true;
}
}
if(error) {
manager.build.buildFailure();
}
Once you have marked the build as failure, you can use the standard email functionality of Jenkins to send an email or format it to look nice using the Email-ext plugin.
When you set up a Jenkins job various test result plugins will show regressions if the latest build is worse than the previous one.
We have many jobs for many projects on our Jenkins and we wanted to avoid having a 'job per branch' set up. So currently we are using a parameterized build to build eg different development branches using a single job.
But that means when I build a new branch any regressions are measured against the previous build, which may be for a different branch. What I really want is to measure regressions in a feature branch against the latest build of the master branch.
I thought we should probably set up a separate 'master' build alongside the parameterized 'branches' build. But I still can't see how I would compare results between jobs. Is there any plugin that can help?
UPDATE
I have started experimenting in the Script Console to see if I could write a post-build script... I have managed to get the latest build of master branch in my parameterized job... I can't work out how to get to the test results from the build object though.
The data I need is available in JSON at
http://<jenkins server>/job/<job name>/<build number>/testReport/api/json?pretty=true
...if I could just get at this data structure it would be great!
I tried using JsonSlurper to load the json via HTTP but I get 403, I guess because my script has no auth session.
I guess I could load the xml test results from disk and parse them in my script, it just seems a bit stupid when Jenkins has already done this.
I eventually managed to achieve everything I wanted, using a Groovy script in the Groovy Postbuild Plugin
I did a lot of exploring using the script console http://<jenkins>/script and also the Jenkins API class docs are handy.
Everyone's use is going to be a bit different as you have to dig down into the build plugins to get the info you need, but here's some bits of my code which may help.
First get the build you want:
def getProject(projectName) {
// in a postbuild action use `manager.hudson`
// in the script web console use `Jenkins.instance`
def project = manager.hudson.getItemByFullName(projectName)
if (!project) {
throw new RuntimeException("Project not found: $projectName")
}
project
}
// CloudBees folder plugin is supported, you can use natural paths:
project = getProject('MyFolder/TestJob')
build = project.getLastCompletedBuild()
The main test results (jUnit etc) seem to be available directly on the build as:
result = build.getTestResultAction()
// eg
failedTestNames = result.getFailedTests().collect{ test ->
test.getFullName()
}
To get the more specialised results from eg Violations plugin or Cobertura code coverage you have to look for a specific build action.
// have a look what's available:
build.getActions()
You'll see a list of stuff like:
[hudson.plugins.git.GitTagAction#2b4b8a1c,
hudson.scm.SCMRevisionState$None#40d6dce2,
hudson.tasks.junit.TestResultAction#39c99826,
jenkins.plugins.show_build_parameters.ShowParametersBuildAction#4291d1a5]
These are instances, the part in front of the # sign is the class name so I used that to make this method for getting a specific action:
def final VIOLATIONS_ACTION = hudson.plugins.violations.ViolationsBuildAction
def final COVERAGE_ACTION = hudson.plugins.cobertura.CoberturaBuildAction
def getAction(build, actionCls) {
def action = build.getActions().findResult { act ->
actionCls.isInstance(act) ? act : null
}
if (!action) {
throw new RuntimeException("Action not found in ${build.getFullDisplayName()}: ${actionCls.getSimpleName()}")
}
action
}
violations = getAction(build, VIOLATIONS_ACTION)
// you have to explore a bit more to find what you're interested in:
pylint_count = violations?.getReport()?.getViolations()?."pylint"
coverage = getAction(build, COVERAGE_ACTION)?.getResults()
// if you println it looks like a map but it's really an Enum of Ratio objects
// convert to something nicer to work with:
coverage_map = coverage.collectEntries { key, val -> [key.name(), val.getPercentageFloat()] }
With these building blocks I was able to put together a post-build script which compared the results for two 'unrelated' build jobs, then using the Groovy Postbuild plugin's helper methods to set the build status.
Hope this helps someone else.
How can I get build time stamp of the latest build from Jenkins?
I want to insert this value in the Email subject in post build actions.
Build Timestamp Plugin will be the Best Answer to get the TIMESTAMPS in the Build process.
Follow the below Simple steps to get the "BUILD_TIMESTAMP" variable enabled.
STEP 1:
Manage Jenkins -> Plugin Manager -> Installed...
Search for "Build Timestamp Plugin".
Install with or without Restart.
STEP 2:
Manage Jenkins -> Configure System.
Search for 'Build Timestamp' section, then Enable the CHECKBOX.
Select the TIMEZONE, TIME format you want to setup with..Save the Page.
USAGE:
When Configuring the Build with ANT or MAVEN,
Please declare a Global variable as,
E.G. btime=${BUILD_TIMESTAMP}
(use this in your Properties box in ANT or MAVEN Build Section)
use 'btime' in your Code to any String Variables etc..
NOTE: This changed in Jenkins 1.597, Please see here for more info regarding the migration
You should be able to view all the global environment variables that are available during the build by navigating to https://<your-jenkins>/env-vars.html.
Replace https://<your-jenkins>/ with the URL you use to get to Jenkins webpage (for example, it could be http://localhost:8080/env-vars.html).
One of the environment variables is :
BUILD_ID
The current build id, such as "2005-08-22_23-59-59" (YYYY-MM-DD_hh-mm-ss)
If you use jenkins editable email notification, you should be able to use ${ENV, var="BUILD_ID"} in the subject line of your email.
One way this can be done is using shell script in global environment section, here, I am using UNIX timestamp but you can use any shell script syntax compatible time format:
pipeline {
agent any
environment {
def BUILDVERSION = sh(script: "echo `date +%s`", returnStdout: true).trim()
}
stages {
stage("Awesome Stage") {
steps {
echo "Current build version :: $BUILDVERSION"
}
}
}
}
Try use Build Timestamp Plugin and use BUILD_TIMESTAMP variable.
Generate environment variables from script (Unix script) :
echo "BUILD_DATE=$(date +%F-%T)"
I know its late replying to this question, but I have recently found a better solution to this problem without installing any plugin. We can create a formatted version number and can then use the variable created to display the build date/time.
Steps to create: Build Environment --> Create a formatted version number:
Environment Variable Name: BUILD_DATE
Version Number Format String: ${BUILD_DATE_FORMATTED}
thats it. Just use the variable created above in the email subject line as ${ENV, var="BUILD_DATE"} and you will get the date/time of the current build.
You can use the Jenkins object to fetch the start time directly
Jenkins.getInstance().getItemByFullName(<your_job_name>).getBuildByNumber(<your_build_number>).getTime()
also answered it here:
https://stackoverflow.com/a/63074829/1968948
BUILD_ID used to provide this information but they changed it to provide the Build Number since Jenkins 1.597. Refer this for more information.
You can achieve this using the Build Time Stamp plugin as pointed out in the other answers.
However, if you are not allowed or not willing to use a plugin, follow the below method:
def BUILD_TIMESTAMP = null
withCredentials([usernamePassword(credentialsId: 'JenkinsCredentials', passwordVariable: 'JENKINS_PASSWORD', usernameVariable: 'JENKINS_USERNAME')]) {
sh(script: "curl https://${JENKINS_USERNAME}:${JENKINS_PASSWORD}#<JENKINS_URL>/job/<JOB_NAME>/lastBuild/buildTimestamp", returnStdout: true).trim();
}
println BUILD_TIMESTAMP
This might seem a bit of overkill but manages to get the job done.
The credentials for accessing your Jenkins should be added and the id needs to be passed in the withCredentials statement, in place of 'JenkinsCredentials'. Feel free to omit that step if your Jenkins doesn't use authentication.
This answer below shows another method using "regexp feature of the Description Setter Plugin" which solved my problem as I could not install new plugins on Jenkins due to permission issues:
Use build timestamp in setting build description Jenkins
If you want add a timestamp to every request from browser to jenkins server.
You can refer to the jenkins crumb issuer mechanism, and you can hack the /scripts/hudson-behavior.js add modify here. so it will transform a timestamp to server.
/**
* Puts a hidden input field to the form so that the form submission will have the crumb value
*/
appendToForm : function(form) {
// add here. ..... you code
if(this.fieldName==null) return; // noop
var div = document.createElement("div");
div.innerHTML = "<input type=hidden name='"+this.fieldName+"' value='"+this.value+"'>";
form.appendChild(div);
}