Related
I just upgraded the jenkins version of our development factory to lts. We were previously in an old 2.x version
In fact we have a stage to load all scripts :
stage('Load Utilities Scripts') {
checkout scm
echo "${job_name}"
modules.messages = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/slack_functions.groovy"
modules.ansible = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/ansible_functions.groovy"
modules.jenkins = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/jenkins_functions.groovy"
modules.pic_env = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/pic_env.groovy"
modules.maven = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/maven_functions.groovy"
modules.git = load "${jenkins_home}/workspace/${job_name}#script/pipelines/utils/git_functions.groovy"
}
it worked perfectly well except that now the scripts are positioned in
/var/jenkins_home/workspace/SAGES2_BUILD_AUTO#script/357f3acc22b0fbb05a66735d46d7d7eb950a2e836ab3762b1905784bc550ee5e on builtin
/var/jenkins_home/workspace/SAGES2_BUILD_AUTO on builtin
I guess it's a security evolution (i know groovy script plugin on jenkins are often deprecated but it's another problem).
How can i get this string : 357f3acc22b0fbb05a66735d46d7d7eb950a2e836ab3762b1905784bc550ee5e ? Is there any vars like job_name or jenkins_home ? Who generates this token and where ? Can i have the hand on this string ?
What does it represent ?
You can get the current directory and build the absolute path by appending the relative part.
modules.messages = load pwd() + "#script/pipelines/utils/slack_functions.groovy"
Or something like this
modules.messages = load "${WORKSPACE}#script/pipelines/utils/slack_functions.groovy"
It seems this has being introduced as a security improvement. So you will have to work around this like below.
script {
// Locate the jenkins folder
// This is done because there is a new sub-folder (like : 17a4ba1ed1ce777b18c5...)
git_jenkins_folder = sh (
script: "find " + WORKSPACE + "#script -type d -name 'jenkins' -printf '%T# %Tc %p\n' | sort -rn | head -1 | cut -d' ' -f9",
returnStdout: true
).trim()
// Load the groovy methods in groovy files
slack = load git_jenkins_folder + "/pipelines/utils/slack_functions.groovy"
}
Credit: Why does Jenkins creates a subfolder within the workspace#script folder to checkout git code instead of the workspace#script itself?
I wanted to show the user who triggered a Jenkins job in the post job email. This is possible by using the plugin Build User Vars Plugin and the env variable BUILD_USER.
But this variable do not get initialized when the job is triggered by a scheduler.
How can we achieve this? I know we have a plugin called - EnvInject Plugin, and that can be used...
But I just want to know how we can use this and achieve the solution...
Build user vars plugin wasn't working for me so I did a quick-and-dirty hack:
BUILD_CAUSE_JSON=$(curl --silent ${BUILD_URL}/api/json | tr "{}" "\n" | grep "Started by")
BUILD_USER_ID=$(echo $BUILD_CAUSE_JSON | tr "," "\n" | grep "userId" | awk -F\" '{print $4}')
BUILD_USER_NAME=$(echo $BUILD_CAUSE_JSON | tr "," "\n" | grep "userName" | awk -F\" '{print $4}')
SIMPLE SOLUTIONS (NO PLUGINS) !!
METHOD 1: Via Shell
BUILD_TRIGGER_BY=$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | egrep '^userId>|^userName>' | sed 's/.*>//g' | sed -e '1s/$/ \//g' | tr '\n' ' ')
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
METHOD 2: Via Groovy
node('master') {
BUILD_TRIGGER_BY = sh ( script: "BUILD_BY=\$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | egrep '^userId>|^userName>' | sed 's/.*>//g' | sed -e '1s/\$/ \\/ /g'); if [[ -z \${BUILD_BY} ]]; then BUILD_BY=\$(curl -k --silent ${BUILD_URL}/api/xml | tr '<' '\n' | grep '^shortDescription>' | sed 's/.*user //g;s/.*by //g'); fi; echo \${BUILD_BY}", returnStdout: true ).trim()
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
}
METHOD 3: Via Groovy
BUILD_TRIGGER_BY = "${currentBuild.getBuildCauses()[0].shortDescription} / ${currentBuild.getBuildCauses()[0].userId}"
echo "BUILD_TRIGGER_BY: ${BUILD_TRIGGER_BY}"
OUTPUT:
Started by user Admin / user#example.com
Note: Output will be both User ID and User Name
This can be done using the Jenkins Build User Vars Plugin which exposes a set of environment variables, including the user who started the build.
It gives environment variables like BUILD_USER_ID, EMAIL, etc.
When the build is triggered manually by a logged-in user, that user's userid is available in the BUILD_USER_ID environment variable.
However, this environment variable won't be replaced / initialized when the build is automatically triggered by a Jenkins timer / scheduler.
Attached a screenshot for details
This can be resolved by injecting a condition to the Job by using Conditional Build Step Plugin / Run Condition Plugin,where in to each job we can add a condition to initialize the variable BUILD_USER_ID only when the build is caused or triggered by the Timer or scheduler, by setting a condition using the regular expression..
Without Plugin ->
def cause = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
echo "userName: ${cause.userName}"
Install 'Build User Vars Plugin' and use like below:- [ See https://plugins.jenkins.io/build-user-vars-plugin ]
Be sure to check mark the Set jenkins user build variables checkbox under Build Environment for your Jenkins job's configuration.
I found similar but really working on Jenkins 2.1.x and easy for my understanding way.
And it works without any plugins.
if (currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')['userId']){
// Will be run only if someone user triggers build
// Because in other cases this contructions returns null
}
You can use in this construction any classes described here.
They will be returns maps with usable values.
This gets the username who clicked "Build Now" in a Jenkins pipeline job.
#NonCPS
def getBuildUser() {
return currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
}
I'm using a combination of the 'Execute Shell' and 'Env Inject' plugin as follows:
Create an 'Execute Shell' build step that uses shell parameter substitution to write default the value and echo that value into a file. Example highlighted in screen shot below.
Use the 'Env Inject' file to read that file as properties to set.
The token $BUILD_CAUSE from the email-ext plugin is what you are looking for.
You can see the full content token reference when you click the ? just after the Attach build log combobox at the email content configuration.
Some tokens get added by plugins, but this one should be aviable by default.
Edit: As pointed out by bishop in the comments, when using the EnvInject plugin, the $BUILD_CAUSE token gets changed to behave differently.
I have written a groovy script to extract the started by which would correctly get the source, regardless if user, scm or timer (could add more). It would recursively navigate the build tree to get the "original" 'started by' cause https://github.com/Me-ion/jenkins_build_trigger_cause_extractor
I wanted to trigger build initiator info to one of my slack/flock group so I used following way to get build initiator email and name by writing in Declarative fashion .
I am just printing here, you can use to store in some environment variable or write in one file giving file path according to your own convenience..
pipeline {
environment {
BRANCH_NAME = "${env.BRANCH_NAME}"
}
agent any
stages{
stage('Build-Initiator-Info'){
sh 'echo $(git show -s --pretty=%ae)'
sh 'echo $(git show -s --pretty=%an)'
}
}
}
Just to elaborate on Musaffir Lp's answer. The Conditional Build Step plugin now supports the Build Cause directly - it requires the Run Condition Plugin also.
If you wanted to detect when the build was started by a timer you can select a Run? value of Build Cause, with Build Cause of: TimerTrigger
This is a little simpler and more robust than using a regex. There are also other triggers you can detect, for example when the build was a result of Source Control Management commit, you can select: SCMTrigger.
This below is working for me.
Install "user build vars plugin"
Build Name = ${BUILD_NUMBER}_${TICKET}_${ENV,var="BUILD_USER_ID"}
I created a function that return the Triggered Job Name:
String getTriggeredJob(CURRENT_BUILD) {
if (CURRENT_BUILD.upstreamBuilds.size() > 0) {
TRIGGERED_JOB = CURRENT_BUILD.upstreamBuilds[0].projectName
if (!TRIGGERED_JOB.isEmpty()) {
return TRIGGERED_JOB
}
}
return "Self"
}
CURRENT_BUILD is env var currentBuild
How to return Username & UserId:
UserName: currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserName()
UserId: currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
There is other way to get user_id, where you don't need to install anything.
BUILD_USER_ID = sh (
script: 'id -u',
returnStdout: true
).trim()
echo "bUILD USER: ${BUILD_USER_ID }"
For declarative pipeline syntax, here is a quick hack, base on #Kevin answer.
For declarative pipeline you need to enclose them in a node, else you will get an error/ build failure
node {
def BUILD_FULL = sh (
script: 'curl --silent '+buildURL+' | tr "{}" "\\n" | grep -Po \'"shortDescription":.*?[^\\\\]"\' | cut -d ":" -f2',
returnStdout: true
)
slackSend channel: '#ci-cd',
color: '#000000',
message: "The pipeline was ${BUILD_FULL} ${GIT_COMMIT_MSG} "
}
The output will be slack notification sent to your slack channel with the git short description
I have a Jenkins job with 100+ builds. I need to search through all the builds of that job to find builds that have a certain string in the console output. Is there any plugin for that? How do I do that?
I often use the Jenkins Script Console for tasks like this. The Groovy plugin provides the Script Console, but if you're going to use the Script Console for periodic maintenance, you'll also want the Scriptler plugin which allows you to manage the scripts that you run.
From Manage Jenkins -> Script Console, you can write a groovy script that iterates through the job's builds looking for the matching string:
JOB_NAME = "My Job"
BUILD_STRING = "Hello, world"
def job = Jenkins.instance.items.find { it.name == JOB_NAME }
for (build in job.builds) {
def log = build.log
if (log.contains(BUILD_STRING)) {
println "${job.name}: ${build.id}"
}
}
If there is no additional requirements I would do it simply in the shell, e.g.:
find $JENKINS_HOME/jobs/haystack -name log -exec grep -l needle {} \; \
| sed 's|.*/\(.*\)/log|\1|'
To search in logs of all jobs:
I enhanced #DaveBacher 's code to be run in the Jenkins script console.
Helped me to locate a sporadic error happening in multiple jobs.
NEEDLE = "string_i_am_looking_for"
for (job in Jenkins.instance.getAllItems(Job.class)) {
for (build in job.builds) {
def log = build.log
if (log.contains(NEEDLE)) {
println "${job.name}: ${build.id}"
}
}
}
Thanks everyone for your valuable solutions. After a bit of additional research i found that there is a plugin in Jenkins to do this.
https://wiki.jenkins-ci.org/display/JENKINS/Lucene-Search
This will save the console output results and users can do search in search box.
There is the Log Parser Plugin
highlighting lines of interest in the log (errors, warnings,information)
dividing the log into sections displaying a summary of number of errors, warnings and information lines within the log and its sections.
linking the summary of errors and warnings into the context of the full log, making it easy to find a line of interest in the log
showing a summary of errors and warnings on the build page
If it is old logs then #jil has the answer assuming you are on Linux.
Just to throw another plugin out there, this blog post pointed me at the TextFinder plugin which allows you to search for text in either a workspace file or the console output, and override the build status as success/failure when the text is found.
The original poster doesn't say what should happen when the text is found, but it was searching for this functionality that brought me here.
To search for a regex pattern in all Jenkins jobs, and print the first matching line:
for (job in Jenkins.instance.items) {
for (build in job.builds) {
try {
def log = build.log
def match = log =~ "\n(.*${PATTERN}.*)\n"
if (match) {
println "Job [${job.name}] - Build [${build.id}]: ${match[0][0]}"
}
}
catch (Exception e) {
println e
}
}
}
For example, searching in my builds for PATTERN = "(TLS|Build).*timeout" I found:
Job [OSP-AWS] - Build [83]: Build timeout: dial tcp
[::1]:6443: connect: connection refused
Job [OSP-GCP] - Build [21]: Unable to
connect to the server: net/http: TLS handshake timeout
Just use Jenkins std search (top right corner) with keyword "console":
console:"whatever you are looking for"
I delete old jenkins builds with rm where job is hosted:
my_job/builds/$ rm -rf [1-9]*
These old builds are still visible in job page.
How to remove them with command line?
(without the delete button in each build user interface)
Here is another option: delete the builds remotely with cURL. (Replace the beginning of the URLs with whatever you use to access Jenkins with your browser.)
$ curl -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The above deletes build #1 to #56 for job myJob.
If authentication is enabled on the Jenkins instance, a user name and API token must be provided like this:
$ curl -u userName:apiToken -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The API token must be fetched from the /me/configure page in Jenkins. Just click on the "Show API Token..." button to display both the user name and the API token.
Edit: As pointed out by yegeniy in a comment below, one might have to replace doDeleteAll by doDelete in the URLs above to make this work, depending on the configuration.
It looks like this has been added to the CLI, or is at least being worked on: http://jenkins.361315.n4.nabble.com/How-to-purge-old-builds-td385290.html
Syntax would be something like this: java -jar jenkins-cli.jar -s http://my.jenkins.host delete-builds myproject '1-7499' --username $user --password $password
Check your home jenkins directory:
"Manage Jenkins" ==> "Configure System"
Check field "Home directory" (usually it is /var/lib/jenkins)
Command for delete all jenkins job builds
/jenkins_home/jobs> rm -rf */builds/*
After delete should reload config:
"Manage Jenkins" ==> "Reload Configuration from Disk"
You can do it by Groovy Scripts using Hudson API.. Access your jenkins instalation
http://localhost:38080/script.
For Example, for deleting all old builds of all projects using the follow script:
Note: Take care if you use Finger Prints , you will lose all history.
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
System.out.println("Deleting all builds of job "+item.name)
for(build in item.getBuilds()){
build.delete()
}
}
else {
System.out.println("Skipping job "+item.name+", currently building")
}
}
Or for cleaning all workspaces :
import hudson.model.*
// For each project
for(item in Hudson.instance.items) {
// check that job is not building
if(!item.isBuilding()) {
println("Wiping out workspace of job "+item.name)
item.doDoWipeOutWorkspace()
}
else {
println("Skipping job "+item.name+", currently building")
}
}
There are a lot of examples on the Jenkins wiki
Is there a reason you need to do this manually instead of letting Jenkins delete old builds for you?
You can change your job configuration to automatically delete old builds, based either on number of days or number of builds. No more worrying about it or having to keep track, Jenkins just does it for you.
The following script cleans old builds of jobs. You should reload config from disk if you delete build manually:
import hudson.model.*
for(item in Hudson.instance.items) {
if (!item.isBuilding()) {
println("Deleting old builds of job " + item.name)
for (build in item.getBuilds()) {
//delete all except the last
if (build.getNumber() < item.getLastBuild().getNumber()) {
println "delete " + build
try {
build.delete()
} catch (Exception e) {
println e
}
}
}
} else {
println("Skipping job " + item.name + ", currently building")
}
}
From Script Console Run this, but you need to change the job name:
def jobName = "name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
From Jenkins Scriptler console run the following Groovy script to delete all the builds of jobs listed under a view:
import jenkins.model.Jenkins
hudson.model.Hudson.instance.getView('<ViewName>').items.each() {
println it.fullDisplayName
def jobname = it.fullDisplayName
def item = hudson.model.Hudson.instance.getItem(jobname)
def build = item.getLastBuild()
if (item.getLastBuild() != null) {
Jenkins.instance.getItemByFullName(jobname).builds.findAll {
it.number <= build.getNumber()
}.each {
it.delete()
}
}
}
def jobName = "MY_JOB_NAME"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().findAll { it.number < 10 }.each { it.delete() }
if you had 12 builds this would clear out builds 0-9 and you'd have 12,11,10 remaining. Just drop in the script console
This script will configure the build retention settings of all of the Jenkins jobs.
Change the values from 30 and 200 to suite you needs, run the script, then restart the Jenkins service.
#!/bin/bash
cd $HOME
for xml in $(find jobs -name config.xml)
do
sed -i 's#<daysToKeep>.*#<daysToKeep>30</daysToKeep>#' $xml
sed -i 's#<numToKeep>.*#<numToKeep>200</numToKeep>#' $xml
done
The script below works well with Folders and Multibranch Pipelines. It preserves only 10 last builds for each job. That could be adjusted or removed (proper if) if needed. Run that from web script console (example URL: https://jenkins.company.com/script)
def jobs = Hudson.instance.getAllItems(hudson.model.Job.class)
for (job in jobs){
println(job)
def recent = job.builds.limit(10)
for(build in job.builds){
if(!recent.contains(build)){
println("\t Deleting build: " + build)
build.delete()
}
}
}
From my opinion all those answers are not sufficient, you have to do:
echo "Cleaning:"
echo "${params.PL_JOB_NAME}"
echo "${params.PL_BUILD_NUMBER}"
build_number = params.PL_BUILD_NUMBER as Integer
sleep time: 5, unit: 'SECONDS'
wfjob = Jenkins.instance.getItemByFullName(params.PL_JOB_NAME)
wfjob.getBuilds().findAll { it.number >= build_number }.each { it.delete() }
wfjob.save()
wfjob.nextBuildNumber = build_number
wfjob.save()
wfjob.updateNextBuildNumber(build_number)
wfjob.save()
wfjob.doReload()
Or the job will not be correctly reset and you have to hit build until you reach next free number in the meanwhile the jenkins log will show:
java.lang.IllegalStateException: JENKINS-23152: ****/<BUILD_NUMBER> already existed;
I recently updated the configuration of one of my hudson builds. The build history is out of sync. Is there a way to clear my build history?
Please and thank you
Use the script console (Manage Jenkins > Script Console) and something like this script to bulk delete a job's build history https://github.com/jenkinsci/jenkins-scripts/blob/master/scriptler/bulkDeleteBuilds.groovy
That script assumes you want to only delete a range of builds. To delete all builds for a given job, use this (tested):
// change this variable to match the name of the job whose builds you want to delete
def jobName = "Your Job Name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
// uncomment these lines to reset the build number to 1:
//job.nextBuildNumber = 1
//job.save()
This answer is for Jenkins
Go to your Jenkins home page → Manage Jenkins → Script Console
Run the following script there. Change copy_folder to your project name
Code:
def jobName = "copy_folder"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
My post
If you click Manage Hudson / Reload Configuration From Disk, Hudson will reload all the build history data.
If the data on disk is messed up, you'll need to go to your %HUDSON_HOME%\jobs\<projectname> directory and restore the build directories as they're supposed to be. Then reload config data.
If you're simply asking how to remove all build history, you can just delete the builds one by one via the UI if there are just a few, or go to the %HUDSON_HOME%\jobs\<projectname> directory and delete all the subdirectories there -- they correspond to the builds.
Afterwards restart the service for the changes to take effect.
Here is another option: delete the builds with cURL.
$ curl -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The above deletes build #1 to #56 for job myJob.
If authentication is enabled on the Jenkins instance, a user name and API token must be provided like this:
$ curl -u userName:apiToken -X POST http://jenkins-host.tld:8080/jenkins/job/myJob/[1-56]/doDeleteAll
The API token must be fetched from the /me/configure page in Jenkins. Just click on the "Show API Token..." button to display both the user name and the API token.
Edit: one might have to replace doDeleteAll by doDelete in the URLs above to make this work, depending on the configuration or the version of Jenkins used.
Here is how to delete ALL BUILDS FOR ALL JOBS...... using the Jenkins Scripting.
def jobs = Jenkins.instance.projects.collect { it }
jobs.each { job -> job.getBuilds().each { it.delete() }}
You could modify the project configuration temporarily to save only the last 1 build, reload the configuration (which should trash the old builds), then change the configuration setting again to your desired value.
If you want to clear the build history of MultiBranchProject (e.g. pipeline),
go to your Jenkins home page → Manage Jenkins → Script Console and run the following script:
def projectName = "ProjectName"
def project = Jenkins.instance.getItem(projectName)
def jobs = project.getItems().each {
def job = it
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
}
This one is the best option available.
Jenkins.instance.getAllItems(AbstractProject.class).each {it -> Jenkins.instance.getItemByFullName(it.fullName).builds.findAll { it.number > 0 }.each { it.delete() } }
This code will delete all Jenkins Job build history.
Using Script Console.
In case the jobs are grouped it's possible to either give it a full name with forward slashes:
getItemByFullName("folder_name/job_name")
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
or traverse the hierarchy like this:
def folder = Jenkins.instance.getItem("folder_name")
def job = folder.getItem("job_name")
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()
Deleting directly from file system is not safe. You can run the below script to delete all builds from all jobs ( recursively ).
def numberOfBuildsToKeep = 10
Jenkins.instance.getAllItems(AbstractItem.class).each {
if( it.class.toString() != "class com.cloudbees.hudson.plugins.folder.Folder" && it.class.toString() != "class org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject") {
println it.name
builds = it.getBuilds()
for(int i = numberOfBuildsToKeep; i < builds.size(); i++) {
builds.get(i).delete()
println "Deleted" + builds.get(i)
}
}
}
Go to "Manage Jenkins" > "Script Console"
Run below:
def jobName = "build_name"
def job = Jenkins.instance.getItem(jobName)
job.getBuilds().each { it.delete() }
job.save()
Another easy way to clean builds is by adding the Discard Old Plugin at the end of your jobs. Set a maximum number of builds to save and then run the job again:
https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
Go to the %HUDSON_HOME%\jobs\<projectname> remove builds dir and remove lastStable, lastSuccessful links, and remove nextBuildNumber file.
After doing above steps go to below link from UI
Jenkins-> Manage Jenkins -> Reload Configuration from Disk
It will do as you need
If using the Script Console method then try using the following instead to take into account if jobs are being grouped into folder containers.
def jobName = "Your Job Name"
def job = Jenkins.instance.getItemByFullName(jobName)
or
def jobName = "My Folder/Your Job Name
def job = Jenkins.instance.getItemByFullName(jobName)
Navigate to: %JENKINS_HOME%\jobs\jobName
Open the file "nextBuildNumber" and change the number. After that reload Jenkins configuration. Note: "nextBuildNumber" file contains the next build no that will be used by Jenkins.
Tested on jenkins 2.293 over linux. It will remove all the build logs but not the corellative build number
cd /var/lib/jenkins/jobs
find . -name "builds" -exec rm -rf {} \;
Be careful with this command because it executes a rm -rf on each find result. You could exec this first to validate if the result are only the builds folder of you jobs
find . -name "builds"
If you are looking for a solution where you have job inside a Folder you can use getItemByFullName function. It also supports white space in folder and job name.
def jobName = "folder_name/job_name"
def job = Jenkins.instance.getItemByFullName(jobName)
job.getBuilds().each { it.delete() }
job.nextBuildNumber = 1
job.save()