Are there any possibilities to trigger my pipeline only, if there exists a special file in my filesystem.
So I want to start a new build on my pipeline at 9 p.m. But only, if there is the file run.xy on my filesystem.
At your Build periodically in Build Triggers section add 0 21 * * *.
And pipeline start by something like this:
def folder = new File( 'filepath' )
// If it does exist
if( folder.exists() ) {
//run build
} else {
println "File doesn't exist"
}
Related
I want to delete old builds for all the jobs I kept inside a folder from the script console, not from Jenkins individual job configuration to discard old builds. I want to keep only the recent 10 builds and remove the older ones. I am asking about the deletion of the old builds of a particular folder and not other jobs in another folder.
The following groovy script will delete all the builds excluding the 10 most recent builds of each job configured in Jenkins.
import jenkins.model.*
import hudson.model.*
Jenkins.instance.getAllItems(AbstractItem.class).each {
def job = jenkins.model.Jenkins.instance.getItem(it.fullName)
if (job!=null){
println "job: " + job;
int i = 1;
job.builds.each {
def build = it;
if (i<=10){
println "build: " + build.displayName + " will NOT BE DELETED";
}else{
println "build: " + build.displayName + " will BE DELETED";
it.delete();
}
i = ++i
}
}
};
The first loop will iterate over all Jenkins items and will store it under job var. If the job is not a Jenkins Job the result can be null, so the if (job!=null) check is there to allow the code to proceed only for Jenkins jobs.
The int i = 1; is the initialization of a counter for each job, that is incremented when a build is found. As far as I know, the build order is preserved, and the most recent build is returned first in the loop. When the counter reaches 10 or more, the if..else enters the else block and deletes the build it.delete()
I want to create a groovy function in my Jenkins job that looks into a folder and deletes all files who are older than X days.
So I start looking in the internet and found different kind of solutions.
At first I create a .groovy file with Visual Studio Code on my local PC to understand how it works. That is the reason why my code looks not similar to the codes in the internet because I changed it so that I understand how the code works.
def deleteFilesOlderThanDays(int daysBack, String path) {
def DAY_IN_MILLIS = 24 * 60 * 60 * 1000
File directory = new File(path)
if(directory.exists()){
File[] listFiles = directory.listFiles()
for(File listFile : listFiles) {
def days_from_now = ( (System.currentTimeMillis() - listFile.lastModified()) /(DAY_IN_MILLIS))
if(days_from_now > daysBack) {
println('------------')
println('file is older')
println(listFile)
}
else{
println('------------')
println('File is not older')
println(listFile)
}
}//End: for(File listFile : listFiles) {
}//End: if(directory.exists()){
}
(I know, the code do not delete something. It is only for my understanding)
The second step was to include this new created function into my Jenkins groovy file. But since then I'm desperate.
I have the problem that I do not get a positive result at the beginning from the code if the folder really exist.
The line:
if(directory.exists()){
makes me a lot of problems and it is not clear for me why.
I have tried so many kind of versions but I haven’t found a solution for me.
I have also used the “Pipeline Syntax” example [Sample Step fileExists] but it doesn’t help for me.
I have included:
import java.io.File
At the beginning of my file.
I have a basic file which I include in the Jenkins job. This file includes my library files. One of this library files is the file.groovy. In the basic Jenkins file I execute the function file.deleteFilesOlderThanDays() (for testing I do not use any parameters).
The code from my function for testing is:
def deleteFilesOlderThanDays() {
dir = '.\\ABC'
echo "1. ----------------------------------------"
File directory1 = new File('.\\ABC\\')
exist = directory1.exists()
echo 'Directory1 name is = '+directory1
echo 'exist value is = '+exist
echo "2. ----------------------------------------"
File directory2 = new File('.\\ABC')
exist = directory2.exists()
echo 'Directory2 name is = '+directory2
echo 'exist value is = '+exist
echo "3. ----------------------------------------"
File directory3 = new File(dir)
exist = directory3.exists()
echo 'Directory3 name is = '+directory3
echo 'exist value is = '+exist
echo "4. Pipeline Syntax ------------------------"
exist = fileExists '.\\ABC'
echo 'exist value is = '+exist
echo "5. ----------------------------------------"
File directory5 = new File(dir)
echo 'Directory5 name is = '+directory5
// execute an error
// exist = fileExists(directory5)
exist = fileExists "directory5"
echo 'exist value is = '+exist
echo "6. ----------------------------------------"
exist = fileExists(dir)
echo 'exist value is = '+exist
File[] listFiles = directory5.listFiles()
echo 'List file = '+listFiles
}
And the Output in the Jenkins Console Output is: (I cleaned it a little bit up….)
1. ----------------------------------------
Directory1 name is = .\ABC\
exist value is = false
2. ----------------------------------------
Directory2 name is = .\ABC
exist value is = false
3. ----------------------------------------
Directory3 name is = .\ABC
exist value is = false
4. Pipeline Syntax ------------------------
exist value is = true
5. ----------------------------------------
Directory5 name is = .\ABC
exist value is = false
6. ----------------------------------------
exist value is = true
List file = null
I only get a true value in step 4 and 6. So I can be sure that the folder really exist.
So it seems to be for me that the command:
File directory = new File(dir)
Not work correct in my case.
I can’t create a listFile variable because the directory would not be initialized correct.
For me is also not clear which kind of commands I should use. The groovy examples use always functions like:
.exists()
But in the Jenkins examples I always find code like this:
fileExists()
Why there are some differences between groovy and Jenkins groovy style? It should be the same ore not?
Does anyone have an idea for me or can told me what I’m doing wrong?
You may benefit from this answer from a similar question:
"
java.io.File methods will refer to files on the master where Jenkins is running, so not in the current workspace on the slave machine.
To refer to files on the slave machine, you should use the readFile method
"
def dir = readFile("${WORKSPACE}/ABC");
Link to original answer
Thanks for all that feedback.
OK, for me is now clear that Jenkins Groovy != Groovy is.
I have read a lot about it that there are different command if you are executing file search on a Jenkins Master or on a Jenkins Slave.
The suggestion from Youg to start after confirmation helps me.
I had problems with deleting the file so at the end I used a primitive batch command to get my function run.
The finally functions looks like now:
def deleteFilesOlderThanXDays(daysBack, path) {
def DAY_IN_MILLIS = 24 * 60 * 60 * 1000
if(fileExists(path)){
// change into path
dir(path) {
// find all kind of files
files = findFiles(glob: '*.*')
for (int i = 0; i < files.length; i++) {
def days_from_now = ( (System.currentTimeMillis() - files[i].lastModified) /(DAY_IN_MILLIS))
if(days_from_now > daysBack) {
echo('file : >>'+files[i].name+'<< is older than '+daysBack+' days')
bat('del /F /Q "'+files[i].name+'"')
}
else{
echo('file : >>'+files[i].name+'<< is not only than '+daysBack+' days')
}
}// End: for (int i = 0; i < files.length; i++) {
}// End: dir(path) {
}// End: if(fileExists(path)){
}
Thanks for helping and best regards,
You can add below script to list the file and folders in current work directory, so that you can confirm the folder ABC is exists or not.
After you confirm the ABC folder exist, then dig into the rest code.
def deleteFilesOlderThanDays() {
// print current work directory
pwd
// if jenkins job run on window machine
bat 'dir'
// if jenkins job run on linux machine
sh 'ls -l'
dir = '.\\ABC'
echo "1. ----------------------------------------"
.....
For fileExists usage, I think the correct way as following:
fileExists './ABC'
def dir = './ABC'
fileExists dir
Should use / as path separator, rather than \ according to its document at below:
I wrote a plugin following
http://www.baeldung.com/jenkins-custom-plugin
And it generates a html report
File artifactsDir = build.getArtifactsDir();
String path = artifactsDir.getCanonicalPath() + REPORT_TEMPLATE_PATH;
File reportFile = new File("path");
// write report's text to the report's file
and for the next build, I want to import this report file to see the changes
I tried these but none of them works
build.getPreviousSuccessfulBuild().getArtifactManager().root() + REPORT_TEMPLATE_PATH
// fail with File not found, but the file is there in bash
build.getPreviousSuccessfulBuild().getArtifactsDir() + REPORT_TEMPLATE_PATH
// null pointer exception, seems to be generated by getArtifactsDir()
build.getPreviousBuild().getArtifactManager().root() + REPORT_TEMPLATE_PATH
So how can I obtain the last successful build report file within current build ?
This is how I did it in an pipeline job. I stripped parts of the original code, I hope I didn't introduce an error. I also removed error handling for clarity:
// find last successful build
def lastSuccessfulBuild = currentBuild.getPreviousBuild()
while (lastSuccessfulBuild && (lastSuccessfulBuild.currentResult != 'SUCCESS')) {
lastSuccessfulBuild = lastSuccessfulBuild.getPreviousBuild()
}
// here I go for a file named 'crc.txt'
// this works only if you have a
// archiveArtifacts artifacts: 'crc.txt', fingerprint: true
// somewhere in your build
def build = lastSuccessfulBuild?.getRawBuild()
def artifact = build.getArtifacts().find { it.fileName == 'crc.txt' }
def uri = build.artifactManager.root().child(artifact.relativePath).toURI()
def content = uri.toURL().text
When I compare our solutions: you don't use child() and you have the relative path in REPORT_TEMPLATE_PATH while I obtain it from the artifact.
What is a good strategy for cleaning Jenkins workspace? After getting started with Jenkins in a large team, available workspace space is shrinking, even through jobs/pipelines seems to do due diligence to clean after themselves.
The current approach I am trying: iterate through every node (including master),
under $WORKSPACE, find and delete directories over given age. Does this seem safe?
I tried online groovy code snippets to run via Jenkins console, and those got rejected with errors. I'd rather stick with simple code that is known to work.
Many thanks.
I have a stage for each run that cleans the work space:
node {
try {
stage('Clean Work Space') {
cleanWs()
sh 'pwd'
sh 'ls'
}
}
}
I would also refer to: https://jenkins.io/doc/pipeline/steps/ws-cleanup/
You can use the "Discard Old Builds" option on the Jobs to limit the number of builds stored for every job. This will reduce the disk space usage.
you can use deleteDir() command to clean the work space.
The best strategy to delete the Workspaces is to either go by Sprint numbers (if you are using agile) or by Date modified (to retain latest top 5).
I have created a script that has Groovy scripts for both of the strategies mentioned above.
Here is my detailed blog on strategies to delete Jenkins workspace for all jobs and individual jobs.
https://medium.com/#vishnuteja/jenkins-cleanup-workspace-reduce-disk-usage-18310097d3ef
Here is the code for cleaning up workspaces by Date modified.
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
//manager.listener.logger.println new Date(System.currentTimeMillis()).format('MM/dd/yyyy hh:mm:ss a') + " / " + " -- Start Time"
//Get value from String Parameter
MAX_BUILDS = manager.build.buildVariables.get("MAX_BUILDS").toInteger()
for (job in Jenkins.instance.items)
{
int count = 0
manager.listener.logger.println "\n ***Job Name: "+job.name+"***"
if(job.workspace!=null && job.workspace!="") //Check if there is a workspace associated with the Job
{
manager.listener.logger.println "Workspace path : " + job.workspace
String workspace = job.workspace
File folder = new File(workspace)
if(folder!=null && folder.exists()) //Check if the Workspace folder exists
{
// Get all files and folders within the Workspace of current job.
//Iterate through only folders and sort em by Modified Date.
File[] files = new File(workspace).listFiles().sort(){
a,b -> b.lastModified().compareTo a.lastModified()
}
.each{
if(!it.isFile()) //Check only for folders
{
if(count < MAX_BUILDS)
manager.listener.logger.println new Date(it.lastModified()).format('MM/dd/yyyy hh:mm:ss a') + " /" + it.name + " -- Save"
else
{
manager.listener.logger.println new Date(it.lastModified()).format('MM/dd/yyyy hh:mm:ss a') + " /" + it.name + " ** Deleted"
it.deleteDir()
}
count++
}
}
}
else
{
manager.listener.logger.println "Workspace is empty or doesn't exist"
}
}
else
{
manager.listener.logger.println "No Workspace associated with this job"
}
}
}
//manager.listener.logger.println new Date(System.currentTimeMillis()).format('MM/dd/yyyy hh:mm:ss a') + " / " + " -- End Time"
I am able to load Jenkinsfile automatically through multi branch pipeline plugin with a limitation of only one jenkinsfile per branch.
I have multiple Jenkinsfiles per branch which I want to load, I have tried with below method by creating master Jenkins file and loading specific files. In below code it merges 1.Jenkinsfile and 2.Jenkinsfile as one pipeline.
node {
git url: 'git#bitbucket.org:xxxxxxxxx/pipeline.git', branch: 'B1P1'
sh "ls -latr"
load '1.Jenkinsfile'
load '2.Jenkinsfile'
}
Is there a way I can load multiple Jenkins pipeline code separately from one branch?
I did this writing a share library (ref https://jenkins.io/doc/book/pipeline/shared-libraries/) containing the following file (in vars/generateJobsForJenkinsfiles.groovy):
/**
* Creates jenkins pipeline jobs from pipeline script files
* #param gitRepoName name of github repo, e.g. <organisation>/<repository>
* #param filepattern ant style pattern for pipeline script files for which we want to create jobs
* #param jobPath closure of type (relativePathToPipelineScript -> jobPath) where jobPath is a string of formated as '<foldername>/../<jobname>' (i.e. jenkins job path)
*/
def call(String gitRepoName, String filepattern, def jobPath) {
def pipelineJobs = []
def base = env.WORKSPACE
def pipelineFiles = new FileNameFinder().getFileNames(base, filepattern)
for (pipelineFil in pipelineFiles) {
def relativeScriptPath = (pipelineFil - base).substring(1)
def _jobPath = jobPath(relativeScriptPath).split('/')
def jobfolderpath = _jobPath[0..-2]
def jobname = _jobPath[-1]
echo "Create jenkins job ${jobfolderpath.join('/')}:${jobname} for $pipelineFil"
def dslScript = []
//create folders
for (i=0; i<jobfolderpath.size(); i++)
dslScript << "folder('${jobfolderpath[0..i].join('/')}')"
//create job
dslScript << """
pipelineJob('${jobfolderpath.join('/')}/${jobname}') {
definition {
cpsScm {
scm {
git {
remote {
github('$gitRepoName', 'https')
credentials('github-credentials')
}
branch('master')
}
}
scriptPath("$relativeScriptPath")
}
}
configure { d ->
d / definition / lightweight(true)
}
}
"""
pipelineJobs << dslScript.join('\n')
//println dslScript
}
if (!pipelineJobs.empty)
jobDsl sandbox: true, scriptText: pipelineJobs.join('\n'), removedJobAction: 'DELETE', removedViewAction: 'DELETE'
}
Most likely you want to map old Jenkins' jobs (pre pipeline) operating on single branch of some project to a single multi branch pipeline. The appropriate approach would be to create stages that are input dependent (like a question user if he/she wants to deploy to staging / live).
Alternatively you could just create a new separate Pipeline jenkins job that actually references an your project's SCM and points to your other Jenkinsfile (then one pipeline job per every other jenkinsfile).