We have a Jenkins job where html reports are generated and placed in workspace/reports. We have been using "Workspace Cleanup Plugin" so we can keep only the last report. We would like to keep the reports created during the past X days and delete the older ones. Is there a way or a plugin that can automate this?
It's not a good idea to put something into job's own workspace as an archive that needs to be accessed later. Someone may eventually clean the workspace (either manually or via jenkins pre- or post-build step) and all your precious data and statistics will be gone.
I'd suggest to publish reports using a different directory and then run a cleanup manually in it. You may as well define Jenkins global parameter (plugin) such as REPORT_ROOT=/home/${USER}/reports and use it in job config to save html reports to ${REPORT_ROOT}/${JOB_NAME}
To cleanup you'd want to run find ${REPORT_ROOT} -type f -mtime +2 -delete provided that there is no parent directory created for each separate report. This can be run in a separate job or as subproject to the job that publishes html report or in the job itself. -mtime +2 means older than 2 days.
If each build has it's own directory for html reports than the cleanup would be find ${REPORT_ROOT}/${JOB_NAME} -type d -mtime +2 -delete
Use PeriodicBackupPlugin.
It helps to take back backup of your data periodically and has a BackupExecutor that will check existing backups in each location and delete number of backups older than X number of days defined in configuration.
Check the image for info on configuration.
There is a Configuration Slicing Plugin that could help you
Related
Though I have seen many were mentioning to use Deleteworkspace plugin to use and corresponding script in post build actions of the pipeline.
here we are using the multibranch pipeline.. I'm facing the challenges to maintain the storage space ..as it is running out for every build.
my management is not recommended to delete the workspace folder..instead choose the files in workspace and delete.
upon using the command du-sh ./*/ ..i can see workspace storage is 2.5 GB, but when i cd into workspace .. the ls -lart ..got to see ...all files with less size only 1 GB approxiamtely ( not upto 2.5GB)..
can someone. assist me, where i have missed ..
and help me to understand to restore the space in Jenkins
I need to archive any .png screenshots of the latest katalon test run in Jenkins as a post-build step of the same test run.
Using the "archive the artifacts" post-build action in Jenkins, I currently have the file path set to Reports/**/[test run name]/**/*.png where Reports is in the workspace directory. However this will just pull every .png file from the current and all previous test runs stored there, of which are kept stored in the workspace for a week before being cleaned out.
I've tried using the "Exclude" field but haven't been able to figure out a way to exclude older files with only being able to use a file path with wildcards.
Is there a way in Jenkins, using archive the artifacts or something else, to archive just the .png files generated by the same job without deleting all previously generated .png files?
From our experience, it's better to keep the artifacts archived with the job and not rely on files residing in the workspace together with a separate job to clean them. You can tell Jenkins to discard old builds (and artifacts) with something like this:
pipeline {
options {
buildDiscarder(logRotator(daysToKeepStr: '7', artifactDaysToKeepStr: '7'))
}
}
And any jobs (and their artifacts) will be cleaned after 7 days.
If you go with that path, you can safely remove any png files after archiving them as artifacts, and you won't need to find out which are new and which are old.
Alternatively, you can order your png-producing step to name the files starting with the job number (available as env.BUILD_NUMBER) and only archive the files starting with this number.
Finally, you can run find command with -name '*png' and -mtime predicate to produce the list of recently modified png files, and use that as an input to the archive step.
I wonder if it is possible to remove only one build (including artifacts) from job workspace.
I tried to "Delete Build" in Build History but all it does is remove build reference from Build History table. I know I can ssh to a server and delete files from the command line but I am looking for a way to do it from Jenkins web interface.
After installing Workspace Cleanup Plugin I am able to wipe out current workspace but I want to keep my other builds in the workspace.
In your Jenkins instance, to be able to have folder/per build - set flag "Use custom workspace" in your job's settings. Here is a brief help info from the setting description:
For each job on Jenkins, Jenkins allocates a unique "workspace directory."
This is the directory where the code is checked out and builds happen.
Normally you should let Jenkins allocate and clean up workspace directories,
but in several situations this is problematic, and in such case, this option
lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be
built on a specific location. While there's no doubt that such a build is
not ideal, this option allows you to get going in such a situation.
...
And your custom directory path would look like this:
workspace\$JOB_NAME\$BUILD_NUMBER ~> workspace\my-job-name\123
where $JOB_NAME will be "my-job-name" and $BUILD_NUMBER is the build number, eq. "123".
There is one nasty problem with this approach and this is why I wouldn't recommend to use it - Jenkins will not be able to reclaim disk space for outdated builds. You would have to handle cleanup of outdated builds manually and it is a lot of hassle.
Alternative approach, that gives you more control, tools and is able to keep disk space usage under control (without your supervision) is to use default workspace settings and archive your build output (files, original source code, libraries and etc.) as a post-build action. Very-very handy and gives you access to a whole bunch of great tools like, Copy Artifact Plugin or ArtifactDeployer Plugin in other jobs.
Hope that info helps you make a decision that fits your needs best.
I also use "General/Advanced/Use custom workspace" (as in #pabloduo's answer) on a Windows machine with something like:
C:\${JOB_NAME}\${BUILD_NUMBER}
Just wanted to add a solution for getting rid of the build job's workspaces.
I use Groovy Events Listener Plugin for this.
Using the plug-in's standard configuration I just use the following Groovy script:
if (event == Event.JOB_DELETED){
new File(env.WORKSPACE).deleteDir()
}
And now the custom workspace is deleted when the build job is deleted.
Just be aware that this would also delete non-custom workspaces (because the event is triggered for all jobs on your Jenkins server).
I'm a little new to Jenkins, and I can't seem to figure this out. I have access to a Jenkins server that uses slaves to perform build jobs.
If a build fails, it stores a generated zip archive in a persistent Workspace directory for further debugging. The zip file is generated by a python script that keeps track of only the last 3 failed builds to conserve memory (i.e. 3 failed builds will result in 3 archives in the folder, but a fourth failed build will delete the oldest archive before adding the new one).
What I'm trying to do is add a download link to a failed Jenkins Run to allow users to quickly download the zip file that was generated for that build. But I'm really confused as to how approach this!
So I guess the question is, how could I add a download link to a Jenkins Run page to a file generated during that Run if it fails?
Example usage scenario:
1. I build some code :)
2. It fails :(
3. I download the zip file (from the Run page) with the generated debug files and find the fix :)
4. Space doesn't get filled up as zip files are kept only for the last 3 builds!
Any help would be greatly appreciated! Thank you! I'm happy to provide more information if needed ^^ I am currently trying to use a system groovy script to do this, but perhaps artifacts would be more appropriate? I really can't seem to find good documentation on this!
There are built in methods in Jenkins to allow this workflow:
you can archive any artifact (in that case the zip) as post build step
data retention strategy can be configured in the job via Discard old builds (Advanced).
in order to send out customized mails on build failure with embedded download link you should review Email Ext Plugin; it allows you to configure individual texts for e.g. build failures where you could add the link to downloading the artifact.
I'm trying to understand the purpose of the archive directory.
From what I've seen, the archive is per build. Meaning if I set the 'Max # of builds to keep' to 3 for example, The archive will also be deleted.
Is that correct?
If so, how can I tell Jenkins to only save my artifact (zip file) created and delete the build directory?
Is there a one archive directory for all builds?
There are two directories for a Jenkins job, by default (there can be just one, builds, in case of a Workflow job, or more, e.g. an additional promotions directory):
JENKINS_HOME/jobs/${JOB_NAME}/builds/ containing sub-dirs according to $BUILD_ID
JENKINS_HOME/jobs/${JOB_NAME}/workspace/
So, there's one workspace per job and one build directory per build.
Don't let the page Administering Jenkins with this change driven by JENKINS-8446 confuse you. The workspace default, as also mentioned in the inline help (of the current v1.635 at the time of this writing), is still:
Manage Jenkins → Configure System → Advanced... → Workspace Root Directory: ${ITEM_ROOTDIR}/workspace
It's NOT ${JENKINS_HOME}/workspace/${JOB_NAME}.
If a discard of old builds takes place the workspace isn't discarded, but, since it exists per job, it is overwritten with every new build. Hence, if you'd like to keep old artifacts you can:
create an "archive" job that uses the Copy Artifact Plugin and connect it as downstream to your build job.
move them to some other location yourself (including creation of uniquely-named sub-dirs to store the artifact(s) in) using a Post-build Action → Groovy Postbuild or → Execute a set of scripts.
There's also the Discard Old Build plugin which enhances Jenkins' built-in discard functionality via a Post-build Action.