Is it possible in Jenkins to only delete specific artifacts created by a build job?
In my situation I create 4 artifacts - 3 of which are 2kb .txt files and one of which is 0.5gb tar.gz. I'd like to keep the 3 smaller artifacts indefinitely, but aggressively remove the tar.gz artifacts after 5 builds.
Using logrotator this doesn't seem to be an option (it can only remove all artifacts or none) but I was hoping there might be a way to write a Groovy library that could be called after each new build, or a pluggin that could handle this for me?
See: https://javadoc.jenkins.io/hudson/tasks/LogRotator.html
Does anyone have any pointers on how to get started with this?
You can safely remove build artifacts directly on filesystem level.
If you want to trigger this directly from within Jenkins, the easiest way is probably to write a Groovy post-build script that takes care of the deletion.
Related
I was recently put in charge of all Jenkins-related work at my job, and was tasked with storing build artifacts from our declarative pipelines in a place where:
- They are accessible to everyone on the team
- They can be stored for long periods of time
Ideally they would be visible on the Jenkins interface, where they appear when using the default 'archiveArtifacts' command. I know this saves them in the JENKINS_HOME directory. The problem is that I have to discard old builds to avoid running out of space and the artifacts are deleted with them. Furthermore, I don't have access to the server that Jenkins runs on because it's managed by a separate team, so I can't go into JENKINS_HOME.
I looked into a few ARMs like Nexus and Artifactory, but from my understanding those are only supposed to be used for full releases. I'm looking to save artifacts after every new merge, which can happen multiple times a day.
I'm currently saving them on a functional user's home directory, but I'm the only one with direct access to it so that's no good. I also looked into plugins like ArtifactDeployer, which doesn't support pipelines and only does as much as a 'cp' command as far as I could tell.
I ended up creating some freestyle jobs that copy artifacts from the pipelines and save them directly in their workspace. This way they're stored on our Jenkins slaves and visible through the interface to anyone who has permission to view job workspaces.
Nexus does not care what kind of artifacts you drop there. Its a good idea to use it.
I have a job to maven build our project, we now have one job per release version. As the number of releases grows, there are too many jobs and very hard to find the one we need.
I wonder if there is a way to launch the same job with different parameters? The problem is one job only has one workspace, so I'm not sure if it's possible?
Thanks.
Use This build is Parameterized option to build the jobs. Using this you can build the same job for different parameters. You will be asked to enter the parameter before building or you can also give a default parameter and you can have multiple parameters.
It is good the archive the artifacts which you need later.
You can also have the option keep build forever, this will keep the builds permanently Ir-respective of the number of builds to keep.
To use above option you should enable Discard old build option.
You can also link your repository directly to Jenkins which will trigger the job whenever a new commit is made to master or a new tag is created.
My understanding
As far as I understood artifacts up to now, is that it was used to know if my project refers to an unfinished build task's artifacts.
Problem
I tried reading more (on jenkins site too) but I'm not sure I understand so easily what they do now. I know that when I promote a build, I can fingerprint artifacts. What does it mean then?
Artifacts - anything produced during the build process.
Fingerprinting artifacts - recording the MD5 checksum of selected artifacts.
You can then use the MD5 checksum to track a particular artifact back to the build it came from.
Adding to #Slav answer, Fingerprinting help Jenkins in keeping track of which version of a file is used by which version of a dependency.
Quoting an example and how it works from Jenkins Page:
For example:
Suppose you have the TOP project that depends on the MIDDLE project, which in turn depends on the BOTTOM project.
You are working on the BOTTOM project. The TOP team reported that bottom.jar that they are using causes an Null Pointer Exception, which you (a member of the BOTTOM team) thought you fixed in BOTTOM #32.
Jenkins can tell you which MIDDLE builds and TOP builds are using (or not using) your bottom.jar #32.
How does it work?
The fingerprint of a file is simply a MD5 checksum. Jenkins maintains a database of md5sum, and for each md5sum, Jenkins records which builds of which projects used. This database is updated every time a build runs and files are fingerprinted.
To avoid the excessive disk usage, Jenkins does not store the actual file. Instead, it just stores md5sum and their usages. These files can be seen in
$JENKINS_HOME/fingerprints
I have a couple jenkins builds that run every second hour or so, since jenkins stores the data and metadata for the builds this takes up a lot of space but most of that space goes to the jars that are stored.
Jenkins keeps every jar for every build and most of them don't really change from one build to another so I was wondering if there's a way to
a) store only the jars that changed, which would be the best case scenario, something using symbolic links or something;
b) don't store the jars at all, we don't really check the builds by using the jars as a debug tool so we don't really need them. Of course I could put a cron to erase them, but I'd prefer do that from inside jenkins if possible.
Jenkins only stores jars and such if you have an "Archive the artifacts" post-build action in your job. If you don't have this, it doesn't archive anything except for logs and results.
If you're wanting to store SOME stuff but just not the jars, you can change the Excludes line in the advanced setting of the "Archive the artifacts" post-build action.
Having real problems creating artifacts in teamcity 6.5 (using TFS & MSBuild as the buildrunner if it makes any odds, which it probably does as any examples I find seem to use SVN...).
The Build works, so long as I enter no checkout rules.
If I understand it, I'll need to set up some artifacts, that themselves rely on checkout rules(?).
I have two builds that are identical other than the way they are kicked off.
One is initiated on check-in
One is initiated manually from within TC. This build is the Test Build
Assembly version numbers come from a single versioninfo.cs file that is a linked file in all projects in the solution. This method is detailed here : http://www.codeproject.com/Articles/328977/The-Right-Way-to-Version-Your-Assemblies and holds the version number thus:
[assembly: AssemblyFileVersion("9.1.0.0")]
Ultimately, I'm unable to copy the output of the test build to another location.
As it stands, the only output of a build is in the teamcity data directory, for example :
C:\TeamCity\buildAgent\work\ceaaf65dc87ff856\Project1\bin\Debug
C:\TeamCity\buildAgent\work\ceaaf65dc87ff856\Project2\bin\Debug
etc
I'd like to copy the output files (exes and DLLs) to an output folder which has the build number of the build on it
For arguments sake, lets say for the version number above, this would be to
c:\BuildServer_Output\SolutionName\9.1.0.0
Currently I have not been able to create artifact paths that actually do anything - i.e. to copy anything anywhere.
For instance I have acoupe of artifact paths, but nothing ever gets put into C:\BuildServer_TestBuilds -
+:Accounts\bin\debug* => C:\BuildServer_TestBuilds
+:BackOffice\bin\debug* => C:\BuildServer_TestBuilds
Am I getting no artifacts (and my artifact paths therefore ignored) because I have no checkout rules?
Any help would be appreciated.
I am pretty sure artifacts and checkout rules are completely independent. Artifacts just deal with what has been built. Checkout rules tell teamcity how to react to and checkout changes in the VCS.
It looks like your artifact paths are beginning with absolute paths. I have always found it easier to use relative paths with wildcards. That way I don't need to worry about where teamcity put the build. We use the following to get all dlls and exes to one folder
**\bin\Debug\*.*=>deploymentdir
Our build configuration page has an artifacts link and when we open it it will have things like
deploymentdir\common\bin\debug\common.dll
deploymentdir\common\bin\debug\common.pdb
deploymentdir\runner\bin\debug\runner.exe
In one of our other builds we use an msbuild script to flatten our output before putting it through the artifact process.
We do use checkout rules but we have not had to change our artifact paths to accommodate them.