I have a system of continuos integration that using ivy/archiva/jenkins.
For a lot of reasons, principally because i work with other people and often we work with the same projects, i have to use the "latest time" strategy for my project's and i use ant ivy to publish/resolve from an archiva repository.
How could i do that?
Is it possible? And if it is, how ivy compare the date between my cached artifacts and the one that's in the repository?
Because when i set that strategy on, my project's starts to continuously downloads every artifacts.
I have notice that this working, because every repo take note of their modules last date revision, so when you publish an artifact the repos update the date of that revision.
Related
As is good practise, I've got Jenkins set up at work to automatically build everything for continuous integration, pulling files from our Git repositories. On our development branches, builds get kicked off automatically whenever anyone commits a change. When we want to do formal testing, we pull the build from Jenkins and use that; and when we want to sign off a change request, we quote the Jenkins build number where the change went in. So far, so good.
The problem we have is that builds are a significant size. For our SDK, we have to build across multiple platforms so that we can check it works on all of them. At maybe 50MB per build, this starts to mount up! Short term I can keep asking IT to give me more storage space, but longer term I'd like a more strategic solution
The obvious answer in Jenkins is to set up deletion rules, whether deleting after some time or after some number of builds. The problem then though is that if we delete that older development build, we lose the traceability of what we tested. I'm sure most engineers at one time or another have had to do a binary chop through older builds to find an obscure bug/regression which was only spotted some time later. For me, it is unacceptable to lose that history.
The important feature of build history though is not the binary build artifacts, but the build log recording what Git commits (or anything else; toolchain versions for example) went into each build. That's what lets us go back to investigate older builds and recreate them if required. The build log is relatively small (and highly compressible, being a text file). We do still need to keep build artifacts for recent builds though, so that testers can use them. So I'm thinking a better alternative would be to preserve the build log in Jenkins for all builds, but to have Jenkins automatically delete build artifacts after some time.
Does anyone know of a way in Jenkins (perhaps a plugin?) which would let us automatically delete/archive build artifacts from older builds, but still keep the build details and log for those builds? I'm happy to do a Jenkins upgrade if necessary to get this feature. And of course this needs to be only for selected development build jobs - all release build jobs need their build artifacts to be preserved forever, as do any builds which have the "keep forever" button ticked.
If it's absolutely necessary, I could set up a separate cron job to do this on the Jenkins file area. That's a nasty hack though, and I suspect it's likely to cause some issues with Jenkins, so I'd rather not do something that brute-force if there's a better alternative.
I think you need this option in your jenkinsfile
buildDiscarder(logRotator(artifactNumToKeepStr: '10'))
artifactNumToKeepStr: This number of builds have their artifacts kept.
After a recent update (both Jenkins and Plug-ins) my Ivy Project settings can no longer be changed due to incompatible layouts (table to div change in a minor version update, from Jenkins 2.263 to 2.264). This broke every plugin that was involved in configuring projects, but went unnoticed for two months because our project settings haven't needed to change in quite a while, and the builds were still working fine in the meantime.
For reference, my build process is based on:
Ant for the build
Ivy for dependency resolution
Artifactory as a dependency repository
Subversion as a code repository (with Jenkins commit triggers)
Junit with Cobertura, Jmeter
FindBugs, CheckStyle, CLOC
Projects are based on Java and JavaDoc
I tried reverting to the earlier version of Jenkins, but this affected nearly every plugin, and I wasn't able to successfully revert to the plugin version combination from prior to the breaking update. After failing to revert the updates, I decided instead to plow forward in updating all of our 68 projects to accommodate the new plugin versions.
Unfortunately, I can't save any configuration changes to Ivy Projects. After trial and error, I've found that I can re-produce my builds using Freestyle Projects. However, Jenkins doesn't seem to offer any way to convert projects from one type to another. If I were to create new projects from scratch to replace my existing projects (all 68, including their dependencies and specific plugin settings), I would lose all of my previous build histories, including the build numbers (which carry over to our deployments) and our project metrics (which we use for performance evaluation). So, I don't want to lose all of that information.
How can I manually change an Ivy Project to a Freestyle Project?
I found a partial solution, but it doesn't seem to work for all projects.
Stop the Jenkins webapp (important).
For each Ivy Project that you want to convert to a Freestyle Project, rename the root element of jobs/[project]/config.xml from <hudson.ivy.IvyModuleSet plugin="ivy#2.1"> to <project> (don't forget to also change the closing tag at the end of the document from </hudson.ivy.IvyModuleSet> to </project>.
Restart Jenkins.
For most projects, I am then able to change the project configuration and save (importantly, Ant/Ivy-Artifactory Integration in a Freestyle Project is a feature-matched substitute for an Ivy Project).
However, three other projects still show up as Ivy Projects after changing the root element tag. What these projects had in common was that they all use the Performance Plugin. In order to finish converting these to Freestyle, I needed to additionally:
Disable the Performance Plugin
Restart Jenkins
Edit/Save the configuration for those projects as above.
Side effects and special considerations:
All of my build timestamps (prior to the change) are now listed as Dec 31, 1969 7:00 PM EDT, with a most recent build time as 50 yr. New build timestamps are correct. This likely was the result of no longer depending on the CloudBees plugin for build pipelines, which mapped build timestamps to build versions to avoid an old regression bug.
Every project immediately changed to red (Failed) on the dashboard, even though no builds had been attempted after the update, and the previous status was blue (Success) or yellow (Unstable). I suspect this is related to the above issue. After the next attempted build, whether successful or not, the status accurately reflects the build status.
No ability to use the Performance Plugin.
Several projects now show up as both an Upstream and Downstream Project, causing endless build cycles. There were three cases of this involving different combinations of projects, and in those cases, one or both projects needed to be removed from the build triggers. I suspect it had been this way for a while but for some reason the endless cycles only happen after the update.
I suddenly have a lot of "Unreadable Data" across all of my Jenkins projects. Unfortunately, discarding it is an all-or-nothing process (can't pick a single project to test). I backed up my jobs directory and clicked Discard, and to my surprise everything still works.
It looks like I'm back in business. My build numbers have been preserved, and the only noticeable side effect is the 50 year old builds. If I encounter any other issues resulting from these changes, I will update this answer.
I have a couple jenkins builds that run every second hour or so, since jenkins stores the data and metadata for the builds this takes up a lot of space but most of that space goes to the jars that are stored.
Jenkins keeps every jar for every build and most of them don't really change from one build to another so I was wondering if there's a way to
a) store only the jars that changed, which would be the best case scenario, something using symbolic links or something;
b) don't store the jars at all, we don't really check the builds by using the jars as a debug tool so we don't really need them. Of course I could put a cron to erase them, but I'd prefer do that from inside jenkins if possible.
Jenkins only stores jars and such if you have an "Archive the artifacts" post-build action in your job. If you don't have this, it doesn't archive anything except for logs and results.
If you're wanting to store SOME stuff but just not the jars, you can change the Excludes line in the advanced setting of the "Archive the artifacts" post-build action.
Currently we are using StarTeam to perform the build as well as versioning. we planned to migrate startteam to TFS2010. We have some script for perform the build. i wanted to change this script according to my requirement. i gone through the TFS but i had lot more confusion.
in StarTeam, we will get a files from "Ready to Build" label and perform the build. In TFS, how we are going to get a files from TFS? What concept should i use to get a files and perform the build? i have gone through the lot of commands like get, check-in, checkout etc..
If we use "tf get" command, we can get all the files from TFS but i have a clarification on that. shall i get all the files from TFS for every build? i hope, this is unnecessary headache.. correct me if i am wrong..
how we perform the build in TFS? i have read some types of build such as manual, gatedcheckin, Continuous Integration and schedule.
Is there any relationship between branch and build activities?
In TFS, What is the meaning of Workspace?
As said, many questions in one. Hope this helps along the way:
A workspace is a mapping between the server and a local storage,
similar to checkout in Subversion, view in ClearCase, etc.
"TF get" normally only fetches those files that have changed since
last update. You can force it to fetch everything - and sometimes
have to - but its not normally done.
Team Build is the recommended system to build with when using TFS. It can take some time to get into (Windows Workflow-based), but is quite powerful. There are default process definitions that set up the most common actions for you.
By default, you can't control whether to build by setting a certain label, but you can define that only this label should be used when builds are triggered. Labels in TFS work a little differently compared to other VCS, though, so maybe there's an 'opportunity' to re-think your build process along the way. If you're set on using a label as before, you'll need to build a Custom build activity.
I have a build project that I run from TeamCity, now it takes the build.number from TeamCity.
I want to be able to use the buildnumber task (Ant) which in my understanding when used, is supposed to change the value of build number property and increment it for next time.
But it seems that I still get the build number from TeamCity.
How can I use the build number from <buildnumber file="mybuild.number"/>?
Use the TeamCity Autoincrementer plugin to share an incrementing build number between configurations.
The ANT buildnumber task sets the ANT property build.number.
I don't use team city, but the documentation states that it is also setting the same property.... Might explain your problem :-(
Perhaps someone knows how to switch this team city feature off.
Is there a good reason to switch? The only alternative I can offer ivy buildnumber task. It calculates the next build/release number based on what was previously published to the repository. This may not be suitable in your case. I prefer it because I don't depend on build server settings or need to keep a property file under revision control.
I'm afraid so far there is no way to disable build.number property from TeamCity.
The only possibility I can imagine is not to use TeamCity Ant runner and use Command-line runner instead. But in this case you'll miss many nicities of the bundled Ant runner (like immediate test reporting).
Why would you want to use buildnumber task of the Ant? To keep build number across the builds, you'd have either to commit mybuild.number to version control or to run the build always on the same agent in the same place.