We are currently developing an app with multiple parallel streams of development. We have a Jenkins job to build each stream/release. So Job-A may be building release 1.1, and Job-B may be building release 1.2.
I think it would be best to have the build number shared across each release, such that if Job-A runs with build number 125, if Job-B runs next it will run with build number 126. The reason I think this is the best strategy is that this is an Android app, which requires its versionCode parameter to be incremented each time it's submitted to Google Play. We use the Jenkins build number for the versionCode value.
Is there any way to configure Jenkins to share a build number across multiple jobs? Or, has anyone come up with a better solution to this problem?
short answer use timestamps or manually set versionCodes, keep things out of the CI server when not necessary. Or force the jenkins build numbers.
long answer I like jenkins to be responsible for automating something that also works on its own. So if I don't need jenkins for the setup, I am happy as well.
Also if you use 2 branches, you probably commit in random orders into them. Trying to tie the jobs together in some ways seems like an unnecessary trouble that could be a problem later on. E.g. what if version 2.0 is built and QAed now, just waiting for the proper release date and marketing team to complete its job, but you need to release a v1.1.1 quick fix after that ? Depending on the solution you pick, you may need to trigger some rebuilds to force a versionCode bump. New build, new QA ?
Your real requirement for the versionCode is for it to be higher than the previous release.
From http://developer.android.com/guide/topics/manifest/manifest-element.html
android:versionCode An internal version number.
This number is used
only to determine whether one version is more recent than another,
with higher numbers indicating more recent versions. This is not the
version number shown to users; that number is set by the versionName
attribute. The value must be set as an integer, such as "100". You can
define it however you want, as long as each successive version has a
higher number. For example, it could be a build number. Or you could
translate a version number in "x.y" format to an integer by encoding
the "x" and "y" separately in the lower and upper 16 bits. Or you
could simply increase the number by one each time a new version is
released.
So here are 2 solutions:
manual bumping. In our projects, I use some sed scripts to automate the bumping of the build number before release. As I also need to change a few things by hand, like versionName prefix, disable/enable debugging mode during development, etc, I manually run a bumpversion script so that next build in my branch has appropriate version and versionCode numbers. Note I use the jenkins build number in versionName instead. This solution prevents you from having the 1.1.1 needs to be out after v2 is ready problem if you pick a large enough versionCode bump for v2.
another more automated yet still simple solution would be to use something out of timestamps. The format YYMMDDHHSS is good enough of an integer (< 2^31), and chances are that whatever version you are going to release next is going to be prepared after the previous one and not within the same minute. So basically when you build v1.1, it gets e.g. 1308131600 and if you build v1.2 the minute after it gets 1308131601. (this obviously doesn't help you against the v1.1.1/v2 scenario)
Here are some ideas for scripts to generate/update versionCode Auto increment version code in Android app.
The jenkins way
Now if you still want jenkins in charge, a simple solution is to use something like https://wiki.jenkins-ci.org/display/JENKINS/Next+Build+Number+Plugin and configure your per branch jobs to have a large enough prefix to ensure no clash. The setup is still pretty simple.
E.g.
110000 for branch 1.1
120000 for branch 1.2
You could look at the multijob plugin where you can add multiple parameterised jobs in to a containing job
https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
You could also look at artifact archiving Archive the artifacts in hudson/jenkins and then pick the files up later
I haven't tried it, but i'm thinking about going on to the build machine and in each of the jobs replacing the nextBuildNumber file with a symlink to a single file somewhere. What could possibly go wrong. Well, concurrent access might be an issue. There might be an issue if Jenkins re-creates the file from scratch, ie with a remove and a create, instead of just opening it as normal.
Related
As is good practise, I've got Jenkins set up at work to automatically build everything for continuous integration, pulling files from our Git repositories. On our development branches, builds get kicked off automatically whenever anyone commits a change. When we want to do formal testing, we pull the build from Jenkins and use that; and when we want to sign off a change request, we quote the Jenkins build number where the change went in. So far, so good.
The problem we have is that builds are a significant size. For our SDK, we have to build across multiple platforms so that we can check it works on all of them. At maybe 50MB per build, this starts to mount up! Short term I can keep asking IT to give me more storage space, but longer term I'd like a more strategic solution
The obvious answer in Jenkins is to set up deletion rules, whether deleting after some time or after some number of builds. The problem then though is that if we delete that older development build, we lose the traceability of what we tested. I'm sure most engineers at one time or another have had to do a binary chop through older builds to find an obscure bug/regression which was only spotted some time later. For me, it is unacceptable to lose that history.
The important feature of build history though is not the binary build artifacts, but the build log recording what Git commits (or anything else; toolchain versions for example) went into each build. That's what lets us go back to investigate older builds and recreate them if required. The build log is relatively small (and highly compressible, being a text file). We do still need to keep build artifacts for recent builds though, so that testers can use them. So I'm thinking a better alternative would be to preserve the build log in Jenkins for all builds, but to have Jenkins automatically delete build artifacts after some time.
Does anyone know of a way in Jenkins (perhaps a plugin?) which would let us automatically delete/archive build artifacts from older builds, but still keep the build details and log for those builds? I'm happy to do a Jenkins upgrade if necessary to get this feature. And of course this needs to be only for selected development build jobs - all release build jobs need their build artifacts to be preserved forever, as do any builds which have the "keep forever" button ticked.
If it's absolutely necessary, I could set up a separate cron job to do this on the Jenkins file area. That's a nasty hack though, and I suspect it's likely to cause some issues with Jenkins, so I'd rather not do something that brute-force if there's a better alternative.
I think you need this option in your jenkinsfile
buildDiscarder(logRotator(artifactNumToKeepStr: '10'))
artifactNumToKeepStr: This number of builds have their artifacts kept.
When starting a set of builds in fast succession, Jenkins interrupts the earlier builds and continues only the latest. Can I switch this feature off, i.e. enforce that every build is built to the full?
Sample screenshot. Builds #2287, #2288, and #2289 started in rapid succession. Jenkins aborted the first two and continued only the latest #2289.
In our larger team, this has repeatedly caused confusion if the latest build fails. First, it is no longer possible to determine exactly which of the three or four builds introduced the problem. Second, if the earlier builds contained important fixes, they were never delivered only because some later, less important build, overruled them.
I think what you are looking for is tbe Random String Parameter plugin
Adds a new parameter type called Random String Parameter which inserts a random string that prevents Jenkins from merging multiple job runs into one.
I don't understand what Promoted build really is and how it works. Can someone please explain to me like to a 10 years old kid. If you can provide some sample examples would help me a lot.
Thanks
In a typical software developing organization with CI system, there are 10's or 100's of continuous builds daily. Only one of those builds (usually the latest stable) is selected and "promoted" to be a Release Candidate (RC), which goes to the next quality gate - usually the QA department. Then, they select one of those RC's (others are dropped) and again, "promote" it to the next level - either to staging environment, validation etc. Then, finally... one of these builds is again "promoted" to be an official release.
Why is that important?
Visibility: You would want to distinguish many "regular", continuous builds from few, selected "RC" builds.
Retention: If you commit often (which is the best practice), you will likely get lot of daily builds, and would like to implement a retention policy (e.g. only keep last 100 builds or only builds from the last 7 days). You will then want to make sure promoted builds (RCs) are locked against retention. This is mostly important if you deploy binaries to customers, and may need the exact binary to reproduce an escaping bug in the future (though you still have the source code in the repository, I've seen cases where escaping bugs relate to the build process rather than the source code - due to rapid changes to the build process, or time-of-build sensitive data like digital signatures).
Permissions: you may want to prevent access to builds with "half baked" features from non-developers.
Binary Repositories: you may want to publish only meaningful builds to an external binary repository.
Builds in Jenkins can be "promoted" either manually or automatically, using plugins like Promoted Builds Plugin. You can also create your entire "promotion" workflow using pipeline scripts. Here's an example:
a "Continuous" job that polls SCM and builds on every change. It has a retention policy to keep only the last 50 builds. Access is restricted only to developers;
a "Release Candidates" job that copies artifacts from a manually selected build (using parameters). Access is allowed to QA testers;
a "Releases" jobs that copies artifacts from a manually selected RC. Access is allowed to the entire organization. Binaries are released to external/public repository.
I hope this answers your question :-)
We have our projects configured with MSBuild script customization to modify the ApplicationVersion property in the project and copy that into the AssemblyInfo.cs file when the project builds. The problem is that we have TFS set up to run on a nightly schedule, with "Build even if nothing has changed since the previous build" unchecked. But since TFS itself is producing this version update, it will rebuild and increment every night. So this is sort of an infinite loop of our own design, but trying to figure out how to get out of it.
If the "changed since the previous build" detection is based on the history timestamp, ideally it'd be nice if when the version gets updated and commits to TFS it does it with a timestamp that precedes the build time. Is that even possible?
If the "changed since the previous build" detection is based on some boolean/bit flag, is there a way to reset it?
Using TFS 2012.
I'm assuming that you're checking in the new version of the assemblyinfo.cs once it's been updated, and this is why TFS is queuing a new build. Have you tried adding a comment to the checkin of ***NO_CI*** This will definitely suppress a CI build but I'm not 100% certain if it will work in your scenario.
Another option is generating the version number via an algorithm rather then just incrementing a counter and checking it back in to Version Control. This circumvents the issue of a new build being triggered
i.e if your version number looks something like
1.2.3.4
Where 1 is Major (modified by a human not the build process)
2 is minor (also modified by a human)
the final 2 digits are then updated by an automated process.
You could use number of days since January 2000 for digit 3 (an arbitrary number but something that would change on a daily basis) and either the latest changeset number in Version Control or the total number of builds performed by TFS for digit 4.
This would fulfill 2 requirements, that version numbers are unique for a given build of an assembly, they always go up.
I would suggest that you don't check the new version number into TFS. There is no value in having the version number in there.
I typically set the checked in assembly info numbers to all zeros. ( 0.0.0.0) and never update them except locally for the build.
This gives you the benefit of always being able to identify locally built DLLs.
I'm trying to tweak some options in my jenkins configuration, which is causing many builds to fail. I'd prefer to not keep these failures around in the build history, since they're not technically failures of the repository. In the past, I've just deleted the build after looking at the log, but this is a little tedious.
Is there a way to start a build with an option to not record the result of the build permanently?
Perhaps there's a URL that can be used to trigger a debugging build, somethign like:
JENKINS_URL/job/JOBNAME/build?DEBUGGING
You can set the "discard old builds" option in your job to only keep 1 build. If you have older builds you want to keep, you can give them the "keep this build forever" property. If you have a large number of jobs to work with, you can use the Configuration Slicing plugin to modify the Max # of builds to keep.