Currently we are using axion-release-plugin to control our release process. Now we would like to drop the SNAPSHOT and introduce the build number in our release process. So that we can achieve CI.
Basically currently we are using version 1.0.0-SNAPSHOT and now we need something like 1.0.0-BUILDNUMBER or 1.0.BUILDNUMBER from Jenkins.
Please provide any solution or plugin we can use with gradle.
I'm actually quite happy with nebula-release-plugin developed by NetFlix. It can generate a unique version number based on branches and tags in your git repo.
By default it counts with using git flow, but you can reconfigure the behaviour. Take a look :)
Related
I want to display non-code differences between current build and the latest known successful build on Jenkins.
By non-code differences I mean things like:
Environment variables (includes Jenkins parameters) (set), maybe with some filter
Version of system tool packages (rpm -qa | sort)
Versions of python packages installed (pip freeze)
While I know how to save and archive these files as part of the build, the only part that is not clear is how to generate the diff/change-report regarding differences found between current build and the last successful build.
Please note that I am looking for a pipeline compatible solution and ideally I would prefer to make this report easily accessible on Jenkins UI, like we currently have with SCM changelogs.
Or to rephrase this, how do I create build manifest and diff it against last known successful one? If anyone knows a standard manifest format that can easily be used to combine all these information it would be great.
you always ask the most baller questions, nice work. :)
we always try to push as many things into code as possible because of the same sort of lack of traceability you're describing with non-code configuration. we start with using Jenkinsfiles, so we capture a lot of the build configuration there (in a way that still shows changes in source control). for system tool packages, we get that into the app by using docker and by inheriting from a specific tag of the docker base image. so even if we want to change system packages or even the python version, for example, that would manifest as an update of the FROM line in the app's Dockerfile. Even environment variables can be micromanaged by docker, to address your other example. There's more detail about how we try to sidestep your question at https://jenkins.io/blog/2017/07/13/speaker-blog-rosetta-stone/.
there will always be things that are hard to capture as code, and builds will therefore still fail and be hard to debug occasionally, so i hope someone pipes up with a clean solution to your question.
I have multiple environments. They are debug, dev, and prod. I'd like to refer to an image by the latest dev (latest) or dev (version 1.1) or prod (latest). How would I go about tagging builds and pushes?
My first thought was to create separate repositories for each environment debug, dev, and prod. But I am starting to wonder if I can do this with just one repository. If its possible to do with one container what would the syntax be when building and pushing?
This is what has worked best for me and my team and I recommend it:
I recommend having a single repo per project for all environments, it is easier to manage. Especially if you have microservices, then your project is composed by multiple microservices. Managing one repo per env per project is a pain.
For example, I have a users api.
The docker repo is users. And this repo is used by alpha, dev and beta.
We create an env variable called $DOCKER_TAG in our CI/CD service and set it at the time the build is created, like this:
DOCKER_TAG: $(date +%Y%m%d).$BUILD_NUMBER => This is in bash.
Where $BUILD_NUMBER is previously set by the build being run when the CI/CD run is triggered. For example, when we merge a PR, a build is triggered, as build no. 1, so $BUILD_NUMBER: 1.
The resulting tag looks like this when used: 20171612.1
so our docker image is: users:20171612.1
Why this format?
It allows us to deploy the same tag on different environments with a
run task.
It helps us keep track when an image was created and what
build it belongs to.
Through the build number, we can find the commit information and map all together as needed, nice for trobleshooting.
It allows us to use the same docker repo per project.
It is nice to know when we created the image from the tag itself.
So, when we merge, we create a single build. Then that build is deployed as needed to the different environments. We don't create an independent build per environment. And we keep track on what's deployed where.
If there's a bug in an environment with certain tag, we pull such tag, build and trobleshoot and reproduce the issue under that condition. If we find an issue, we have the build number in the tag 20171612.1 so we know the build no. 1 has the issue. We check our CI/CD service and that tells us what commit is the most current. We check out that commit hash from git and debug and fix the issue. Then we deploy it as a hotfix, for example.
If you don't have a CI/CD yet, and you are doing this manually, just set the tag in that format manually (pretty much type the full string as is) and instead of a build number, use a commit short git hash (if you are using git):
20170612.ed73d4f
So you know what is the most current commit so you can troubleshoot issues with a specific image and map back to the code to create fixes as needed.
You can also define any other suffix for your tag that maps to the code version, so you can easily troubleshoot (e.g. map to git tags if you are using those).
Try it, adjust it as need it and do what it works best for you and your team. There's many ways to go around tagging. We tried many and this one is our favorite so far.
Hope this is of help.
There's two schools of thought, stable tagging, where you update a single tag, and unique tagging. Each have their pros and cons. Stable tags can create instability when deploying to self healing clusters as a new node might pull a new version, while the rest of the cluster is running a slightly older version. Unique tagging is a best practice for deployment. However, to manage base image updates of OS & Framework patching, you'll want to build upon stable tags in your dockerfile, and enable automatic container builds. For a more detailed walk through, with visuals, here's a post:
https://blogs.msdn.microsoft.com/stevelasker/2018/03/01/docker-tagging-best-practices-for-tagging-and-versioning-docker-images/
I think "lastest" is for the last productive image. It is what I expect in the docker hub, although there are no images in development.
On the other hand you can use tags like for example 0.0.1-dev. When this image is finished you can do the tag again and push, and then the repository will detect that the layers are already in the repository.
Now when you are about a candidate version to come out to production, you have to only have the semantic version despite not being in an environment pruduccion. That's what I would do.
I’m new to Jenkins and I like some help (reassurance) about how I think I should setup my jobs.
The end goal is fairly simple.
Objective 1: When a developer commits code to a mercurial repo Jenkins pulls the changes, builds the project and runs the unit tests. This happens continuously throughout the day so developers get the earliest possible feedback if they break something.
Objective 2: Nightly, Jenkins pulls the last stable build from above and runs automated UI tests. If those tests pass it publishes the nightly build somewhere.
I have a job configured that achieves objective 1 but I’m struggling with objective 2.
(Not the publishing part, the idea of seeding this job with the last stable build of objective 1).
At the moment, I’m planning to use branches in the HG repo to implement this.
My branches would look something like Main >> Int >> Dev.
The job in objective 1 would work on the tip of the Dev branch.
If the build succeeds and the tests pass it would commit to the Int branch.
The job in objective 2 could then simply work on the tip of the Int branch.
Is this how it’s generally done?
I’ve also been looking at/considering:
- plugins like Promoted Builds and Copy Artifacts
- parameterised builds
- downstream jobs
IMO my objectives are fairly common but I can’t find many examples of this approach online. Perhaps it’s so obvious there was no need but I just wanted to check.
In the past I've stored generated artifacts like this in an artifact repository. You could use something like Nexus or Artifactory for this, but I've also just used a flat file system.
You could put the build artifacts in source control, like you said, but there usually isn't a reason to have version control on compiled builds (you should be able to re-create them based on rev numbers) - they usually just take up a lot of space in your repo.
If your version numbers are incremental in nature your nightly job should be able to pull the latest one fairly easily.
Maybe you can capture the last good revision ID and post it somewhere. Then the nightly build can use that last known good revision. The method to go about doing this can vary but its the concept of using revision ID that I want to communicate here. This would prevent you from having to create a separate branch.
We are currently developing an app with multiple parallel streams of development. We have a Jenkins job to build each stream/release. So Job-A may be building release 1.1, and Job-B may be building release 1.2.
I think it would be best to have the build number shared across each release, such that if Job-A runs with build number 125, if Job-B runs next it will run with build number 126. The reason I think this is the best strategy is that this is an Android app, which requires its versionCode parameter to be incremented each time it's submitted to Google Play. We use the Jenkins build number for the versionCode value.
Is there any way to configure Jenkins to share a build number across multiple jobs? Or, has anyone come up with a better solution to this problem?
short answer use timestamps or manually set versionCodes, keep things out of the CI server when not necessary. Or force the jenkins build numbers.
long answer I like jenkins to be responsible for automating something that also works on its own. So if I don't need jenkins for the setup, I am happy as well.
Also if you use 2 branches, you probably commit in random orders into them. Trying to tie the jobs together in some ways seems like an unnecessary trouble that could be a problem later on. E.g. what if version 2.0 is built and QAed now, just waiting for the proper release date and marketing team to complete its job, but you need to release a v1.1.1 quick fix after that ? Depending on the solution you pick, you may need to trigger some rebuilds to force a versionCode bump. New build, new QA ?
Your real requirement for the versionCode is for it to be higher than the previous release.
From http://developer.android.com/guide/topics/manifest/manifest-element.html
android:versionCode An internal version number.
This number is used
only to determine whether one version is more recent than another,
with higher numbers indicating more recent versions. This is not the
version number shown to users; that number is set by the versionName
attribute. The value must be set as an integer, such as "100". You can
define it however you want, as long as each successive version has a
higher number. For example, it could be a build number. Or you could
translate a version number in "x.y" format to an integer by encoding
the "x" and "y" separately in the lower and upper 16 bits. Or you
could simply increase the number by one each time a new version is
released.
So here are 2 solutions:
manual bumping. In our projects, I use some sed scripts to automate the bumping of the build number before release. As I also need to change a few things by hand, like versionName prefix, disable/enable debugging mode during development, etc, I manually run a bumpversion script so that next build in my branch has appropriate version and versionCode numbers. Note I use the jenkins build number in versionName instead. This solution prevents you from having the 1.1.1 needs to be out after v2 is ready problem if you pick a large enough versionCode bump for v2.
another more automated yet still simple solution would be to use something out of timestamps. The format YYMMDDHHSS is good enough of an integer (< 2^31), and chances are that whatever version you are going to release next is going to be prepared after the previous one and not within the same minute. So basically when you build v1.1, it gets e.g. 1308131600 and if you build v1.2 the minute after it gets 1308131601. (this obviously doesn't help you against the v1.1.1/v2 scenario)
Here are some ideas for scripts to generate/update versionCode Auto increment version code in Android app.
The jenkins way
Now if you still want jenkins in charge, a simple solution is to use something like https://wiki.jenkins-ci.org/display/JENKINS/Next+Build+Number+Plugin and configure your per branch jobs to have a large enough prefix to ensure no clash. The setup is still pretty simple.
E.g.
110000 for branch 1.1
120000 for branch 1.2
You could look at the multijob plugin where you can add multiple parameterised jobs in to a containing job
https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin
You could also look at artifact archiving Archive the artifacts in hudson/jenkins and then pick the files up later
I haven't tried it, but i'm thinking about going on to the build machine and in each of the jobs replacing the nextBuildNumber file with a symlink to a single file somewhere. What could possibly go wrong. Well, concurrent access might be an issue. There might be an issue if Jenkins re-creates the file from scratch, ie with a remove and a create, instead of just opening it as normal.
I have a requirement to do a dependent build using Jenkins Following is the requirement:
Project 1 has a branch which is used among two release lines. For example project1 development branch ikt/master is share in two release line rel1.2_4GB and rel_1.2_2JB.
When ever a change is submitted in ikt/master of project1 it should trigger build of both the release line rel1.2_4GB and rel_1.2_2JB simultaneously.
Build results should wait for other build to pass means both builds should be green.
Please suggest me steps using both plugin as well as without plugin (if possible).
Kind Regards,
I think your best option is to use the Parameterized Trigger Plugin to do this.
It's very simple and easy to use and you can trigger several child jobs and wait for their results. Based on their results, you can choose to fail or pass the build.
I suggest you read some more about it and do some experimenting. It works very well for me.
I hope this helps.
NOTE - I suggest not wasting time looking for a non plugin solution. If you have a good tool, use it. Don't loose time trying to be smarter...