Build once deploy many with Jenkins and Github (specific branching workflow) - jenkins

Im creating Jenkins pipeline for building and deploying our app.
I want to have the same build for the staging and production environment(buld-once-deploy-many approach).
Merging feature branch into staging branch would build an app for both staging and production, but it will deploy it only to staging bucket.
After testing, it would be good that developers merge the staging branch into master which will take the previous staging build and deploy it to production.
Alternative would be to have one branch, and devs would manually trigger another job on Jenkins that would deploy the build to production.
I want to avoid devs going into jenkins and triggering build, since most of them find it intimidating, there are also some nasty vpn configuration steps they need to go through to have access to Jenkins etc.
Would this be a bad practice? Do you have any suggestions how to achieve something like this?
Thanks

This is more of a devops question than a Jenkins question, but here is my take...
I can't decide if you are saying that you would re-build from the master branch, or only use the commit to the master branch to trigger a deployment of the original staging artifact. So I'll address both.
For this situation:
If you build an artifact in a staging branch, and test it. Everything looks green on the tests. So you merge those changes to another branch, re-build, and deploy to production.
The problem: Can you be 100% sure that there was nothing else in the master branch that is not different than what was built AND tested in staging?
You are risking the jello view anti-pattern because unknown and untested changes could sneak into your production artifact. It becomes terribly difficult to troubleshoot why it worked fine it staging, but now fails in production.
For the second situation:
If you are saying that you wouldn't rebuild from the master branch, then merging back to master doesn't buy you anything except a trigger to kick off a new build, because you are never generating an artifact from Master.
If you are going to to it this way, I think you could commit to a single branch, and then tag a release that is meant to go to production, then trigger off of the tag.
Either way, this seems like a strange pattern, and would be difficult to accomplish in an automated fashion in Jenkins. Somehow the newly triggered build would have to find the previously built artifact and deploy that to production.
Possible solution:
There are a few ways to solve this, but one of the easiest is to build once and deploy the same artifact all the way down the line, as you mentioned as an alternative option. But this would likely require some approvals or additional triggering in Jenkins.
If you don't want developers to have to touch Jenkins, then the more likely solution is to build and run your unit tests and smoke tests on the staging environment. Then when a developer wants to promote a build to production, they commit it to the master branch, where the same build is kicked off, and testing is all performed again, in addition to more advanced integration, functional, and acceptance testing. If it doesn't pass, it doesn't go to production.
Your initial tests in staging give the developer quick feedback, but don't serve as the official tests, which only run on the production build.
With a pipeline script, you could easily accomplish this with a single Jenkinsfile and single multibranch pipeline job with stages that are only run when the branch pattern matches.
stage ("Acceptance Testing" ) {
when { branch "master" }
echo "Do testing here"
}

Related

Jenkins how to deploy same code to different servers (that I can specify)?

in the software company I work we use jenkins to deploy to different servers, the way we do that is ever single branch from git repository deploy to the specific server based on the name of the branch and in the specifications on the jenkinsfile. But we are in the process of unification of this branchs in just one: Master, but how we can configure jenkins to catch the same code and deploy to the servers we are interested in, without changing the code? I think we should separate code from deploy, but the pipeline still have to exist in some way.
2 solutions come to mind:
I believe you may be using SCM polling to get the builds started. With git diff you can check what was changed and based on that start the specific deployment.
If you are running the builds manually you can parameterize the build and specify this way which one you want to deploy.
From experience you might want to set the pipeline so when you commit to the repository only testing and building is done and deploy is not done (or only on a test env) and the proper prod deploy is done only manually (and can be parameterized).

Delete not recently run jobs from Jenkins

In part of our testing setup we are build our artifacts needed and then copying a template job and settings its name so it recognizable.
Build artifact -> copy test template -> ending with a job for each
test case
that means i'm ending up with lots of jobs with Test_Client${BRANCHNAME}_Server${BRANCHNAME}
I'm running through these jobs alot while testing that branch, but as soon as it's merged it's not going to be touched again, which is why i would like to create a job of sorts that simply deletes the jobs that havn't been run for the 14 days or so.
Does anyone know a way of doing this? and not just cleaning out the workspace.
Thanks!
It may be a big change, but this is an ideal case for the Multibranch pipeline.
On one project, we have a master branch and version branches. The developers branch to short-lived feature branches or other branching purposes. As the branches are created and pushed to github, the multibranch job picks them up and starts building them. Developers get quick feedback that the changes will pass the build. When they merge them to master or to a version branch, then delete the branch, the multibranch job goes away.

How to achieve Roll back using Jenkins

I know this Forum is not to provide strategy's.
Using Jenkins I have set up CI and CD to my Dev,QA and Staging environments. I am stuck up with Rollback strategy for all my environments.
1- What happens if my build fails in Dev
2- What happens if my build fails in QA and passed in Dev.
3- What happens if my build fails in Staging and passed in Dev and QA.
How should I roll back and get things done considering DB in not in place. I have created sample workflow but not sure its an right process.
Generally you can achieve this in 2 ways:
Setting up some sort of release management tool that tracks every execution of your pipeline and snapshots the variables, artifacts, etc... that was used on that exact execution, then you can just run an earlier release of it (check tools like octopus deploy)
If you are using a branch strategy with tags you can parameterize your jobs, passing the tag you wanna build, and build the "earlier tag" if something fails. Check the rebuild option for older job executions.

How do you manage multiple releases in multiple environments in continuous integration/delivery?

I am trying to wrap my head around this. Most CI/CD examples/projects have a single master that is always released, and have some variant of, e.g. git-flow, to have a develop branch. Once tagged, it goes to master.
Either way, master is always released to production.
But in the real world as I see it, there are human gates for release to production and other environments. What mechanism do you use to manage the deployment of different versions?
For example:
v1.5 is the current production release
v1.6 has passed all tests, artifacts are ready, it is tagged as valid, but business decides to deploy it only to staging, awaiting an opportune moment to deploy
v1.5 is deployed to a demo environment
v2.0 has also passed all tests, but is in UAT, subject to the customer being happy, as it is a major release
There could be many more such environments - production, staging, UAT, demo, demo2, etc.
What mechanism do you use to handle the tagging of a particular version for a particular environment, and the actual deployment thereof?
Although there a probably a few ways to do it, I use the build pipeline plugin https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin Along with the copy artifacts plugin https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
With these, you can create individual jobs for each piece of your environment, and link them altogether.
So as in your example, the pipeline would look like:
Build -> Test and Deploy to UAT (2.0) -> deploy to staging(1.6) -> demo(1.5) -> prod (1.5)
Each piece represents a different build in jenkins. The idea behind continuous integration is you create the binaries once, and you carry it down the pipeline, only changing configuration pieces along the way. In a build job, the artifacts are created and then archived. In any jobs after, the artifact is picked up from the upstream job, some stuff is done, and then it get's re-archived for the next downstream job. So the deploy to staging would go to the Test and Deploy to Uat job to get its binary. The entire concept of Continuous Delivery boils down to the the build pipeline. http://en.wikipedia.org/wiki/Continuous_delivery (and yes I did just cite wikipedia).
As for tagging individual binaries for specific environments, that is by definition, not continuous integration. A binary is suppose to be created in a way that it can easily be propagated from one environment to the next. So unfortunately, individual builds for specific environments can never be continuous delivery. You can use jenkins as a CI server all you want, but if your process does not match, you will never achieve true continuous integration.
Braching, merging and checkins always seems to be a touchy subject when it comes to Continuous Integration, so I won't go into it much. But a lot of people share the idea that : "If different members of the team are working on separate branches, then by definition, they not participating in continuous integration process." http://eugenedvorkin.com/continuous-integration-strategies-for-branching-and-merging/
EDIT
For Flagging specific builds, it sounds like your looking to take use of this feature : https://wiki.jenkins-ci.org/display/JENKINS/Fingerprint ... Which gets the job done effectively, giving you the entire life of any individual artifact. A bit more complex solution would be artifactory, which is essentially artifact source control.
I explained the concept of the deployment process above, and without information on your specific environment it is hard to go much further. But for me, for java applications deployed to tomcat containers, the deploy plugin works great https://wiki.jenkins-ci.org/display/JENKINS/Deploy+Plugin
You shouldn't have to worry about selection of which artifact to deploy. The pipeline should be setup to always deploy the latest artifact that was archived in its corresponding upstream job.
Maybe Docker can help you out with this issue. It is able to deploy images of projects to a specific environment. If that environment has a docker client or a docker deamon you are able to request specific information about that environment and the project (to be) deployed on it.
Jenkins can still play a huge part in your pipeline for the integration part and you could let docker do the delivery part.
Docker: https://www.docker.com
Docker plugin for jenkins: https://wiki.jenkins-ci.org/display/JENKINS/Docker+build+step+plugin
Docker also has support for windows machines and .NET.

TFS or Teamcity, how to automate deployment to various environments?

Looking for advice on how to handle this scenerio.
We have 3 environments: Dev, QA and Production.
Currently pushing the code to each environment is a manual process, wondering how something like Cruisecontrol or TeamCity could streamline this process.
How can we push to the various environments in an automated way?
How should TFS be setup to make this happen? i.e. master branch, feature branches etc.
Scenerio:
Developer#1 pushes their changes to the Dev and QA servers.
Developer#2 pushes their changes to the Dev and QA servers.
Now we need to only push Developer#1's changes to production.
Should the main branch have only the code that should be going to production?
To control what gets pushed to each environment KMoraz's approach would be the correct one, using branches and merging.
Now, for build and deployment automation the latest setup I've been using is with Team City.
My setup is:
Trunk build: compiles on every commit, runs all unit tests, generates code coverage reports, runs FxCop
Static analysis build: runs nightly against Trunk, executing Duplicate Finder (Team City), ConQAT code clone analysis, StatSVN, and Resharper Code Inspections (Team City)
DEV Deployment (dependency on Trunk build): on every commit, if the Trunk build is successful, the application is automatically deployed to a DEV environment, using MS WebDeploy with config transformations.
QA Deployment: triggered manually through Team City's interface (click of a button), when moving to QA. Deploys the application to the QA server using MS WebDeploy with config transformations.
You would also set up builds for different branches, depending on your needs, especially for branches created for releases of stable versions.
The key part, is having different visual studio build configurations (just as you have "Release" and "Debug", you should have "Dev", "QA", etc), which you should use along with web.config transformations in order to get WebDeploy to configure your environment for you.
That way you'd have different web.Dev.config, web.QA.config transformations, one for each build configuration, with specific settings.
There's an excellent series of posts by Troy Hunt called "You're deploying it wrong!" which guides you through the setup of automated builds and deployments.
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity.html
It was very useful to me when setting this up.
Now we need to only push Developer#1's changes to production.
-Developer #1 checked-in his code to the Dev branch. After QA verified his changes, now you merge the changes to the Main branch and build a release for production from the Main.
Should the main branch have only the code that should be going to
production?
-Yes. Ideally, production releases should be built from the Main branch.
How can we push to the various environments in an automated way?
-In TFS, a common practice is defining a build defintion per branch and/or build type. Apart from the source and build type, each defintion can also have its own tasks, I.e: run unit tests, publish to certain folders, deploy build artifacts to Lab Management, etc.
ProjectName-Main-Gated
ProjectName-Dev-CI
ProjectName-Dev-Nightly
ProjectName-Test-CI

Resources