Looking for advice on how to handle this scenerio.
We have 3 environments: Dev, QA and Production.
Currently pushing the code to each environment is a manual process, wondering how something like Cruisecontrol or TeamCity could streamline this process.
How can we push to the various environments in an automated way?
How should TFS be setup to make this happen? i.e. master branch, feature branches etc.
Scenerio:
Developer#1 pushes their changes to the Dev and QA servers.
Developer#2 pushes their changes to the Dev and QA servers.
Now we need to only push Developer#1's changes to production.
Should the main branch have only the code that should be going to production?
To control what gets pushed to each environment KMoraz's approach would be the correct one, using branches and merging.
Now, for build and deployment automation the latest setup I've been using is with Team City.
My setup is:
Trunk build: compiles on every commit, runs all unit tests, generates code coverage reports, runs FxCop
Static analysis build: runs nightly against Trunk, executing Duplicate Finder (Team City), ConQAT code clone analysis, StatSVN, and Resharper Code Inspections (Team City)
DEV Deployment (dependency on Trunk build): on every commit, if the Trunk build is successful, the application is automatically deployed to a DEV environment, using MS WebDeploy with config transformations.
QA Deployment: triggered manually through Team City's interface (click of a button), when moving to QA. Deploys the application to the QA server using MS WebDeploy with config transformations.
You would also set up builds for different branches, depending on your needs, especially for branches created for releases of stable versions.
The key part, is having different visual studio build configurations (just as you have "Release" and "Debug", you should have "Dev", "QA", etc), which you should use along with web.config transformations in order to get WebDeploy to configure your environment for you.
That way you'd have different web.Dev.config, web.QA.config transformations, one for each build configuration, with specific settings.
There's an excellent series of posts by Troy Hunt called "You're deploying it wrong!" which guides you through the setup of automated builds and deployments.
http://www.troyhunt.com/2010/11/you-deploying-it-wrong-teamcity.html
It was very useful to me when setting this up.
Now we need to only push Developer#1's changes to production.
-Developer #1 checked-in his code to the Dev branch. After QA verified his changes, now you merge the changes to the Main branch and build a release for production from the Main.
Should the main branch have only the code that should be going to
production?
-Yes. Ideally, production releases should be built from the Main branch.
How can we push to the various environments in an automated way?
-In TFS, a common practice is defining a build defintion per branch and/or build type. Apart from the source and build type, each defintion can also have its own tasks, I.e: run unit tests, publish to certain folders, deploy build artifacts to Lab Management, etc.
ProjectName-Main-Gated
ProjectName-Dev-CI
ProjectName-Dev-Nightly
ProjectName-Test-CI
Related
We have Dev, QA pre_prod (stage) and Prod(Trunk) branch.
This is how our process works , please help in improving process to avail CI&CD features using TFS..
Dev changes are merged to QA branch on schedule basis say end of sprint. then published from QA branch. QA team does testing if found any defect , then bug-fix is done in QA branch then get merged with Dev branch.
if QA is done then QA branch is merged with pre-Prod and Acceptance testing is done by client then merged to prod branch and published to client.
if it comes production bug, hot fix is made in pre-prod branch -> published for acceptance testing (acceptance env where QA gets notified) ->all good then then merge to prod branch and publish to client.
hot fix is not merged to QA as it may add other QA ongoing check-ins. since hot fix needs instant fix it goes through Acceptane testing directly where QA also can test .. if all good then get merged to QA branch.
Cons:
1. All development has to be done to give build to QA . so QA is idle. no CI.
2. Another team to do merging between branches. publish it and if error again merging it. resource not free...
how can we implement TFS CI& CD features here ?
CI better works with one branch pattern where check-in will trigger build then QA will test and approve then promote to prod. but here if QA rejects then dev has to fix since it is single branch it will get merge with ongoing daily check-ins and if triggers some more error in QA the this cycle will repeat all the time and features can't be promoted to prod.
Second solution could be to establish branch specific CI & CD as changes are done in individual branch. but here we can't avail the CD feature.... as there is no link between environments ...
third concern is QA needs a stable environment to test all tasks..if in mid new deployment is added through CD then QA testing will get hampered.
please help...and provide your thoughts to it if process needs to be changed...
Im creating Jenkins pipeline for building and deploying our app.
I want to have the same build for the staging and production environment(buld-once-deploy-many approach).
Merging feature branch into staging branch would build an app for both staging and production, but it will deploy it only to staging bucket.
After testing, it would be good that developers merge the staging branch into master which will take the previous staging build and deploy it to production.
Alternative would be to have one branch, and devs would manually trigger another job on Jenkins that would deploy the build to production.
I want to avoid devs going into jenkins and triggering build, since most of them find it intimidating, there are also some nasty vpn configuration steps they need to go through to have access to Jenkins etc.
Would this be a bad practice? Do you have any suggestions how to achieve something like this?
Thanks
This is more of a devops question than a Jenkins question, but here is my take...
I can't decide if you are saying that you would re-build from the master branch, or only use the commit to the master branch to trigger a deployment of the original staging artifact. So I'll address both.
For this situation:
If you build an artifact in a staging branch, and test it. Everything looks green on the tests. So you merge those changes to another branch, re-build, and deploy to production.
The problem: Can you be 100% sure that there was nothing else in the master branch that is not different than what was built AND tested in staging?
You are risking the jello view anti-pattern because unknown and untested changes could sneak into your production artifact. It becomes terribly difficult to troubleshoot why it worked fine it staging, but now fails in production.
For the second situation:
If you are saying that you wouldn't rebuild from the master branch, then merging back to master doesn't buy you anything except a trigger to kick off a new build, because you are never generating an artifact from Master.
If you are going to to it this way, I think you could commit to a single branch, and then tag a release that is meant to go to production, then trigger off of the tag.
Either way, this seems like a strange pattern, and would be difficult to accomplish in an automated fashion in Jenkins. Somehow the newly triggered build would have to find the previously built artifact and deploy that to production.
Possible solution:
There are a few ways to solve this, but one of the easiest is to build once and deploy the same artifact all the way down the line, as you mentioned as an alternative option. But this would likely require some approvals or additional triggering in Jenkins.
If you don't want developers to have to touch Jenkins, then the more likely solution is to build and run your unit tests and smoke tests on the staging environment. Then when a developer wants to promote a build to production, they commit it to the master branch, where the same build is kicked off, and testing is all performed again, in addition to more advanced integration, functional, and acceptance testing. If it doesn't pass, it doesn't go to production.
Your initial tests in staging give the developer quick feedback, but don't serve as the official tests, which only run on the production build.
With a pipeline script, you could easily accomplish this with a single Jenkinsfile and single multibranch pipeline job with stages that are only run when the branch pattern matches.
stage ("Acceptance Testing" ) {
when { branch "master" }
echo "Do testing here"
}
I am creating a build using the new TFS 2015 Build definitions. I have msbuild tasks as well as npm/gulp tasks. I am looking at using variables to allow me to build and deploy to each environment, with DEV being the only one that runs on check-in. However, I don't want anyone to be able to start a deploy for production. How would I go about limiting the users that can start a deploy to production? I'd prefer to only have one build definition, for maintenance.
Use the Release hub capabilities for deployments and create an approval workflow for your environment pipeline.
I am trying to wrap my head around this. Most CI/CD examples/projects have a single master that is always released, and have some variant of, e.g. git-flow, to have a develop branch. Once tagged, it goes to master.
Either way, master is always released to production.
But in the real world as I see it, there are human gates for release to production and other environments. What mechanism do you use to manage the deployment of different versions?
For example:
v1.5 is the current production release
v1.6 has passed all tests, artifacts are ready, it is tagged as valid, but business decides to deploy it only to staging, awaiting an opportune moment to deploy
v1.5 is deployed to a demo environment
v2.0 has also passed all tests, but is in UAT, subject to the customer being happy, as it is a major release
There could be many more such environments - production, staging, UAT, demo, demo2, etc.
What mechanism do you use to handle the tagging of a particular version for a particular environment, and the actual deployment thereof?
Although there a probably a few ways to do it, I use the build pipeline plugin https://wiki.jenkins-ci.org/display/JENKINS/Build+Pipeline+Plugin Along with the copy artifacts plugin https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin
With these, you can create individual jobs for each piece of your environment, and link them altogether.
So as in your example, the pipeline would look like:
Build -> Test and Deploy to UAT (2.0) -> deploy to staging(1.6) -> demo(1.5) -> prod (1.5)
Each piece represents a different build in jenkins. The idea behind continuous integration is you create the binaries once, and you carry it down the pipeline, only changing configuration pieces along the way. In a build job, the artifacts are created and then archived. In any jobs after, the artifact is picked up from the upstream job, some stuff is done, and then it get's re-archived for the next downstream job. So the deploy to staging would go to the Test and Deploy to Uat job to get its binary. The entire concept of Continuous Delivery boils down to the the build pipeline. http://en.wikipedia.org/wiki/Continuous_delivery (and yes I did just cite wikipedia).
As for tagging individual binaries for specific environments, that is by definition, not continuous integration. A binary is suppose to be created in a way that it can easily be propagated from one environment to the next. So unfortunately, individual builds for specific environments can never be continuous delivery. You can use jenkins as a CI server all you want, but if your process does not match, you will never achieve true continuous integration.
Braching, merging and checkins always seems to be a touchy subject when it comes to Continuous Integration, so I won't go into it much. But a lot of people share the idea that : "If different members of the team are working on separate branches, then by definition, they not participating in continuous integration process." http://eugenedvorkin.com/continuous-integration-strategies-for-branching-and-merging/
EDIT
For Flagging specific builds, it sounds like your looking to take use of this feature : https://wiki.jenkins-ci.org/display/JENKINS/Fingerprint ... Which gets the job done effectively, giving you the entire life of any individual artifact. A bit more complex solution would be artifactory, which is essentially artifact source control.
I explained the concept of the deployment process above, and without information on your specific environment it is hard to go much further. But for me, for java applications deployed to tomcat containers, the deploy plugin works great https://wiki.jenkins-ci.org/display/JENKINS/Deploy+Plugin
You shouldn't have to worry about selection of which artifact to deploy. The pipeline should be setup to always deploy the latest artifact that was archived in its corresponding upstream job.
Maybe Docker can help you out with this issue. It is able to deploy images of projects to a specific environment. If that environment has a docker client or a docker deamon you are able to request specific information about that environment and the project (to be) deployed on it.
Jenkins can still play a huge part in your pipeline for the integration part and you could let docker do the delivery part.
Docker: https://www.docker.com
Docker plugin for jenkins: https://wiki.jenkins-ci.org/display/JENKINS/Docker+build+step+plugin
Docker also has support for windows machines and .NET.
I have multiple branches and for every one i have specific build process.
How do i set up gated build process for specific branch?
example
Development branch
-> fast for developers
build process this has main intergation tests
Release branch
-> checkin before production.
contain load testing and records steps contains automated UI tests
Now for my development i want to run tests every time when developer commits changes. but i do not want release build to run every time.
How do i set this up?
You need to have two separate build definitions: One building your 'Development' branch & one building your 'Release' branch.
Both can base on the same Build process template and their differences are:
Dev-build targets in 'Workspace' the path of your development branch - Rel-build targets the path of your release branch
(This is you main question) Trigger = "Gated check-in" for your Dev-build, Trigger = "Manual" (or some other) for your Rel-build
Rel-build will probably be tasked with additional tests.
A basic walkthrough on how to setup a build definition in TFS2010 can be found here