I am looking for a specific strategy / convention that works for concurrent development with Feature branches and a database deployment tool such as DbUp, DbDeploy, ReadyRoll, etc.
We run multiple feature branches for concurrent project development.
Each branch has a dedicated development, QA and UAT environment which is deployed via Octopus deploy.
I am trying to tackle automatic database deployment with Octopus using a tool that will handle the changes applied in each of the branches.
Database changes will occur in all of the branches (including the release branch).
Most of the tools that I have seen thus far use a sequenced based approach of scripts that are checked into the VCS and deployed by the tool. The tool for the most part applies the script in an ascending filename order and most that I have seen specify to follow an approach of 1, 2, 3, etc.
This works fine for one branch.
My issue will be when feature A has 1 and Feature B has 1 - both get merged into the main branch. I now have two #1 scripts. Making it even more fun - our path to production is to merge one more time which may also have a 1. So now we have 3 #1 scripts.
There is also the problem of backward merging. Once a project is done - merge the release branch back into main and then merge again to the feature branch to reset it for the next project. In this scenario - I have two additional #1 scripts that have not been applied to the target feature branch database.
My initial solution to this is to use a julian date as the leading prefix for the sql filenames that are checked into source. I was also thinking of applying the branch name to the file along with the work item. So the sql file would follow a convention of {XXXXX_Y_ZZZZZZ.sql} where xxxxx is the julian date, y is the branch and zzzzzz is the work item from TFS.
I am looking for a specific solution to this problem. Has anyone else solved this? What did you do? What are the drawbacks? What tools did you use?
Have you looked into Readyroll's semantic versioning?
We are implementing Readyroll now, we are a small team in which it's easy to communicate and confine changes between feature branches.
We integrate to a development branch and detect conflicting changes early on and basically rewrite earlier migration scripts if necessary (which requires have a baseline of your database project to which you can revert)
On the Redgate website there is some more information about working with branches but doesn't cover your scenario exactly, switch branches with readyroll.
When looking in to tools I basically came to the conclusion it a choice between a state based approach or a migration based approach. I like readyroll's take on it, which offers a mix.
I very curious what you end up with!
Related
We are using visualstudio.com TFS to store the source code. We have DEV, STG and PROD branches. now I want to automate build and release to IIS server.
But it looks like to me that TFS compiles the code from the branch where I commit to, and that's the only build, after that I can install the same binary on all the servers.
But it doesn't seem to be ok, because what if we find a bug on the live code, so we need to fix it, but we already have new code in DEV, and we don't want to revert, and install the old code with the fix on the dev server?
What I think we should do is to merge the code from DEV to STG, from STG to PROD, but I couldn't find any module for that. And it looks strange, I would be surprised if I am the only person whats to do this, especially because it is doable with Jenkins.
thanks
I typically discourage automating merging of code between branches; merging should be a deliberate action. When someone expresses that desire, it's usually a sign of a problem with the branching model and deployment model they're using.
Code promotion branching (where you have one branch that corresponds to each environment, as in the scenario you described) is a bad practice because it encourages you to build different sets of binaries for each environment. You build something for a lower environment, test it, validate it, then totally disregard that testing and rebuild from source. That should terrify you, especially for production branches -- you have no idea if what you just built is going to work properly.
Current thinking is to minimize the number of branches you have, and instead isolate in-progress work behind feature toggles so that features that aren't ready for production can simply be disabled, but still allow deployments to take place. With sufficiently mature feature toggling practices and testing practices in place, you can actually eliminate branching entirely and work out of a single branch, promoting binaries to production whenever your automated and manual QA process deems the version they're testing is good.
Assuming you're using TFVC, if you want to maintain a code promotion branching strategy, then you'll need to maintain multiple builds and releases, one for each branch/environment. Typically, in this kind of scenario, I'd eliminate the stage branch entirely. Anything that goes to the Dev branch is built and deployed to Dev. Anything that goes into the Prod branch is built and follows a deployment pipeline from staging->production. Hotfixes can go directly into the prod branch and be merged backwards. There are tons of strategies for branching; you'll need to read up on the subject and design a branching and release strategy that works best for your team.
I've currently got a number of Team Projects all happily managed by TFS.
I have one project, a windows app, that is currently in use and before I ported the code base to TFS, we manually used to maintain both the production build and a new development build using scripts to copy/merge the files.
e.g. App1 v1.x - production and
App1 v2.0 in development.
Before TFS we manually "merged" to bug fixes from the development build into the production build - so where applicable bug fixes from v1.x were also fixed in v2.
In this particular case v2 is quite different, re-factored ui, etc. The question I have is what is the best way of porting this scenario to TFS.
As I see it I have two options:
Create a new Team Project and continue manually "merging" applicable
code files.
Create a brand from the current project v1 and replace / overwrite the project piece by piece in VS so that the source control can manage the changes back to the Main team project.
V2 has a few additional class libs too - if that makes a difference.
Typically, this is the strategy we use to do branching and merging:
The project starts. All team members are working on version 1.0 in the single CURRENT branch.
Project approaches a milestone or release. Make a branch that will be used to stabilized the code for production in PROD 1.0 branch. Now half of the team stabilizes the product on the branch and half of the team continues doing new (riskful) stuff on the CURRENT branch.
PROD 1.0 branch is ready and delivered in production. The branch stays intact to provide maintenance and support on the delivered version. Fixes are made on the PROD 1.0 branch and merged into the CURRENT branch.
Project approaches another milestone or release. Make a new branch that will be used to stabilized the code for production in PROD 2.0 branch. The same mechanism is used as described before.
Using this branch-per-release strategy, you always have a branch per shipped and to be maintained version in which fixes can easily be propagated forward and backward between versions and the mainstream CURRENT development line.
With this respect, we use a TFS team project for multiple releases of a certain product. This limits the overhead creating and maintaining the project spaces.
In short...To branch!
It seems your code between versions is that much similar that you should be fine by simply branching your new release.
In general, changing to a new Team Project makes sense if you have one or more of the following:
a totally new bread of requirements and/or different Work Item types
completely different technology, in other words you have radically changed the design & merging is more loss than gain
need to store docs & other tokens in another SharePoint site
different development methodology - process template
different teams of people working on both sides, possibly want to restrict users from viewing v1 or v2
plan to release both products in completely different release cycles
need for totally separated reports
Reading your post I don't recognize the need to go after creating a new Team Project - of course I might be wrong.
In case you decide to go with the branch variant you could be greatly benefited from the impressive Visual Studio TFS Branching Guide 2010, on how to shape the structure of your codebase.
There are several/many questions regarding TFS branching strategy, but I am haven't been able to come up with a strategy that fits with my scenario. My TFS project consists of a single solution that contains a Web Project, a Business Layer Project, and a Data Layer Project. The project is a portal of reports. Reports are largely isolated in subfolders within the project. There are however some features across the entire project such as session management. Over a given period of time, the workflow may occur as follows:
Stable snapshot of code.
Development of Report A begins.
Development of Report B begins.
The project with the inclusion of Report A needs to be pushed to our qa environment.
The project with the inclusion of Report A and Report B needs to be pushed to our qa environment.
The project with only the inclusion of Report B needs to be pushed to our prod environment.
So basically, each report is on a completely independent timetable. I need to be able to independently publish a branch of code to our different environments. Currently, we don't have branching - we just don't add a link to a new feature if the project gets published when a report isn't ready but is included in the project. Not the best scenario.
My initial go at a branching strategy was to have Main sit between the QA and prod environments, basically as just a container to merge before branching to a production branch for a production publish. Each report would be developed on a branch from main. For both our test and qa environments, a branch from main would be created and the appropriate development branch(es) would be merged into this "proposed updates" branch. This doesn't work though because I am merging development/feature branches into a branch that isn't the parent branch. I can't have Main at this level because a Report may be in development for weeks while another may be on a timetable that has it developed and pushed through the process to production in only a few days. My "proposed update" branches for test and qa need to be able to be independently created from a merging of only the appropriate dev branch(es).
My only experience with branching/merging is a main+dev pair of branches, so I'm very out of my element here. How can I setup my branching in such a way that I am able to merge features in on independent timetables without getting stuck and code being published to an environment before it is ready?
If it matters, we are on TFS 2008 right now and hope to go to TFS 2010 soon. This is an immediate need to get going on our current TFS 2008 server though.
I'm not exactly clear on everything; reading comprehension and all.
As I understand it, your current process is Dev -> Test -> QA -> Production. Devs work on code, push it to an environment where they can test on it. Once satisfied, they push it to QA, and when code passes it moves into production.
In addition, you have several "teams" (1 or more devs) that must work on separate reports, each of which must eventually be moved through the above process into Production. Teams may be working on code that is distinct from all others, or teams may find they cannot move their code forward until other teams reach stability.
If I were in charge of branching for this solution, I would recommend the following.
First, create a Production branch. This branch only contains production code. Only your QA team touches this branch.
Next, create a QA branch. This branch is also maintained solely by QA teams. They manually merge test code into this branch, run their quality assurance tests, then merge with Production. Every time they merge with Production, or test code is accepted into QA, a label is applied to the branch. If test code fails, the branch is reverted back to the prior label.
Development teams manage their own branches. They are created by branching from QA at the latest label. This assures they are working with the latest approved code. Developers work with and test on this branch. If teams have a dependency on each other, they should work on the same branch, unless it becomes clear that creating secondary branches from their shared Dev branch would be easier. Once a Dev branch meets the milestones set for the developers, QA should be informed that the branch is ready for merging with QA for testing.
Alternatively, depending on how complex development is, you might even consider uniting the QA and Production branches. Often, it is a simple matter to add a label to a branch to indicate a stable, production worthy build. It also keeps the branching strategy as simple as possible, which is always a good thing.
I think you should look at the Branching Guidance put together by the VS ALM Rangers.
http://tfsbranchingguideiii.codeplex.com/
This should alswer all of your questions. You are looking at quite an advanced branch plan. I also have some good practical guidance on my blog. I know that I am talking about Scrum teams, but it is basicaly Feature branching based on the Guidance.
http://blog.hinshelwood.com/archive/2010/04/14/guidance-a-branching-strategy-for-scrum-teams.aspx
If you get a chance please vote over
at the new "Visual Studio ALM"
StackExchange over in Area51 as we are
trying to setup a place dedicated to
answering these questions with the
Visual Studio ALM MVP's and Visual
Studio ALM Rangers on hand to answer
your questions.
http://area51.stackexchange.com/proposals/15894/visual-studio-alm
[I know this question is old but this may help others that come across this question]
Most "branch per feature" teams use one "main" branch and break away from the branch per environment approach. Environment releases can be handled by clearly labeling each environmental release on the MAIN branch.
The QA release is the result of merging all features/issue branches considered fully tested and ready for QA into MAIN. As bugs are found, new bug branches are branched off of MAIN, which are fixed and merged back to MAIN. When all QA releases are deemed ready for production, a PROD build is made from MAIN. In short MAIN is the one source of truth for code.
If you need to work ahead, an integration-test branch (TEST) can be used to determine what features are "production ready," but feature/issue branches should be merged to MAIN on a case-by-case basis, rather than in bulk from the TEST branch.
Hotfixes can be branched from the PROD label, then fixed, tested and merged back to MAIN for a new PROD release.
I'm going to refer you to a great video on Branch per Feature. It focuses on GIT, but the strategies do not depend on GIT and can be used just as effectively on TFS SCC: https://youtu.be/9SZ7kSQ2424
We have 3 environments: dev, test, and staging.
I want to check in and out of TFS. When we make changes, I want to promote the code to the dev web server. Next I want to promote the changes to test, then to staging. Would it be possible to do this with Team Foundation Server?
Why on earth are people suggesting branching? You branch for different features or static branches for release snapshots.
Surely the differences between these environments are configuration items/files and settings within these. All you need to do is get your deployment and release management process in order.
Create appropriate MSBuild tasks and use TFSBuild (Continuous Integration) to call these to take care of outputting the correct config files for the Environment/Configuration you are building. You can trivially add another MSBuild target that deals with the appropriate deployment to the respective target environments.
You can manually checkout the code from each branch, make your changes subsequently to each branch, and checkin. Very carefully.
Much better is to have these 3 environments be branches of each other. (Typically you start with dev, and branch to the other 2 in turn). Then you can use the Merge functionality to merge (for example) your dev changesets directly to test, etc. At this point your Test modules (that need to be changed to match dev) are checked out, with the changes. Then simply commit the changes. Then repeat for staging and rinse. This is the suggested methodology for this common scenerio.
Two important notes:
Even though TFS if very server-centric (compared to SVN, for example), this merging functionality happens on the client. You need to have each branch mapped to your machine. After the Merge process is completed, you'll have uncommitted changes in the target branch until you check in.
In Microsoft's vision and in the example I give here, these branches are permanent. This was a change from my previous practice using SVN, where whole branches were created/promoted/retired all the time. In the TFS way, you create the Test branch and it remains, indefinately, the Test branch. It's never promoted; its changes are merged elsewhere.
Building is a separate action. You need to set up a separate build for each situation, though of course once you set up the first one the other two will be trivial. After your merge to staging, you'll then run the staging build. (From Team Explorer or in the Build menu). TFS is a bit heavy but once it is set up it does handle this situation very well, easy for a distributed team to merge and build quickly (with automated build tests, etc.).
Yes, this is possible, but you must manually check them in from one branch to another.
Simply put I have the following branch setup:
MAIN
|--- DEV
|--- PROD
Most developments are done in the DEV branch. When the code is ready to test, everything is merged to the MAIN branch and published to our test environment. When tests are completed, a merge to PROD is done and everything is published to the production server. Every now and then changes (mostly bugfixes) are made on the MAIN or PROD code, but this is an exception.
I have been asked to think out a system for feature and bugfix merging. This means that separate changes in the DEV should be merged across MAIN and PROD. With our current setup this information is lost: for example features A, B and C are implemented in the DEV branch. Let's say every feature has two corresponding changesets: A1, A2, B1, B2, C1, C2. With our current way of working, everything is merged to the MAIN branch in one go. So when we want to "cherry pick" features which have to go from MAIN to PROD we can't do this because there's only one changeset on MAIN: the checkin of the merge.
How would you fix this? Do I need to change something to my branching strategy?
I'm using TFS for source control.
So when we want to "cherry pick" features which have to go from MAIN to PROD we can't do this because there's only one changeset on MAIN: the checkin of the merge.
You can write a tool to piece through the merge history, if you like, but the real answer is don't do that. When you cherry pick, you lose any guarantee that the code you tested and stabilized in the source branch will perform the same way in the target branch. Sometimes that's ok, but in your case it defeats the whole purpose of having an intermediate branch sitting between raw untested Dev checkins and your live PROD deployments.
As discussed in my favorite branch/merge video, your guiding principle should be "merge down, copy up." That is, whenever the need arises to deconstruct and/or apply code diffs, let unstable branches take the hit. (Cherry picking features out of an otherwise integrated app is one example.) Meanwhile, code that's promoted up toward stable branches like Main & Prod should always be a straight copy that matches what you've already worked so hard to stabilize in the source branch. Sounds like you're following this strategy currently; preserving it in the face of cherry picks would be my #1 motivation for using feature branches, even moreso than insulating feature teams from each others' breakages.
Managing dependencies between features is an issue, as Jim mentioned. If you can identify them in advance, the usual solution is to make sub-branch(es) that are shared by the features with the common dependency.
Feature1
\
LibA---
/ \
Feature2 \
DEV -- MAIN -- PROD
Feature3 /
\ /
LibB---
/
Feature4
Software doesn't always go as planned, of course. And this doesn't work at all if the branches that need to share code are on opposite sides of the tree (e.g. if Feature1 depends on LibA and LibB, but Feature2 is ill equipped to be part of B for structural or technical reasons).
I don't think there's any magic sauce here, you've just got to find a system where you have a revision on main for each unit you might like to cherry pick.
This can be done trivially by merging each revision individually, which is a pain, but gets you what you want.
Alternatively, you can up the granularity, by merging each feature into main one at a time. This requires that you work sequentially on features, which may be ok if you're on your own, but will be a pain if there's a few of you, since you'll have to go through a code freeze where some people have finished and others haven't.
Another way of working which you may or may not find more manageable is to have a DEV branch for each feature. In this sense, instead of having an ever existant DEV branch, have a collection of ephemeral DEV branches that only exist until the feature is completed.
The reintegration of each DEV branch will give you a clear revision in main which can be cherry picked.
You can get dependencies between dev branches. Say branch devA needs some implementation from branch devB, you'll have to merge the required parts of devB into main and then merge them down into devA. However, devA shouldn't be needing unfinished work from devB, so you should (in theory) be able to RI those parts happily anyway. And of course, since you're cherry picking into PROD, these partial integrations don't have to get published.
Given your branching strategy, I guess you've already found this, but if not, it's worth reading:
http://branchingguidance.codeplex.com/wikipage?title=html&referringTitle=Home