Please read carefully as I believe my use case is unique and I have tried searching a lot on how to do this, but I am still unsure.
Generally, I am trying to set up a repo for a group of developers to work on and have it contain CI checks and require reviewers. However, I have run into some issues with how Github enforces branch protection. On top of that, I cannot use Actions as we are using a self-hosted Enterprise Github through an organization.
My desires:
Use Jenkins (which is already set up and building) builds as checks for PRs. If the checks don't pass, you cannot click merge on the PR.
Allow pushing to a branch that I have a PR up for so the author can push changes based on PR comments.
Require two approvals from maintainers. You cannot click merge without these.
It seems I could protect a master branch for example. However, I would like to enforce the PR checks whenever a person chooses to make a PR. For example, from one dev branch into another dev branch.
Use case:
An author sets up a PR for merging a branch some-work into dev. Jenkins builds the HEAD of the branch some-work to evaluate the checks. I would like to enforce this PR to have two approvals from maintainers. So, those reviewers make some comments and request some changes. The author makes those changes and pushes a new commit to the some-work branch. Jenkins runs on the new HEAD to reevaluate the checks. Then, if-and-only-if the two reviewers approve and the checks pass can the merge button be clicked.
What I have tried:
Using Github branch protections: the required approvals and the required status checks. However, this prevents any pushing or force pushing to the branch being developed on. I could just apply these protections to master, but I also want these checks part of any PR (even dev2->dev1, for example).
Github actions, but these are not available in the self-hosted enterprise Github I have to use.
What I understand:
I understand that I can protect master, for example, in this manner with the native Github branch protection. However, if I want master to be something that always works, it is understandable that developers would break up a feature into multiple branches. They would also want their follow developers to review it when merging it from their branch to an intermediate (non-master) branch. Then the actual branch being merged into master consists of code written by many developers.
Thank you all in advance for your time and help. :)
Using Github branch protections: the required approvals and the required status checks.
However, this prevents any pushing or force pushing to the branch being developed on
But... that issue (not being able to push a protected branch) could be part of a possible solution.
I would make jenkins create/reset a PR branch based on a push on a topic branch (like some-work-pr, based on some-work)
some-work is not protected, and can receive commits at any time
some-work-pr is created by Jenkins protected, and cannot be modified: compilation/test/review/approval happens here.
PR would only be done from xxx-pr branches (protected PR branches created/managed by Jenkins), while other topic branches continue to evolve.
You can only enforce these policies with branch protections, so if you want to enable required CI checks before merging a PR for all branches, then you need to protect all branches (e.g., with the pattern **). In that case, you'll need to have developers use a forking model for your repositories so that they can push code to their forks and then merge in the changes via pull requests only.
Note that if you adopt an approach where projects are implemented as a set of small, incremental changes that are merged frequently and use feature flags to control whether the code is enabled, then as a practical matter developers will only merge into the main branch and you can get away with only protecting the main branch.
Related
I'm working with git flow and a doubt arouse about the master branch. I have development and user acceptance working on develop branch and when I finish the test, I merge the develop with master branch and put into production. My question is if something goes wrong after this merge with master branch, I will not have tested it. What should I do to have the master branch tested? How to ensure that the master is not broken? Is there any pattern?
tl;dr: No, you already tested it.
Details:
In a properly implemented Git-Flow strategy, anything you merge into master should be fully ahead of master 1, so you would never need to test what you'll get after merging into master, as it will be the identical state before and after the merge. (And presumably you have fully tested the branch before merging into master!) The reason this is true is that when hotfix branches are merged into master, they should be immediately merged back down into develop too (or into a release branch if one currently exists). This means there should never be any code that is only on master and not yet in the branch you're about to merge into master. As documented by Git Flow regarding completing hotfix branches:
When finished, the bugfix needs to be merged back into master, but also needs to be merged back into develop, in order to safeguard that the bugfix is included in the next release as well. This is completely similar to how release branches are finished.
And also,
The one exception to the rule here is that, when a release branch currently exists, the hotfix changes need to be merged into that release branch, instead of develop.
Side Note: You mentioned that you are merging develop directly into master, and you can do that if it's working for you, but I just want to point out that typically in Git Flow you use a release branch instead. Doing so enables simultaneous development on develop while you are hardening a release as it awaits deployment. Here's a question that discusses this, and my answer there provides some tips for how to avoid creating release branches in Git Flow, if you normally don't need them.
1 In Git we oftentimes say that one branch is "fully ahead" of another branch when the first branch can reach the tip commit of the second branch. For example, develop is ahead of master if the tip commit of master is in the history of the develop branch. In this case however what we actually mean is that all commits with state changes in master are present on develop, and this distinction exists only because of the --no-ff requirement when merging into master, so it's possible that master might have some merge commits that don't yet exist on develop. Those merge commits do not contain any new state though, so we can say from a practical point of view that develop is fully ahead of master.
From a purity standpoint, my personal preference is to always have every commit on master be present in develop or release before merging into master, so that we can say "fully ahead" and mean it from a commit ID standpoint as well. To achieve this, you can slightly tweak the documented Git Flow by, instead of merging release and hotfix branches back to develop after merging into master, merge master back down into develop. This achieves the identical state but also achieves the "fully ahead" meaning that we like to see in Git. It also means anytime you merge anything into master, you could do a fast forward merge, but we choose to use --no-ff instead to maintain a historical record of when the merge occurred, and exactly what was merged. Using this tweak, the fact that you could have done a fast-forward merge is the proof you need that you don't have re-test master after the merge.
If the master branch changed since you forked the develop branch, you certainly could break master after merging develop (especially if you had to sift through merge conflicts), even if develop was working fine on its own. The way to make sure that master isn't broken after merging is to have good unit tests for master to make sure no existing functionality is broken, and then add and run additional unit tests for develop which test the new feature works after its merged.
For our projects have Integration and Master branch. For deployment we cut a RC branch off Integration. Once deploy is complete we create Pull request for RC to Master.
However, now Master shows as 1 ahead because it has a commit that does not exist in Integration. I considered to create Pull request RC to Integration as well, but TFS does not allow it since there are no changes.
Is there a way to avoid this situation with Master? Can behind/ahead only check the code changes and not commits? To fix this I now have to create Pull Master to Integration, and that is a pain for all the projects we do.
Afraid not able to avoid this situation. It's also not possible to make behind/ahead only check the code changes and not commits. Since you are using Pull Request (which execute git merge --no-ff).
To be honest, it's not necessary to resolve Integration branch behind/ahead master.
you can have two mostly independent branches without any problems. The important measure of differences between branches is given by git diff. If this reports no differences, then it's Ok.
You could also take a look at this similar question: VSTS Git: Is it necesary to resolve dev branch behind/ahead master and if so how?
If you insist on avoid the ahead on Master, you may have to create Pull Master to Integration as you have pointed out in the question.
My team's policy is to not check in broken code into TFS. But what if two team members are working on the same file and each member's work represents a partial solution ? One members changes won't work without the other.
How is the best way to combine the code and still follow policy of not checking in broken code ?
Both developers could create a shelveset of their changes once their own coding is complete. Once it's time to merge their code, you have 1 developer unshelve the other developer's shelveset into their own workspace. Doing this will also trigger the Merge Tool in case there are any merge conflicts that can't be automatically resolved. This should allow the developer to combine work with the other without checking in broken code.
Note that both developers should make a shevleset of their own work before unshelving the other's, in case the unshelve of the other developers work goes terribly wrong in the merge and they need to revert their own work to the un-merged state.
Here's some documentation on creating and unshelving shelvesets
Consider allowing broken code on branches. The team members could commit code to branches and merge those branches together before merging into master. Your primary or master branch should require a passing build before a pull request can be merged in.
Look at the feature branch model: https://learn.microsoft.com/en-us/vsts/git/concepts/git-branching-guidance. This does not depend on Git but it is convenient as branching is cheap.
Both developers work on the feature branch which is allowed to be broken. When they are finished they issue a pull request to the master branch. The merge is only allowed if all tests pass to make sure the master branch stays healthy.
When using Gerrit Code Review - is it possible to checkout all unreviewed changes for the master branch in one go? I know it is possible to checkout a specific patch set (https://gerrit-documentation.storage.googleapis.com/Documentation/2.12.2/intro-quick.html#_trying_out_the_change), but I would like to get all pending-review commits for the master branch.
Background: we currently aren't using Gerrit yet but we are going to. We have both automated and manual testing of the master branch and I would like to be able to keep on using that, before the review step, so I need to be able to checkout a branch with all pending-review commits.
AFAIK, Gerrit does not provide that feature. In fact, I am not sure if that is even possible without human interaction. If there are N independent unreviewed commits to master, there could be conflicts between them that would make it impossible to automatically build up a branch made of all N commits on top of master.
Is it possible to do pull request in bitbucket style in Phabricator?
Eg. to branch off of some existing branch and then to create pull (merge) request to merge new branch back?
I see that Phabricator differential tool only allows to submit some manually entered diff to some branch. Is it the only way?
You can mix git and arc together, but it does go against how arc diff is meant to be used.
You could leverage audit instead, although I haven't gone into any details about that below (nor have I used audit, yet).
Below, I've tried to explain our new workflow, coming from git-flow to an adapted version using git and arc diff.
Background
Before we started using Phabricator we used Gitlab and created merge requests. These would get reviewed by another developer. We use JIRA and our workflow incorporates a review required stage which has several checks before progressing into testing.
At this point, we've pushed the branch to the remote, requested a review and awaiting testing to occur (we have a mix of both manual and automated testing).
Once the review has been accepted and tests have passed, the feature branch is merged into origin/develop.
New workflow
Our new workflow removes the need to create the merge request within Gitlab, especially for reviewing.
My team still uses a workflow that incorporates git-flow, however, we've introduced the arc diff command. This creates the diff within Phabricator. The developer pushes his branch to the remote, but doesn't raise a merge request.
We run the following commands when creating a diff (merge into origin/develop)
git checkout -b feature/foo
git add <files>
git commit -m "A useful commit message"
git push origin feature/foo
arc diff origin/develop # this creates the diff within diffusion
Once the review has been accepted, we don't merge (or arc land) the branch, we await all testing to take place. This allows us to update the diff should testing fail and the developer who reviews can easily see which commits need reviewing.
Once testing has passed we can simply merge, either using gitlab merge request or the command line. We usually run arc close-revision <revision-id> to close the revision within Phabricator itself.
Additional notes
I believe the philosophy of arc diff is that you don't push your local branch. Instead you create a diff which diffusion displays. This is classed as a pre-push workflow.
Phabricator also has a post-push workflow which incorporates audit. You can simply mark commits ready for audit by amending your commit messages.
No, see https://secure.phabricator.com/T5000 to track this feature request.
Differential's primary input should be Arcanist, the Phabricator command line tool. It wraps git and provides lint, unit, and other precommit checks that help reduce time spent reviewing code. For example, it can emit patches and amend code before submission for review.
https://secure.phabricator.com/book/phabricator/article/arcanist/