we have a team of 3 and have about 10 clients for our branded/customized apps
All of the customized apps have a central code base in git (master).
We would like to use git as the central repository for all of the apps.
What we would like to achieve is:
we will set up a code base as master, it has its dummy graphics and data.
Each time we have a new client, we will make a new branch (client1). Update graphics and customize the app, then we deploy to the production.
So developers will work on their local machines, make a sub branch (client1-1, client1-2) etc
then commit back to parent branch (client1).
Later, if we want to commit a change from client 1-1 all the way back to master, is that possible?
Also, is that the standard development process for iOS app ?
I feel that is not a good idea because, when you deploy to production or merge with the master, you will be practically be spending a lot of time on merge conflicts, then getting the code to the production. If i were you, I will have 10 different git repositories and if a new customer comes, then i will create a new one. Since, git is free, creating a repository will not be difficult. Saves you a lot of time and tension as well. Once the repository is ready, then clear all your production to make it as if new. Then push the ready repository. Hope this helps...
Related
How is image promotion handled with 'dev' images?
Like many others, we have three environments for deployments – Dev, Stage, Production.
Stage always tracks the latest master, and production deploys are triggered when releases are tagged in Github. It's easy enough to promote images between stage and production.
Where I'm having trouble understanding this is with Dev. Dev images are built on pull requests in our case, but there's a good chance the branch behind the PR is some number of commits behind master, so promoting this image through the ranks would cause outdated code to end up on stage and prod.
What is the typical practice here? Are dev images kept separate, in a separate image repository and never mixed with the stage/production images? Is there some way to promote them that I'm missing?
Stage always tracks the latest master, and production deploys are triggered when releases are tagged in Github
Dev images are built on pull requests in our case, but there's a good chance the branch behind the PR is some number of commits behind master
This is all a bit confusing to me. I like to keep things simple, and I usually advocate for one branch per environment.
In this scenario, I would advise you to have the following branches:
dev
stage
production
This will allow you to make changes to one environment at a time, progressively. So first you would push your changes to the dev branch, and if they are successful, you would then push the same changes to staging; likewise, if the changes are successful in staging, you would then push the changes to production. The pushing to the branch can and should be done via merge requests.
Each branch would have its own pipeline associated to it, which would build, test and release the image to the environment specific registry. Keeping registries separate amongst environments may seem overkill but it would make your life easier when it comes down to permissions management.
Having one branch per environment also has the added benefit that you will be able to restrict access to certain branches, e.g. everyone should be able to merge to dev but not everyone should be able to merge to prod.
We're a small team of 4 developers and 2 testers and I'm a team lead of the team. Developers do their tasks each in separate branch. Our stack is ASP.NET MVC, ASP.NET Core, Entity Framework 6, MSSQL, IIS, Windows Server. We also use Bitbucket, Jira software to store code and manage issues.
For example, there is a task "add an about window". A developer creates branch named "add-about-window" and put all the code there. Once the task is done, I do code review and in case all was good, I merge the branch into some accumulating branch let's name it "main". As a next step, I then manually deploy the updated "main" branch to test server with installed IIS, MSSQL. Once done I notify testers to test freshly uploaded app to make sure "add about window" is done correctly and works good. If testers find a bug, I have to revert the task branch merge from "main" branch and tell the developer to fix the bug in task's branch. Once the developer fixed it, I merge the branch into "main" branch again and ask testers to check again. In the end the task branch gets deleted.
This is really inconvenient, time consuming and frustrating. I have heard about git flow (maybe this is kind of what we have now).
Ideally, I would like this process to be as this:
Each developer still do work in separate branches.
Once a task is done and all the task code is in task branch I do code review.
Once code review is done and all found issues are fixed I just click "deploy"
There is a Docker image which contains IIS, MSSQL, Windows. It also with some base version of the application we work on, fully tested and stable. Let's say it's on a state of some date, like start of the year.
The Docker image is taken and a new container starts.
This Docker container gets fully initialized and then the code from a branch gets installed on the running container.
This container then has own domain name like "proj-100.branches.ourcompany.com" ("proj-100" is task's ID in Jira) which testers can go on and test.
This would definitely decrease time I spend on deployment and also will make the process more convenient and comfortable.
Can someone advice some resources I can learn about similar deployment models? Or maybe someone can share info on this. Any info will be very appreciated.
regardless of your stack, and before talking about the solutions, what you describe is the basic use case of any CI-CD process. all the exhausting manual steps you described, can be done with any CI tool.
now, let's consider what you already have, and talk about the steps for your desired solution - you're using bitbucket, which already gives you at least steps 1 and 2 - merging only approved PRs into master/main.
step 3 is where we start the CI automation process - you define a webhook upon certain actions in the bitbucket repo, which triggers a CI job/pipeline(can be a Jenkins server, gitlab-ci, or many other CI solutions). this way, you won't even need a "deploy" button, since the merging action can trigger the job, which can automatically run unit tests, integration tests(if you define them), build artifacts/docker-images and finally deploy.
step 4 needs some basic understanding of the docker containers design - a docker image is not a VM. it has its use cases and relevant scenarios, and more importantly an advised architecture guideline to follow.
to make it short, I'll only mention the principle of separation - each service should be in a separate container. it allows upscaling and easier debugging, and much more. which means - what you need is not a docker image that contains your entire system, but an orchestration of containers, each containing an independent software unit, with a clear responsibility. and here Kubernetes comes into play.
back to the CI job - after the PR merge, the job starts, running the pre-defined unit tests, building the container, and uploading to your registry.
moving to CD - depending on your process, after the updated and tested docker images are in your registry(could be artifactory/GitLab registry/docker registry...), the CD job can take any image it needs, and deploy them in your Kubernetes cluster. and that's it! the process is done.
A word of advice - if you don't have a professional DevOps team, or a good understanding of docker, CI-CD process, and Kubernetes, and if your dev team is small(and unfortunately it seems so) you may want to consider hiring a DevOps company to build the DevOps/CICD infrastructure for you, preferably with a completely managed DevOps solution and then do a handover. everything I wrote is just the guideline and basic points, to give you the big picture. good luck!
All the other answers are here great still I would like to add my piece of advice.
Recently I was also working on a product and we were three team members. It was a node.js project. If you are on AWS then you can use the AWS pipeline. This will detect pushes from a specific GitHub branch and the changes will get deployed to the server. The pipeline service has a build stage too. You can also configure slack notifications.
But you should have at least two environments production and dev to check if deployment is working properly on dev.
AWS also has services like AWS Code Commit and AWS Code Deploy.
This is just a basic solution. You don't actually need fancy software to set up ci/cd.
This kind of setup is usually supported by a CICD tool coupled with Kubernetes.
Either an on-premise one, like Jenkins+Kubernetes, or its Jenkins Kubernetes plugin, which runs dynamic agents in a Kubernetes cluster.
You can see an example in "How to Setup Jenkins Build Agents on Kubernetes Pods" by Bibin Wilson.
Or a Cloud one, like Bitbucket pipeline deploying a containerized application to Kubernetes
In both instances, the idea remains the same: create a ephemeral execution environment (a Docker container with the right components in it) for each pushed branch, in order to execute tests.
That way, said tests can take place before any merge between a feature branch and an integration branch like main.
If the tests pass, Jenkins itself could trigger an automatic merge (assuming the feature branch was rebased first on top of the target branch, main in your case)
We have similar process in our team.
We use gitlab-ci.
Hence there are some out of docker infrastructure (nginx with test stand dns),
we just create dev1, dev2 ... stands (5 stands for team of 10 developer and more than 6 microservices). For each devX stand and each microservice we have deploy to devX button in our CI-CD. And we just reserve in slack devX for feature Y on time of tests after deploy. Whan tests are done and bugs are fixed we merge to main branch and other feature brunch can be deployed and tested on devX stand.
As a next step, I then manually deploy the updated "main" branch to test server with installed IIS, MSSQL.
Once done I notify testers to test freshly uploaded app to make sure "add about window" is done correctly and works good.
If testers find a bug, I have to revert the task branch merge from "main" branch and tell the developer to fix the bug in task's branch.
If there are multiple environments then devs could deploy themselves into them. Even a single "dev" environment they can deploy to would greatly help. The devs should be able to deploy themselves and notify the testers without going through you.
That the deploy is "manual" is suspicious. How manual? Ideally it should just be a few button clicks. Sometimes you can even have it so that pushing to a branch does a deploy (through webhooks).
You should be able to deploy from branches besides main. What that means or looks like can vary a lot but the point is that if you're forcing everything there and having to revert when it doesn't work you're creating a lot of unneeded work. Ideally there should be some way to test locally. If there really can't be then you need to at least allow a way to deploy from any branch (or something like force pushing to a branch called 'dev' or something).
From another angle, unless the application gets horribly broken you don't necessarily need to rollback changes unless a release is coming soon. You can just have it fixed in another pull request.
All in all the main problem here sounds like there's only a single environment for testing, the process to deploy to it is far too manual, and the devs have no way to deploy to it themselves. This sort of thing is a massive bottle neck. Having a burdensome process to even begin to test things takes a big toll on everyone's morale -- which can be worse than the loss in velocity. You don't necesscarily need every dev to be able to spin up as many environments as they want at the push of a button but devs do need some autonomy to be able to test.
Having the application run in Docker containers can greatly help with running it locally as well as making the deployment process simpler. I've tried to stay away from specific product suggestions because this is more of a process problem it sounds like.
We are using visualstudio.com TFS to store the source code. We have DEV, STG and PROD branches. now I want to automate build and release to IIS server.
But it looks like to me that TFS compiles the code from the branch where I commit to, and that's the only build, after that I can install the same binary on all the servers.
But it doesn't seem to be ok, because what if we find a bug on the live code, so we need to fix it, but we already have new code in DEV, and we don't want to revert, and install the old code with the fix on the dev server?
What I think we should do is to merge the code from DEV to STG, from STG to PROD, but I couldn't find any module for that. And it looks strange, I would be surprised if I am the only person whats to do this, especially because it is doable with Jenkins.
thanks
I typically discourage automating merging of code between branches; merging should be a deliberate action. When someone expresses that desire, it's usually a sign of a problem with the branching model and deployment model they're using.
Code promotion branching (where you have one branch that corresponds to each environment, as in the scenario you described) is a bad practice because it encourages you to build different sets of binaries for each environment. You build something for a lower environment, test it, validate it, then totally disregard that testing and rebuild from source. That should terrify you, especially for production branches -- you have no idea if what you just built is going to work properly.
Current thinking is to minimize the number of branches you have, and instead isolate in-progress work behind feature toggles so that features that aren't ready for production can simply be disabled, but still allow deployments to take place. With sufficiently mature feature toggling practices and testing practices in place, you can actually eliminate branching entirely and work out of a single branch, promoting binaries to production whenever your automated and manual QA process deems the version they're testing is good.
Assuming you're using TFVC, if you want to maintain a code promotion branching strategy, then you'll need to maintain multiple builds and releases, one for each branch/environment. Typically, in this kind of scenario, I'd eliminate the stage branch entirely. Anything that goes to the Dev branch is built and deployed to Dev. Anything that goes into the Prod branch is built and follows a deployment pipeline from staging->production. Hotfixes can go directly into the prod branch and be merged backwards. There are tons of strategies for branching; you'll need to read up on the subject and design a branching and release strategy that works best for your team.
Using TFS, we have the following:
A main baseline
A development branch for each development effort. These get merged back to the baseline.
A release branch that is created with each release. Bug fixes are made here, released, and merged back to the baseline.
Using shelvesets, we can share code across development branches if needed without contaminating the baseline. Useful for code reviews.
When we deliver our development changes to baseline we have an automated build that kicks off and automatically places our changes on the test server.
The problem is that the business analysts can't see our changes until they're on the test server, and currently the only way to get our changes on the test server is to check them into baseline. So if the BA's find something wrong, the code is, unfortunately, already in baseline and we would have to go through the trouble of taking it back out.
Is there a way we can change our branching strategy or process to get the BA's what they want to see without contaminating our baseline?
Your branching strategy sounds exactly what we decided on at my company. I don't think the issue is with your branching strategy, I think the issue is that you have to check changes into the baseline in order to apply them to the test server.
At my company, changes aren't checked into the baseline until they are promoted and running in production. Release branches are what are deployed to the test servers... if bugs are found, or the BAs want to change something, we don't have to go through the pain of removing the changes from the baseline.
However, if you have a lot of concurrent releases, this can become a pain to merge all of the releases together before moving them to production, since you aren't merging into the baseline until later in the process. At my company, we have a very strict release schedule, and try to only have a single release working its way to production at a time. Because of this, waiting to merge the release into the baseline until the release has been promoted into production hasn't created any issues for us, or extra work so far...
How often do you do releases? Would you be able to deploy release branches onto your test servers, and have the baseline represent what is currently deployed in production?
(I'd make this a comment, but I'm still working on earning that privilege...)
I would not prefer this approach, I would suggest:
A main baseline which contains stabilized code. The code will be merged into this branch from respective release branch only after successful release.
A Release branch which gets created from Main for each release. This branch will be used to generate Release Builds and will be deployed to test environment.
A Development Branch created from Release Branch, it will be used for Development efforts and will be merged to Release when I'm ready to give my build to test.
I'm getting lost with Git branching model and the workflow that I want to create for my team (developers + designers).
Suppose that the project is based on a MVC pattern, so we have a structure similar to :
Models/
Controllers/
Views/
Developers works on the M & C parts with some basic/generated views (Rails, Django or CakePHP app for example)
and Designers works on the V part
How can I manage that developers works on M&C and keep some basic crappy views, and in the same time, designers make sexy views based on controllers actions coded and added by developers progressively ?
I tried to make it works with 3 branches :
master (production-ready)
dev
ui
but no idea how a designer working on ui branch can keep the code elsewhere than /views updated an a working app...
Thanks folks for help !
With git, there's no reason for developers to work on a separate branch or to have mocked up views. Have designers and developers work in the same branch, on the same codebase. When a view is done (or at least improved and not crashing) have the designer commit and push them to a master repository. The same is true for developers: when a local change is 'done', have them commit it and push it.
Before pushing, each side needs to pull (to ensure there are no conflicts). If the two groups are working in mutually-exclusive pieces of code (separate files or even separate parts of the same files), then the pull will simply update the local copy and all will work well.
With this, both sides are always seeing the most up-to-date codebase, and contributing directly towards the exact end goal, watching it evolve.
Git is so simple to use, theres no reason everyone should not have their own branch to work out of. Hell, thats one of the main reasons to use a version control system. In Git, commits are cheap.
Typically, we have a master, and anyone who is working on updates or features will branch from the master and branch a branch if need be, then a release master (someone like me) will take care of merging them all back down, checking for conflicts, testing the release and merging back to master.
As you work on it, others can receive your changes by doing a fetch/pull against your branch to pull in the changes.
Git isn't magic. It doesn't let your designers use code that the developers are actively writing. The developers still have to write, test and commit their code, and push it some place the developers can pull it from.
Typically you'll have a "bare" repository that all parties push their work to when it's ready to be shared. Everybody else pulls that work down. It might be the designers job to pull the developer's work and merge the dev branch into the ui branch, for example:
git checkout ui
git fetch
git merge dev
If you really want to enforce things like branch and path rights, I suggest you checkout gitolite
This will allow you to manage access at all sorts of levels.