I keep reading about how important nightly builds (and automated builds in general) are to the development process, and I would love to implement them on my projects. The problem is, I develop MVC web applications. I cannot imagine how I would be able to easily run an arbitrary build. I would need to deploy it to a server including any dll, view page, and DB changes necessary, and possibly seed the DB with appropriate test data. That does not sound automatic or even useful.
Is there a useful way to do nightly builds in a web application? The best solution would be one where I can access the application at an address like nightlybuild.myapplication.com/buildnumber and the server would figure out what DLL, view pages, and DB to use.
Related
I would like to know what the best practice is for taking a ruby on rails application in production and adding a feature to it or debugging a broken feature?
What I mean is, say you have a working application and you have lots of people using it. You want to add a new feature to this app. You clone your application to your local machine. Create a new feature (or w/e) branch.
Now what do you change/do so you don't destroy your database and so you are able to test and debug this application on your local machine?
Also, let's say this is an older rails application with an older ruby version.
I would also like to note that I am having trouble finding any information this and am willing to read books and lots of text to learn if it is a very involved task.
Although the complexity of this type of operation varies quite a bit, usually based on the complexity of the application itself, I think a few generalizations can be made.
Tests
Obviously, do not break any existing tests. Write tests for you new functionality, even if they are the first tests in the application.
Data
Ideally, you will have data to work with that very closely mirrors your production data. In some cases (CMS) this may be an actual dump of the production database and assets, restored locally. In other cases (billing portal for a hospital), you will probably need to rely on well-constructed seed data. Once your automated tests pass, you can perform manual QA against the (possibly simulated) production data.
Staging
If you do not have a staging environment that 100% mirrors your production environment, set one up now. This should be set up as close as you possibly can to your production environment, using the database guidelines from above. Merge your feature branch into staging prior to merging it into production. This will allow you to do a final QA test in a near-production environment. This can be used to test not only new application features, but new server versions, ruby version, etc.
CI/CD
It is becoming very common to use CI/CD to automate the testing and deployment of feature branches. This can help enforce code quality guidelines. It can also allow you to run the tests in an environment that matches production, for extra peace of mind.
Backups
Obviously, even with all of this, things can still go wrong. Keeping up-to-date backups is vital, for worst case scenarios.
I am not a developer, but reading about CI/CD at the moment. Now I am wondering about good practices for automated code deployment. I read a lot about the deployment of code to a pre-existing environment so far.
My question now is whether it is also good-practice to use e.g. a Jenkins workflow to deploy an environment from scratch when a new build is created. For example for testing of a newly created build, deleting the environment again after testing.
I know that there are various plugins to interact with AWS, Azure etc. that could be used to develop a job for deployment of a virtual machine.
There are also plugins to trigger Puppet to deploy infra (as code) and there are plugins to invoke an infrastructure orchestration.
So everything is available to be able to deploy the infrastructure and middleware before deploying code (with some extra effort of course).
Is this something that is used in real life? How is it done?
The background of my question is my interest in full automation of development with as few clicks as possible, and cost saving in a pay-per-use model by not having idle machines.
My question now is whether it is also good-practice to use e.g. a Jenkins workflow to deploy an environment from scratch when a new build is created
Yes it is good practice to deploy an environment from scratch. Like you say, Jenkins and Jenkins pipelines can certainly help with kicking off and orchestrating that process depending on your specific requirements. Deploying a full environment from scratch is one of the hardest things to automate, and if that is automated, it implies that a lot of other things are also automated, such as infrastructure, application deployments, application configuration, and so on.
Is this something that is used in real life?
Yes, definitely. A lot of shops do this. The simpler your environments, the easier it is, and therefore, a startup with one backend app would have relatively little trouble achieving this valhalla state. But even the creation of the most complex environments--with hundreds of interdependent applications--can be fully automated; it just takes more time and effort.
The background of my question is my interest in full automation of development with as less clicks as possible and cost saving in a pay-per-use model by not having idling machines.
Yes, definitely. The "spin up and destroy" strategy benefits all hosting models (since, after full automation, no one ever has to wait for someone to manually provision an environment), but those using public clouds see even larger benefits in terms of cost (vs always leaving AWS environments running, for example).
I appreciate your thoughts.
Not a problem. I will advise that this question doesn't fit stackoverflow's question and answer sweet spot super well, since it is quite general. In the future, I would recommend chatting with your developers, finding folks who are excited about this sort of thing, and formulating more specific questions when you all get stuck in the weeds on something. Welcome to stackoverflow!
All is being used in various combinations; the objective is to deliver continuous value to end user. My two cents:
Build & Release
It depends on what you are using. I personally recommend to use what is available with the tool. For example, VSTS (Visual Studio Team Services) offers complete CI/CD pipeline. But if you have a unique need which can only be served by Jenkins then you must use that and VSTS offers that out of the box.
IAC (Infrastructure as code)
In addition to Puppet etc. You can take benefits of AZURE ARM (Azure Resource Manager) Template to Build and destroy an environment. Again, see what is available out of the box with the tool set you have.
Pay-per-use
What I have personally used is Azure Dev/Test Labs and have the code deployed to that via CI/CD pipeline. Later setup Shutdown policy on the VM so it will auto-start and auto-shutdown based on time provided. This is a great feature to let you save cost on the resources being used and replicate environments.
For example, UAT environment might not needed until QA is signed off. But using IAC you can quickly spin up the environment automatically and then have one-click deployment setup to deploy code to UAT.
I'm working asp.net web based application, I have deployed this application on server, Its getting response on port 80 from a outside client.
I want the to fix the bugs so I want to run this application in Debug mode so that I can attach the worker process with the application and this is making the Performance down and its disturbing the QA team.
So can I have two application one can run in release mode so that QA activity does not get disturbed and parallelly I can debug the build and fix the bugs or can do further development.
I'm facing the same problem during the development activity, If multiple developers are working paralley , only one is able to debug the application other one has to wait.
So please suggest me, If I can get rid of this situation.
I have only one server on which I can test this application.
This is a way too long discussion, but I will try to offer you a few ideas:
Each developer should develop on his own machine (sources and database should be local).
In order to sync your work you should use:
a. a source control solution like TFS or SVN (this is free) for your sources.
b. database changes can easily be synced by generating update scripts using SQL Schema Compare directly from Visual Studio (you will need SQL Server Data Tools for this), Redgate SQL Compare or another application that can compare the database strucure (there are many available online, some of them free).
You should have a separate server (DB and app) to testing.
You should have a separate server (DB anb app) for production.
You say you have one server to test the application. But I suppose each developer has his own computer, right? In this case you need to skip #3 and use the same server for testing and production, but with different databases and applications.
I suggest you check this website for similar answers (see Best practice for test and production environments for example) to find the best solution that applies in your case.
I have an MVC app that using EF6 Code First. I want to deploy this app to multiple datacenters. On deployments that have migrations, I can write a script to migrate them all as simultaneously as possible, but if one datacenter is slower, then the calls could all be rejected since the schema no longer matches. A script that tried to coordinate would also make rolling upgrades impossible.
Is there a way to make EF at least attempt to run the query even though the schemas don't match? Is there a different way I can/should approach this?
UPDATE:
Let's see if I can word this better. I want to have my MVC app in multiple datacenters. Let's assume that I deploy the app to each datacenter individually.
Option 1
Deploy to DC A
Code first migration runs on centralized DB
Requests made to DC A succeed, but requests to DC B fail
Option 2
Deploy to DC A
Do not automatically run migration
Requests made to DC A fail and requests to DC B continue to succeed
How do I develop a deployment strategy that will make it so that requests to either DC will work?
BTW: I am using Azure Web Sites, if a platform-specific solution is needed.
In your post, it seemed like you were concerned with how it would behave during the actual upgrade. Nothing about testing. But in comments you are asking about doing a partial deployment then doing testing. So on one hand you'd want to deploy as quickly as possible to minimize downtime. On the other hand, it sounds like you want to deploy to one site, test, and have the other sites continue to function while you are verifying the first deployment?
Verifying a deployment is reasonable, but fairly complex. I'm not sure you will find much in the way of automation for this. I think you should test prior to production deployment thoroughly, and then simply deploy as quickly as possible in production. If there were an issue you found only when deploying to production, you'd be in a bad situation, because now your site is down until you can fix it. Even if you could get the other instance to work with the new database, that is risky as it is going to be modifying things against a schema it doesn't completely understand. Additionally, if you do need to rollback the DDL then you will almost certainly lose any data that was modified since the deployment. So it is really best that all instances for the old schema fail until they are upgraded, to prevent them from modifying data that is at risk of being lost.
Usually you should have done a deployment to a staging environment that is as close to your production as possible to test the database migration process. This is called pre-production testing, and sometimes involves restoring the most recent backup from production into staging to ensure new constraints/structures are valid for existing data. By deploying to this staging environment, you should have a very high level of confidence that production deployment will go successfully.
You additionally safe guard yourself against production deployment issues by taking backups prior to deployment so that you can rollback as necesary(although this is worst case scenario as it might mean throwing out important data that came in between backup/deployment and realization that there is an issue). I imagine EF migrations uses a transaction to run the DDL scripts so they should rollback all-or-nothing if there is an issue.
I have started at a new site that is using .Net applications for the first time. As a developer I am used to VSS but this product is dying a death so we are using TFS (BASIC) instead.
I have been using TFS for source control up until now. But now we are having new servers installed for a live environment.
Now I am not sure what I should be doing. There are no books on TFS 2010 that I can find and I am wondering what tips you can give me. Does TFS need to be installed again, or should I use the existing installation? I am thinking I ought to set up a daily build for a test server. I have not been using TDD up until now, but for the next project this may change.
What must I absolutely get right, and what pitfuls should I avoid?
Without being there in your environment, it's hard to make appropriate recommendations. I've made some assumptions about what your installation based on what you said, but these may be wildly wrong.
You say you're using TFS (BASIC)-- I'm not sure what you mean by that, but if you are using TFS installed on one of the developers workstations, and you're starting to move towards a more robust development environment, I would recommend that you get a separate server (or servers) for your TFS installation.
It sounds like you're relatively small, so having your application tier and your data tier on the same machine shouldn't be that much of an issue. Just make sure that you have enough RAM on the machine to support both processes, and that you have enough disk space allocated for the growth of the database.
You talk about Test Driven Development (TDD), but what I think you're actually talking about is Continuous Integration (CI). When you have a CI environment set up, builds happen automatically based on either a schedule, or triggered by check-ins. Having this set up is never a bad idea, and would recommend that you get into the rhythm of CI builds as soon as possible.
If you're looking for a build server, you are probably going to be ok hosting the build agent on the combined application/data tier. If you find that you're getting performance hits when you do builds, you can move your builds to a different server without much effort.
You will also want to look at migrating your source code repository from your current environment to your future environment. The TFS installation wizard might be able to help you with that. If not, there are other options available, such as moving the database files to the new machine, or using the codeplex-based TFS Integration Platform.