I'm looking to get TFS to build and deploy to our manual and auto TEST environments.
manual TEST - overnight updates
auto TEST - continuous integration
Once deployed, I want TFS to deploy and run a set of web-integration tests (SpecFlow / Selenium / nUnit) on another server (acting pseudo client of the website).
I then want TFS to collate the results of these tests and report back.
What's the best approach to get this working?
You'd normally change the build process template to do what you want. Adding steps to deploy could be one of these customozations. Have a look at the Build Release Deploy build template from the Microsoft ALM Rangers which includes steps to publish the build to another machine.
You can also make use of WebDeploy and add a set of publish profiles to your solution. Team Build will happily execute those steps during build, if told to do so.
In TFS you can use a Test Agent to then execute tests from a remote machine and have the results added to the test results for your build.
Seeing that you'd like to use NUnit ad SpecFlow I suggest using TFS 2012 in combination with Visual Studio 2012, which have built-in support for executing 3rd party based tests. You'll need to install the NUnit adapter and put the assemblies for it in source control so that Team Build will pick them up.
Related
I want to be able to run a StyleCop analysis as a build step on our build definition, but am coming across some issues.
I can't use this because our TFS policies forbid external build steps.
I would prefer to not use this method either as it appears that it runs an analysis every time it builds (and we don't want this to happen on development machines).
Ideally I'd like to be able to run the analysis from batch or PowerShell as a build step, but that doesn't appear to be an option.
What methods are available that allow me to run a StyleCop analysis on an existing project (which already has StyleCop installed) as a build step in TFS Build, without it running each time the software is compiled on a development machine, and without adding too much bloat to the software?
You could check StyleCopCmdLine. StyleCopCmdLine is wrapper project to make it possible to easily call StyleCop from a command prompt or a PowerShell script.
For a discussion of its usage within a TFS 2015 vNext build see this blog post.
Our development team uses Visual studio 2013 Pro along with TFS 2010. We need the following abilities for our continous integration / delivery initiative:
1. Automatically build our web project on every checkin.
2. Automatically deploy green builds to development and/or test environments.
3. Provide build/deployment notifications via email.
Extra Credit: Run unit tests with each build.
We don't have a budget, the reason Jenkins is appealing. The setup is new to me and not sure how the pieces fit and what other tools are needed to preform the tasks above?
If you have MSDN subscriptions, you can use TFS for free. TFS 2015 can provide all these features. TFS 2010 can provide most of this as well.
Use a gated checkin or CI build definition in TFS (2010 and 2015) to automatically build on checkin and run tests. The build definition has the option to run tests as part of the build.
Use MSDeploy to package up your components into an automated package. Most people don't know you can use it for more than web applications. (For example, databases)
TFS also provides email alerts on many events.
Regarding automating the deployments, we used separate TFS (in 2010) build definitions to trigger deployments to each environment. Now we use TFS 2015 which supports release definitions that adds many more options.
Our plan is to use Team Foundation Build service to do automatic builds, then use the testing facility to automatically perform testing on the build server then release that build onto the application server.
So far we have
Team Foundation Server with TF Build Controller configured
Build server with win2012, Visual Studio 2013 and Build agent configured.
SQL Server with SQL 2013 installed
Application Server with Win2012 and .netframework installed
My question is what do I need to do to set up automatic builds, and to execute the unit test harness once compilation is successful.
Also the deployment target machine will initially be DEV, however we would like to quickly build for test env and prod etc.
This is what I got so far.
Build Controller (Already set up I believe)
Build Agent (Already installed on build server)
Build Process Template (Do I need to do anything with this. Is this what controls the whole lot)
Team Build Definition (I had a look at this, and it seems to use the build process template)
Drop Folder (I am assuming this is where the executables will be dropped into).
At the moment I have bits and pieces of info, what I would like to know is how this whole thing is hanging together. From the moment the developer wants to do the build to the moment that exe is placed into the DEVAPPSERV (Development application server).
Is anyone able to point me in the right direction or give a summary of what I need to make this happen?
Many thanks,
Dalibor
Install TFS Server (TFS Disk) Create a Team Project Collection and any desired Projects
Install TFS Controller + Agents onto a dedicated machine (TFS Disk) Configure only the build options if on a different machine to the TFS Server
Configure Build Controller to connect to a Specific Team Collection on your TFS Server
Install VS Premium or higher on build machine, if you want code coverage results for your tests
Add some code to TFS Source Control
Create a Build Definition using the default template.
Configure the build definition.
Set the working folder for the build, include only what you need as this will speed up the process
Point the definition to your .sln or proj file.
Ensure testing is enabled and that your test assembly names will match the regex used to identify test dll's i.e. name your test assemblies with the word test.
Set the trigger to be CI or what ever flavour of build you require i.e. gated build
Save the build definition
Trigger a manual build and debug any issues
you should have the basics done and a repeatable build created.
That should cover the basics, you may want to customise the build template (see Ewald Hoffman's guide for tips), you may want to narrow down your code coverage (look for runsettings file info).
If you follow these steps you should be able to get a basic build created and running from these, if you hit any issues you can come back and ask specific questions about a particular area
In order to do automatic builds you should check the CI build option ( under the trigger build option ) and third party automated testing can be run by executed by a post build script.
See the following TFS article about post build scripts.
http://msdn.microsoft.com/en-us/library/dn376353.aspx
Another few TFS questions if anyone can help even remotely :-)
We currently have TFS 2012 installed. We have 2 servers (both VMs running Windows Server 2008 R2 x64). One server is TFS with the application layer and database (running SQL Server 2008 R2) installed. The other server is our BUILD server with the build controller and build agent installed on it.
We can connect to TFS 2012 OK from our local developer PCs using Visual Studio 2012 Ultimate.
We then have various complex build definitions that compiles our code, runs unit tests, generates real-time sandcastle documentation and installs to our multi tiered, multi server TEST or LIVE environments using InstallShield (takes SSL and performs pre-install conditions etc)
We are using the UpgradeTemplate.xaml build process definition along with TFSBuild.proj file. FYI it was originally developed in TFS 2008 and has served us well hence why we don't WANT to upgrade to a modified defaulttemplate.xaml
We now want to include "post deployment tests" (aka smoking testing or system testing or end-to-end testing I believe) in the form of Coded UI tests.
We can record these tests locally, add asserts etc and check them into TFS etc but my problem is how to invoke them from TFS Build at the end of our build sequence without upgrading to a defaulttemplate.xaml? We use the true build property to invoke our unit tests but this is run prior to deployment. What other property(s) do i need to set? is this at all possible?
I have googled/MSDN'd an believie we may need a third server with a test controller and test agent on it (that isnt locked with a screen saver, build service account has permissions and the build service is configured to run as an internative process and not a service). Is this correct?
If we want to video record the tests as they are run have people found this useful? Does it function/configure as easy as articles seem to imply?
Can visual studio 2012 Ultimate do everything that Microsoft Test Manager 2012 does? and vice-versus? I haven't used this and wonder if this a testers tool only really?
1.
We can record these tests locally, add asserts etc and check them into TFS etc but my problem is how to invoke them from TFS Build at the end of our build sequence without upgrading to a defaulttemplate.xaml? We use the true build property to invoke our unit tests but this is run prior to deployment. What other property(s) do i need to set? is this at all possible?
If the coded ui test is used as a stand alone unittest in the unittest dll, mstest should pick up the coded ui tests. If the coded ui test is however an automation of a test case, an alternative is to use the TCM command line tooling to kick of the test case and thereby executing the coded ui test.
2.
I have googled/MSDN'd an believie we may need a third server with a test controller and test agent on it (that isnt locked with a screen saver, build service account has permissions and the build service is configured to run as an interactive process and not a service). Is this correct?
Yes: You would need an interactive build agent.
No: the current build agent could also be set to interactive.
3.
If we want to video record the tests as they are run have people found this useful? Does it function/configure as easy as articles seem to imply?
Yes: It works as the articles imply
No: The intellitrace collectors and logs are usually more useful than a video recording. The articles also forget to mention that your TFS server might grow out of proportions if not handled with care. http://geekswithblogs.net/terje/archive/2011/11/15/guide-to-reduce-tfs-database-growth-using-the-test-attachment.aspx
4.
Can visual studio 2012 Ultimate do everything that Microsoft Test Manager 2012 does? and vice-versus? I haven't used this and wonder if this a testers tool only really?
No, there not the same and don't try to pretend that (like blend). The focus of the tools are different; MTM is focused on testing by a tester (functional, manual, exploratory), testing in visual studio is focused on testing as a developer (unittest, webtesting, automating tests). Visual Studio ultimate therefore can't edit and create test cases and suites. MTM can't create develop (coded) unittest tests. Installing visual studio ultimate also installs MTM, so you can use both if needed.
Hi I am using TFS and want do deploy two web applications after build to a test server.
After the deployment I want to call a web page at one of the web applications to trigger some work like, delete all data and insert test data.
How can I accomplish this?
You could have a look at TFSDeployer. If you are comfortable with Powershell, you can write a deployment script that does whatever you want, and TFSDeployer will execute it when it detects changes to the build quality (i.e. you set build quality to "deploy to test" for example and it will run the script to deploy to the test environment). There's more information on the site, and this is what I use almost always for deployments.
You can MS Deploy to deploy your applications to the test environments: http://www.ewaldhofman.nl/post/2010/04/12/Auto-deployment-of-my-web-application-with-Team-Build-2010-to-add-Interactive-Testing.aspx.
To call a web application, you can either create a codedui that you run as part of your testing, or you add an activity to your build template that calls the web site: http://www.ewaldhofman.nl/?tag=/build+2010+customization
You can use DeployToAzure. It publishes specified cloud project to Azure as a part of your TFS build workflow. Publishing is executed after solution is built and unit tested are passed. You can include some steps (like call web page) after that.
And you don't need any Powershell programming for that.
Check here for additional information: Deploying Azure Project from TFS 2010 Build Server