I have a project with unit tests running on TFS with scheduled builds. I receive e-mail notifications like:
Standard email notification on build completion
I need to add a list of tests with test results and test error message to the notification mail. Is it possible to do it with TFS?
If not, how can i configure an extra step to the build definition with unit tests data like "unit test name: result" (completed/failed) + error message.
It would also be a good point to add an attachment with test results to the email.
Please provide me with information/instructions/examples.
The project with unit tests is coded using c# and MSTests.
The build alert template is located at:
c:\Program Files\Microsoft Team Foundation Server 14.0\Application
Tier\TFSJobAgent\Transforms\1033\BuildCompletedEvent.xsl
Find the template you could do customization in it.
Also, You could write a powershell script to get test results then send emails. Then run this script in TFS build.
To get test results using from TFS RESI API: How can I get the last test run id from TFS to my powershell script
Send email using powershell: https://practical365.com/exchange-server/powershell-how-to-send-email/
Here is also another blog with codes: http://chamindac.blogspot.sg/2015/10/send-test-result-email-after-running.html
Related
I have a TFS build definition setup to run the task "Visual Studio Test" using the Nunit3 Test Adapter. This runs successfully and publishes test results. However, neither the TestFixture name or the Test's Fully Qualified Name are available in the test reuslts. This makes it difficult to determine what exactly what test failed without using lengthy test names.
Is there a way to include the Test Fixture names in the published test results from the build?
Please refer to the Tests summary tab of your build. It shows the detail information of test results of your Nunit3 tests.
You could use the Filter to find out those Failed or Succeeded tests.
You could also click the link in the picture below to check detail data for this test run.
Good day.
I have a TFS build and a Visual Studio Test step there. All test are run, but I can't get detailed info about them, only quantity, passed\failed\others, run duration.
Also, I see "Outcome: Failed" and "This build doesn't have test results for the outcome you've selected".
How to configure this outcome?
You can't directly use the build-in outcome to analysis the detail test result.
You need to run the Publish Test Result task. First generate an XML file which contains the results. And then use the publish results step and pointing to that file so that the test results will show up in the build output.
More details about how to use this, please refer this similar tutorial with xUnit test: Execute and publish xUnit Tests results with .NET Core and VSTS
Note: For now VSTS/TFS does not support NUnit3 format. Source Link: Support NUnit2 format
Using the default build template on TFS 2013, we use the Post-test script to collect trx files from protractor tests, in order to include those in the build report summary.
We use vs2015 MsTest:
MSTest.exe
/publishresultsfile:...\TRXTestResult\CHROME-54.0.2840.71-results.trx
/publish:https://.../tfs/defaultcollection
/teamproject:...
/publishbuild:...
/platform:AnyCpu
/flavor:Release
Result:
Waiting to publish...
Publishing results of test run (test name) to
https://.../tfs/defaultcollection ...
Publish completed successfully.
However, the published test result wont show up in the build report (only the default from unit test).
This can't be achieved by a simple test script. You need to create a custom activity to handle this.
Inside this activity you can handle everything that is necessary to run the protractor tests and publish the test results. With the custom activity you are also able to integrate the automated protractor tests very easy into different build templates. This improves the reusability in a great way.
Then you need to create the custom build template and integration into a TFS Build. More details please refer the tutorial from this blog: Automated UI testing of AngularJS applications using TFS
At the moment we have our Jenkins Jobs configured so the Test Results get written to a specific file and then published. The problem with this is the next time the build runs it replaces the previous test results. This means the link the the email notification points to the same test results file and we are not able to debug why the tests failed.
Is there a way to create test results with a unique name like a time stamp and publish these?
You can use the archive function of Jenkins. Add the post build action "Archive artifacts" to your job and give the filename pattern of your test file. Now the file is archived for every job. The link in the mail notification may still point to the newest file, but you can download the archived files in the web interface of Jenkins (if you go to a specific build, you see all archived artifacts of that build).
We have a Visual Studio test controller with 3 registered test agents in a specific test environment setup for our nightly automation runs. I've seen ample documentation on having the build agents run the tests, but we need the test execution to go through the controller and run from the test agents instead.
My thought was to edit the build process template so it would trigger the execution of these remotely executed tests and then wait for the test run results, but I have no experience with build templates and I've been unable to find any examples showing how I might accomplish this. And this is of course assuming that editing the build process is the best/correct solution in the first place.
Can someone with experience with triggering remote execution of tests at the end of a build/deploy cycle point me in the right direction please?
Actually, you don'have to change anything to your template. Just make sure your build definition refers to the correct tests and testsettings file that are configured for remote execution.
Step 1:
Please open http://msdn.microsoft.com/en-us/library/ee256991.aspx and scroll down to the section "Add a test settings for remote execution or data collection to your solution". Follow this to create a test settings file for remote execution.
Step 2:
Edit your build definition: go to the Process page, under heading "2. Basic", open the Automated Tests dialog by clicking the "..."at the end. It the Automated Tests dialog, click "Add". Then browse for your test settings file (for remote execution, the one you just created) and confirm your choices.
Now save your build definition and queue your build. Automagically, your tests are now performed on the remote system, because your testsettings file tells your build system to do so.
Hope that is enough to start your remote tests to work.