When trying to run the unit tests as a build step in TFS Build 2015 (vNext), I get the following warning:
Warning: Unable to create DiaSession for <assembly>
No source location data will be available for this assembly.
The test assembly is then run, but the unit tests inside fail because they can't locate the references, I'm assuming because of the 'No source location data will be available' part of the warning.
The NUnit NuGet package I am using for the unit tests is: "NUnitTestAdapter.WithFramework" v2.0.0. The project is made in C#.
I've seen this question, but I don't really understand what they are saying, and it doesn't look like they are using TFS Build 2015 (vNext) build definitions.
Any ideas?
The question you referenced is talking about the old TFS build XAML not vNext build the one you are using.
However you could also use the same MSBuild arguments in vNext Visual Stduio Build or MSbuild task.
You need to use add the /p:NoWarn=warningNumber in MSBuild Arguments
in the build definition. Or use the /p:WarningLevel=0 argument to
suppress all warnings.
According to your error info, seems the issue should not related to TFS build side. Since unit tests inside fail because they can't locate the references, please try to directly run your tests on the build agent locally(remote to). This will help you to narrow down the issue. Besides, also Enable Verbose Debug Mode for TFS Build vNext by adding system.debug=true to see if there are more details log info for troubleshooting.
Related
I have a TFS 2017 Update 2 on-prem server with VS 2017 Enterprise installed. Our Build contains a Unit-Testing project which contains a Fakes assembly and works locally on the dev machines.
When build on TFS, the build fails with
The type or namespace name 'Fakes' does not exist in the namespace
'Our.Product' (are you missing an assembly reference?)
(i.e. misses/fails to generate the Our.Product.Assembly.Fakes.dll file)
I have additionally set up Fakes as suggested here.
What baffles me is that there are no mentions of Fakes in the build log (except for the error itself and the paths where MSBuild looks for the file), even with debug verbosity. He doesn't event try to generate the Fakes assembly.
Are there other steps I need to take? We use the vNext build definitions.
Update:
This is the build step that works now, before the /tv:15.0 /p:VisualStudioVersion=15.0 were not there.
Update
Use the latest version for msbuild arguments such as /tv:15.0 /p:VisualStudioVersion=15.0 could solve this issue.
First try to use build service account RDP to remote build agent and manually run the build to see if the assembly generated.
This will narrow down if it's related to your environment on the build agent. Make sure it's the same as your locally dev machines.
If the assembly is also generated, then the issue may related to your build definition settings. Check if you are using the right msbuild version. For vNext build, also enable verbose Debug Mode with system.debug=true
I am using the TFS 2013\VS 2013 professional editions for continuous build. Am looking to use an open source tool like OpenCover for code coverage. I have no prior experience in code coverage tools. I installed the OpenCover UI from Nuget but not sure how to include Codecoverage in integrated build in TFS. Getting "No Code Coverage Results" when the build runs.
I enabled code coverage under Automated tests node in build definition.
Any suggestion on how to implement code coverage in TFS\VS 2013 Profession edition would be greatly appreciated.
If you mean Enabled Code Coverage in the build definition just like below screeshot.
This is VS build-in code coverage, according to the Compare VS offerings site, only Visual Studio Enterprise has Code Coverage feature, so if you use TFS for your CI build and want to use the built-in code coverage feature, installing VS Enterprise on your build agent machine is required.
If you want to integrate OpenCover with TFS XAML build, you need to customize your build process template, add a custom activity in the build definition to trigger the opencover command for generating the report.
Created a RunCoverage.cmd at the root of my source folder
Installed / copied in TFS Servers the Opencover binaries and added into Path(must restart TFS Services to enable TFS to see it).
Created a new actitivity in the build definition, that is, editing the build template (before sonar execution activity) to run the cmd:
Running command line statements from TFS custom activity
http://msdn.microsoft.com/en-us/library/gg265783.aspx
There is the cmd contents:
#REM #echo suppresses command line. ^ is line continuation character
#ECHO Executing OpenCover...
OpenCover.Console.exe -register:user ^
-target:"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" ^
-targetargs:"/testcontainer:%~dp0test.dll /usestderr /resultsfile:%~dp0MSTestsResults.trx" ^
-output:%~dp0opencovertests.xml
More details please refer Naim Raja Díaz's answer in this similar question:Integration of OpenCover with TFS
We have a CI system using TFS 2015, and all was working well.
However, we added some new tests, and the "Gather Artifacts" stage stopped working.
I tracked this down to the fact that the MSTest command-line gets built thus:
MsTest.exe /testcontainer:blah /test:test1 /test:test2 [...] /resultsfile:"c:\blah.trx"
So when we go above a certain number of tests, the command-line ends up too long and becomes truncated, losing the /resultsfile switch and possibly some of the /test: switches also.
In this instance, TFS reports the suite as having passed, despite the fact that all tests did not run.
I see in the documentation for MSTest.exe that there is a /testlist: parameter that lets you specify which tests run by pointing at a file.
Can anyone help me work out how I switch over to using a testlist, or some other way of solving this issue?
I found the solution, it was to switch from using "Test Plan" to "Test Assembly"
Just as DaveShaw said in the comment, the limitation is not of MSTest or TFS, it's related to Windows Command Line AFAIK.
We can use both VSTEST and MSTEST to run automated unit and coded UI tests from a command line.
VSTest.Console.exe is optimized for performance and is used in place
of MSTest.exe in Visual Studio.
After finish running the test. It won’t save the results and there is no .trx file generated in the target directory.
MSTest.exe You can use the MSTest.exe program to run automated tests
in a test assembly from a command line.
MSTest is used for load tests and for compatibility with Visual Studio 2010 test projects.
MSTest can also be used to view the test results from these test runs, save the results to disk, and save your results to Team Foundation Server.
More detail info you could refer this link: How to choose between tcm.exe, mstest.exe and vstest.console.exe
Since you are using TFS2015, and if you don't have any VS2010 test projects,
we encourage you to use VStest instead of MStest.
I've been scouring various Nunit articles for the past several days trying to figure out the issue with why my Nunit tests are not running.
I am using TFS 2015 Build vNext engine and the Visual Studio Test step to execute my Nunit test cases. The VS 2015 solution used for generating the Nunit test dlls contains the several .net projects and 2 Nunit test projects. Nunit3.0 and Nunit3.0 Test Nuget packages are being used to create and execute the tests. Here is the nuget configuration and nunit.framework reference in my Visual Studio 2015 solution. The test cases run fine in the Visual Studio IDE and compile without error when the solution is build in vNext.
When the VS Test step executes in Build vNext, I get the following warning:
Warning: Dependent Assembly nunit.framework of E:\TFSBuild_Directory\14\s\EBS\B_WebEnable\ABC.Data\ABC.Data.Core.Nunit.Tests\obj\UnitTesting\ABC.Data.Core.Nunit.Tests.dll not found. Can be ignored if not a NUnit project.
The VS Test step is configured:
VS Test Step Configuration
I have confirmed on the build server that the nuget packages are getting restored to the build server and the Nunit.framework.dll appear to be in the expected file path as compared to my local path.
Any ideas as to what could be causing this issue? Any input is appreciated!
I have restored Nunit and Nunit3TestAdapter from Nuget to have a test on my side, but couldn't reproduce your issue. I'd like to share my steps here for your reference:
Restore Nunit and Nunit3TestAdapter from Nuget in VS for the test project (as your first screenshot), and check in the packages.config file.
In "Visual Studio Test" task, specify $(Build.SourcesDirectory)\packages in "Path to Custom Test Adaptors". As if there is a packages folder in the sources directory, it is automatically searched for testadapters (It seems you have a long path, you may need to check it).
I use the default VS template:
No error or warning occurred during this process. You can compare your steps with mine and correct yours. Hope this helps you.
Ok - after stepping away from the issue and coming back to it with fresh eyes, I found the issue causing the nunit.framework.dll issue.
After going step by step through the build definition, I forgot that I had an MSBuild parameter in the build step that was copying the compiled output to another location /p:OutDir=$(build.stagingDirectory)\ABC. Once I changed the test step to point to the new location, everything worked fine. Thanks everyone for the feedback!
I am new to TFS and want to integrate OpenCover with TFS. If any has done this please help!
This question is rather old but maybe you are still interested. With current Version of TFS (2015 Update 2) this is now possible as a "vsts Extension".
See here for details: https://github.com/RobertK66/vsts-opencover
Since the answer doesn't specify the version of TFS, here is an answer for 2015/2017.
OpenCover can be run from TFS using the Powershell build step. You need to get the contents of the OpenCover NuGet package onto TFS and run OpenCover.console.exe from there.
Since TFS doesn't support the format produced by OpenCover, you need to take one additional step and convert the results to Cobertura format. It's possible using the OpenCoverToCoberturaConverter NuGet package.
I've described the whole process in much more detail on my blog:
http://codewithstyle.info/setting-up-coverage-reports-on-vsts-with-opencover/
OpenCover is just a console application so you can just modify your scripts to get OpenCover to run your unit tests.
I haven't used TFS for several years and it has changed since then however this blog post should help
To incorporate coverage measurement of OpenCover the build process of TFS (second half)
The original is in Japanese but if you are familiar with TFS then the screens will probably be obvious.
OpenCover also comes with an MSBuild task that may help you with your integration.
I've just integrated opencover with TFS Build, to generate a xml with the results that will be processed by sonar:
Created a RunCoverage.cmd at the root of my source folder
Installed / copied in TFS Servers the Opencover binaries and added into Path(must restart TFS Services to enable TFS to see it).
Created a new actitivity in the build definition, that is, editing the build template (before sonar execution activity) to run the cmd:
Running command line statements from TFS custom activity
http://msdn.microsoft.com/en-us/library/gg265783.aspx
There is the cmd contents:
#REM #echo suppresses command line. ^ is line continuation character
#ECHO Executing OpenCover...
OpenCover.Console.exe -register:user ^
-target:"C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe" ^
-targetargs:"/testcontainer:%~dp0test.dll /usestderr /resultsfile:%~dp0MSTestsResults.trx" ^
-output:%~dp0opencovertests.xml
But i'm facing three issues (that are related with my concrete implementation, but you may face them):
The Tests are runned twice (one for template itself, and other for OpenCover)
The MsTest.exe in the TFS Servers is not in the same path, so when the Controller asigns an agent (if the match is done by tags) then if the agent executing the build is in a TFS Server that does not have the MSTest in the right path it will fail.
How to inject in cmd the corresponding test runner (MsTest, XUnit, Nunit, etc...) depending on the test project
Hope that it helps! (and someone can help me ;-)