We are working with TFS + MTM to run some automation.
We have succesfully created Test Plan, Test Suite and Automated Test Cases. If we run them from MTM everything is fine but there is one issue regarding the drop location.
Right now, MTM picks the assemblies from \TFS\MyBuildDefinition\bin automatically while in our drop location we have a different structure so sometimes MTM can't find the test assembly.
If I configure an automated test there isn't much to play around with so I was wondering how can I specify a custom location for the assemblies that need to be tested.
You can specify additional files that are located in specific folders and should be deployed before running automated tests using Test Settings:
Related
I have 2 executables, an app and an automation test app who will perform actions on the app. My automation tests are basically NUNIT tests who calls Chrome Web Driver.
Everything is hosted at TFS. In my build definition, i run sanity checks for every PR. I wanted to expand that. The automation tests are divided into many different categories (being Sanity a category). I've been seeing some stuff related to TFS Test Suits, and my idea was to make in a way whenever when someone makes a PR, he could choose some test category to run on that PR using that build. So in a easyer way of sayng, if my PR changes how 'blue buttons work' ill run the 'blue buttons test suit' on my pr.
Would using Test Suits be the best solution for this ? Has any1 done this or have any nice information on how to achieve this ?
Thanks for any responses !! Best regards !
You could be able to use the Visual Studio Test Agent Deploy and Run Functional tests steps in your build definition to run auto tests on build agents.
Associated test methods with test cases in Visual Studio.
Create a build definition to build your project and add the 2 steps I mentioned above. In the Run Functional Test step, select the test suites which contains those test cases in step1.
More details please refer this blog: Executing Automated tests in Build vNext using Test Plan, Test Suites
Using the default build template on TFS 2013, we use the Post-test script to collect trx files from protractor tests, in order to include those in the build report summary.
We use vs2015 MsTest:
MSTest.exe
/publishresultsfile:...\TRXTestResult\CHROME-54.0.2840.71-results.trx
/publish:https://.../tfs/defaultcollection
/teamproject:...
/publishbuild:...
/platform:AnyCpu
/flavor:Release
Result:
Waiting to publish...
Publishing results of test run (test name) to
https://.../tfs/defaultcollection ...
Publish completed successfully.
However, the published test result wont show up in the build report (only the default from unit test).
This can't be achieved by a simple test script. You need to create a custom activity to handle this.
Inside this activity you can handle everything that is necessary to run the protractor tests and publish the test results. With the custom activity you are also able to integrate the automated protractor tests very easy into different build templates. This improves the reusability in a great way.
Then you need to create the custom build template and integration into a TFS Build. More details please refer the tutorial from this blog: Automated UI testing of AngularJS applications using TFS
How to integrate specflow unit test with MTM test cases.
1) I have created the automated unit tests in specflow-C#-BDD style.
2) I already have manual test cases in MTM
3) I am connected to TFS and my project is checkedin.
I need to make sure the unit test methods (automation) are run against the manual test cases in the mtm from my local machine. I have referred various articles over the internet but unable to reach to a solution. It was found that i need to create test agents , controllers and environment which i am unable to do.
Please guide me from here.
You can integrate any MS Test Unit Test with a test case.
If you are using Specflow you need to make sure that you implement using MS Test. This is fairly strait forwards and allows you to execute the automation in a test environment. In visual studio if you open your solution that contains the tests you can also open the test case and associate on the automation tab.
You then add your environment within which you want to run the tests to Lab Manager as a Standard Environment.
http://nakedalm.com/standard-environments-for-automated-deployment-and-testing/
I have a problem of comprehension.
Unit tests are coded by developers in order to test a class (Java).
Integration tests are aimed to know if the different classes work together.
My problem is:
Based on continuous integration: I have Subversion (SVN) linked to Jenkins, and Sonar linked to Jenkins.
How are the integration tests created? Who does them? Are these tests already available in Sonar, or developers have to code them? Sonar launches integration tests thanks to Jenkins? How does it work...?
Integration tests are also coded by developers, to test multiple classes at one time, conceptually a "module", whatever that means in your world.
In my world, unit tests are tests that exercise one class, and have no dependencies externally. We allow file system access for mock data and logging, but that's all.
If a test exercises an actual database, or a running executable somewhere (e.g. web service) it is an integration test. We write them with junit, same as a unit test.
We find it works best for us to have separate Jenkins jobs linked in a pipeline to build, execute unit tests, execute integration tests, and load Sonar. While SonarQube is able to run tests for you, we prefer the separation which allows us to manually execute either set of tests via Jenkins without updating Sonar at the same time.
We have a set of user acceptance tests (written in MSTest) which run fine locally, they're referring to some test files which the acceptance tests load in and process.
However, when running on team build (TFS 2010) the path to the files we're using is obviously different and the file cannot be found. This is still the case with using relative paths as the depth at which the local tests happen is different to the depth at which they happen on the build server.
What's the best way to refer to a file in tests so that it works both locally and during team build test runs.
I've considered putting them in the team project but I don't even know what the path to that will be.
Any ideas much appreciated!
Try the [DeploymentItemAttribute][1]