I have a problem of comprehension.
Unit tests are coded by developers in order to test a class (Java).
Integration tests are aimed to know if the different classes work together.
My problem is:
Based on continuous integration: I have Subversion (SVN) linked to Jenkins, and Sonar linked to Jenkins.
How are the integration tests created? Who does them? Are these tests already available in Sonar, or developers have to code them? Sonar launches integration tests thanks to Jenkins? How does it work...?
Integration tests are also coded by developers, to test multiple classes at one time, conceptually a "module", whatever that means in your world.
In my world, unit tests are tests that exercise one class, and have no dependencies externally. We allow file system access for mock data and logging, but that's all.
If a test exercises an actual database, or a running executable somewhere (e.g. web service) it is an integration test. We write them with junit, same as a unit test.
We find it works best for us to have separate Jenkins jobs linked in a pipeline to build, execute unit tests, execute integration tests, and load Sonar. While SonarQube is able to run tests for you, we prefer the separation which allows us to manually execute either set of tests via Jenkins without updating Sonar at the same time.
Related
In our (grails 2.x) projects, we often have plugins, that come with their own unit tests, but are accompanied by testapps, which include the plugin inline, and run functional and integration tests.
To run the tests, we first run the unit tests on the plugin project, and then run the integration and functional tests on the test application.
I would like to start using code coverage metrics, but I need my data to be representative of all the tests that are running, not just unit tests that come embedded in the plugin source tree.
I have not yet chosen a particular code coverage solution, so this is a free variable that could help solve the issue.
Any help or insight appreciated
I have 2 executables, an app and an automation test app who will perform actions on the app. My automation tests are basically NUNIT tests who calls Chrome Web Driver.
Everything is hosted at TFS. In my build definition, i run sanity checks for every PR. I wanted to expand that. The automation tests are divided into many different categories (being Sanity a category). I've been seeing some stuff related to TFS Test Suits, and my idea was to make in a way whenever when someone makes a PR, he could choose some test category to run on that PR using that build. So in a easyer way of sayng, if my PR changes how 'blue buttons work' ill run the 'blue buttons test suit' on my pr.
Would using Test Suits be the best solution for this ? Has any1 done this or have any nice information on how to achieve this ?
Thanks for any responses !! Best regards !
You could be able to use the Visual Studio Test Agent Deploy and Run Functional tests steps in your build definition to run auto tests on build agents.
Associated test methods with test cases in Visual Studio.
Create a build definition to build your project and add the 2 steps I mentioned above. In the Run Functional Test step, select the test suites which contains those test cases in step1.
More details please refer this blog: Executing Automated tests in Build vNext using Test Plan, Test Suites
I am currently working on a distributed system consisting of two Grails apps (3.x), let's call them A and B, where A depends on B. For my functional tests I am wondering: How can I automatically start B when I am running the test suite of A? I am thinking about something like JUnit rules, but I could not find any docs on how to programmatically start/manage Grails apps.
As a side note, for nice and clean IDE integration I do not want to launch B as part of my build test phase, but as part of my test suite setup.
A couple of months later and more deeply in the topic of Microservices I would suggest not to consider system tests as candidates for one single project - while I would still keep my unit- and service-level tests (i.e. API testing with mocked collaborators) in one project with the affected service, I would probably spin up a system landscape via gradle and docker and then run an end-to-end test suite in the form of UI tests.
How to integrate specflow unit test with MTM test cases.
1) I have created the automated unit tests in specflow-C#-BDD style.
2) I already have manual test cases in MTM
3) I am connected to TFS and my project is checkedin.
I need to make sure the unit test methods (automation) are run against the manual test cases in the mtm from my local machine. I have referred various articles over the internet but unable to reach to a solution. It was found that i need to create test agents , controllers and environment which i am unable to do.
Please guide me from here.
You can integrate any MS Test Unit Test with a test case.
If you are using Specflow you need to make sure that you implement using MS Test. This is fairly strait forwards and allows you to execute the automation in a test environment. In visual studio if you open your solution that contains the tests you can also open the test case and associate on the automation tab.
You then add your environment within which you want to run the tests to Lab Manager as a Standard Environment.
http://nakedalm.com/standard-environments-for-automated-deployment-and-testing/
When running unit tests, Gradle can execute multiple tests in parallel without any changes to the tests themselves (i.e. special annotations, test runners, etc.). I'd like to achieve the same thing with ant, but I'm not sure how.
I've seen this question but none of the answers really appeal to me. They either involve hacks with ant-contrib, special runners set up with the #RunWith annotation, some other special annotations, etc. I'm also aware of TestNG, but I can't make the Eclipse plug-in migrate our tests - and we have around 10,000 of them so I'm not doing it by hand!
Gradle doesn't need any of this stuff, so how do I do it in ant? I guess Gradle uses a special runner, but if so, it's set up as part of the JUnit setup, and not mentioned on every single test. If that's the case, then that's fine. I just don't really want to go and modify c. 10,000 unit tests!
Gradle doesn't use a special JUnit runner in the strict sense of the word. It "simply" has a sophisticated test task that knows how to spin up multiple JVMs, run a subset of test classes in each of them (by invoking JUnit), and report back the results to the JVM that executes the build. There the results get aggregated to make it look like a single-JVM, single-threaded test execution. This even works for builds that define their own test listeners.
To get parallel test execution in Ant, you would need an Ant task that supports this feature (not sure if one exists). An alternative is to import your Ant build into Gradle (ant.importBuild "build.xml") and add a test task on the Gradle side.