I have a test suite with tests ordered as:
Suite1---A
B
C
How can I change the order of the test cases?
I have tried refactoring but that is not helping.
It doesn't look this is possible...
Remember, that the order tests run is alphabetical. So in the above example, TestThree will not run because TestTwo actually comes after TestThree
--FitNesse > UserGuide > TestSuites > TagsAndFilters
...but it shouldn't matter. Tests should be independent and not rely upon previous tests (other than SuiteSetUp).
Related
I use ginkgo to write some tests, including:
books_suite_test.go
install_test.go
reading_test.go
isbn_test.go
uninstall_test.go
how to run these tests in a specific order as following:
install_test.go -> reading_test.go -> isbn_test.go -> uninstall_test.go
I found ginkgo can not ensure the test order. Is there any way to do this?
User the Ordered keyword in your Describe-block to run the block as a "Ordered Container".
https://onsi.github.io/ginkgo/#ordered-containers
You should not build tests that rely on tests of other files. Because the dirty answer to your question is that it's possible to put Containers and It-blocks into functions and call them in the order you desire
Ordered Container
In Mocha we can use --grep flag for selecting specific test for running:
mocha --grep 'my test'
But when numbers of tests raises, come specific test cases can share the same name, what makes single grep insufficient. I'd love to have something like "nested grep" to be able to select tests more specifically, regarding its parents. Can it be possible? Or maybe there are some other options to select a test for running more specific way?
Okay, fortunately I discovered that we can do something like this:
mocha /path/to/specific/file -g 'pattern for specific test case'
It solves my problem as long as case names don't repeat in the scope of one file (but they generally shouldn't).
I am trying to get my geb-spock functional tests to run in a specified order because SpecA will create data required for SpecB during its run.
This question is about running the specifications in order, not the individual test methods within the specification.
I have tried changing the specification name to indicate execution order but that didn't work. I found a solution where a Test Suite was used, and the tests were added to the suite in order, but I can't find how to make a test suite work in Grails.
Explicitly specifying them as grails test-app functional: SpecA SpecB , is not a long term option, as more specs will be added.
For sequential or whatever the sequence you want to run your tasks, I do the following thing in my build.gradle file:
def modules = ["X", "Y", "Z", "ZZ"]
if (modules.size() > 1) {
for(j in 1 .. modules.size()-1 ) {
tasks[modules[j]].mustRunAfter modules[values[j-1]]
}
}
Hope that helps. Cheers!
Not really an answer to your question but a general advice - don't do this. Introducing data setup dependencies between test classes will make your suite brittle in the long run. Reasoning about what the state is at a given point will get harder and harder as the amount of tests grows and the global state size with it. Later on hanging a test or introducing a new one might break many tests downstream. This is just asking for trouble.
Ideally, you want to setup the data needed by a test immediately before that test and tear it down afterwards. Grails Remote Control plugin and test data fixture builders are your friends here.
You should define your initialization code in a single place, and if it's shared between both Specs, it may be a good idea to create a superclass with methods you can call in each Spec's set up methods, or a whole class devoted to declare testing methods to reuse.
In any case, the purpose of a unit test is only to test a single functionality, and it shouldn't be responsible of setting up other tests as well.
Since updating my gems, one of my cucumber tests appears to be failing i.e. in the one-line printout of test outcomes it prints a red F:
However the test summary shows that all the tests passed which seems to contradict the red F.
What's a good way to try and unwrap what's going on here and to get some output from that apparently failing test?
Run your tests with cucumber --format=pretty ... and check out the full output
Is there a way to basically mark a TestUnit test as a "todo"? I thought I could do it by using pending at the top of the test, but that still runs the test (and thus it's red). I'd like to basically do the same thing, but not actually run the test (just skip over it).
Does anybody know of a way to do this?
Thanks!
if these are test case stubs, you could just put assert true as the only assertion and a puts in the test to describe what needs to be done. this of course wouldn't help if your tests have failing assertions unless you comment them out.