I am using Spec Runner to run my test cases, the scenrios are getting called twice.
What might be the issue?
Please find the below scenario and test results attached
ScenarioTestResults
Are they failing scenarios? In the standard configuration SpecFlow+Runner retries failing tests.
To disable the retry of a scenario, you have to set the retryCount parameter in the execution element to 0. See http://www.specflow.org/plus/documentation/SpecFlowPlus-Runner-Profiles/#Execution
Full disclosure: I am one of the developers of the SpecFlow+Runner.
Related
Let me start by stating I am very wet behind the ear with gherkins and cucumber.
I've put together a PoC for my company of an integration a Jenkins projects that will build and execute tests when there is a check in a Git repository. When the tests have completed Jenkins will then update the test managed in Xray for Jira.
The tests are cucumber written using gherkins. I have in vain attempted to cause a single test to produce a failure just to be able to add that to the demo I am going to be giving to upper management.
Here is the contents of my file HelloWorld.feature:
Feature: First Hello World
#firsth #hello #XT-93
Scenario Outline: First Hello World
Given I have "<task>" task
And Step from "<scenario>" in "<file>" feature file
When I attempt to solve it
Then I surely succeed
Examples:
| task | scenario | file |
| first | First Hello | First Feature |
Currently all the tests I have pass. I have attempted to modify that test so that it would fail but thus far have only been able to get it to show in Xray as EXECUTING or TO DO.
I have searched to see if it was possible to create a test that would always result in a test failure but have not been able to find anything.
I know do not know gherkins, I'm only using what was given to me to work with, so please forgive my question.
Thank you for any guidance anyone might be able to provide.
Cucumber assumes a step passes if no exception is thrown. Causing a test to fail is easy. Just throw an exception in the step definition.
Most unit testing frameworks give you an explicit way to fail a test. You haven't mentioned the tech stack in use, but MS Test for .NET gives you Assert.Fail("reason for failure goes here.");
Or simply throw an explicit exception: throw new Exception("fail test on purpose");
As long as the step throws an exception the entire scenario should fail.
Hello i am new to jbehave and thucydides and the issue i am facing is all the steps are executed in the .java file but it only skips the #when step due to which my test gets skipped. I tried several options but it always marks when as pending when i run the test.
After executing tests case, check your console or report file for story/step error annotation.
Tests that contain no steps are considered to be pending. If one of steps (the "given-when-then" structure) gets PENDING while executing, then whole test gets labelled as SKIPPED. http://www.wakaleo.com/thucydides-one-page/thucydides.html#_defining_high_level_tests_in_junit - 6.2.1
From my experience, most pending steps ("given-when-then") are from bad spelling of a step name/title. Step from .story file and implementation file of a story (your_story.java depending on language) are different. Like "xx yy" =/= "xx yv"
I am trying to work on whats wrong with my lithium current setup. I have installed the Xdebug and verified that remote host can establish the connection as requested.
http://myinstance.com/test/lithium/tests/cases/analysis/logger/adapter/CacheTest?filters[]=lithium\test\filter\Coverage
Please note in fresh installation in local environment , "Coverage" Filter is working as expected.
I added some test code inside the "apply" function in coverage.php but it is not even called !!!! Can some have experience in debugging the above URL ?
I am not able to understand why coverage filter is not called up and executed ...Any hints are highly appreciated !
The filters in the query string are added to the options in lithium\test\Controller::__invoke() and then passed into the test Report object created by the test Dispatcher. The Report object finds the test filter class and then runs the applyFilter() method for that test filter as can be seen in lines 140 to 143 of the current code. So those lines would be another place to debug. Those should wrap the run() method of your tests with this filter code inside the apply() method that uses xdebug_get_code_coverage() and related functions. You said you added test code in the apply method and it isn't called. I'm not sure what the issue is. Are you sure you are pointing to the right server and code location? It is possible to run tests from the command line. Maybe you should try that. See code comments in lithium\console\command\Test or run li3 test --help for info on how to use the command-line test runner.
I can confirm on nginx I also have /test/lithium/tests/cases/analysis/logger/adapter/CacheTest?filters[]=lithium\x5Ctest\x5Cfilter\x5CCoverage in my access log. The \x5C is expected url encoding of the backslash character.
I have a test suite which has the init and end functions implemented in it.
When I run the suite it produces some html outputs to show the results of the test cases (pass and fail etc.) from the suite.
But in the log the init_per_suite and end_per_suite are also counted as test cases and their run result is shown in the log. Is there a way to avoid this? I guess there might be a flag in Erlang common test which can be used to disable this.
No, you can't disable it. Besides it may be important information if start_per_suite/end_per_suite succeeds or or fails.
Also you can see that start_per_suite/end_per_suite are not included in general numeration of testcases in resulting html. May be it'll help you if you want to parse the html output. Also you can sort cases by their numbers so the unnumered cases will be on the top/bottom.
From time to time I run into the issue that Grails integration tests the name of which ends in "IntegrationTests" don't work coming up with exceptions that show that GORM methods have not been added to domain classes. After renaming those tests to "*IntegrationTest" (no s at the end) they work fine.
A short example:
class MyIntegrationTests {
#Test
void myTest() {
assert MyDomainClass.count() == 0
}
}
Will fail with the following exception:
Failure: myTest(de.myproject.MyIntegrationTests)
groovy.lang.MissingMethodException: No signature of method: de.myproject.MyDomainClass.count() is applicable for argument types: () values: []
Possible solutions: count(), ident(), print(java.io.PrintWriter), print(java.lang.Object), getCount(), wait()
at de.myproject.MyIntegrationTests.myTest(MyIntegrationTests.groovy:9)
After renaming MyIntegrationTests to MyIntegrationTest the test passes.
Is there some kind of magic happening according to the test's name? All I found in Grails documentation is: "Tests can also use the suffix of Test instead of Tests. " Any ideas?
I eventually found the cause for the different behaviour of "*Test" and "*Tests" myself: Different postfixes change the order in which the tests are being run. To make things worse, the exact order is platform-dependent. Thus, my tests ran locally (OSX) in a different order than on my CI machine (Linux), and thereby produced different results.
Why the exception occurrs in some order is a totally different problem, though, which I haven't figured out (yet).
This should work how you had it originally as long as the file is in the integration folder. Are you sure you didn't have it in the unit test folder and then move it into the integration folder on the rename? Or possibly that you're using intellij and you did a "junit" test run rather than a "grails" one?
The error you're getting seems to imply that grails didn't start up when running your test.
Your test won't be executed if it does not has the suffix Tests.
Copied From Grails documentation home page (http://grails.org/doc/latest/guide/testing.html):
The default class name suffix is Tests but as of Grails 1.2.2, the
suffix of Test is also supported.
j-