All scenarios do what they are supposed to but, until now, I only had 1.
Now that I have a second scenario, the following problem arises.
If the scenario 1 fails an assertion, the subsequent scenarios are skipped.
See below the skipped scenarios.
Setup: ConfigFileTransformation (thread #0)
Scenario: Shows telephone numbers on the header in ShowContactInformation
(target: BrowserBrowserStack_Win8_Chrome_52) -> Succeeded on thread #0
Scenario: Shows telephone numbers on the contact sidebar box in
ShowContactInformation (target: BrowserBrowserStack_Win8_Chrome_52)
-> Failed on thread #0
[ERROR] 2 assertions failed.
Restore: ConfigFileTransformation (thread #0)
Scenario: Shows telephone numbers on the header in ShowContactInformation
(target: BrowserBrowserStack_Win8_Firefox_42)
-> Skipped on thread #-1
Scenario: Shows telephone numbers on the contact sidebar box in
ShowContactInformation
(target: BrowserBrowserStack_Win8_Firefox_42)
-> Skipped on thread #-1
Result: 1 failed
Total: 4
Succeeded: 1
Ignored: 0
Pending: 0
Skipped: 2 <--- **** Other scenarios must be executed regardless of anything else.
Failed: 1
How do I prevent specflow from skipping subsequent scenarios?
Updated the Execution element of the test profile to:
<Execution retryFor="None" stopAfterFailures="0" testThreadCount="1" testSchedulingMode="Sequential" />
The key thing seems to be stopAfterFailures="0".
Related
I'm trying to figure out how to verify sequenceEqual method in Mono.
My code is as follow:
Mono<Boolean> mono = Mono.sequenceEqual(Mono.just(1), Mono.just(1));
StepVerifier.create(mono)
.expectNext(true)
.expectComplete()
.verify();
And here is the stacktrace I got
- java.lang.Throwable: Multiple exceptions
- java.lang.AssertionError: expectation "defaultOnSubscribe" failed (expected: onSubscribe(); actual: onNext(true))
- java.lang.AssertionError: expectation "expectNext(true)" failed (expected: onNext(true); actual: onComplete())
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.validate(DefaultStepVerifierBuilder.java:1337)
at reactor.test.DefaultStepVerifierBuilder$DefaultVerifySubscriber.verify(DefaultStepVerifierBuilder.java:827)
at reactor.test.DefaultStepVerifierBuilder$DefaultStepVerifier.verify(DefaultStepVerifierBuilder.java:533)
at reactor.test.DefaultStepVerifierBuilder$DefaultStepVerifier.verify(DefaultStepVerifierBuilder.java:508)
This operator isn't well-behaved with regard to subscription signal propagation, and since the StepVerifier expects a well-behaved Publisher all the way, you get an error.
To work around this bug, you could regularly subscribe to sequenceEqual and have a custom subscriber capture the events in AtomicReference or similar then assert those...
Once running my test I faced this:
=ERROR REPORT==== 29-Jul-2016::01:34:22 ===
** Generic server <0.141.0> terminating
** Last message in was {send_req,
{{url,
***************,
*****************,443,
undefined,undefined,"/api",https,hostname},
[{"X-API-Version","0.5"},
{"Accept","*/*"}],
post,<<>>,
[{is_ssl,true},
{ssl_option,
[{certfile,
<<"/Users/slava/src/api/cert_dev.pem">>},
{password,"nil"}]}],
15000}}
** Reason for termination ==
** {{badmatch,false},
[{ibrowse_lib,get_value,2,
[{file,"/Users/slava/Documents/src/api/deps/ibrowse/src/ibrowse_lib.erl"},
{line,184}]},
{ibrowse_http_client,check_ssl_options,2,
{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,629}]},
{gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,661}]},
{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}
Then I ran the same test and it failed with small error message. Very uninformative.
Can't figure out what actually forced this verbose report.
Usual one:
1) test get vehicle data by vin
test/adapter_test.exs:14
match (=) failed
code: %HTTPotion.Response{} = Adapter.request(:get, 'vehicles', vin: #vin)
rhs: %HTTPotion.ErrorResponse{message: "closed"}
stacktrace:
test/remoto_adapter_test.exs:15: (test)
While coding with the STS IDE, many times this error message is been prompted. Whether it is related to tool tip for java code or compiler error?
#Kris request hereby I am pasting the complete log details of this error, in comment it is not possible to paste this big error message:
!ENTRY org.eclipse.core.jobs 4 2 2016-05-27 15:42:26.837
!MESSAGE An internal error occurred during: "Compute launch button tooltip".
!STACK 0
java.lang.NoSuchMethodError: org.eclipse.jdt.internal.compiler.env.NameEnvironmentAnswer.<init>([Lorg/eclipse/jdt/internal/compiler/env/ISourceType;Lorg/eclipse/jdt/internal/compiler/env/AccessRestriction;)V
at org.eclipse.ajdt.core.codeconversion.ITDAwareNameEnvironment.find(ITDAwareNameEnvironment.java:115)
at org.eclipse.jdt.internal.core.SearchableEnvironment.findType(SearchableEnvironment.java:305)
at org.eclipse.jdt.internal.core.CancelableNameEnvironment.findType(CancelableNameEnvironment.java:45)
at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:174)
at org.eclipse.jdt.internal.compiler.lookup.PackageBinding.getTypeOrPackage(PackageBinding.java:214)
at org.eclipse.jdt.internal.compiler.lookup.Scope.getTypeOrPackage(Scope.java:3228)
at org.eclipse.jdt.internal.compiler.lookup.Scope.getType(Scope.java:2940)
at org.eclipse.jdt.internal.compiler.ast.SingleTypeReference.getTypeBinding(SingleTypeReference.java:51)
at org.eclipse.jdt.internal.compiler.ast.TypeReference.internalResolveType(TypeReference.java:495)
at org.eclipse.jdt.internal.compiler.ast.TypeReference.resolveType(TypeReference.java:586)
at org.eclipse.jdt.internal.compiler.ast.TypeReference.resolveTypeArgument(TypeReference.java:598)
at org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveLeafType(ParameterizedSingleTypeReference.java:254)
at org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.internalResolveType(ParameterizedSingleTypeReference.java:165)
at org.eclipse.jdt.internal.compiler.ast.ParameterizedSingleTypeReference.resolveType(ParameterizedSingleTypeReference.java:378)
at org.eclipse.jdt.internal.compiler.ast.TypeReference.resolveType(TypeReference.java:582)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.resolveTypeFor(SourceTypeBinding.java:1759)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.fields(SourceTypeBinding.java:934)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.internalFaultInTypeForFieldsAndMethods(SourceTypeBinding.java:903)
at org.eclipse.jdt.internal.compiler.lookup.SourceTypeBinding.faultInTypesForFieldsAndMethods(SourceTypeBinding.java:899)
at org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope.faultInTypes(CompilationUnitScope.java:448)
at org.eclipse.jdt.internal.core.hierarchy.HierarchyResolver.resolve(HierarchyResolver.java:827)
at org.eclipse.jdt.internal.core.hierarchy.HierarchyResolver.resolve(HierarchyResolver.java:628)
at org.eclipse.jdt.internal.core.hierarchy.HierarchyBuilder.buildSupertypes(HierarchyBuilder.java:116)
at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.build(IndexBasedHierarchyBuilder.java:151)
at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.compute(TypeHierarchy.java:315)
at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.refresh(TypeHierarchy.java:1286)
at org.eclipse.jdt.internal.core.CreateTypeHierarchyOperation.executeOperation(CreateTypeHierarchyOperation.java:90)
at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:724)
at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:790)
at org.eclipse.jdt.internal.core.SourceType.newSupertypeHierarchy(SourceType.java:743)
at org.eclipse.jdt.internal.core.SourceType.newSupertypeHierarchy(SourceType.java:695)
at org.eclipse.jdt.internal.launching.JavaLaunchableTester.hasSuperclass(JavaLaunchableTester.java:467)
at org.eclipse.jdt.internal.launching.JavaLaunchableTester.test(JavaLaunchableTester.java:611)
at org.eclipse.core.internal.expressions.Property.test(Property.java:58)
at org.eclipse.core.internal.expressions.TestExpression.evaluate(TestExpression.java:100)
at org.eclipse.core.internal.expressions.CompositeExpression.evaluateOr(CompositeExpression.java:68)
at org.eclipse.core.internal.expressions.OrExpression.evaluate(OrExpression.java:22)
at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
at org.eclipse.core.internal.expressions.AndExpression.evaluate(AndExpression.java:31)
at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
at org.eclipse.core.internal.expressions.IterateExpression.evaluate(IterateExpression.java:160)
at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
at org.eclipse.core.internal.expressions.WithExpression.evaluate(WithExpression.java:75)
at org.eclipse.core.internal.expressions.CompositeExpression.evaluateAnd(CompositeExpression.java:53)
at org.eclipse.core.internal.expressions.EnablementExpression.evaluate(EnablementExpression.java:55)
at org.eclipse.debug.internal.ui.launchConfigurations.LaunchShortcutExtension.evalEnablementExpression(LaunchShortcutExtension.java:281)
at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager.getShortcutsForSelection(LaunchingResourceManager.java:454)
at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager.computeLabels(LaunchingResourceManager.java:240)
at org.eclipse.debug.internal.ui.contextlaunching.LaunchingResourceManager$1$1.run(LaunchingResourceManager.java:142)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
When I log into my ruby applicaiton, the first page url is http://localhost:3000/tree_display/list. So , when I run the test to assert the same, I am getting this error.
[user1 project]$ rake test
DEPRECATION WARNING: String based terminators are deprecated, please use a lambda. (called from included at /home/.gem/ruby/2.1.3/bundler/gems/authlogic-09163c7d2a9b/lib/authlogic/session/callbacks.rb:66)
DEPRECATION WARNING: String based terminators are deprecated, please use a lambda. (called from included at /home/.gem/ruby/2.1.3/bundler/gems/authlogic-09163c7d2a9b/lib/authlogic/session/callbacks.rb:67)
Started
FAIL["test_login_and_access_bookmarks/managing_bookmarks", BookmarksTest, 0.678676891]
test_login_and_access_bookmarks/managing_bookmarks#BookmarksTest (0.68s)
expecting <"tree_display/list"> but rendering with <["content_pages/view", "shared/_header", "shared/_navigation", "auth/_login", "shared/_flash_messages", "shared/_footer_message", "layouts/application"]>
test/integration/bookmarks_test.rb:15:in `block in <class:BookmarksTest>'
1/1: [=========================================================================================================] 100% Time: 00:00:00, Time: 00:00:00
Finished in 0.68149s
1 tests, 2 assertions, 1 failures, 0 errors, 0 skips
[user1 project]$
It is rendering with <["content_pages/view", "shared/_header", "shared/_navigation", "auth/_login", "shared/_flash_messages", "shared/_footer_message", "layouts/application"]>. These are supposed to be on the first page when I enter the username and password and then I go to http://localhost:3000/tree_display/list
I noticed that the session variable contains this :#host="www.example.com",
What is the problem here? Why is it not rendering the tree_display/list? Appreciate any help.
Probably login failed in your test or it is taking time to login while you are asserting on previous page in the test. Try adding delays in assertion and login step.
I'm using Gallio/MbUnit framework for my web testing, and the tests are kicked off from Jenkins. I've installed the Gallio/MbUnit plugin and it's publishing the xml report. I'm trying to find a way to display test log or console messages in the "Test Result" section so the team can easily read failures instead of digging into the "Console Output" for any failed test run.
When I run these Gallio/MbUnit tests from my local machine using Icarus everything is pretty nicely formatted, but not so much with Jenkins. I'd like to keep using it and improve how we display the errors. Suggestions?
For a failed test:
Failed
...MainMenuTests.AcctClaimsItems
Failing for the past 4 builds (Since Failed#128 )
Took 47 sec.
add description
Error Message
Expected value to be false.Actual Value : d:\Jenkins\jobs\...\workspace\WebTesting\Base\Helpers.cs:line 90d:\Jenkins\jobs\...\workspace\WebTesting\TigerEye\Tests\MainMenuTests.cs:line 329true
Stacktrace
at WebTesting.Base.Helpers.Click(IWebDriver driver, IWebElement element) in
From the raw console output:
Start time: 4:21 PM
Initializing the runtime and loading plugins.
Verifying test files.
Initializing the test runner.
Running the tests.
[failed] Test WebTesting/MainMenuTests/AcctClaimsItems
Expected value to be false.
Found System.Web Exception after click to url
.../Accounting/FETReport.aspx
Actual Value : true
at WebTesting.Base.Click(IWebDriver driver, IWebElement element) in d:\Jenkins\jobs\...\workspace\WebTesting\Base\StaticHelpers.cs:line 90 at WebTesting...\Tests.MainMenuTests.AcctClaimsItems() in d:\Jenkins\jobs\...\workspace\WebTesting\TigerEye\Tests\MainMenuTests.cs:line 329
Code:
if (driver.PageSource.Contains("System.Web - Exception"))
{
TestLog.Write("Found exception on page {0}", driver.Url);
TestLog.Write(driver.PageSource.ToString());
Console.Write("Found exception on page {0}", driver.Url);
Console.Write(driver.PageSource.ToString());
Assert.IsFalse(driver.PageSource.Contains("System.Web - Exception"), "Found System.Web Exception after click to url {0}",driver.Url);
}
There is a general console parsing plugin you can use to add a post build step https://wiki.jenkins-ci.org/display/JENKINS/Log+Parser+Plugin