When my test runs into a critical failure such as tapping an invalid element the Automation Instrument attempts to restart the test from the beginning which results in a lot of errors and can even lag my system, making it difficult to stop the test. I don't have the repeat option enabled. Is there a way of preventing this behavior?
I reckon what you can do is: try capturing when your test runs into a failure by means of a try/catch block.
When your test fails, it will jump inside the catch block and you can stop it there.
Maybe something like this.
try {
// Run your tests
} catch (exception){
UIALogger.logFail("Test failed with error message: " + exception.message);
}
I think that the logFail() method should be enough to keep your tests from running indefinately.
Related
I've hit an assertion in code and wondering if there's a way to create a wrapper around the assert that would enable breaking out and continuing execution or some other function that would enable a way to suppress the assert through the lldb debugger.
assert(condition(), makeCriticalEvent().event.description, file: fileID, line: UInt(line))
This is the standard assertion in apples libraries here. When I hit this assertion I tried continuing execution but it stays stuck at the assertion. I'd like to silence the assertion (likely through the lldb debugger by typing some command). Anyone have an idea how to do this?
You have to do two things. First, you have to tell lldb to suppress the SIGABRT signal that the assert delivers. Do this by running:
(lldb) process handle SIGABRT -p 0
in lldb. Normally SIGABRT is not maskable, so I was a little surprised this worked. Maybe because this is a SIGABRT the process sends itself? I don't think there's any guarantee suppressing SIGABRT's has to work in the debugger so YMMV, but it seems to currently. Anyway, do this when you've hit the assert.
Then you need to forcibly unwind the assert part of the stack. You can do that using thread return, which returns to the thread above the currently selected one w/o executing the code in that frame or any of the others below it. So just select the frame that caused the assert, go down one frame on the stacks and do thread return.
Now when you continue you won't hit the abort and you'll be back in your code.
Is it possible to catch the following exception?
"Failed to get matching snapshots"
Most of the stability issues with XCUITest is due to not having a proper method to wait for element to exist. Tried exists(), waitforexistence(),xctwaiter waits etc. In all cases it fails randomly with above error. Is there a way we have handle this exception do a retry in our tests itself.
You should stick to combination of two methods
Use it for every flaky element
button.waitForExistence()
button.tap()
How can I stop Xcode iOS unit tests if a fatalerror is hit?
That is in case I have 10 unit tests, but it happens that the code it calls for unit test number 5 has a coding problem (** coding issue in this case is in the test case and setup code **) and is throwing a fatalError. So in this case the unit testing stops there and does not continue to other test cases in that test class.
(not sure if this is the intended operational / process for good unit testing or not? )
Try
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
continueAfterFailure = false
}
A related problem I had was stopping the unit test with a breakpoint when it fails, so that I can see the issue, without having to click or scroll through tons of test output.
Making a unit test failure breakpoint is simple, but wasn't easy to find.
Xcode 8+ there is a new breakpoint that you can add called a "Test Failure Breakpoint".
Click on Breakpoints (Left Panel ... Command + 7)
Click on + (Bottom left corner)
Click on "Test Failure Breakpoint"
References
Debugging Tests with Xcode
There is no easy fix for this. In case of an uncaught (objc) exception or a failed assert the process that runs the unit tests did receive either a mach exception or a unix signal.
Matt Gallagher has a nice solution for this, which he presents in this blog post.
It is good practice to write safe code everywhere, including your unit tests.
Regarding your problem, as Oleg Danu replied, you should set continueAfterFailure = false.
The next step is to add following before your test can crash.
var optionalVariable: Int?
XCTAssertNotNil(optionalVariable)
I recommend to add it into setUp()
In this way your test will stop before crashing Xcode, and you don't need to set any breakpoints.
Have a system build using C++ Builder 2010 that after running for about 20 hours it starts firing of assertion failures.
Assertion failed: xdrPtr && xdrPtr == *xdrLPP, file xx.cpp, line 2349
Tried google on it like crazy but not much info. Some people seem to refer a bunch of different assertions in xx.cpp to shortcomings in the exception handling in C++ Builder. But I haven't found anything referencing this particular line in the file.
We have integrated madExcept and it seems like somewhere along the way this catches an out of memory exception, but not sure if it's connected. No matter what an assertion triggering doesn't seem correct.
Edit:
I found an instance of a if-statement that as part of it's statement used a function that could throw an exception. I wonder if this could be the culprit somehow messing up the flow of the exception handling or something?
Consider
if(foo() == 0) {
...
}
wrapped in a try catch block.
If an exception is thrown from within foo() so that no int is returned here how will the if statement react? I'm thinking it still might try to finish executing that line and this performing the if check on the return of the function which will barf since no int was returned. Is this well defined or is this undefined behaviour?
Wouldn't
int fooStatus = foo();
if(fooStatus == 0) {
...
}
be better (or should I say safer)?
Edit 2:
I just managed to get the assertion on my dev machine (the application just standing idle) without any exception about memory popping up and the app only consuming around 100 mb. So they were probably not connected.
Will try to see if I can catch it again and see around where it barfs.
Edit 3:
Managed to catch it. First comes an assertion failure notice like explained. Then the debugger shows me this exception notification.
If I break it takes me here in the code
It actually highlights the first code line after
pConnection->Open();
But it seems I can change this to anything and that line is still highlighted. So my guess is that the error is in the code above it somehow. I have seen more reports about people getting this type of assertion failure when working with databases in RAD Studio... hmmmm.
Update:
I found a thread that recursively called it's own Execute function if it wasn't able to reach the DB server. I think this is at least part of the issue. This will just keep on trying and as more and more worker threads spawn and also keep trying it can only end in disaster.
If madExcept is hinting that you have an out of memory condition, the assert could fail if the pointers are NULL (i.e. the allocation failed). What are the values of xdrPtr and xdrLPP when the assert occurs? Can you trace back to where they are allocated?
I would start looking for memory leaks.
I have a plugin in Jenkins that operates a remote server via rest.
How can I send one last request to the server on build abort? Thus, gracefully finishing the work of the plugin?
The only reference to the 'Abort sequence' that I found is this.
Which makes me think that the procedure is very rough, and that I can't catch the signal before it terminates the child (my plugin).
I have a similar need, which I solved with the PostBuildScript Plugin.
I choose to run a build step, but you can run several other options. Very easy to use.
I have a script I must run regardless of how the build ended (success, fail or abort). It works great for me.
I hope this helps.
When the build is aborted, an InterruptedException is thrown. You can catch it like any other exception, and not only gracefully exit, but prevent the abort, if you so wish.
In your boolean perform() method, set up a
try {
... // Whatever your plugin does
}
catch (InterruptedException e) {
// Code to handle build being aborted.
}
statement, which will handle any aborts that occur while your build step is being run. In the catch statement you can then throw an InterruptedException again to continue aborting the build, or just return and let the build continue.