I'm writing an iOS app, that uses GoogleMaps. I've written some unit tests to test separate business logic modules. They look like this:
func testIfStationsExistNearPoint1() {
let comMangager = CommunicationManager()
comMangager.getListOfStations(coordinates: Coordinates(longitude: Constants.Point1CoordinateLongitude, latitude: Constants.Point1CoordinateLatitude), distance: 50, completion: { (json) in
if (json != nil) {
let stations = StationWrapper.sharedStationWrapper.jsonToStations(json: json!)
assert(stations.count > 0, String(format:"%ld stations", stations.count))
}
else {
XCTFail("json is nil")
}
}) { (error) in
XCTFail(String(format:"List of stations returned error: %#", error))
}
}
Note, that those unit tests are completely separated from UI (Where GoogleMaps are used. Coordinates is a custom class, not GoogleMaps CLLocationCoordinate2D).
Also note, that application itself works ok. And I'm 100% sure, that in main app, all code, that works with map drawing, is called in main thread.
Yet somehow I receive this error, when trying any of my unit tests:
com.google.Maps.LabelingBehavior (15): -[UIApplication
applicationState] must be used from main thread only
When I set a breakpoint in the very first line of this unit test, it's not even hit. Same thing for both simulator and real device.
I wonder, how's that even possible, since I'm not using Google Maps anywhere in unit tests? I've tried to search for similar cases, yet haven't found any, that was related unit testing.
Any ideas what's wrong here because I feel completely stuck?
XCode launches your application before executing unit tests.
So this error is produced by your application before Xcode starts executing unit tests. That's why your breakpoint at the beginning of test is not hit.
Placing exception breakpoint should lead you to source of the issue.
Related
I want to write path of test artifacts at the end of output in console. It would be handy. And I'm also just curious.
It seems that XCTest somehow terminate the app, app delegate doesn't receive lifecycle callbacks, program exits before main function from main.m.
Take a look at Test Execution and Observation specifically testBundleDidFinish.
If you define an object which adopts XCTestObservation you can add it to the XCTestObservationCenter.shared instance at any point during your test suite's run and receive a call when the entire bundle has finished.
Jonah helped me, I also want to place some code here for other people.
To start observation before any test there is a proper way to do it.
There is NSPrincipalClass key in test bundle's Info.plist. Put there a name of your "principal class" (google it for more info). In my case it is PrincipalClass. When test bundle is loaded, init is called in the principal class. Here's my PrincipalClass:
#objc(PrincipalClass)
final class PrincipalClass: NSObject {
override init() {
TestObservationEntryPoint.instance.startObservation()
}
}
I was able to make cool reporting system with Allure2. Reporting is not what I wanted at the time I asked this question, but it was possible because I found an answer. It is a good application of test observation in XCTest.
For me the answers were very confusing.
func testYourTest() {
let expectation = XCTestExpectation(description: "Your action")
API.sharedInstance.yourRequestWithCompletionAndFailure(parameter: parameter, completion: { (response) in
//XCTAssert with response
expectation.fulfill()
}) { (error) in
//XCTAssert with your error
expectation.fulfill()
}
wait(for: [expectation], timeout: 30.0)
}
With expectation you listen to a response. Don't forget to set a timeout.
More Details/Source: https://developer.apple.com/documentation/xctest/asynchronous_tests_and_expectations/testing_asynchronous_operations_with_expectations
I wrote a mock Coredata manager in order to test some classes in unit tests.
I have about 10 classes that get's NSManagedObjectContext from a class called DatabaseManager. I have decided if unit tests are running, don't deal with actual Coredata NSManagedObjectContext but redirect to Mock Coredata Class to get the NSManagedObjectContext.
func getContext() -> NSManagedObjectContext {
if ProcessInfo.processInfo.environment["XCTestConfigurationFilePath"] == nil
{
return persistentContainer.viewContext
}
else
{
return MockDatabaseController.instance.managedObjectContext()
}
}
This works pretty well in Unit tests and debugging and when distributed thru adhoc too.
But my concern is if it ever fails to get the correct value from ProcessInfo.processInfo.environment["XCTestConfigurationFilePath"] app will probably be useless.
How viable is it to use ProcessInfo.processInfo.environment["XCTestConfigurationFilePath"] in production code ?
I would use Swift conditional compilation along with -D flags passed in the build arguments to make sure that the code was only active in test environments and never had the opportunity to make it into production.
How can I stop Xcode iOS unit tests if a fatalerror is hit?
That is in case I have 10 unit tests, but it happens that the code it calls for unit test number 5 has a coding problem (** coding issue in this case is in the test case and setup code **) and is throwing a fatalError. So in this case the unit testing stops there and does not continue to other test cases in that test class.
(not sure if this is the intended operational / process for good unit testing or not? )
Try
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
continueAfterFailure = false
}
A related problem I had was stopping the unit test with a breakpoint when it fails, so that I can see the issue, without having to click or scroll through tons of test output.
Making a unit test failure breakpoint is simple, but wasn't easy to find.
Xcode 8+ there is a new breakpoint that you can add called a "Test Failure Breakpoint".
Click on Breakpoints (Left Panel ... Command + 7)
Click on + (Bottom left corner)
Click on "Test Failure Breakpoint"
References
Debugging Tests with Xcode
There is no easy fix for this. In case of an uncaught (objc) exception or a failed assert the process that runs the unit tests did receive either a mach exception or a unix signal.
Matt Gallagher has a nice solution for this, which he presents in this blog post.
It is good practice to write safe code everywhere, including your unit tests.
Regarding your problem, as Oleg Danu replied, you should set continueAfterFailure = false.
The next step is to add following before your test can crash.
var optionalVariable: Int?
XCTAssertNotNil(optionalVariable)
I recommend to add it into setUp()
In this way your test will stop before crashing Xcode, and you don't need to set any breakpoints.
I have an app with a couple of thousand lines and within that code there are a lot of println() commands. Does this slow the app down? It is obviously being executed in the Simulator, but what happens when you archive, submit and download the app from the app store/TestFlight. Is this code still "active", and what about code that is "commented out"?
Is it literally never read or should I delete commented out code when I submit to test flight/app store?
Yes it does slow the code.
Both print and println decrease performance of the application.
Println problem
println is not removed when Swift does code optimisation.
for i in 0...1_000 {
println(i)
}
This code can't be optimised and after compiling Assembly code would perform a loop with 1000 instructions that actually don't do anything valuable.
Analysing Assembly code
The problem is that Swift compiler can't do optimal optimisation to the code with print and println commands.
You can see it if you have a look on generated Assembly code.
You can do see assembly code with Hopper Disassembler or by compiling Swift code to the Assembly with by using swiftc compiler:
xcrun swiftc -emit-assembly myCode.swift
Swift code optimisation
Lets have a look on few examples for better understanding.
Swift compiler can eliminate a lot of unnecessary code like:
Empty function calls
Creating objects that are not used
Empty Loops
Example:
class Object {
func nothing() {
}
}
for i in 0...1_000 {
let object = Object3(x: i)
object.nothing()
object.nothing()
}
In this example Swift complier would do this optimisation:
1. Remove both nothing method calls
After this the loop body would have only 1 instruction
for i in 0...1_000 {
let object = Object(x: i)
}
2. Then it would remove creating Object instance, because it's actually not used.
for i in 0...1_000 {
}
3. The final step would be removing empty loop.
And we end up with no code to execute
Solutions
Comment out print and println
This is definitely not the best solution.
//println("A")
Use DEBUG preprocessor statement
With this solution you can simple change logic of your debug_print function
debug_println("A)
func debug_println<T>(object: T) {
#if DEBUG
println(object)
#endif
}
Conclusion
Always Remove print and println from release application!!
If you add print and println instruction, the Swift code can't be optimised in the most optimal way and it could lead to the big performance penalties.
Generally you should not leave any form of logging turned on in a production app, it will most likely not impact performance but it is poor practice to leave it enabled and unneeded.
As for commented code, this is irrelevant as it will be ignored by the compiler and not be part of the final binary.
See this answer on how to disable println() in production code, there is a variety of solutions, Remove println() for release version iOS Swift
As you do not want to have to comment out all your println() calls just for a release, it is much better to just disable them, otherwise you'll be wasting a lot of time.
printLn shouldn't have much of an impact at all as the bulk of the operation has already been carried out before that point.
Commented out code is sometimes useful, although it can make your source difficult to read it has absolutely no bearing on performance whatsoever and I've never had anything declined for commented out code and my stuff is full of it.
In Xcode, is there a way for me run a single test case n times automatically?
Reason for doing this is that some of my beta testers are encountering random crashes in my app. I see the crash logs in TestFlight, along with the stack trace, but I can't reproduce the crash.
The crash happens infrequently but when it does, it always happens when users are trying to create a DB record, which then gets uploaded to a server. The problem with the crash logs is that my code does not make an appearance in their stack traces (all UIKit & CoreFoundation stuff - and different each time).
My solution is to run the test for that part of the app 100s of times, with the exception breakpoint set, to try to trigger the bug in my dev environment. But I don't know how to do this automatically.
It took 7 years, but as of Xcode 13, support for test repetition is now built-in.
From the Xcode 13 release notes:
Enable test repetition in your test plan, xcodebuild, or by running your test from the test diamond by Control-clicking and selecting Run Repeatedly to bring up the test repetition dialog.
You can read my answer here.
Basically you need to override invokeTest method
override func invokeTest() {
for time in 0...15 {
print("this test is being invoked: \(time) times")
super.invokeTest()
}
}
In Xcode as such, no.
You can create an XCTestCase class that hooks into the test-running methods it inherits to return multiple runs, but that tends to be annoying and mostly undocumented.
It's probably easier to instead have a "meta-test" that calls out to your other test method repeatedly:
func testOnce() {}
func testManyTimes() {
for _ in 0..<1000 { testOnce() }
}
You might need to call out to some per-test setup/teardown methods. You could perhaps work around that by instead making the loop body be something like:
let test = XCTestCase(selector: #selector(testOnce))
test.invokeTest()
This would lean on the XCTest machinery that your standard tests use, but it might gripe about not being wired into an XCTestCaseRun (or not).