I've a problem and I don't know the solution even after several hours of try and error and googling and stackoverflowing.
I have a view controller. I would like to pass per dependency injection an object. This object derives from a protocol. In general it is not a problem to setup a unit test. Also mocking works and the unit tests are running. So where's the problem?
I am testing only one class in my primary target. This class has absolutely nothing to do with view controller. But the code coverage is showing me a decent percent value of covering the view controllers. After a while I found out that when I hit the "test" button the project gets executed as if I push the "run" button. And because of that the view controller gets initialized and created and I have no chance to pass another dependency first, or before the tests getting executed.
So I need a method to distinguish between a test run and a real run, to pass in one case a real object and in the other case the fake object.
And my question is, how to to that? I wonder why nobody have this problem. I mean what gives me the code coverage tool if it shows me that methods are covered even though I haven't tested them.
The one and only class that I am testing:
And these are the coverage results (The bars are just gray because Xcode lost focus during screenshot. Otherwise they are blue.):
So I was expecting to see covered in the results just the class I am testing and not everything else. I know why this problem persists. The view controller has a dependency and this dependency after it gets initialized creates some more classes and so on. What I would like to do is to pass a fake object during unit testing and a real object during a real run. Just like It works in Visual Studio for non ui tests: If the tests are executed the application does NOT start up. The test runner just initialize the subjects under test and that's all. And this is what I want to achieve for iOS unit tests. I guess I've missed sth. very important :(
For all of us who have or will have the same problem. The solution is to specify environment variables for the test run. After that you can check if you are running your unit tests with code like this (assuming you have created an environment variable called "InTestMode" and set the value to "1" during test run:
let dict = NSProcessInfo.processInfo().environment
if let env = dict["InTestMode"] as? String?
{
return env == "1"
}
return false
Related
I have a hard time to grasp how to implement unit tests in a class where all my fields are private.
The class is calculating a user's position with BLE and CoreLocation - not that important. I have a protocol, which when a new location is found I'm calling it and all the classes which conform to that protocol will receive a room id and room name. So, what that means is that literally all the fields in my class are private, because yeah, there's no reason any outside class should access them right? But that also means I can literally test nothing in that class, even though there are quite a few functions which I would like to test. I mean, I could just make the variables internal instead of private, but it just seems wrong to do that just to unit test. I've heard about dependency injection, but it just seems like so much effort.
For example I have this function:
private var beacons: [AppBeacon] = []
private var serverBeacons:[Beacon] = []
private func addBeacons(serverBeacons: [Beacon]){
for beacon in serverBeacons {
let beacon = AppBeacon(id: beacon.id, uuid: beacon.uuid, building: beacon.building, name: beacon.name)
beacons.append(beacon)
}
}
there's no way I can test whether the beacons array was actually filled up as I wanted to or not for example. The public features of my class are basically a function called startLocating and the result which is the room id and name and I know in black box testing which unit testing imitates (right?) I should not care about the intermediate steps, but honestly, with this much functionality should I just say, doesn't matter? And assume i did populate the beacons with some rssi values of my choice, the actual location algorithm is executed on a node.js server, so I honestly don't know what to test client side?
It's classic MVC and there's no way I can change it architecture until the deadline that I have, so I don't know what's the best way to go from here? Just don't test the functionalities? Make the fields internal instead of private? We do testing of the algorithm itself server side, so testing whether the the room id is the expected room id, is already tested.
I read on another post the following:
"Unit testing by definition is black box testing, which means you don't care about the internals of the unit you test. You are mainly interested to see what's the unit output based on the inputs you give it in the unit test.
Now, by outputs we can assert on several things:
the result of a method
the state of the object after acting on it,
the interaction with the dependencies the object has
In all cases, we are interested only about the public interface, since that's the one that communicates with the rest of the world.
Private stuff don't need to have unit tests simply because any private item is indirectly used by a public one. The trick is to write enough tests that exercise the public members so that the private ones are fully covered.
Also, one important thing to keep in mind is that unit testing should validate the unit specifications, and not it's implementation. Validating implementation details adds a tight coupling between the unit testing code and the tested code, which has a big disadvantage: if the tested implementation detail changes, then it's likely that the unit test will need to be changed also, and this decreases the benefit having unit test for that piece of code."
And from that I essentially understand it as that I should just not unit test this?
If you have a private var that would help you write unit tests, change it to private(set) var so that it can be read (but not changed).
Revealing the innards may bother you. If it does, it's possible that there's another type waiting to be extracted from the view controller.
First, the definition of 'unit-test' you have quoted is quite unusual: All definitions from the literature that I am aware of consider unit-testing a glass-box/white-box testing method. More precisely, unit-testing as such is actually neither black-box or white-box - it is the method used for designing the test cases that makes the difference. But there is no reason not to apply white-box test design techniques for unit-testing. In fact, some white-box test design techniques make only sense when applied for unit-testing. For example, when you investigate on unit-testing, you will encounter a lot of discussions about different code coverage criteria, like, statement coverage, branch coverage, condition coverage, MC/DC - for all of which it is essential to know the implementation details of the code and consider these implementation details during test case design.
But, even taking that into account, it does not change much for your particular case: The variables in your code are still private :-) However, you are approaching the testing problem in a too restrictive way: You try to test the function addBeacons for itself, in total isolation even from the other functions in that component.
To test addBeacons, your test cases should not only contain a call to addBeacons, but also some other function call, the result of which would show you if the call to addBeacons was successfull. For example, if you also have a function to find a beacon by name or position, you would first call addBeacon and, to check that this has succeeded, call one of these function to see if your added beacon will be found. These two calls would be part of the same test case.
As expected, I normally have difficulties when following TDD (i.e. writing the tests first) with the view layer.
Namely, in order to observe or trigger certain visible changes (layout or styling) I would need to make the view's internals public. This breaks encapsulation and allows for client code do some thing like myView.label.text = "User".
To avoid this, I either add getter methods to UIView class:
var userName: String{ return label.text }
Or do some extensions that are added only to the test framework:
extension MyView{
//avoids making a getter just for the sake of testing, while keeping it private and interchangeable
var userName : String{
return (viewWithTag(someLabelTage) as! UILabel).text
}
A different approach is to skip the TDD work flow (i.e. test manually after the feature is done) and add snapshot testing (see https://github.com/pointfreeco/swift-snapshot-testing) to the increase coverage and have a safety net when refactoring.
Considering all this, I was wondering if there are any other patterns or approaches, that I can use to be more efficient while keeping my confidence in the code.
Not an expert in Swift, but regardless of the language/framework, something tells me you're actually making your task harder.
I normally have difficulties when following TDD with the view layer.
This is expected. The View support to be simple and have no behavior (i.e domain logic) at all. It should be simple user interaction, and binding data to the controls in the view. So you don't need TDD or to be more precise Unit Tests around the View in my opinion. You're not going to get much value out of trying to write Unit tests for the View.
You View can be tested more effectively using a UI test framework, such as Selenium, or your own UI test framework, which tests User interactions. This way you more return in investment (ROI) than attempting to TDD the View layer.
I don't have much to add to Spock's answer. Tests should be written to help development and maintenance of the code in the future. If they get in the way then either the code is not architected properly or it's code for which the ROI is low.
A pattern that is worth mentioning is the humble view. There are a few ways to approach it, each with different trade offs, but the gist is to have data structures defining how the view should look like, and then let the view read these data structure and set its properties from them.
This allows you to test that the data structure driving the view is generated properly, without having to test the view itself, because it's nothing but an humble object that reads and sets values.
I have created a test project and noticed that XCode generates some test class named: YourProjectName-ProjectTests
This is how it looks like:
- (void)setUp {
[super setUp];
// Put setup code here. This method is called before the invocation of each test method in the class.
}
- (void)tearDown {
// Put teardown code here. This method is called after the invocation of each test method in the class.
[super tearDown];
}
- (void)testExample {
// This is an example of a functional test case.
XCTAssert(YES, #"Pass");
}
- (void)testPerformanceExample {
// This is an example of a performance test case.
[self measureBlock:^{
// Put the code you want to measure the time of here.
}];
}
Would anyone be able to explain me how to use these automated testing methods? Is there an official documentation on this?
Are those test meant to be used as UI testing or as app code testing? Or both?
You an perform UI tests through XCTest but it's designed for unit testing. Instruments has UI Automation which is designed for UI testing.
Opening up XCTestAssertions.h will give you a pretty good reference for what's included in XCTest. Generally all the assertions follow a similar pattern:
XCTAssertSomething(somethingBeingTested, #"A message to display if the test fails");
Hitting ⌘ 5 will bring up the test navigator which lets you run all of your tests, all of your failed tests, or individual tests. You can also run a test by clicking on the diamond shape to the left of the method name in the editor.
Here are a few tips I've come across when unit testing with XCTest:
The last argument in assertions is optional but in some cases it's useful to describe the behaviour you're looking for.
Tests are not run in any specific order, but your tests shouldn't rely on each other anyway.
Mock objects let you test far, far more than basic assertions. I like OCMock.
You can use instruments on your tests for debugging.
Finally, if you have access to another machine you can set up OSX Server to automatically run your tests daily or on every commit and notify you if any of the tests fail.
This is how Unit Testing is implemented in Xcode.
The setUp and TearDown methods are to prepare any mock data necessary to test a unit (which is select part of code).
The two test methods there are just an example. Usually as an author of the unit of code, you know well what the unit supposed to do and what it should not.
Using test methods you test how the assumptions and expectations are satisfied by the unit. The convention is to have a separate test method for each assumption/expectation of a behaviour of the unit.
For example given class that handles strings that has a method to to make all strings CAPS, you would test that passing any random nonCaps string would still return ALL CAPS. An putting none truing as a parameter would return nil or error.
So basically you test your units to behave as expected.
There's a whole theory behind Unit Testing which is out of scope of this thread. Just Google it.
I have a class which can be configured to do 2 slightly different things. I want to test both paths. The class is a descendant of UIViewController and most of the configuration takes place in Interface Builder. i need to verify that both Storyboard Scenes and their Outlets are wired up in the same way, but also need to check for the difference in behavior.
I'd like to use shared XCTest suites for this purpose.
One is aimed for use with the left hand, one for the right. Both appear after another when using the app. The first one (right hand) triggers a segue to the other. The last one (left hand) should trigger a different segue. This is where it differs, for example.
Now I want to verify the segues with tests. I'd like to create a BothHandSharedTests suite which both view controller instance tests use to verify everything they have in common. However, the BothHandSharedTests class is treated as a self-containing test suite, which it clearly isn't.
I came up with these strategies:
inherit from an abstract XCTest descendant, like described above (doesn't seem to be that easy),
write a test auite for the common properties and use one of the two as the Object Under Test, and add two smaller suites for the differences.
How would you solve this problem?
Here's a solution in Swift:
class AbstractTests: XCTestCase {
// Your tests here
override func perform(_ run: XCTestRun) {
if type(of: self) != AbstractTests.self {
super.perform(run)
}
}
}
I don't have a conclusive answer, but here's what I ended up doing.
I first tried the subclassing route. In the parent test (the "AbstractTestCase") I implemented all the tests that would be executed by the the AbstractTestCase subclasses, but added a macro so they don't get run by the actual parent test:
#define DONT_RUN_TEST_IF_PARENT if ([[self className] isEqualToString:#"AbstractTestCase"]) { return; }
I then added this macro to the start of every test, like so:
- (void)testSomething
{
DONT_RUN_TEST_IF_PARENT
... actual test code ...
}
This way, in the ConcreteTestCase classes which inherit from AbstractTestCase, all those tests would be shared and run automatically. You can override -setUp to perform the necessary class-specific set-up, of course.
However – This turned out to be a crappy solution for a couple reasons:
It confuses Xcodes testing UI. You don't really get to see a live representation of what's running, and tests sometimes don't show up as intended. This makes clicking through to debug test failures difficult or impossible.
It confuses XCTest itself – I found that tests would often get run even when I didn't ask them too (if I were just trying to run a single test) and the so the test output wouldn't be what I would expect.
Honestly it felt a little janky to have that macro – macros which redirect flow control are never really that good of an idea.
Instead, I'm now using a shared object, a TestCaseHelper, which is instantiated for each test class, and has has a protocol/delegate pattern common to all test cases. It's less DRY – most test cases are just duplicates of the others – but at least they are simple. This way, Xcode doesn't get confused, and debugging failures is still possible.
A better solution will likely have to come from Apple, unless you're interested in ditching your entire test suite for something else.
I am dealing with a case where my tests pass or fail based on the order of declaration. This of-course points to not properly isolated tests. But I am stumped about how to go about finding the issue.
The thing is my junit tests derive from a class that is belongs to a testing framework built on junit and has some dependency injection container. The container gets reset for every test by the base class setup and so there are no lingering objects at least in the container since the container itself is new. So I am leaning toward the following scenario.
test1 indirectly causes some classA which sets up classA.somestaticMember to xyz value. test obj does not maintain any references to classA directly- but classA is still loaded by vm with a value xyz when test1 ends.
test2 access classA and trips up on somestaticmember having xyz value.
The problem is a) I dont know if this is indeed the case- how do I go about finding that ? I cannot seem to find a reference to a static var in the code...
b) is there a way to tell junit to dump all its loaded classes and do it afresh for every test method ?
You can declare a method with #Before, like
#Before public void init()
{
// set up stuff
}
and JUnit will run it before each test. You can use that to set up a "fixture" (a known set of fresh objects, data, etc that your tests will work with independently of each other).
There's also an #After, that you can use to do any cleanup required after each test. You don't normally need to do this, as Java will clean up any objects you used, but it could be useful for restoring outside objects (stuff you don't create and control) to a known state.
(Note, though: if you're relying on outside objects in order to do your tests, what you have isn't a unit test anymore. You can't really say whether a failure is due to your code or the outside object, and that's one of the purposes of unit tests.)