Per the comment within the default template for XCTestCase regarding setUp :
Put setup code here; it will be run once, before the first test case.
However, in XCTestCase.h, the comment above setUp states differently:
Setup method called before the invocation of each test method in the class.
To confirm the actual behavior, I put an NSLog withinsetUp to count how many times it was called:
static int count = 0;
- (void)setUp
{
[super setUp];
count++;
NSLog(#"Call Count = %d", count);
}
This resulted in the setUp method being called before every test method (confirming the comment on XCTestCase.h).
I wanted to use the setUp method to create test/mock objects once (e.g. to setup a Core Data test stack). Creating these over and over again would be processor intensive and potentially very slow.
So,
1) What is setUp actually intended to be used for? Surely developers aren't creating objects in it over and over?
2) How can I create these objects only once within an XCTestCase?
There are a couple of points for discussion here: the behaviour of the setUp methods, and general best testing practices.
There are actually two setUp methods:
+ (void)setUp;
- (void)setUp;
The class method (+ (void)setUp) is only run once during the entire test run.
The instance method (- (void)setUp) is the one in the default template; it's run before every single test. Hopefully, in a hypothetical future version of Xcode, this comment will have been changed to // Put setup code here. This method is called before the invocation of each test method in the class. WINK WINK
So through these two methods, both of the behaviours you described are possible.
Regarding your comment:
"Surely developers aren't creating objects in it over and over?"
My answer would be "Yes, they usually are". A popular acronym for 'good' unit tests is FIRST:
Fast
Isolated
Repeatable
Self-Verifying
Timely
Isolated is key to this discussion: your tests shouldn't rely on any previous state left behind by other tests. Ideally, you should tear down and recreate your in-memory Core Data stack for every test, so you know that you're starting from a clean slate. A good example is in this post by Graham Lee. You want to use an in-memory stack because a) you can easily throw it away, and b) it should be very fast because it's just in-memory and not hitting your disk.
If you find that your tests are running slowly as a result of this (don't optimize prematurely), then I think a reasonable next step would be to create the stack in your + (void)setUp method, but create a brand new context every time in your - (void)setUp method.
I came here with almost the same problem: how to perform the setUp just once and in Swift. Here is the solution:
override class func setUp() {
super.setUp()
// ...
}
override class func tearDown() {
// ...
super.tearDown()
}
Right now I'm still looking for a solution for an asynchronous setUp!
Related
I am trying to unit test a two functions in a view controller class. The two functions will create a user and sign in a user respectively. My tests are not UI related.
As of now, I simply need to create a test that passes when one of the functions is called. The goal is to do this without changing the current view controller implementation if possible and just keep it all in the testing class/function.
How do I go about this?
Thanks
I'm assuming a view controller with a method that invokes either "create user" or "sign in user". (You say it's not UI related, but we can easily test button taps if that's the trigger.)
class MyViewController: UIViewController {
func triggeringMethod() {
// Calls one of the two methods below
}
func createUser() {
// Does stuff
}
func signInUser() {
// Does stuff
}
}
And it sounds like you want to test the flow, but not the effect. That is, you want to test "Was createUser() called?" There are several ways to get what you want, but the better ways will require you to change your view controller implementation.
Making a Partial Mock
A standard trick from Working Effectively With Legacy Code by Michael Feathers is "Subclass and Override Method". Let's start there. In test code, we can make
class TestableMyViewController: MyViewController {
override func createUser() {
}
override func signInUser() {
}
}
So far, this is a way of stubbing out the effects of these methods. But we can now add the mocking techniques from my try! Swift Tokyo talk.
class TestableMyViewController: MyViewController {
var createUserCallCount = 0
var signInUserCallCount = 0
override func createUser() {
createUserCallCount += 1
}
override func signInUser() {
signInUserCallCount += 1
}
}
Now you can call the triggering method, and check the call counts.
(Changes you may have to make: The class can't be final. The methods can't be private.)
Moving the Workers
While this is a fine place to start, don't stop there. What we've created is a "partial mock". That's where we've kept most of the functionality, but mocked out a couple of the methods. This is something to avoid. The reason is that we end up with class that mixes production code and test code. It would be far too easy to end up in a situation where you're inadvertently testing test code instead of testing production code.
What the partial mock makes clear is that we're missing a boundary. The view controller is doing too much. The actual work of "create user" and "sign in user" should be performed by another type (possibly even 2 types). In Swift, we can define this boundary with a protocol. That way production code can use the real functionality, while for test code we can inject mocks.
This means the production code should avoid deciding for itself who does the actual work. Instead, we should tell it who does the work. That way, tests can provide alternative workers. Specifying these dependencies from the outside is called "Dependency Injection".
Passing Back Effects
Another option lets us avoid mocks altogether. Instead of testing whether something was called, we can describe the desired effect in an enumeration. Then we can define effects like
enum Effect {
case createUser(CreateUserRequestModel)
case signInUser(SignInUserRequestModel)
}
Instead of the triggering method calling createUser() or signInUser(), it would call a delegate. (Another option is to pass in closures instead of specifying delegates.)
protocol Delegate {
perform(_ effect: Effect)
}
Then in the triggering method,
delegate?.perform(.createUser(parameters))
This means it's up to the actual delegate to transform these enumeration values into actual work. But it makes the tests easy to write. All we need is to provide a testing implementation that captures the Effect value.
I'm going through a great book learning about test-driven development in Swift. My ultimate goal is to get a better understanding of OOP architecture. As I'm reading the book, a section early on states that the setUp() method is fired before each test method which I understand does the setup of objects to run the test for a pass or fail result. What I'm unsure of is how is this even possible I'm guessing from an architecture stand-point? How was Apple able to make a class that has a method which is fired before every other method in the class?
Here is some sample code:
import XCTest
#testable import FirstDemo
class FirstDemoTests: XCTestCase {
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
}
override func tearDown() {
// Put teardown code here. This method is called after the invocation of each test method in the class.
super.tearDown()
}
func testExample() {
// This is an example of a functional test case.
// Use XCTAssert and related functions to verify your tests produce the correct results.
}
func testPerformanceExample() {
// This is an example of a performance test case.
self.measure {
// Put the code you want to measure the time of here.
}
}
}
I think XCTest class has a lifecycle just like UIViewController for example.
Same as viewDidLoad() called after init(coder:) when the view is loaded into memory, setUp() method is also called after the test class is loaded (once during the life of the object).
You can also investigate native and third party test frameworks source code on github:
https://github.com/apple/swift-corelibs-xctest
https://github.com/google/googletest
You are Subclassing a Class called XCTestCase. Usally Testframework introspects Classes in which Test-Methods are defined and run them in a particular order.
Anyway, im not 100% sure if the XCTest is working this way, but you could try to have a look at the source code and try to dig deeper there:
https://github.com/apple/swift-corelibs-xctest
For each test to be tested: setup() is called, then the actual test, then teardown().
Then again setup another test from that class and tearDown again...until all** tests of your test class get ran.
The sequence of the lines won't affect which test gets called first.
The reason that it's made this way is because 2 tests should be 100% independent. You don't want testExample do something to the object/sut/system_under_test (mutate its state) you're performing your tests on but, not undo what it's done. Otherwise your testPerformanceExample would be loaded from a state you weren't
expecting.
I have the following problem. I want to execute a piece of code before all test classes are executed. For instance: I don't want my game to use the SoundEngine singleton during executing, but the SilentSoundEngine. I would like to activate the SilentSoundEngine one time not in all tests. All my tests look like this:
class TestBasketExcercise : XCTestCase {
override func setUp() {
SilentSoundEngine.activate () // SoundEngine is a singleton
}
// The tests
}
-Edit-
Most of the answers are directed at providing custom superclass for the TestCase. I am looking for a more general and cleaner way to provide the environment that all tests need to execute. Isn't there a "main" function/ Appdelegate like feature somewhere for tests?
TL;DR:
As stated here, you should declare an NSPrincipalClass in your test-targets Info.plist. Execute all the one-time-setup code inside the init of this class, since "XCTest automatically creates a single instance of that class when the test bundle is loaded", thus all your one-time-setup code will be executed once when loading the test-bundle.
A bit more verbose:
To answer the idea in your edit first:
Afaik, there is no main() for the test bundle, since the tests are injected into your running main target, therefore you would have to add the one-time-setup code into the main() of your main target with a compile-time (or at least a runtime) check if the target is used to run tests. Without this check, you'd risk activating the SilentSoundEngine when running the target normally, which I guess is undesirable, since the class name implies that this sound-engine will produce no sound and honestly, who wants that? :)
There is however an AppDelegate-like feature, I will come to that at the end of my answer (if you're impatient, it's under the header "Another (more XCTest-specific) approach").
Now, let's divide this question into two core problems:
How can you ensure that the code you want to execute exactly one time when running the tests is actually being executed exactly one time when running the tests
Where should you execute that code, so it doesn't feel like an ugly hack and so it just works without you having to think of it and remember necessary steps each time you write a new test suite
Regarding point 1:
As #Martin R mentioned correctly in his comments to this answer to your question, overriding +load is not possible anymore as of Swift 1.2 (which is ancient history by now :D), and dispatch_once() isn't available anymore in Swift 3.
One approach
When you try to use dispatch_once anyway, Xcode (>=8) is as always very smart and suggests that you should use lazily initialized globals instead.
Of course, the term global tends to have everyone indulge in fear and panic, but you can of course limit their scope by making them private/fileprivate (which does the same for file-level declarations), so you don't pollute your namespace.
Imho, they are actually a pretty nice pattern (still, the dose makes the poison...) that can look like this, for example:
private let _doSomethingOneTimeThatDoesNotReturnAResult: Void = {
print("This will be done one time. It doesn't return a result.")
}()
private let _doSomethingOneTimeThatDoesReturnAResult: String = {
print("This will be done one time. It returns a result.")
return "result"
}()
for i in 0...5 {
print(i)
_doSomethingOneTimeThatDoesNotReturnAResult
print(_doSomethingOneTimeThatDoesReturnAResult)
}
This prints:
This will be done one time. It doesn't return a result.
This will be done one time. It returns a result.
0
result
1
result
2
result
3
result
4
result
5
result
Side note:
Interestingly enough, the private lets are evaluated before the loop even starts, which you can see because if it were not the case, the 0 would have been the very first print. When you comment the loop out, it will still print the first two lines (i.e. evaluate the lets).
However, I guess that this is playground specific behaviour because as stated here and here, globals are normally initialized the first time they are referenced somewhere, thus they shouldn't be evaluated when you comment out the loop.
Another (more XCTest-specific) approach
(This actually solves both point 1 and 2...)
As the company from Cupertino states here, there is a way to run one-time-pre-testing setup code.
To achieve this, you create a dummy setup-class (maybe call it TestSetup?) and put all the one time setup code into its init:
class TestSetup: NSObject {
override init() {
SilentSoundEngine.activate()
}
}
Note that the class has to inherit from NSObject, since Xcode tries to instantiate the "single instance of that class" by using +new, so if the class is a pure Swift class, this will happen:
*** NSForwarding: warning: object 0x11c2d01e0 of class 'YourTestTargetsName.TestSetup' does not implement methodSignatureForSelector: -- trouble ahead
Unrecognized selector +[YourTestTargetsName.TestSetup new]
Then, you declare this class as the PrincipalClass in your test-bundles Info.plist file:
Note that you have to use the fully qualified class-name (i.e. YourTestTargetsName.TestSetup as compared to just TestSetup), so the class is found by Xcode (Thanks, zneak...).
As stated in the documentation of XCTestObservationCenter, "XCTest automatically creates a single instance of that class when the test bundle is loaded", so all your one-time-setup code will be executed in the init of TestSetup when loading the test-bundle.
From Writing Test Classes and Methods:
You can optionally add customized methods for class setup
(+ (void)setUp) and teardown (+ (void)tearDown) as well, which run before
and after all of the test methods in the class.
In Swift that would be class methods:
override class func setUp() {
super.setUp()
// Called once before all tests are run
}
override class func tearDown() {
// Called once after all tests are run
super.tearDown()
}
If you want to call the setUp method only once for all UI Tests in the class, in Xcode 11 you can call override class func setUp()
This approach is part of Apple documentation for UI testing:
https://developer.apple.com/documentation/xctest/xctestcase/understanding_setup_and_teardown_for_test_methods
If you build a superclass for your test case to be based on, then you can run a universal setup in the superclass and do whatever specific setup you might need to in the subclasses. I'm more familiar with Obj-C than Swift and haven't had a chance to test this yet, but this should be close.
// superclass
class SuperClass : XCTestCase {
override func setUp() {
SilentSoundEngine.activate () // SoundEngine is a singleton
}
}
// subclass
class Subclass : Superclass {
override func setUp() {
super.setup()
}
}
I am going through an application and adding Unit Tests. The application is written using storyboards and supports iOS 6.1 and above.
I have been able to test all the usual return methods with no problem. However I am currently stumped with a certain test I want to perform:
Essentially I have a method, lets call it doLogin:
- (IBAction)doLogin:(UIButton *)sender {
// Some logic here
if ( //certain criteria to meet) {
variable = x; // important variable set here
[self performSegueWithIdentifier:#"memorableWord" sender:sender];
} else {
// handler error here
}
So I want to test that either the segue is called and that the variable is set, or that the MemorableWord view controller is loaded and the variables in there are correct. The variable set here in the doLogin method is passed through to the memorableWord segues' destination view controller in the prepareForSegue method.
I have OCMock set up and working, and I am also using XCTest as my unit testing framework. Has anyone been able to product a unit test to cover such a situation??
It seems that Google and SO are pretty bare in regards to information around this area.. lots of examples on simple basic tests that are pretty irrelevant to the more complex reality of iOS testing.
You're on the right track, your test wants to check that:
When the login button is tapped doLogin is called with the loginButton as the sender
If some criteria is YES, call performSegue
So you should actually trigger the full flow from login button down to performSegue:
- (void)testLogin {
LoginViewController *loginViewController = ...;
id loginMock = [OCMockObject partialMockForObject:loginViewController];
//here the expect call has the advantage of swallowing performSegueWithIdentifier, you can use forwardToRealObject to get it to go all the way through if necessary
[[loginMock expect] performSegueWithIdentifier:#"memorableWord" sender:loginViewController.loginButton];
//you also expect this action to be called
[[loginMock expect] doLogin:loginViewController.loginButton];
//mocking out the criteria to get through the if statement can happen on the partial mock as well
BOOL doSegue = YES;
[[[loginMock expect] andReturnValue:OCMOCK_VALUE(doSegue)] criteria];
[loginViewController.loginButton sendActionsForControlEvents:UIControlEventTouchUpInside];
[loginMock verify]; [loginMock stopMocking];
}
You'll need to implement a property for "criteria" so that there is a getter you can mock using 'expect'.
Its important to realize that 'expect' will only mock out 1 call to the getter, subsequent calls will fail with "Unexpected method invoked...". You can use 'stub' to mock it out for all calls but this means it will always return the same value.
IMHO this seems to be a testing scenario which has not properly been setup.
With unit tests you should only test units (e.g. single methods) of your application. Those units should be independent from all other parts of your application. This will guarantee you that a single function is properly tested without any side effects.
BTW: OCMock is great tool to "mock out" all parts you do not want to test and therefore create side effects.
In general your test seems to be more like an integration test
IT is the phase of software testing, in which individual software modules are combined and tested as a group.
So what would I do in your case:
I would either define an integration test, where I would properly test all parts of my view and therefore indirectly test my view controllers. Have a look at a good testing framework for this kind of scenario - KIF
Or I would perform single unit tests on the methods 'doLogin' as well as the method for calculating the criteria within your if statement. All dependencies should be mocked out which means within your doLogin test, you should even mock the criteria method...
So the only way I can see for me to unit test this is using partial mocks:
- (void)testExample
{
id loginMock = [OCMockObject partialMockForObject:self.controller];
[[loginMock expect] performSegueWithIdentifier:#"memorableWord" sender:[OCMArg any]];
[loginMock performSelectorOnMainThread:#selector(loginButton:) withObject:self.controller.loginButton waitUntilDone:YES];
[loginMock verify];
}
Of course this is only an example of the test and isn't actually the test I am performing, but hopefully demonstrates the way in which I am having to test this method in my view controller. As you can see, if the performSegueWithIdentifier is not called, the verify with cause the test to fail.
Give OCMock a read, I have just bought a book from amazon about Unit Testing iOS and its really good to read. Looking to get a TDD book too.
I am aware that blocks are one of the latest feature added in ios. But I am really finding a
tough time learning it .
I have seen people doing the following
typedef void(^CallBackBlk) (NSString *);
#property(copy,nonatomic)CallBackBlk block;
and in .m class
-(void)doSomething:(CallBackBlk )cb{
self.block=cb;
}
I never understood what is the use of assigning it to cb here. Can't I simply do the following
-(void)doSomthing{
block(#"my string");
}
I am really not getting the purpose of storing the block in instance variable. Can any help
me with an example. Any help is greatly appreciated
In your doSomething method, where does block come from?
Answer that, and you'll have your reason.
Ah -- the commentary makes the question clear. Snark served a purpose (snark and too lazy to type out a real answer on my iPhone at 7AM :).
An instance variable is just a slot to put things. Nothing is in that slot to start with.
In your case, you could implement:
-(void)doSomething:(CallBackBlk )cb{
cb();
}
However, typically, a callback is used when you do something asynchronously. For example, you might do:
[myObject doSomething:^{
NSLog(#"did something");
}];
And then:
-(void)doSomething:(CallBackBlk)cb {
dispatch_async(... global concurrent queue ..., ^{
... do some work ...
cb();
});
}
That is, doSomething: will return as soon as the dispatch_async() happens. The callback block is used to callback to let you know that asynchronous operation is done.
Of course, still no need for an instance variable. Take that class that does something a bit further; make it some kind of relatively complex, state transitioning, engine. Say, like your average internet downloader or compute heavy simulation engine. At that point, lumping all your background work into a single method would be overly complex and, thus, shoving the callback block(s) (there may likely be more than one; a progress updater, a completion block and/or an error block, for example) into instance variables allow the class's implementation to be subdivided along lines of functionality more cleanly.
What is the use of storing the block in an instance variable
Perhaps to be able to access it later?
You would do that if you want to invoke the block later, after the method that assigns it has already returned.
Consider for example an object that manages a download. You might want to have a block that gets invoked when the download completes (e.g. to update the UI), but you don't want the download method to have to wait until that happens (because it might take a long time).
maybe and example of use will help..
one use for storing it as a variable i have found is if you have multiple views that all access another view (for me it was a map on the next view) i used blocks that were setup by the previous view (set the default location for the map, initialise markers and so forth) then passed it through to the next view, where it would run it, setting up the map. it was handy having the block use the local variables of the previous view to access certain attributes. it wasnt the only way to do it, but i found it was a nice clean way of going about it.
and here is an example of what gets run in the viewDidLoad of the mapview
if(setupMap){
setupMap(mapView);
}
if(gpsUpdate){
gpsUpdate(mapView);
}
if(addMarker){
addMarker(mapView);
}
now if those blocks were assigned (the if statement check if they are nil), it would run them and do the appropriate setup for the map. not every view needed to do those, so they would only pass to the map view what needed to be done. this keeps the map view very general purpose, and code gets reused a lot. write once use lots!
To use the block, you call your doSomething: method:
CallBackBlk laterBlock = ^(NSString *someString) {
NSLog(#"This code is called by SomeClass at some future time with the string %#", someString);
};
SomeClass *instance = [[SomeClass alloc] init];
[instance doSomething:laterBlock];
As you code the implementation of your class, it will presumably reach some condition or finish an action, and then call the laterBlock:
if (someCondition == YES) {
self.block("Condition is true");
}