OCMock. Why does the second of these test pass? - ios

I am getting up to speed of mock-ing with iOS using OCMock. I wrote a simple mock-ing unit test that passes when - based on the expectation I set - it should fail.
Here is the code:
// Scientific Calculator class
#implementation ScientificCalculator
- (NSNumber *)sqrt:(NSNumber *)input {
return [NSNumber numberWithDouble:sqrt([input doubleValue])];
}
#end
// unit test
-(void) testScientificCalculatorSqrt {
NSNumber *input = [NSNumber numberWithDouble:4];
ScientificCalculator *scientificCalculator = [[ScientificCalculator alloc] init];
NSNumber *expected = [NSNumber numberWithDouble:2];
NSNumber *result = [scientificCalculator sqrt:input];
STAssertEquals([result doubleValue], [expected doubleValue], nil);
}
// unit test
-(void)testScientificCalculatorMocking {
// setup
NSNumber *input = [NSNumber numberWithDouble:44];
id scientificCalculator = [OCMockObject mockForClass:[ScientificCalculator class]];
// expect
NSNumber *expected = [NSNumber numberWithDouble:2];
[[[scientificCalculator expect] andReturn:expected] sqrt:input];
// test
NSNumber *result = [scientificCalculator sqrt:input];
// verify
STAssertEquals([result doubleValue], [expected doubleValue], nil);
[scientificCalculator verify];
}
The testScientificCalculatorMocking unit test uses 44 as input to sqrt. I set the expected values to 2. Both STAssertEquals and [scientificCalculator verify] pass successfully.
Can someone please explain what I am doing wrong here?
Thanks,
Doug

You've told the mock to return 2 (expected) when sqrt is called with 44 (input). The result variable will therefore contain the value 2. You're then comparing result (2) and expected (2), which will pass.

Your first test is the correct way to test something like this. There's no reason to mock your calculator, because its behavior is what you're testing.
Generally you mock things that are orthogonal to what you're testing and may have side effects if you execute them in your test. For example, say you had a method that calculated the square root of Apple's stock price. You might mock the web service call to retrieve the stock price, so your test doesn't make network calls and the price is static in your test. Then you'd pass the pre-defined price to your calculator and verify that the square root value it gives you is correct.
As to stub vs. expect, expect means this method must be called once. stub means it may or may not be called, any number of times. In both cases, the andReturn part just defines what will be returned when the method is invoked on your mock. Remember, a mock is dumb, so you need to tell it what to return.

Related

Objective C - Best practice for Unit Testing with multiple test inputs

I'm writing unit test codes for an existing project. The project is in Objective-C and I have to test few functions with a number of inputs to the test cases. For example, I have a test case to test a function calculator where two parameters are inputted. Currently I create array to store the set of input values to run the test. The code used are as follows:
- (void)setUp {
[super setUp];
self.vcToTest = [[BodyMassIndexVC alloc] init];
input1 = [[NSMutableArray alloc] initWithObjects:#"193", #"192", #"192", #"165", #"155", #"154", nil];
input2 = [[NSMutableArray alloc] initWithObjects:#"37", #"37", #"36", #"80",#"120", #"120", nil];
}
- (void)testCalculatorSuccess {
for (int i=0; i<input1.count; i++) {
NSArray *expectedResult = [[NSArray alloc] initWithObjects: #"9.93", #"10.04", #"9.77", #"29.38", #"49.95", #"50.60", nil];
NSString *actualResult = [self.vcToTest calculateResult:input1[i] andInput2:input2[i]];
XCTAssertEqualObjects(actualResult, expectedResult[i]);
}
}
I searched for best practices online but was not able to find any. Can someone help me with this? Am I running the test in the right way? What is the best practice to be followed in such cases? Should I create a test case for every set of input?
In my experience, the key consideration should be how easy it will be to maintain the test suite over time. Your current approach will cause problems down the road in two ways:
If you want to use different numbers for you BMI calculation you need to use a different test class (because you're locking in the values in the setup method).
If you ever decide to use different math or multiple BMI equations you'll have to update the arrays anywhere you're checking those values.
What I would suggest is to instead create a CSV or plain text file that has the height, weight, and expected BMI values in it. That keeps the test data together. Then in your test methods, load the file and check your actual BMI against the expected BMI.
You have the flexibility here to mix and match test data, or use different test files for different BMI equations. Personally, I also like the fact that you can keep old data files around as you change things, in case you ever want to rollback or add legacy algorithm support.
A quick and dirty version would look something like this:
- (NSArray *)dataFromFileNamed:(NSString *)filename {
NSString *path = [[NSBundle bundleForClass:[self class]] pathForResource:filename ofType:nil];
// load the data however its formatted
// return the data as an array
return loadedData;
}
- (void)testBMIFormulaInCmAndKgSuccess {
NSArray *testData = [self dataFromFileNamed:#"BMI_data_01.txt"];
for (int i=0; i < testData.count; i++) {
NSArray *dataSet = testData[i];
CGFloat height = dataSet[0];
CGFloat weight = dataSet[1];
CGFloat expectedBMI = dataSet[2];
NSString *actualResult = [self.vcToTest calculateBMIforHeight:height andWeight:weight];
XCTAssertEqual(actualResult, expectedBMI);
}
}
Your test class should be targeting one specific thing to test which is the sut (system under test). In your case the sut variable should be your vcToTest. The way I would go about testing is by dependency injection rather than storing all the arrays you are testing as instance variables. So you would create a test method that takes in the parameters that you want to test. This way you don't need to keep creating instance variables, you only create local variables that are relevant to the test method.
- (void)setUp {
[super setUp];
// Put setup code here. This method is called before the invocation of each test method in the class.
self.sut = [[BodyMassIndexVC alloc] init];
}
- (void)tearDown {
// Put teardown code here. This method is called after the invocation of each test method in the class.
self.sut = nil;
[super tearDown];
}
- (void)testBMIFormulaInCmAndKgSuccess {
// local variables that are only seen in this method
NSArray *heightsInMetres = [[NSMutableArray alloc] initWithObjects:#"193", #"192", #"192", #"165", #"155", #"154", nil];
NSArray *weightsInKg = [[NSMutableArray alloc] initWithObjects:#"37", #"37", #"36", #"80",#"120", #"120", nil];
NSArray *expectedResults = [[NSArray alloc] initWithObjects: #"9.93", #"10.04", #"9.77", #"29.38", #"49.95", #"50.60", nil];
for (int i=0; i<heightsInMetres.count; i++) {
[self xTestHeightInMeters:heightsInMetres[i] weightInKg:weightsInKg[i] expecting:expectedResults[i]];
}
}
// the variables that are used to test are injected into this method
- (void)xTestHeightInMeters:(NSString *)height weightInKg:(NSString *)weight expecting:(NSString *)expectedResult {
NSString *result = [self.sut calculateBMIforHeight:height andWeight:weight];
XCTAssertEqual(result, expectedResult);
}
If I was you I wouldn't create arrays to run the tests. Arrays are messy and become hard to understand what is going on, and easy to make mistakes. I would create specific test methods that test one thing to make sure the sut is working properly. For example in TDD you create a test method that will fail, then modify your sut to fix this failure in the most simple way. Usually this means your fix will just return exactly what you are expecting. Then you make another test that tests the exact same thing with a different value, it should now fail because your sut is simply returning what they previous test was looking for. Now you modify your sut again to make both tests pass. After this in most situations you won't need any addition tests since it has proven to work in two unique ways.
I know you said you are testing software that was written already, but I strongly recommend you check out Test Driven Development. Even if you don't actually apply TDD, you will learn how to create meaningful tests. This helped me learn tdd
It's usually best to avoid for-loops in unit test code. This rule usually leads to separate assertions.
But in your case, you want to exercise a function with various inputs. Your approach is not bad at all. We can simplify the arrays by using literals:
NSArray<NSString *> *heightsInMetres = #[ #"193", #"192", #"192", #"165", #"155", #"154"];
NSArray<NSString *> *weightsInKg = #[ #"37", #"37", #"36", #"80", #"120", #"120"];
NSArray<NSString *> *expectedResults = #[#"9.93", #"10.04", #"9.77", #"29.38", #"49.95", #"50.60"];
I also normally avoid funny formatting. But aligning the columns helps us see the values in a table-like format.
Finally, I wouldn't put these values in setUp unless they're used across multiple tests.
If you need many tests like this, it may be worth exploring a test format that uses actual tables, like Fitnesse. Table data could be in spreadsheets or in wiki format, driving tests.

Ordering unit test using XCTest in Xcode 6

I would like to know if there is any way to tell Xcode to run unit tests in a specified order. I mean not in a same XCTestCase class file, but between all the class file.
For example I want to run the SitchozrSDKSessionTest before running SitchozrSDKMessageTest.
I looked over few threads on stack or on Apple documentation and I haven't found something helpful.
Thanks for your help.
It's all sorted alphabetically (classes and methods). So when You need some tests running last, just change the name of Class or Method (for (nasty) example by prefixing 'z_').
So...You have Class names:
MyAppTest1
-testMethodA
-testMethodB
MyAppTest2
-testMethodC
-testMethodD
and they run in this order. If you need to run MyAppTest1 as second, just rename so it's name is alphabetically after MyAppTest2 (z_MyAppTest1) and it will run (same for method):
MyAppTest2
-a_testMethodD
-testMethodC
z_MyAppTest1
-testMethodA
-testMethodB
Also please take the naming as example :)
It is true that currently (as of Xcode 7), XCTestCase methods are run in alphabetical order, so you can force an order for them by naming them cleverly. However, this is an implementation detail and seems like it could change.
A (hopefully) less fragile way to do this is to override +[XCTestCase testInvocations] and return your own NSInvocation objects in the order you want the tests run.
Something like:
+ (NSArray *)testInvocations
{
NSArray *selectorStrings = #[#"testFirstThing",
#"testSecondThing",
#"testAnotherThing",
#"testLastThing"];
NSMutableArray *result = [NSMutableArray array];
for (NSString *selectorString in selectorStrings) {
SEL selector = NSSelectorFromString(selectorString);
NSMethodSignature *methodSignature = [self instanceMethodSignatureForSelector:selector];
NSInvocation *invocation = [NSInvocation invocationWithMethodSignature:methodSignature];
invocation.selector = selector;
[result addObject:invocation];
}
return result;
}
Of course, the downside here is that you have to manually add each test method to this instead of them being picked up automatically. There are a few ways to improve this situation. If you only need to order some of your test methods, not all of them, in your override of +testInvocations, you could call through to super, filter out those methods that you've manually ordered, then tack the rest on to the end of the array you return. If you need to order all the test methods, you could still get the result of calling through to super and verify that all of the automatically picked up methods are covered by your manually created, ordered result. If not, you could assert, causing a failure if you've forgotten to add any methods.
I'm leaving aside the discussion of whether it's "correct" to write tests that must run in a certain order. I think there are rare scenarios where that makes sense, but others may disagree.
its ordered by function names letter orders, doesn't matter how you order it in your code.
e.g.:
-(void)testCFun(){};
-(void)testB2Fun(){};
-(void)testB1Fun(){};
the actual execute order is :
testB1Fun called
testB2Fun called
testCFun called
In addition to andrew-madsen's answer:
I created a 'smart' testInvocations class method which searches for selectors starting with "test" and sorts them alphabetically.
So you don't have to maintain an NSArray of selector names separately.
#import <objc/runtime.h>
+ (NSArray <NSInvocation *> *)testInvocations
{
// Get the selectors of this class
unsigned int mc = 0;
Method *mlist = class_copyMethodList(self.class, &mc);
NSMutableArray *selectorNames = [NSMutableArray array];
for (int i = 0; i < mc; i++) {
NSString *name = [NSString stringWithFormat:#"%s", sel_getName(method_getName(mlist[i]))];
if (name.length > 4
&& [[name substringToIndex:4] isEqualToString:#"test"]) {
[selectorNames addObject:name];
}
}
// Sort them alphabetically
[selectorNames sortUsingComparator:^NSComparisonResult(NSString * _Nonnull sel1, NSString * _Nonnull sel2) {
return [sel1 compare:sel2];
}];
// Build the NSArray with NSInvocations
NSMutableArray *result = [NSMutableArray array];
for (NSString *selectorString in selectorNames) {
SEL selector = NSSelectorFromString(selectorString);
NSMethodSignature *methodSignature = [self instanceMethodSignatureForSelector:selector];
NSInvocation *invocation = [NSInvocation invocationWithMethodSignature:methodSignature];
invocation.selector = selector;
[result addObject:invocation];
}
return result;
}
Footnote: It's considered bad practice making your unit tests depend on eachother
For you exemple you can just rename the tests files on your project like this :
SitchozrSDKSessionTest -> t001_SitchozrSDKSessionTest
SitchozrSDKMessageTest -> t002_SitchozrSDKMessageTest
Xcode treat the files using alphabetic order.
Since Xcode 7, XCTestCase subclasses are alphabetically ordered. If you want to ensure that the test methods themselves are also run in alphabetical order, you must override the testInvocations class method and return the invocations sorted by selector.
#interface MyTestCase : XCTestCase
#end
#implementation MyTestCase
+ (NSArray<NSInvocation *> *) testInvocations
{
return [[super testInvocations] sortedArrayUsingComparator:^NSComparisonResult(NSInvocation *invocation1, NSInvocation *invocation2) {
return [NSStringFromSelector(invocation1.selector) compare:NSStringFromSelector(invocation2.selector)];
}];
}
- (void) test_01
{
}
- (void) test_02
{
}
#end
The OP did not mention whether CI is available or whether a command line answer would be sufficient. In those cases, I have enjoyed using xctool [1]. It has syntax to run tests down to just a single test method:
path/to/xctool.sh \
-workspace YourWorkspace.xcworkspace \
-scheme YourScheme \
test -only SomeTestTarget:SomeTestClass/testSomeMethod
I will do this to ensure we test the items we think are easiest first.
1.[] facebook/xctool: A replacement for Apple's xcodebuild that makes it easier to build and test iOS or OSX apps. ; ; https://github.com/facebook/xctool

Dot syntax and square brackets

I am doing a tuturial on Lynda.com for objective-c, and ran accross this example code. This is a part of the ViewController.m file. The idea behind the exercise was to create a picker object with custom elements in it.
The following code works just fine and gives me a picker with "happy" and "sad" as the options:
#implementation ViewController
-(NSInteger)numberOfComponentsInPickerView:(UIPickerView *)pickerView{
return 1;
}
-(NSInteger)pickerView:(UIPickerView *)pickerView numberOfRowsInComponent:(NSInteger)component{
return [[self moods] count];
}
-(NSString *)pickerView:(UIPickerView *)pickerView titleForRow:(NSInteger)row forComponent:(NSInteger)component{
return self.moods[row];
}
- (void)viewDidLoad
{
[super viewDidLoad];
// Do any additional setup after loading the view, typically from a nib.
self.moods = #[#"happy",#"sad"];
}
However, I prefer square brackets to dot syntax and, as you can see I experimented in a few different places with it. Thereturn [[self moods] count was written as return [self.moods count] in the tutorial, but I wanted to use square brackets to verify that it still worked and I understood what was going on, so I changed it and it worked just fine. HOWEVER, I have been trying to do the same thing with the self.moods = #[#"happy",#"sad"]; because I don't like how it looks. I tried:
[[self moods] initWithObjects: #"happy",#"sad", nil];
But I just got a blank picker and a warning "expression result unused". I tried putting _moods = before that expression, and still got a blank picker. What is wrong here?
The reason that [[self moods] initWithObjects: #"happy",#"sad", nil]; is not doing what you expect is due to a misunderstanding in what is happening with regards to dot syntax and how it relates to message sending using square brackets.
Dot syntax is the "syntactic sugar" and recommended way of accessing properties of classes, such as the mood property from your question. Dot syntax is simply a shorthand for accessors (setters / getters) in Objective-C. A quick example might help clear this up.
When dot syntax finds itself on the right hand side of an assignment operator OR as the receiver of a message, the getter method is invoked.
// These two are equivalent
NSArray *allMoods = self.moods
NSArray *allMoods = [self moods]
// These two are equivalent
NSUInteger count = [self.moods count];
NSUInteger count = [[self moods] count];
When dot syntax finds itself on the left hand side of an assignment operator, the setter method is invoked.
// These two are equivalent
self.moods = #[#"happy", #"sad"];
[self setMoods:#[#"happy", #"sad"];
Using dot syntax is not only a nice shorthand, it makes your intentions clearer and newcomers to your code immediately aware that moods is a property of your class.
Also, the reason that [[self moods] initWithObjects: #"happy",#"sad", nil]; is not valid is because -initWithObjects: is an initializer of NSArray that should be called immediately following +alloc. In the piece of code above, [self moods] is returning an NSArray that already exists or lazily instantiating one. For completeness, -initWithObjects should be used as follows:
NSArray *myArray = [[NSArray alloc] initWithObjects:#"happy", #"sad", nil];
I assume you declared #property (strong, nonatomic) NSArray *moods; in the interface since self.moods works.
Setter and getter methods setMoods and getMoods are created automatically.
Here's how the dot syntax boils down to
// These are equivalent
self.moods = #[#"happy",#"sad"];
[self setMoods:#[#"happy",#"sad"]]; // Literal declaration
[self setMoods:[[NSArray alloc] initWithObjects:#"happy",#"sad",nil]]; // Full declaration
This works because you were using the "literal" way of declaring an NSArray* which includes both "allocation" and "initialization".
- (instancetype)initWithObjects: is an instance method which should be called on an instance variable already allocated with alloc. You tried to initialize a variable which has never been allocated in memory.
An slightly cleaner alternative would be:
[self setMoods:[NSArray arrayWithObjects:#"happy",#"sad",nil]];
arrayWithObjects: include both allocation and initialization.
the [self moods] way of referencing it can only be used on the right hand side of an expression, it's calling the getter for the property. self.moods = ... is actually syntactic sugar for [self setMoods:...]
so try [self setMoods:#[#"happy",#"sad"]]
You'll want to read up on the #property declaration and how it "synthesizes" getter and setter methods. What you want to do is "set" the moods property:
[self setMoods: #[#"happy",#"sad"]];

How can I unit test a method that depends on a volatile variable in Objective-C?

Here's a simplified version of my class:
#interface RTMovieBuilder : NSObject
#property (atomic, getter = isCancelled) volatile BOOL cancelled;
#property (nonatomic, weak) id<BuilderDelegate>delegate;
- (void)moviesFromJSON:(id)JSON;
- (Movie *)movieFromDictionary:(NSDictionary *)dict;
- (void)cancel;
#end
#implementation RTMovieBuilder
- (void)moviesFromJSON:(id)JSON
{
// Check for errors -> If good, then do...
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
[self syncrouslyCreateMoviesFromJSON:JSON];
});
}
- (void)syncrouslyCreateMoviesFromJSON:(id)JSON
{
NSMutableArray *movies = [NSMutableArray array];
for (NSDictionary *dict in JSON)
{
if ([self isCancelled])
return;
else
[movies addObject:[self movieFromDictionary:dict]];
}
[self notifyDelegateCreatedObjects:movies];
}
- (Movie *)movieFromDictionary:(NSDictionary *)dict
{
Movie *movie = [[Movie alloc] init];
// Set movie properties based on dictionary...
return movie;
}
- (void)cancel
{
[self setCancelled:YES];
}
// ... Other methods omitted for brevity's sake
#end
The property cancelled is atomic and volatile because it may be accessed by other threads (i.e. the main thread may call cancel method to stop the operation). (I believe these are needed, if not, please note why it's not in your answer.)
I am trying to write unit tests to make sure this will work before writing the view controller class.
How can I write a unit test that will simulate a call to cancel while RTMovieBuilder is in the middle of creating movies?
Edit
Here's a unit test I have already written which tests to make sure that notifyDelegateCreatedObjects: isn't called if cancel is called first.
- (void)testIfCancelledDoesntNotifyDelegateOfSuccess
{
// given
RTMovieBuilder *builder = [[RTMovieBuilder alloc] init];
builder.delegate = mockProtocol(#protocol(BuilderDelegate));
// when
[builder cancel];
[builder notifyDelegateCreatedObjects:#[]];
// then
[verifyCount(builder.delegate, never()) builder:builder createdObjects:anything()];
}
I'm using OCHamcrest and OCMockito. This test passes.
I would avoid trying to simulate thread timing in unit tests and focus more on figuring out what all the possible end states could be regardless of where the timing falls, and write tests for code under those conditions. This avoids endless complexity in your tests, as bbum points out as well.
In your case it seems the condition you need to be testing for is if the call to notifyDelegateCreatedObjects happens after the action is canceled, because the cancel came too late. So instead just unit test the handling of that scenario downstream in your notifyDelegateCreatedObjects method, or whatever class is being notified of that aborted event because of the thread timing.
I know this is not a specific answer to your question but I think its a better approach to achieve the same unit testing goal.
There is no reason to use volatile if your property is atomic and you always go through the setter/getter.
As well, this is a bit of re-inventing the wheel, as noted in the comments.
In general trying to unit test cancellation with any hope of full coverage is very hard because you can't really effectively test all possible timing interactions.

iphone nslog corrupted data

I am having a weird problem with variable values. This is the code (it is part of a class method):
MyAppDelegate *pDelegate = [[UIApplication sharedApplication] delegate];
SomeDictionaryData *appData = [pDelegate.theData retain];
NSLog(#"my instance var: %#",cardIndex); // outputs "my instance var: 4"
NSDictionary *currentCard = [[NSDictionary alloc] initWithDictionary:[appData.cards objectAtIndex:cardIndex]];;
// the above line breaks the app
[currentCard release];
[appData release];
I am using the debugger with the objc_exception_throw breakpoint. The input received by objectAtIndex in there shows as having value = 13760640. The cards attribute of appData is an NSArray and it clearly does not have ten million + items so I get an out of bounds error. I tried casting with (int)cardIndex with no better results. Weird thing is a similar code in some other class works fine.
This is some data I want to use throughout my app so I have a Model class that gets initialized in the AppDelegate as theData and then is accesed by different ViewControllers. This error shows up after one successful access on some other ViewController (that one also does retain/release).
Any help will be appreciated.
Use [cardIndex unsignedIntValue] for the objectAtIndex: line.
You cannot give objectAtIndex: a pointer because it expects an unsigned integer.
For example:
NSDictionary *currentCard = [[NSDictionary alloc] initWithDictionary:[appData.cards objectAtIndex:[cardIndex unsignedIntValue]]];
EDIT:
It sounds like cardIndex is an int but somewhere along the lines it is being set as an NSNumber instance. As a hack, use [(id)cardIndex unsignedIntValue]. If this works, then it means you are using the wrong type for cardIndex (it should be NSNumber, not int).

Resources