guys:
I'm working on an application, and building it with unit testing.
However, I'm now in a situation where I need to test asynchronous calls.
For example,
- (void)testUserInfoBecomesValidWhenUserIsBuiltSuccessfully
{
if ( ![userBuilder userInfoUpToDate] )
{
[userBuilder buildUser];
}
STAssertTrue([userBuilder userInfoUpToDate], #"User information is not valid before building the user");
}
What is the general practice for testing such things?
userInfoUpToDate is expected to be updated asynchronously.
Thanks!
William
Sometimes there is a temptation to test things which you don't usually test using Unit Testing. This basically comes from misunderstanding and desire to test everything. And then you realize you don't know how to test it with unit-testing.
You would better ask yourself - what do I test here?
Do I test that the data is not available until request completes?
Then you can write non-async version of the test which will check that the data is available after request completes.
Do I test that the response saved correctly after request?
You can also test it using flags in your logic.
You can do all logic tests without running asynchronous tests.
So at the bottom I would even ask you why do you think you need to test async call?
The unit tests supposed to run quickly - so consider it as another reason to not test async calls. Imagine continuous integration system which runs these test - it will need extra time.
And reading your comments to another answer - I think it's not common to use async in testing at all. E.g. Kent Beck in TDD book. mentioned that Concurrent Unit Testing is possible but very rare case.
So - what & why you really want to test?
Use a run loop, polling until completion or a timeout is reached:
https://codely.wordpress.com/2013/01/16/unit-testing-asynchronous-tasks-in-objective-c/
Related
I am setting up a test suite for an iOS app. I am using Xcode's XCTest framework. This is just a UI test suite. Right now I have a test target with one file TestAppUITests. All of my test cases reside in this file (first question: should all of my test cases be in here or do I need more files?)
Within this file, I have a bunch of test cases that work as if it's a user using the app. Logging in/ Create account then login -> Navigate around app -> Check if UI elements loaded -> Add additional security like secondary email address -> logout.
How should these be ordered?
I've researched all around, found some gems here and there but still have questions. Within the test target, should you have multiple files? I just have the one within my UITest target.
My other big question is a bit harder to explain. Each time we run the test suite from the beginning the app starts in a state where you are not logged in, so, for example, in order to test something like navigate WITHIN the app, I need to run that test to login first. Right now I have it setup so that the login test runs once, then all other tests after it, then ends with logout. But that one file TestAppUITests is getting very long with tons of test cases. Is this best practice?
Soo. Lets divide this into more parts:
1/ Should all of my test cases be in here or do I need more files?
Well - your tests are same as any other app code, you have. Do you have all of the app code in one file? Probably not, so a good practice is to divide your tests into more classes (I do it in the way of what they test - LoginTests class, UserProfileTests class etc etc.).
To go further with this - I have test classes and methods divided into separate files - f.e. I have a method, that will do a login in UI test, so I have the method in UITestCase+Login extension (UITestCase is a class, that is extended by all these UITestCase+Something extensions) and I have the test, that will do the login in LoginTests in which I am calling the login method from the UITestCase+Login extension.
However - you don't necessarily need more test classes - if you decide to have all of your UI tests in one class, thats your choice. Everything's gonna work, but having the tests and methods that these tests use in separate files is just a good practice for future development of tests and methods.
2/ ... Add additional security like secondary email address... How should these be ordered?
Order them into methods and call these in tests.
This is my method for expecting some UI message, when I use invalid login credentials:
func expectInvalidCredentialsErrorMessageAfterLoggingWithInvalidBIDCredentials() {
let alertText = app.alerts.staticTexts[localizedString("mobile.account.invalid_auth_credentials")]
let okButton = app.alerts.buttons[localizedString("common.ok")]
wait(
until: alertText.exists && okButton.existsAndHittable,
timeout: 15,
or: .fail(message: "Alert text/button are not visible.")
)
okButton.tap()
}
And this is my use of it in test:
func testTryToLoginWitMissingBIDAndExpectError() {
let inputs = BIDLoginInputs(
BID: "",
email: "someemail#someemail.com",
departureIATA: "xxx",
dayOfDeparture: "xx",
monthOfDeparture: "xxx",
yearOfDeparture: "xxx"
)
loginWithBIDFromProfile(inputs: inputs)
expectInvalidCredentialsErrorMessageAfterLoggingWithInvalidBIDCredentials()
}
You can see, that tests are very readable and that they (almost fully) consists of methods, which are re-usable in more tests.
3/ Within the test target, should you have multiple files?
Again - this is up to you, however having everything in one file is not really good for maintenance and future development of these tests.
4/ ... Each time we run the test suite from the beginning the app starts in a state where you are not logged in...Right now I have it setup so that the login test runs once, then all other tests after it, then ends with logout...
Not a good approach (in my humble point of view ofc.) - put the functionality into methods (yes, I repeat myself here :-) ) and divide the test cases into more files (ideally by the nature of their functionality, by "what they do").
Hope, this helped you, I struggled a lot with the same questions, when I was starting with iOS UI tests too.
Oh. And btw - my article on medium.com about advanced tactics and approaches to iOS UI tests with XCTest is coming out in a few days, I can put a link to it, once it is out - that should help you even further.
Echoing this answer, it is against best practices to store all of your tests for a complex application in a single file, and your tests should be structured so that they are independent of each other, only testing a single behaviour in each test.
It may seem counter-intuitive to split everything up into many tests which need to go through re-launching your app at the beginning of every test, but this makes your test suite more reliable and easier to debug as the number of unknowns in a test is minimised by having smaller, shorter tests. Since UI tests take a relatively long time to run, which can be frustrating, you should try to minimise the amount that you need by ensuring your app has good unit/integration test coverage.
With regards to best practices on structuring your XCTest UI tests, you should look into the Screenplay or Page Object models. The Page Object model has been around a long time, and there are many posts around it although many tend to be focused on Selenium or Java-based test frameworks. I have written posts on both the Page Object model and the Screenplay model using Swift and XCTest, they should help you.
I am writing an app that makes plenty of network requests. As usual they are
async, i.e. the call of the request method returns immediately and the result
is delivered via a delegate method or in a closure after some delay.
Now on my registration screen I sent a register request to my backend and
want to verify that the success UI is shown when the request finishes.
Which options are out there to wait for the request to finish, verify the
success UI and only after that leave the test method?
Also are there any more clever options than waiting for the request to finish?
Thanks in advance!
Trivial Approach
Apple implemented major improvements in Xcode 9 / iOS 11 that enables you to wait for the appearance of a UI element. You can use the following one-liner:
<#yourElement#>.waitForExistence(timeout: 5)
Advanced Approach
In general UI and unit tests (referred to as tests here) must run as fast as possible so the developer can run them often and does not get frustrated by the need to run a slow test suite multiple times a day. In some cases, there is the possibility that an (internal or security-related) app accesses an API that can only be accessed from certain networks / IP ranges / hosts. Also, most CI services offer pretty bad hardware and limited internet-connection speed.
For all of those reasons, it is recommended to implement tests in a way that they do no real network requests. Instead, they are run with fake data, so-called fixtures. A clever developer realizes this test suite in a way that source of the data can be switched using a simple switch like a boolean property. Additionally, when the switch is set to fetch real backend data the fixtures can be refreshed/recorded from the backend automatically. This way it is pretty easy to update the fake data and quickly detect changes of the API.
But the main advantage of this approach is speed. Your test will not make real network requests but instead run against local data what makes them independent on:
server issues
connection speed
network restrictions
This way you can run your tests very fast and thus much more often - which is a good way of writing code ("Test Driven Development").
On the other hand, you won't detect server changes immediately anymore since the fake data won't change when the backend data changes. But this is solved by simply refreshing your fixtures using the switch you have implemented because you are a smart developer which makes this issue a story you can tell your children!
But wait, I forgot something! Why this is a replacement for the trivial approach above - you ask? Simple! Since you use local data which is available immediately you also can call the completion handler immediately too. So there is no delay between doing the request and verifying your success UI. This means you don't need to wait which makes your tests even faster!
I hope this helps some of my fellows out there. If you need more guidance regarding this topic don't hesitate and reply to this post.
Cya!
I'm new to iOS, xcode, KIF framework, Objective C. And my first assignment is to write test code using KIF. It sure seems like it would be a lot easier if KIF had conditional statements.
Basically something like:
if ([tester existsViewWithAccessibilityLabel:#"Login"]) {
[self login];
}
// continue with test in a known state
When you run one test at a time KIF exits the app after the test. If you run all your tests at once, it does not exit in between tests - requiring testers be very, very careful of the state of the application (which is very time consuming and not fun).
Testing frameworks typically don't implement if conditions as they already exist in their native form.
You can look at the testing framework's source code to find how it does the "If state checks". This will teach you to fish on how to do most things you may want to do (even if is not always a good idea to do them during a functional test). You could also look here: Can I check if a view exists on the screen with KIF?
Besides, your tests should be assertive in nature as follow the following workflow:
given:
the user has X state setup
(here you write code to assertively setup the state)
It is OK and preferred to isolate your tests and setup
the "given" state (e.g. set login name in the model directly without going
through the UI) as long as you have covered that behavior in another test.
When:
The user tries to do X
(here you tap something etc..)
Then:<br>
The system should respond with Z
(here you verify the system did what you need it)
The first step in every test should be to reset the app to a known state, because it's the only way to guarantee repeatable testing. Once you start putting conditional code in the tests themselves, you are introducing unpredictability into the results.
You can always try the method tryFindingViewWithAccessibilityLabel:error: which returns true if it can find it and false otherwise.
if ([tester tryFindingViewWithAccessibilityLabel(#"login") error: nil]){
//Test things
}
I use unit test. Also I use asynchronous requests for retrive data from server. My questions is how can I test this request, cause now my tests work without waiting retriving data.
I using ASIHTTPRequest for get data from the server. There is how I check my request
- (void)testSignIn {
self.request = [[Requests alloc] init];
[self.request signInWithEmail:#"some.email#gmail.com" andPassword:#"123456"];
}
Here I have a problem. If in this method I will insert STFail befaore I get response it will complete with error in each time (because retrive data will take it some time and this is not a synchronize reuest)
How to resolve this problem?
If you really want to test network code in a unit test:
Switch to using GHUnit, which actually contains support for waiting for asynchronous events.
GHUnit Examples
But really, you shouldn't be doing this. In the above test, what are you testing? Your backend service or your Requests class?
Strictly speaking, Unit Tests should test a unit of code not being going off to the network to retrieve data.
If possible, restructure your tests to provide static test data that will test your functions. You want your tests to be as reliable and consistent as possible. By introducing the network, you are setting yourself up for inconsistent tests (when the network is down, or slow), which may lead to your tests being disabled (assuming you're working in a team)
I develop an iOS app that uses a REST API. The iOS app requests data in worker threads and stores the parsed results in core data. All views use core data to visualize the information. The REST API changes rapidly and I have no real control over the interface.
I am looking for advice how perform integration tests for the app as easy as possible. Should I test against the API or against Mock data? But how to mock GET requests properly if you can create resources with POST or modify them with PUT?
What frameworks do you use for these kind of problems? I played with Frank, which looks nice but is complicated due to rapid UI changes in the iOS app. How would you test the "API request layer" in the app? Worker threads are NSOperations in a queue - everything is build asynchronously. Any recommendations?
I would strongly advise you to mock the server. Servers go down, the behavior changes, and if a test failure implies "maybe my code still works", you have a problem on your hands, because your test doesn't tell you whether or not the code is broken, which is the whole point.
As for how to mock the server, for a unit test that does this:
first_results = list_things()
delete_first_thing()
results_after_delete = list_thing()
I have a mock data structure that looks like this:
{ list_things_request : [first_results, results_after_delete],
delete_thing_request: [delete_thing_response] }
It's keyed on your request, and the value is an array of responses for that request in the order that they were seen. Thus you can support repeatedly running the same request (like listing the things) and getting a different result. I use this format because in my situation it is possible for my API calls to run in a slightly different order than it did last time. If your tests are simpler, you might be able to get away with a simple list of request/response pairs.
I have a flag in my unit tests that indicate if I am in "record" mode (that is, talking to a real server and recording this data-structure to disk) or if I am in "playback" mode (talking to the datastructure). When I need to work with a test, I "record" the interactions with the server and then play them back.
I use the little-known SenTestCaseDidStartNotification to track which unit test is running and isolate my data files appropriately.
The other thing to keep in mind is that instability is the root of all evil. If you have code that does things with sets, or gets the current date, and such, this tends to change the requests and responses, which do not work in an offline scenario. So be careful with those.
(Since nobody stepped in yet and gave you a complete walkthrough) My humble advice: Step back a bit, take out the magic of async, regard everything as sync (api calls, parsing, persistence), and isolate each step as a consumer/producer. After all you don't wan't to unit-test NSURLConnection, or JSONKit or whatever (they should have been tested if you use them), you want to test YOUR code. Your code takes some input and produces output, non-aware of the fact that the input was in fact the output genereated in a background thread somewhere. You can do the isolated test all sync.
Can we agree on the fact that your Views don't care about how their model data was provided? If yes, well, test your View with mock objects.
Can we agree on the fact that your parser doesn't care about how the data was provided? If yes, well, test your parser with mock data.
Network layer: same applies as described above, in the end you'll get an NSDictionary of headers, and some NSData or NSString of content. I don't think you want to unit-test NSURLConnection or any 3'rd party networking api you trust (asihttp, afnetworking,...?), so in the end, what's to be tested?
You can mock up URLs, request headers and POST data for each use-case you have, and setup test cases for expected responses.
In the end, IMHO, it's all about "normalizing" out asyc.
Take a look at Nocilla
For more info, check this other answer to a similar question