I am attempting to run a UI Test in XCTestCase and when I am running it, I would like to run it multiple times with multiple different inputs. Is there a straightforward way to run a test multiple times with different inputs?
I want to run a test with different inputs but only write the test once. An example of this would be that I am trying to pass different user names in to verify behavior.
It's kind of sloppy because it executes setUp() once at the beginning unnecessarily but otherwise it gets the job done...
func testLoop() {
for parameter in myParameters {
setUp()
testToBeLooped(parameter: parameter)
tearDown()
}
}
Instead of directly running your assertions for each input in the same test I will advise to use XCTContext.runActivity to specify activity name for each parameter input:
func testToBeLooped(parameter: Int) {
print(parameter)
XCTAssertEqual(0, parameter % 2)
}
func testLoop() {
let myParameters = [1, 2, 3]
for parameter in myParameters {
XCTContext.runActivity(named: "Testing for parameter \(parameter)") { activity in
setUp()
testToBeLooped(parameter: parameter)
tearDown()
}
}
}
This allows you to visualize exactly which input parameters didn't pass assertions and which of the assertions failed for each in the test report:
Related
I am using RxSwift for reactive programming and want to test my View Model.
I have debounce like this:
In View Model:
let personNameMessage = input.personName.debounce(0.5).map { name -> String in
return "Person Name = \(name)"
}
return Driver.just(personNameMessage)
and then in Test Case:
let scheduler = TestScheduler(initialClock: 0)
let personNameInput = scheduler.createHotObservable([next(100, "John Doe")])
let personObserver = scheduler.createObserver(String.self)
output.personNameWelcome
.asObservable()
.subscribe(personObserver)
.disposed(disposeBag)
scheduler.start()
print(observer.events)
But it did not emitting the event. I've try adding scheduler.advanceBy(550) after scheduler.start() but the results are the same.
Please help.
Thank you.
Given the fact you're using debounce without specifying a scheduler, I'm assuming personName is actually a Driver - in which case, it will use DriverSharingStrategy.scheduler as its default scheduler.
By default, that leads to MainScheduler.instance, but in a test scenario, you can simply mock using the SharingScheduler class.
SharingScheduler.mock(scheduler: yourTestScheduler) {
/// make all your assertions here.
/// the `mock()` will make `DriverSharingStrategy.scheduler` return
/// your test scheduler, instead of the default one.
}
The issue here is not in debounce method itself
however, if your issue in testing the debounce method you have to change the resolution of the test scheduler to know more about scheduler resolution
But the real issue here is to test Observable works on different schedulers since debounce usually uses a different scheduler
Solution : to use SharingScheduler.make()
import RxCocoa
let personNameMessage = input.personName.debounce(0.5,scheduler: SharingScheduler.make()).map { name -> String in
return "Person Name = \(name)"
}
return Driver.just(personNameMessage)
//The test case
func testDebounce() throws {
let schedular = TestScheduler(initialClock: 0, resolution: 0.001)
SharingScheduler.mock(scheduler: schedular) {
let observer = schedular.createObserver(Double.self)
schedular.createColdObservable([.next(0, 0)]).bind(to: observer).disposed(by: bag)
schedular.start()
XCTAssertEqual(oObserver.events, [.next(700, 100)])
}
}
When you really think about it, you don't need to test whether the debounce works (of course it works, RxSwift has several tests that prove it works,) all you need to test is that it is part of the chain. For that, you can simply load the text of the .swift file and run a regex over it.
In the following code, I want to test whether "DisplayHelper.displayAlert" has been called. I am dependency-injecting DisplayHelper and AuthService mock objects, and using PromiseKit
In My ViewController:
#IBAction func submitButtonPressed(sender: AnyObject) {
AuthService.updateUser(["zipcode": self.zipcodeTextField.text!])
.then { user -> Void in
// sucess code
}.error { error -> Void in
self.DisplayHelper.displayAlert("Zipcode Not Found", message: "We can't find that zipcode... please try again.", callingViewController: self)
}
}
Here are the not-working tests:
func testSubmitButtonServerCantFindZipcode() {
vc.zipcodeTextField.text = "00000"
vc.submitButtonPressed(self)
// called AuthService (passes)
XCTAssertEqual(authServiceMock.updateUserCalled, true)
// called displayAlert (fails because tests run before DisplayAlert is called)
XCTAssertEqual(displayHelperMock.displayAlertCalled, true)
}
How do I get the tests to wait for all of the code to execute before the assertions?
When testing asynchronous code with XCTest you need to use XCTestExpectation.
You can rewrite the test code like this:
let e = expectationWithDescription("display alert")
waitForExpectationsWithTimeout(3) { error in
XCTAssertEqual(displayHelperMock.displayAlertCalled, true)
}
Now the only missing piece to make the test work is finding a place where to call expectation.fulfill(). The most appropriate place would be in your AuthService mock, after the success and failure callbacks are run.
If I can suggest you something though, writing tests that asser whether certain methods have been called is not a safe approach to testing, as you are just testing implementation rather than behaviour.
What could be a better approach is to test the two componets independently. TestAuthService making sure that both the success and failure path execute as expected, and the DisplayHelper to make sure that the alert view is actually added to the view hierarchy.
This article could be a useful place to start to find out how unit test alerts, and this post is a good read on why and how to avoid mocks.
I'm a bit confused with the new UI Unit Testing scheme that apple released in their XCode7 Beta. I think it's an awesome idea, but I have a couple questions.
this is one testing method I have...
func testMetricsProperties() {
// Used some of the metrics for testing for reference
let app = XCUIApplication()
app.scrollViews.descendantsMatchingType(.Unknown).containingType(.StaticText, identifier:"rim").childrenMatchingType(.Button).element.tap()
app.textFields["_XCUI:Secure"].typeText("")
app.typeText("\r")
app.buttons["dash metrics"].tap()
let element = app.descendantsMatchingType(.Unknown).containingType(.Image, identifier:"darkBackground.png").childrenMatchingType(.Unknown).element.childrenMatchingType(.Unknown).elementBoundByIndex(1).childrenMatchingType(.Unknown).element.childrenMatchingType(.Unknown).element
let offPlanRevenue = element.childrenMatchingType(.Unknown).elementBoundByIndex(0).staticTexts["OFF PLAN REVENUE"]
offPlanRevenue.tap()
XCTAssert(offPlanRevenue.exists);
XCTAssertEqual(offPlanRevenue.value as! String, "");
}
However, in the next testing method, it seems that I have to load the entire app again,
let app = XCUIApplication()
app.scrollViews.descendantsMatchingType(.Unknown).containingType(.StaticText, identifier:"im").childrenMatchingType(.Button).element.tap()
app.textFields["_XCUI:Secure"].typeText("")
app.typeText("\r")
app.buttons["dash metrics"].tap()
}
Is there anyway I can avoid this? This can be troublesome if i'm trying to run a full test on an entire suite.
I believe what you are looking for is using the setUp() and tearDown() methods. setUp() gets called before each test method and tearDown() gets called after each test method for a class.
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
}
override func tearDown() {
// Put teardown code here. This method is called after the invocation of each test method in the class.
super.tearDown()
}
Use these to clean up between testing methods back to the app's original state.
Just learning Test Driven Development in Swift.I created a class that is subclass of XCTestCase in the "ProjectNameTests" group.
class BasicFunctionTest: XCTestCase {
var values : [Int]?
override func setUp() {
super.setUp()
// Put setup code here. This method is called before the invocation of each test method in the class.
}
override func tearDown() {
// Put teardown code here. This method is called after the invocation of each test method in the class.
super.tearDown()
}
func testExample() {
// This is an example of a functional test case.
XCTAssert(true, "Pass")
}
func testPerformanceExample() {
// This is an example of a performance test case.
self.measureBlock() {
// Put the code you want to measure the time of here.
for i in 1...100000{
println("ok i need to know the time for execution")
}
}
}
}
I want to perform performance testing of these method
func testPerformanceExample() {
// This is an example of a performance test case.
self.measureBlock() {
// Put the code you want to measure the time of here.
for i in 1...100000{
println("ok i need to know the time for execution")
}
}
}
and want to find out the time for execution of the for loop.I run the project on going to Product -> Perform Action - > Test Without Building.
I can see the execution time as 3.50 sec but when i try to set the baseline clicking on the Stroke Area as below image:
I get warning as as
Would you like to update the baseline values for all tests on all
devices? You'll have a chance to review the changes the next time you
commit to source control.
Now when i click on Update the error is as below:
How can i set Baseline of the time for my specific method as Here
Here is my test project :
https://drive.google.com/file/d/0Bzk4QVQhnqKmUzYyclp6ZnJvekE/view?usp=sharing
Can you check whether BasicFunctionTest file Target membership includes your test target. You should be able to check this by selecting the file and navigating to File Inspector (target membership).
I'm writing a test method where I want the SUT to throw an exception when under certain conditions. The code looks like this:
- (void) testCantStartTwice
{
XCTAssertThrows([self.sut start], #"");
}
Now, all is good and the test passes. However, I have Xcode set an Exception Breakpoint for all ObjC exceptions, which is pretty useful when testing out an app in the debugger. As you now, now when I execute my test suite with ⌘U, now it stops at that test and looks like if it's failing, even though it says "Test Succeeded".
Any way of making the breakpoint not stop at that test?
Thanks and all the best
Unfortunately, it does not seem possible to have exception breakpoints ignore anything wrapped in XCTAssertThrows (or now XCTAssertThrowsError).
Workaround 1: Deal with the breakpoints or script around them
When you execute these tests, you can either do the following:
Disable the exception breakpoint.
Hit run every time it hits one of your breakpoints.
Both are not ideal as you'll be doing some manual work.
There are suggestions for how to disable breakpoints for Swift XCTAssertThrowsError assertions here. I have not tested those.
Workaround 2: Return Result<T, Error> instead
My original method looked like this:
struct Truck {
static func makeTruck(numberOfWheels: Int) throws -> Self {
if numberOfWheels < 4 {
throw TruckError.tooFewWheels
}
// ...
return .init()
}
}
// Code example:
let truck: Truck? = try? Truck.makeTruck(numberOfWheels: 1)
// Test example:
XCTAssertThrowsError(try Truck.makeTruck(numberOfWheels: 1))
To work around this problem of breakpoints, I created a duplicate internal function that is more testable (since it won't cause the breakpoint problem) by returning a Result instead of throwing an Error.
struct Truck {
static func _makeTruck(numberOfWheels: Int) -> Result<Truck, Error> {
if numberOfWheels < 4 {
return .failure(TruckError.tooFewWheels)
}
// ...
return .success(.init())
}
static func makeTruck(numberOfWheels: Int) throws -> Self {
switch _makeTruck(numberOfWheels: numberOfWheels) {
case .failure(let error):
throw error
case .success(let value):
return value
}
}
}
// Code example:
let truck: Truck? = try? Truck.makeTruck(numberOfWheels: 1)
// Test example:
XCTAssertEquals(Truck._makeTruck(numberOfWheels: 1), .error(TruckError.tooFewWheels))
Workaround 3: Return T? instead
Finally, if you have a simpler use-case, and your function's failure is quite obvious (e.g. there is only one type of error it can return), consider just returning an optional instead of needing the complexity of the Result type:
struct Truck {
static func makeTruck(numberOfWheels: Int) -> Truck? {
if numberOfWheels < 4 {
return nil
}
// ...
return .init()
}
}
// Code example:
let truck: Truck? = Truck.makeTruck(numberOfWheels: 1)
// Test example:
XCTAssertNil(Truck.makeTruck(numberOfWheels: 1))
Perhaps you could use a Test Failure Breakpoint instead of an Exception breakpoint when you are testing. The wwdc 2013 video on unit testing outlined a pretty good workflow for inspecting test failures. It essentially said:
Set a Test Failure Breakpoint which will allow you to inspect the conditions that caused the failure.
If needed, set a manual breakpoint earlier in the test and rerun so you can step through the statements leading to the failure as usual.
This isn't really a direct answer, but as far as I know, I don't think there is a way to create exceptions to the exception breakpoint. Hope it helps
I had the same issue and I was looking for solution for 2 hours. Probably we can't do anything with that.