I am confused how unit testing for gestures should be done in iOS. In my application, panning a scrollview triggers an event handler that then does some stuff. When unit testing this am I supposed to:
Call the event handler directly and check to see if it does what its supposed to
Simulate the panning with some UI automation testing framework, check to see if the event handler is called, and then finally see if its the desired outcome
I would not call the eventhandler directly but instead use test frameworks doing the gestures for you.
The Apple Framework is UIAutomation - you can code your UITests via javascript just with instruments and even test your code immediately. Unfortunately it doesn't integrate well with hudson / jenkins because command line support is not really good.
What I am using in one of my projects is KIF - it works very well on you CI Server and you can code your Tests with xcode and objective-c
Related
Code Coverage is commonly used with tests in Xcode. I would like to use it for manually executed app. Can I do it, possibly with third-party tools?
For example: I build and launch the app on device, perform some actions with it and then look at code coverage results.
The solution was suggested by someone here: https://github.com/SlatherOrg/slather/issues/209
You could try having a XCUITest that sleeps forever, then manually use the app, and see if on termination the coverage files are generated.
I simply tried the solution and it worked perfectly for me:
class FooTests: XCTestCase {
override func setUp() {
super.setUp()
let app = XCUIApplication()
app.launch()
}
func testBalances() {
sleep(30)
XCTAssert(true)
}
}
After the test succeeded on XCTAssert(true), I could see the code coverage for the manual use cases. You can play around with the sleep(30) to match your requirement.
Probably you might have already figured it out, but it was possible before Xcode7. The way we achieved this was, to set the "Instrument Program Flow" and "Generate Test Coverage files" flags to Yes, on your project, and later add a "flush" code somewhere inside you application, to write the coverage data files. This "flushing" part actually writes the coverage data files, which can be used later by other tools such as gcovr or lcov, to get your coverage data. Once you interact with the app, either manually or through automated tests, the coverage data gets written.
However, with Xcode7, looks like the coverage data is limited to only Xcode unit tests. I am still trying to figure out, if there is any way to gather the coverage data, by interacting with the application manually, or through automated tests.
Code coverage will record which parts of your code your test have run. But you could Build some UITests which would preform some actions as you said. UI can record UI tests to repeat the actions you perform on the simulator then when you run the test it will repeat what you did. The coverage will then show which parts of the code where excited during the UITests.
Look for some info on UITesting in Xcode 7. There is a good demo in one of the developer sessions from wwdc15
https://developer.apple.com/videos/wwdc/2015/?id=406
So I downloaded the beta of XCode 7 and I've created some UI tests, but I can't find the functionality of how to take screenshots of my app/UI during the test.
Can someone please help?
UI Testing in Xcode automatically takes screenshots of your app after every step.
Simply go to one of the tests that has already ran (Report Navigator > choose a Test), then start expanding your tests. When you hover your mouse over the steps, you will see eye icons near each step that has a screenshot.
Here's an example... in this case, notice the eye icon at the end of the gray row. If I were to tap on it, I would see a screenshot of the app right when the button in my app was tapped (since the step is Tap the "Button" Button).
If you want to generate screenshots, you can also use snapshot, which describes how to trigger screenshots in UI tests: https://github.com/fastlane/fastlane/tree/master/snapshot#how-does-it-work
It basically rotates the device to .Unknown (Source), which triggers a snapshot without actually modifying your app's state.
Comparing the output with the generated plist file enables you to even properly name the screenshot
Facebook's ios-snapshot-test-case and KIF both run as Unit Tests, and therefore are in the same process as the app. As such, they can directly access views and use something like renderView: or snapshotViewAfterScreenUpdates. Xcode UI Testing runs in a separate process, and therefore cannot directly access the views.
UI Automation, Apple's now-deprecated Javascript UI testing library, had a function calledcaptureScreenWithName.
Unfortunately, the new Xcode UI Testing lacks any similar function in its testing library, which to me seems like a glaring omission and I encourage you to submit a Radar for it, as taking screenshots is fundamental to perceptual difference tests (which it sounds like you're trying to do). I hope (and expect) that able will address this deficiency in later Xcode updates.
In the meantime, there are more creative approaches to taking screenshots. See this stack overflow response for a workaround involving taking the screenshot in the app itself and then sending it to the test process.
I' ve created a tool that saves the tests last n screenshots and generates the JUnit tests results report, parsing TestSummaries plist file from test logs. https://github.com/nacuteodor/ProcessTestSummaries
Maybe, that helps you.
Facebook's FBSnapshotTestCase can be alternative solution:
https://github.com/facebook/ios-snapshot-test-case
I am trying to use UIAutomation for testing an iOS 8 Today extension widget. I can change the target to the extension, launch it, but then unable to do anything after that.
Has anyone had any success in using UIAutomation with the extensions?
I hope I have the right end of the stick on this one.
I think you want to create a set of UI Tests which run like you can do with a normal application.
I found that I was not able to create a specific UI Tests target to then set the target application as WidgetExtension, as you would normally.
However I was able to set the WidgetExtension target to build and run onto a device (>= iOS 14) then within my UI Tests, I was able to record some steps which would allow me to write a clearer UI Test (place the cursor inside a test function to use record, you might already know this).
The tricky bit would be keeping the WidgetExtension target up-to-date onto the testing simulator to run your automated tests.
In my iOS app, I have a webview.
In the project, there is a specific target that runs integration test for javascript and native code communication. These tests are written in javascript, and ran inside the WebView.
I can detect when javascript is done testing. What I'd like to do is exit the application, so that the xcodebuild -scheme MyJSTests would run and exit properly.
What's the best way to exit an iOS application when the tests are done?
EDIT:
I have no choice on the test framework whatsoever, as we receive the test files from another project. All I can do is implement a method that's responsible for reporting and exiting.
The tests are executed in full javascript, inside the webview. The test scripts are loaded through a custom URLProtocol. At the end of the tests, a method call reportResults() is done, that is forwarded to native code.
It is against apple's guidelines to end a program through code. Exiting a program by their guidelines should be done only by the user, otherwise it can be interpreted as an undesired action by the user.
I am using UIAutomation to test an app, and I would like to find out my code coverage. But since javascript has no preprocessor, that means that gcov and similar are not an option. Has anyone come up with a solution for this?
For Xcode version 4.5 and newer:
Set the “Generate Test Coverage Files” build setting to Yes.
Set the “Instrument Program Flow” build setting to Yes.
This will generate code coverage files every time you run your application in the simulator and exit the application. A detailed explanation of these two steps can be found at the beginning of http://qualitycoding.org/xcode-code-coverage/.
For any unit tests, code coverage files will be generated every time you hit the test button and the tests complete. For UIAutomation, it is a little bit more tricky. You have to ensure the application exits at the conclusion of your tests. The easiest way I found to do this is to turn off multitasking. Add UIApplicationExitsOnSuspend in your MyAppName-Info.plist file and set this option to 'YES'. Run your UI automation test and at the end of it you can exit the app either by manually pressing the home button in the simulator or using the UIATarget.localTarget().deactivateAppForDuration() method.
Note: if your app has any UI Automation tests that rely on the deactivateAppForDuration() method, the tests will terminate upon running the command.
Code Coverage is only used for Unit Testing, there is no Code Coverage for UIAutomation because there is no way to tell how many elements on screen has been "touched" by UIAutoamtion