I've noticed that my Xcode 7.1 code coverage results see to be unaffected by any UI tests I've written. I can set a break point to make sure a particular line of code is hit during a UI test, but the coverage statistics don't reflect it properly.
I read in some of the 'beta' notes for Xcode 7.0 that Code Coverage was supposed to extend to UI Testing? Did that get dropped, or is there a setting I'm missing?
You need to Turn on code coverage in the scheme editor for a target.
https://developer.apple.com/library/prerelease/tvos/documentation/ToolsLanguages/Conceptual/Xcode_Overview/CheckingCodeCoverage.html
Related
I have an iOS app in Xcode 8.2. It has a test target / scheme, for which “Gather coverage data” is checked in the scheme’s Test / Info settings. Coverage data is not gathered. I see how many times a line was iterated over in the gutter as usual, but the Report navigator’s test runs don’t indicate any coverage at all.
I’m wondering if this is because I’ve set the tests to run hostless, i.e. without needing to actually fire up my app – they’re pure logic tests.
Is this possible?
Yes a hostless XCTest target should gather code coverage data.
'iOS Unit Testing' (aka XCTest) bundles, which test a dynamic framework or something else which doesn't require the application environment to exist to run, should happily collect code coverage data and display it in Xcode. Even of the Host Application is set to None. This works either when running Xcode > Product > Test on the Scheme for the framework under test or on the Scheme for the unit tests themselves (if the test bundle is listed in the Test pane of the Scheme editor).
Your problem must be somewhere else, sorry. Its hard for me to guess what the problem is, I suggest you try making a fresh project and see if you can reproduce the problem.
I use XCode 7.2 and Swift 2.0, no extra pods for testing, just XCTestCase class. I don't have #testable import myApp and adding it doesn't fix my problem.
I have everything set up correctly to see code coverage for my Swift project (I have set it up following this post - How to use code coverage in Xcode 7? ).
And it works, when I run all tests (with Command-U). However, when I try to run a particular test class (e.g. CuriousUITests: XCTestCase) or a specific test (e.g. testMyFavouriteButton), the coverage shows as if none of the code is covered!
Why does that happen?
I want to be able to see code coverage when running just one test, so that after implementing a new test, I don't have to run all of my tests to see if it indeed does cover what I expected (It just takes too long to run them all).
Thanks for all your help!
da-na
I have enabled code coverage and everything else in my project.
When I tested my app on a simulator, The test passed.
But when I went to check the report, it says that none of my classes and methods are executed? Everything in zero and all my classes and methods have been given he red colours.
Why can't I see any code coverage for my test?
Code Coverage is commonly used with tests in Xcode. I would like to use it for manually executed app. Can I do it, possibly with third-party tools?
For example: I build and launch the app on device, perform some actions with it and then look at code coverage results.
The solution was suggested by someone here: https://github.com/SlatherOrg/slather/issues/209
You could try having a XCUITest that sleeps forever, then manually use the app, and see if on termination the coverage files are generated.
I simply tried the solution and it worked perfectly for me:
class FooTests: XCTestCase {
override func setUp() {
super.setUp()
let app = XCUIApplication()
app.launch()
}
func testBalances() {
sleep(30)
XCTAssert(true)
}
}
After the test succeeded on XCTAssert(true), I could see the code coverage for the manual use cases. You can play around with the sleep(30) to match your requirement.
Probably you might have already figured it out, but it was possible before Xcode7. The way we achieved this was, to set the "Instrument Program Flow" and "Generate Test Coverage files" flags to Yes, on your project, and later add a "flush" code somewhere inside you application, to write the coverage data files. This "flushing" part actually writes the coverage data files, which can be used later by other tools such as gcovr or lcov, to get your coverage data. Once you interact with the app, either manually or through automated tests, the coverage data gets written.
However, with Xcode7, looks like the coverage data is limited to only Xcode unit tests. I am still trying to figure out, if there is any way to gather the coverage data, by interacting with the application manually, or through automated tests.
Code coverage will record which parts of your code your test have run. But you could Build some UITests which would preform some actions as you said. UI can record UI tests to repeat the actions you perform on the simulator then when you run the test it will repeat what you did. The coverage will then show which parts of the code where excited during the UITests.
Look for some info on UITesting in Xcode 7. There is a good demo in one of the developer sessions from wwdc15
https://developer.apple.com/videos/wwdc/2015/?id=406
I am using UIAutomation to test an app, and I would like to find out my code coverage. But since javascript has no preprocessor, that means that gcov and similar are not an option. Has anyone come up with a solution for this?
For Xcode version 4.5 and newer:
Set the “Generate Test Coverage Files” build setting to Yes.
Set the “Instrument Program Flow” build setting to Yes.
This will generate code coverage files every time you run your application in the simulator and exit the application. A detailed explanation of these two steps can be found at the beginning of http://qualitycoding.org/xcode-code-coverage/.
For any unit tests, code coverage files will be generated every time you hit the test button and the tests complete. For UIAutomation, it is a little bit more tricky. You have to ensure the application exits at the conclusion of your tests. The easiest way I found to do this is to turn off multitasking. Add UIApplicationExitsOnSuspend in your MyAppName-Info.plist file and set this option to 'YES'. Run your UI automation test and at the end of it you can exit the app either by manually pressing the home button in the simulator or using the UIATarget.localTarget().deactivateAppForDuration() method.
Note: if your app has any UI Automation tests that rely on the deactivateAppForDuration() method, the tests will terminate upon running the command.
Code Coverage is only used for Unit Testing, there is no Code Coverage for UIAutomation because there is no way to tell how many elements on screen has been "touched" by UIAutoamtion