I use XCode 7.2 and Swift 2.0, no extra pods for testing, just XCTestCase class. I don't have #testable import myApp and adding it doesn't fix my problem.
I have everything set up correctly to see code coverage for my Swift project (I have set it up following this post - How to use code coverage in Xcode 7? ).
And it works, when I run all tests (with Command-U). However, when I try to run a particular test class (e.g. CuriousUITests: XCTestCase) or a specific test (e.g. testMyFavouriteButton), the coverage shows as if none of the code is covered!
Why does that happen?
I want to be able to see code coverage when running just one test, so that after implementing a new test, I don't have to run all of my tests to see if it indeed does cover what I expected (It just takes too long to run them all).
Thanks for all your help!
da-na
Related
While I am running unit tests with XCTest in Swift, they run fine when code coverage is turned off. However once I try to enable code coverage, I have a failed build/test with 4 classes giving the following error message: Command failed due to signal: Segmentation fault: 11.
Here is what worked for me (as all the other suggestions did not work in my case). I was getting a segmentation fault 11 on a particular Swift class when trying to run unit tests with code coverage turned ON. It turns out that we had a ternary expression on a class's property like so:
let cellBorder : CGFloat = MyHelperClass.isIPad() ? 10.0 : 6.0
Making it a lazy var fixed the problem:
lazy var cellBorder : CGFloat = MyHelperClass.isIPad() ? 10.0 : 6.0
To be clear, the code compiled and worked fine until we tried turning on code coverage.
I also found this Open Radar and this guy's post that outlined the solution. Appears to be an Apple bug.
Without code, build settings, etc. it's hard to say for sure but one thing you should check is to make sure that you are using a #testable import flag in your unit test classes.
For instance with a project named MyApp at the top of your unit test class you would include with the following import #testable import MyApp.
You also want to check to ensure that you've followed the process for enabling coverage all the way through. That information is documented on Apple's developer portal:
Code Coverage | Apple Developer
See this bug report on similar issue. https://bugs.swift.org/plugins/servlet/mobile#issue/SR-1825
I got the same error when implementing a protocol which required an optional variable that I implemented as a lazy var.
In my case, i was building through the cli running the xcodebuild command with Release configuration and no provitionings configured, when i switched to Debug config build and tests worked just fine
Code Coverage is commonly used with tests in Xcode. I would like to use it for manually executed app. Can I do it, possibly with third-party tools?
For example: I build and launch the app on device, perform some actions with it and then look at code coverage results.
The solution was suggested by someone here: https://github.com/SlatherOrg/slather/issues/209
You could try having a XCUITest that sleeps forever, then manually use the app, and see if on termination the coverage files are generated.
I simply tried the solution and it worked perfectly for me:
class FooTests: XCTestCase {
override func setUp() {
super.setUp()
let app = XCUIApplication()
app.launch()
}
func testBalances() {
sleep(30)
XCTAssert(true)
}
}
After the test succeeded on XCTAssert(true), I could see the code coverage for the manual use cases. You can play around with the sleep(30) to match your requirement.
Probably you might have already figured it out, but it was possible before Xcode7. The way we achieved this was, to set the "Instrument Program Flow" and "Generate Test Coverage files" flags to Yes, on your project, and later add a "flush" code somewhere inside you application, to write the coverage data files. This "flushing" part actually writes the coverage data files, which can be used later by other tools such as gcovr or lcov, to get your coverage data. Once you interact with the app, either manually or through automated tests, the coverage data gets written.
However, with Xcode7, looks like the coverage data is limited to only Xcode unit tests. I am still trying to figure out, if there is any way to gather the coverage data, by interacting with the application manually, or through automated tests.
Code coverage will record which parts of your code your test have run. But you could Build some UITests which would preform some actions as you said. UI can record UI tests to repeat the actions you perform on the simulator then when you run the test it will repeat what you did. The coverage will then show which parts of the code where excited during the UITests.
Look for some info on UITesting in Xcode 7. There is a good demo in one of the developer sessions from wwdc15
https://developer.apple.com/videos/wwdc/2015/?id=406
I am running test cases in application with enabled code coverage data Xcode 7 Beta 2. But I am able to get only few files coverage data while my all test cases are running successfully.
Some files has covered all codes by unit test cases but still showing 3% code coverage.
For example:
This is the result of code coverage, as you can see on the right side, there is an info how many times these lines of code was called during tests. In this case - 0.
But...
here is a place in tests where we can see that this function was called indeed. How many times? oh... at least once. This number is delivered by info on the right side.
So the code above should be marked as called, and not be grayed out:-)
Can anyone explain this? Why does this happen?
IT WORKS.
Since Apple released #testable keyword to import your project into test target, you don't have to add your files to both target anymore:
So just remove every file from your test target:
Wherever you need access to your file from your test target just import your target using: #testable import MyApp
Do this for every file in your project.
Then code coverage will be working fine.
Read more from Swift 2 + Xcode 7: Unit Testing Access Made Easy!!!!
If you need to know how to work with code coverage read How to use code coverage in Xcode 7?
As #Gerd Castan mentioned earlier is: "So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested."
Solution is simple. Do not let compiler think that this file is included in more that one target, instead import your module using #testable keyword.
I think I found out what XCTest coverage ist doing and it makes some sense:
My setup:
class1 compiled into target1
class2 compiled into target1 and into target2
Test setup:
import XCTest
#testable import target1
class MyTests: XCTestCase {
func testSomething() {
someMethodFromClass1()
someMethodFromClass2()
}
}
What I find is that class1 (compiled into target1) shows test coverage and class2 (compiled into target1 and into target2) shows no test coverage.
So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested.
And this makes a lot of sense, because testing a method in a target doesn't say anything about how it behaves in a different target.
Apple wants us to test all targets.
Update
One more hint to back this theory:
go to the report navigator
and click on coverage.
If you have more than one target, you see your files grouped by target.
And if you have one file in two targets, you see your file twice.
If you have one file in both targets, the code coverage of this one file is shown for both targets. And (at least in my projects) one file has different blue lines in each target:
coverage in target 1:
coverage of same file in the same project in the same test run in target 2:
If you look at your test coverage in the source editor, apple has to decide which coverage it shows to you.
I think showing the target with the lowest coverage is the best apple can do in the source editor.
simple fix for a special case:
If your only second target is your test target: don't compile into your test target and use #testable import.
For all other cases you have to test each target.
I checked at Apple developers forums for the subject and after reading through various posts I guess I came across the solution.
In order for the thing to work it is necessary to:
Remove all your application source files from the test target
In your unit-test sources put #testable import <NameOfYourModule>
Re-build and re-run tests
I tested this out with my current project, and the results are much better.
Original solution recipe can be found at: http://natashatherobot.com/swift-2-xcode-7-unit-testing-access/
Also it seems that the functionality is a bit raw-ish, hence bugs possible, and Apple suggests submitting bug reports when things do not work as expected:
I've personally seen code coverage results for some very large projects. Right now the support works best for applications and frameworks. If that's what you're testing, then it would be best if you could file a bug report at https://bugreport.apple.com so that we can investigate your particular circumstances. For that matter, a bug report would be a good thing no matter what type of project you have. If possible, it's best to actually in the project with the report. If you can't do that, describe its setup in as much detail as possible. Pictures are good.
Original thread: https://forums.developer.apple.com/message/9733#9733
Bear in mind that there are multiple ways to cover code with tests, you may test all functions, or you may be covering all instructions in the functions, but you may not be covering all the execution paths that are possible.
Or Xcode coverage stuff may be broken, but it's hard to tell if you don't give details on what kind of coverage are you expecting it to check.
This happens because .swift file of your project selected for both targets by default.
Manually select and remove test target for files works for me.
I've got a large iOS project set up with OCUnit tests, some of which are imported from a dependent project, and some of which are local. When I have a failing test in the dependent project, I can click the error, and be transported to the line that's breaking. This isn't working for the local tests. It just takes me to the file, but not the breaking line.
Does anyone know if there is something special I need to do in my unit tests, or configuration of XCode, to get jumping to the broken test working?
(I'm on XCode 4.6.2)
It's not you. Xcode 4 doesn't correctly interpret relative paths like proj1/foo/../../proj2/bar/file.m for unit test failures.
I've found plenty of people using Xcode 3 to do this, but it no longer seems to work in Xcode 4. Or, it works partially…
I've added a preprocessor macro "TEST_TARGET" for Debug and Release under my Test target, and if I use #ifdef TEST_TARGET in the actual unit tests, that works as expected.
However, I really want to log some extra information from a source file that's just part of the main app when it's being run under unit tests (i.e. just a standard source file, not a unit test file). That source file doesn't appear to "see" the define. I've stepped through with the debugger, and the code within #ifdef is never executed.
Is there a way to tell my app is being run under a unit test target?
Here's how you can do it: you can test for something that's loaded when the tests are loaded, but not when they aren't.
For instance:
if (NSClassFromString(#"SenTest")) {
NSLog(#"Extra info when running tests");
}
You could also add categories to your classes that are only present in the test target, which might also be helpful.