I am running test cases in application with enabled code coverage data Xcode 7 Beta 2. But I am able to get only few files coverage data while my all test cases are running successfully.
Some files has covered all codes by unit test cases but still showing 3% code coverage.
For example:
This is the result of code coverage, as you can see on the right side, there is an info how many times these lines of code was called during tests. In this case - 0.
But...
here is a place in tests where we can see that this function was called indeed. How many times? oh... at least once. This number is delivered by info on the right side.
So the code above should be marked as called, and not be grayed out:-)
Can anyone explain this? Why does this happen?
IT WORKS.
Since Apple released #testable keyword to import your project into test target, you don't have to add your files to both target anymore:
So just remove every file from your test target:
Wherever you need access to your file from your test target just import your target using: #testable import MyApp
Do this for every file in your project.
Then code coverage will be working fine.
Read more from Swift 2 + Xcode 7: Unit Testing Access Made Easy!!!!
If you need to know how to work with code coverage read How to use code coverage in Xcode 7?
As #Gerd Castan mentioned earlier is: "So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested."
Solution is simple. Do not let compiler think that this file is included in more that one target, instead import your module using #testable keyword.
I think I found out what XCTest coverage ist doing and it makes some sense:
My setup:
class1 compiled into target1
class2 compiled into target1 and into target2
Test setup:
import XCTest
#testable import target1
class MyTests: XCTestCase {
func testSomething() {
someMethodFromClass1()
someMethodFromClass2()
}
}
What I find is that class1 (compiled into target1) shows test coverage and class2 (compiled into target1 and into target2) shows no test coverage.
So it appears to me that a tested method shows a coverage of 0 when there exists at least one target where this method is not tested.
And this makes a lot of sense, because testing a method in a target doesn't say anything about how it behaves in a different target.
Apple wants us to test all targets.
Update
One more hint to back this theory:
go to the report navigator
and click on coverage.
If you have more than one target, you see your files grouped by target.
And if you have one file in two targets, you see your file twice.
If you have one file in both targets, the code coverage of this one file is shown for both targets. And (at least in my projects) one file has different blue lines in each target:
coverage in target 1:
coverage of same file in the same project in the same test run in target 2:
If you look at your test coverage in the source editor, apple has to decide which coverage it shows to you.
I think showing the target with the lowest coverage is the best apple can do in the source editor.
simple fix for a special case:
If your only second target is your test target: don't compile into your test target and use #testable import.
For all other cases you have to test each target.
I checked at Apple developers forums for the subject and after reading through various posts I guess I came across the solution.
In order for the thing to work it is necessary to:
Remove all your application source files from the test target
In your unit-test sources put #testable import <NameOfYourModule>
Re-build and re-run tests
I tested this out with my current project, and the results are much better.
Original solution recipe can be found at: http://natashatherobot.com/swift-2-xcode-7-unit-testing-access/
Also it seems that the functionality is a bit raw-ish, hence bugs possible, and Apple suggests submitting bug reports when things do not work as expected:
I've personally seen code coverage results for some very large projects. Right now the support works best for applications and frameworks. If that's what you're testing, then it would be best if you could file a bug report at https://bugreport.apple.com so that we can investigate your particular circumstances. For that matter, a bug report would be a good thing no matter what type of project you have. If possible, it's best to actually in the project with the report. If you can't do that, describe its setup in as much detail as possible. Pictures are good.
Original thread: https://forums.developer.apple.com/message/9733#9733
Bear in mind that there are multiple ways to cover code with tests, you may test all functions, or you may be covering all instructions in the functions, but you may not be covering all the execution paths that are possible.
Or Xcode coverage stuff may be broken, but it's hard to tell if you don't give details on what kind of coverage are you expecting it to check.
This happens because .swift file of your project selected for both targets by default.
Manually select and remove test target for files works for me.
Related
I have a few UIView class which is coming under coverage even though. I haven't added them under unit test target. Neither in unit test nor UI unit test. but still those file showing.
I do not know whether they are included in test coverage.
Q: How we can remove them from unit test coverage?
Q: Are they involve in unit test coverage score?
Class included in unit test coverage:
Classes not added in test targets:
I think that your problem is related with gather code coverage for all targets.
create new target - You can duplicate existing one.
Change Host Application to brand new target.
Change your test scheme settings to new target as source of code coverage. Please have a look at screenshots:
Remember add to your test file #testable import UnitTestExampleCodeCoverage
Add to new target files which you want.
In my example proj i have: AppDelegate, ViewController, ViewController2, ViewController3, ViewController4, View, View2, View3, View4. I added to new target: AppDelegate, ViewController3, ViewController4, View3, View4 so only this five classes should be base to calculate code coverage.
Add tests to your classes.
Code coverage report contain only classes added to new target. Please remember that old target can be your main target - the new one is only kind of abstract layer to achieve your goal: "remove some classes from code coverage report".
Here is whole project: https://github.com/mStaruchowicz/SO55723759
Happy testing ;)
Hi #Sandy, to answer your questions:
How we can remove them from unit test coverage?
As of Xcode 10.2 there is no way to exclude files from the test coverage report.
Are they involve in unit test coverage score?
Yes they are. All the files contribute to the total coverage %.
If you want to compute a test coverage score that doesn't take into account certain files, your best chance is to write a script to parse the xccov output.
I can't find any official documentation of how to interact with it but this post is a good place to start.
When you active coverage in your project, all files from the target will compute the coverage.
I recommend for you to use XCOV:
https://github.com/nakiostudio/xcov
You can run the coverage and put your ignored files to improve a correct coverage.
If you want customizable UT coverage report where you can ignore the files you wish, Slather can be a best tool.
you can ignore the files from coverage calculation by configuring the files in .slather.yml file under ignore option.
Please find the details here how to ignore.
When unit tests run,
Your app is launched
Tests are injected and run
The problem is the app launch fires up your normal sequence, creating views and maybe initiating some network calls. This not only interferes with code coverage reporting — it potentially poisons your unit tests. For example, if you test notifications by having a unit test fire off a notification, you need to know that the only objects listening for that notification were created by the test, not lying around from app launch.
Another problem with ignoring coverage of certain files is: How are you going to see what your unit test coverage is of those files?
The solution is to use a different app delegate for testing that does almost nothing.
I once asked a question related to XCTests. And in one of the answers I was told that it is a common practice to use a separate test target (other than the main app) when running unit tests (at least, in iOS development). I tried to find some sources about it, but I couldn't
I understand, that it is probably a best practice, so I would really like to understand it. Could someone explain to me why is it important, what benefits do I get from it and how should I go about doing it? Links to some articles explaining the issue will be much appreciated.
P.S. I understand that I need special environment for tests (fake in-memory database, mocked networking layer, etc.), but up until now I managed to achieve it without a separate test host. But I believe that there might be a better way.
To answer your points:
Why is using a separate test target important?
Separation of concerns. A unit test target does not have the same kind of structure as a regular app target - it contains a test bundle, which references the app target under test, as well as the test classes and any helper classes required. It is not a duplicate of the app target with test code added to it - in fact it should not even contain the code under test. This means that you don't have to make any special effort to keep the test target in sync with the main app target - the fact that the test bundle loads the main app handles all that for you. (You made a comment on Richard Ross's answer to your previous q suggesting you've run into the difficulties duplication causes already).
How should I go about doing this? (checked on Xcode 7).
Assuming you are wanting to add a target to an existing project that doesn't have any tests, select the main project, and then click on the '+' beneath the list of targets in the project. You can also use the File->New->Target menu option.
You'll see a menu asking you to choose a template for your new target. Select 'Test' and within test select 'iOS Unit Testing Bundle'.
Check the fields in the next screen - they should be correct by default - but you might want to double check the 'Target to be Tested' field value is correct if you have a lot of targets in the project/workspace. Click 'OK' and you should have a functioning unit test bundle, with an example test that you should be able to run using Apple-U or Product->Test You'll still need to #import the app classes if you're using ObjC.
If you are creating a new project, all you need to do is tick the 'Include Unit Test' box when creating the project - no other steps are required.
Apple Docs (with links to relevant WWDC presentations)
Tutorials. Most of the tutorials around are a bit out of date. But not that much has changed, so just look at the docs and figure it out. The two below might be useful, otherwise just google. From a quick glance, most of them seem to assume that the project had unit tests set up at the beginning
http://www.raywenderlich.com/22590/beginning-automated-testing-with-xcode-part-12 (2012/iOS 6, but the process is still broadly the same. Also deals with Jenkins, Git and running tests from the CLI).
Unit testing in OSX - most recent post - not iOS, I know, but more up to date than most of the iOS tutorials (Oct 2015), and gives step by step instructions (including setting the unit test target in the build schemes, which probably won't be necessary in your case). Probably worth a look anyway.
I use XCode 7.2 and Swift 2.0, no extra pods for testing, just XCTestCase class. I don't have #testable import myApp and adding it doesn't fix my problem.
I have everything set up correctly to see code coverage for my Swift project (I have set it up following this post - How to use code coverage in Xcode 7? ).
And it works, when I run all tests (with Command-U). However, when I try to run a particular test class (e.g. CuriousUITests: XCTestCase) or a specific test (e.g. testMyFavouriteButton), the coverage shows as if none of the code is covered!
Why does that happen?
I want to be able to see code coverage when running just one test, so that after implementing a new test, I don't have to run all of my tests to see if it indeed does cover what I expected (It just takes too long to run them all).
Thanks for all your help!
da-na
I have recently implemented Localytics to get a better understanding of how our users are using our app.
The integration guide is pretty straight forward. However the unit tests can't be built any longer, when I run them.
The error is familiar to me. This error usually happens if the tested class is not part of the test target membership.
But the SDK _OBJC_CLASS_$_LocalyticsSession provides only a .h file. In order to make it part of the target membership, I needed the .m file, which I don't have.
Has anyone else utilised Localystics and can advice me how to proceed this regarding? Thanks
UPDATE
coneybeare's answer has actually made me try this:
The test target is set to None. But if I change it to target the app itself:
Then it works. However everytime I intend to run the unit tests the whole app has to start in the simulator, which is very irritating (and slow). Am I missing something? How else can I associate the .a code libraries with the test targets?
I don't use Localytics, but inspecing the SDK downloads shows a few .a code libraries. Ensure those are associated with the test targets.
I've found plenty of people using Xcode 3 to do this, but it no longer seems to work in Xcode 4. Or, it works partially…
I've added a preprocessor macro "TEST_TARGET" for Debug and Release under my Test target, and if I use #ifdef TEST_TARGET in the actual unit tests, that works as expected.
However, I really want to log some extra information from a source file that's just part of the main app when it's being run under unit tests (i.e. just a standard source file, not a unit test file). That source file doesn't appear to "see" the define. I've stepped through with the debugger, and the code within #ifdef is never executed.
Is there a way to tell my app is being run under a unit test target?
Here's how you can do it: you can test for something that's loaded when the tests are loaded, but not when they aren't.
For instance:
if (NSClassFromString(#"SenTest")) {
NSLog(#"Extra info when running tests");
}
You could also add categories to your classes that are only present in the test target, which might also be helpful.