My tests are XCTests and I'm using Objective-C instead of Swift.
I've seen some articles that tackle the topic, but the seem focused on older versions of Xcode, for example:
Visualizing Code Coverage with Xcode
Code Coverage Fixed for Xcode 5.1
What would be the recommended approach to get code coverage on Xcode 6? Does Apple have something built in for code coverage, maybe via Xcode CI via OSX Server?
Take a look at frankencover.it
Simple script that can be run from dev's cmd-line or CI build server.
Produces console output as well as a detailed report in HTML format. This can be viewed locally or published as an artifact by the build server.
Optionally includes a checker that will 'fail' the build if required coverage is not met. (Feedback only. This is a prompt to review coverage and look for useful tests that can be added or reduce the required amount. Either option may be correct. )
Free for both commercial and open-source projects. No hosting, sponsorship or subscription required.
Usage:
FTW, it has an easy-to-remember dogue-speak-esque command line interface:
groovy http://appsquickly.github.io/frankencover.it/with --source-dir MyProject/Source
Terminal Output:
HTML Report:
We use Xcoverage for this..Check on link below, if this helps..
Xcoverage
This is an update, From Xcode 6 Apple having in-build code coverage tool, But Have a look at coverStory it is easy to configure test locally before push.
And providing a line by line coverage.
Related
I have the sapplication developed in Polymer and unit testing with WCT. I want to generate the code coverage report for that.
I tried with the istanbul(web-component-tester-istanbul), but it is not supporting it, I am using Polymer Version 1.2.
And I tried with Karma + Grunt, but I am not able to genrate the code coverage report.
Can anybody tell me how we can generate code coverage with Karma? I search on google but not able to find WCT with Karma.
You can use the modified version for now till the original Istanbul is fixed to work with Polymer. wct-Istanbub
I am using Appium for automation test cases in iOS project. I would like to create a report of code coverage. I have gone through appium documentation as well as on internet forums, but didnt find any way.
Is there any way to get it for iOS (Appium)?
It is not possible to do a code coverage because appium doesn't test the actual source code. Appium tests only from the UI level. You might have to manually map the functional test cases and see which all functions it get called when the test is done. But certainly there is no way to figure out like UnitTests with a report to see percent wise code covered report.
I need to know if I can do Continuous Integration with XCode server. In example: set run the tests every night or when someone commit changes, and more..
I am trying to decide one iOS Ui Automation tool to integrate with my Xcode server
Thanks
There are a few problems here:
UIAutomation has no built in support in Xcode Server. I've filed bugs, I've chased down people at WWDC. Most I've ever gotten on this problem is basically "shrug." I'm not sure UIAutomation is a priority for Apple right now. So you're not going to get any official support.
As was noted, you might be able to use a trigger. The trigger won't be able to add anything to the Xcode Server report, besides possibly the error logging. But you're not going to get anything added to the nice report table.
Running on actual devices has traditionally been a problem (if you care about that.) The loading the app part has been a problem for us, but Xcode Server might be able to preload the app for you. In addition, it seems like this might have changed in the iOS 8 SDK.
There is just a lot of uncertainty around this sort of workflow. I'm hoping Apple eventually makes an announcement or adds a new tool, but the best answer I've gotten is if you want to go down this path, use UI Unit Tests. That's a shame because it requires knowledge of Obj-C or Swift, and means interacting with the app at an API level instead of an abstract level, but if you're looking for the direction Apple wants to see people go, that's it.
Edit 7/4/2015: As of WWDC 2015, there is a new UI Testing component as part of Xcode 7 that, in my experience, seems totally supported, and is promising Xcode Server support. I would very strongly recommend using that, and not using the Instruments UIAutomation tool.
With Xcode6 right around the corner they are adding some features to XCode Server specifically it looks like "Triggers" will be helpful for running iOS UIAutomation tools. Since you can run UI automation scripts from cmd line it should be possible to utilize triggers to run your scripts post builds. This along side the logic for when a bot should run will let you decide if it should be nightly or on every commit.
https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/WhatsNewXcode/Articles/xcode_6_0.html#//apple_ref/doc/uid/TP40014509-SW1
I wrote a framework around UIAutomation called Illuminator to handle tasks like nightly test runs, pull request tests, and other automated conveniences.
It provides a flexible and extensible command line that can execute any particular subset of tests that you'd like, and produces reports in JUnit format (used by Jenkins).
I am working on automating my iOS project's build process using Jenkins and xcodebuild command line tool. I am also able to test the application on simulator using UIAutomation testing scripts (instruments command line utility). But I would like to use other instruments like Time Profiler, Leaks, etc.. along with the automation to automatically test for irregular behavior.
Although I can make a custom template and run it with automation, but the problem is I can not parse the results of these other Instruments. Is there a way, for example, to know that there is a leak in my code, and trigger an email (from jenkins)?
It's possible (I've done it), but it is terribly complicated to do well. You need something to translate the Instruments results plist to something Jenkins can understand. You can use XSLT (I wouldn't recommend it), or write a script in perl/python/etc.
The Jenkins XCode plugin is an excellent tool for automating Xcode from Jenkins.
https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin
That aside, building a CI environment around UIAutomation is doable, but there are a lot of gotchas - the Jenkins user has to have the correct set of permissions to use the dev tools, has to be in the right group, and has to be logged into the UI for the simulator to run correctly from Instruments. Make sure you can get that far before you invest too much time into parsing the output of the Instruments run.
I'm just starting to try figure out how to go about continuous integration and test driven development, specifically in Flex 4.
I'd like to run FlexUnit4 tests from an ant script initiated in Hudson. I can get hudson to compile my app but how do I then go about running/seeing results of my tests?
I did get this working, based on these answers I got from over on the adobe forums:
http://forums.adobe.com/thread/504046?tstart=0
http://forums.adobe.com/thread/505090?tstart=0
We run our flash/flex tests on hudson by using the command-line AIR test runner that is running a wrapped version of our flash/flex test application and having that write out a file in the jUnit file format. Hudson can make a nice visualisation if you use that format.
If you are on a linux host you've got to remember to set up Xvfb to give AIR a 'screen', otherwise it will not work.
Google 'hudson air unittest' or for example http://www.subotnik.com/blog/?p=100 for more.
One of the developers from the FlexUnit4 team has suggested a solution using ant/maven on the adobe forums. I'm trying this out now.