I'm having many issues with unit testing in XCode 5 and want to verify if anybody else if having the same problems (and any possible solutions...) I'm using the Kiwi framework developing for ios7 and its a new project therefore is using XCUnit underneath. I'm on a 2011 macbook air on OSX 10.8
The first time I run a test on startup, it runs all tests, even if I only select one case/test class - it even runs all the disabled ones.
Upon adding new unit tests, there is ~1/3 chance that XCode crashes on running all tests.
Clicking on failed unit tests very rarely takes me to the failed unit tests. It acts as if the code has been deleted.
When unit tests have been fixed, errors are often left over both in the issue navigator and in the editor however test is now reporting to succeed.
Upon successful running of all unit tests Xcode often reports that Tests have failed however all show as successful in the navigator.
Unit tests quite regularly get 'stuck' and execute forever. Its then impossible to run/build anything else until have restarted xcode.
Anybody with a similar setup having the same issues? Any solutions? What a bloody mess.
It seems that Xcode 5.1 (DP) will solve your issues.
Yes I get similar problems, also using Kiwi. No idea if its Kiwi related, but certainly I experience crashes when running unit tests, perhaps about 1 in 5 runs, and more often if the program hits a breakpoint and I run the tests again with Cmd-U.
I haven't been using the 'test single' option, so can't comment there.
The errors left over problem appears to be generic to unit tests, not just Kiwi. If you delete the block that contains the left over error, wait a few secs for Xcode to recompile in the background and paste it back it, you'll get rid of it. Or exit and restart Xcode if you have a bunch of them.
Related
I recently added a UI Testing target to our existing project (which already has an iOS app target and a couple of other testing targets). Everything is working as expected and UI tests run without problems, but if a test fails the errors are never displayed in the file they occur and the Test Report shows them at <unknown>:0 while the Issue Navigator shows them at <unknown>:
It does mark the correct test as failed, but doesn't shown an error inside the editor. I looked through the build settings but couldn't find anything out of the ordinary.
Naturally, this makes debugging the tests much more annoying and harder. The usual clean/derived data/xcode restart magic hasn't done anything.
Update: After investigating further, it turns out that this is NOT a problem for assertion failures, which show up correctly.
This has been happening with xcode 9 and xcode10 now, using Quick and XCTest for unit tests they will run all the way fine but throw an exception/crash at the end. If I continue the tests will succeed.
I've spent so many hours trying to figure this out with no luck. I don't see anything helpful in the logs or stack trace. It's a big code base with 2,000+ tests so it's hard to figure out if it's just one causing it. I've tried:
Removing all files from test target, adding 1 by 1, at some point it breaks again but moving back to previous OK version now starts to fail.
deleting all async tests from both Quick and XCTest
Deleting all XCTEst cases, to see if it's an issue of both in 1 target.
I'm working on an app for iOS. I have 3 targets:
App — the app itself
AppTests — unit tests
AppUITests — UI tests
The third one is currently empty, but I intend to use it later. There are lots of tests in the second one.
Before yesterday, when I pressed Cmd+U, Xcode would happily run all tests for me (on a simulator, usually), and give me comprehensive report in the Debug area down below.
All of a sudden that changed. Now I see output from unit tests (AppTests) for less than a second, then it disappears, and a bit later the output from UI tests appears instead — which is completely useless, since there are no UI tests.
I'm not OK with this. UI tests sometimes print valuable information (not normally, but sometimes I use that). I need to see this output. And I kinda can: I see that it's still there in the log file, like DerivedData/App/Logs/Test/<long-guid-here>/Session-2016-06-16_18\:34\:44-EjeDwZ.log, along with lots and lots of other information that I don't understand. And there is another file, like Session-2016-06-16_18:34:44-Wlv6KP.log, in the same folder, with, as it seems, output from UI tests.
I want to see both, as it was before. I'm not OK with grepping through the giant log file.
What I tried:
Removing DerivedData folder entirely.
Cloning the same project from git into a new directory and starting it from there.
Reverting to an earlier version of the code, which worked fine before.
Removing ~/Library/Caches/com.apple.dt.*.
Trying tests on a different simulator.
Doesn't matter.
Another symptom is that the "Tests" panel in Xcode doesn't work anymore. It still correctly displays test suites and tests themselves, but small buttons to the right of tests, intended to run them, don't do anything. I still can run the suite by right-clicking on it and choosing Run AppTests (and I see the output in that case), but not by clicking on buttons. Also, I can run individual tests by clicking on the buttons in the code editor, to the left of test functions.
Does anybody have any idea how to fix it?
Xcode 7.3, Swift, if it matters.
Update:
After fiddling with "Edit Scheme" — specifically, changing "Debug" to "Release", running tests, and then changing back — "Run" buttons in the Tests Navigator suddenly started working. Issue with two different logs still remains, and I have no idea what to make of this.
Update 2:
"Run" buttons stopped working AGAIN, for no obvious reason.
Update 3:
OK, I figured out how to view tests output in the report instead of debug area.
I've got a large iOS project set up with OCUnit tests, some of which are imported from a dependent project, and some of which are local. When I have a failing test in the dependent project, I can click the error, and be transported to the line that's breaking. This isn't working for the local tests. It just takes me to the file, but not the breaking line.
Does anyone know if there is something special I need to do in my unit tests, or configuration of XCode, to get jumping to the broken test working?
(I'm on XCode 4.6.2)
It's not you. Xcode 4 doesn't correctly interpret relative paths like proj1/foo/../../proj2/bar/file.m for unit test failures.
I have enabled test coverage with no problem using the Generate Test Coverage Files and Instrument Program Flow with the fopen$UNIX2003 and fwrite$UNIX2003 hack.
But the problem is, when you use XCode to run tests it ends up launching the simulator which launches your app. When that happens the output for the test coverage is not truly correct because it thinks certain parts of the code are touched because they are executed when the app launches, not because a test touched them.
Is there a better way to see what code was actually touched by a test?
EDIT:
So not really an answer to my question, but a somewhat "solution" that will at least get better coverage numbers can be found here: Run logic tests in Xcode 4 without launching the simulator