I'm currently writing UITests for an iPhone app and currently use NSLog to print when there is an unexpected situation in helper methods, this causes a false assertion at the test level so it's clear where the error occurs in the test. Then I just read the log to see what the issue was. Is there a way to print this to the test report so I can check all failures once all the tests run? Right now the only log results are the ones that the test generates automatically (such as "Tap 'myButton' Button" or "Find: Elements matching typeButton")
I'm happy to make any necessary clarifications.
First of all, you can check your
TestSummaries.plist file
which is generated in
/Users/username/Library/Developer/Xcode/DerivedData/project-name/Logs/Test/TestSummaries.plist
If you open the file in xcode, you will see all your test summaries but in little bit bored way. You have to expand all the nodes. But you can make an html report by using
xcpretty
just use this command in your terminal
xcodebuild test -project projectname -scheme schmefilename -destination 'platform=iOS Simulator,name=iPad 2' | xcpretty --report html --output outputfolderdestination/report.html
To install xcpretty, use the below command
gem install xcpretty
You can use XCUITestHTMLReport. It creates an Xcode like HTML report
You use it like this
$ xcodebuild test -workspace XCUITestHTMLReport.xcworkspace -scheme XCUITestHTMLReportSampleApp -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.0' -resultBundlePath TestResults
$ ./xchtmlreport -r TestResults
Report successfully created at TestResults/index.html
Related
I am using Jenkins job for running current test cases for iOS app. using,
xcodebuild test ... | xcpretty
Although all tests are successful but Jenkins still marked build as failure.
How can I see the actual result?
Edit-1:
Even when i removed | xcpretty, still jobs marked as failure. Here are the full shell command,
xcodebuild test -workspace App.xcworkspace -scheme App -destination 'platform=iOS Simulator,name=iPhone 6, OS=10.2'
Finally the problem was solved by placing below line at the very begining of jenkins shell
#!/bin/sh
I am running the following command from command line to build/test my workspace. The idea is to port it eventually to a CI system. But what happens is that one of the runs succeed but all of the following ones fail. They fail in an odd manner in that the build completes as part of the command but then, I see the message which kicks off the simulator and I see the simulator up but the tests are never run on it. The command is either hung on it forever or it fails with no error message. I tried using xcrun simctl erase all before the command to make sure the simulator is clean. I also tried avoiding specifying derivedDatapath and OBJROOT hoping that would make the builds clean but so far, I am stuck here.
xcodebuild test -workspace <workspace-name> -scheme <scheme-name> -configuration Coverage -sdk iphonesimulator -destination platform='iOS Simulator',OS=9.0,name='iPhone 6'
The final error I see is /usr/bin/touch -c /home-folder/Library/Developer/Xcode/DerivedData/.../Build/Products/Coverage-iphonesimulator/Tests.xctest ** TEST FAILED **
The code runs smoothly from within XCode and the tests always pass. I believe this is some issue with the terminal app connecting to the simulator or the simulator being in a weird state after an earlier run but I could not figure out what could be causing the problem. Any suggestions would be great.
XCode Version: 7.0.1
I don't know if that is the issue, but shouldn't test be at the end of the line? Like this:
xcodebuild -workspace <workspace-name> -scheme <scheme-name> -configuration Coverage -sdk iphonesimulator -destination platform='iOS Simulator',OS=9.0,name='iPhone 6' test
Has anyone been able to successfully pass the KW_SPEC variable to xctool. I'm trying to run a single KIWI spec by using https://github.com/kiwi-bdd/Kiwi/wiki/Kiwi-FAQ#q-how-do-i-run-a-single-spec-describecontextit.
I can run all the tests successfully with xctool but it doesn't seem to pick up the KW_SPEC value. I've tried it in a lot of different places with the command line but no luck.
e.g.:
xctool -destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject KW_SPEC=NewAssessmentTests.m:12 test
-only SampleProject_Acceptance_Tests
Using Kiwi v2.3.1 and xctool 0.2.3
Cheers,
Mo
Update 11/03/2015
#OhadSchneider's comment made me realise that KW_SPEC worked for me because I had set it in the test configuration for my scheme (Edit scheme->Test->Arguments->Environment variables). Setting the variable from shell doesn't work as that variable applies only to the actual build, and not when executing the unit test target.
But there's a workaround to this, by modifying the test phase of your scheme and adding a KW_SPEC environment variable with the value $KW_SPEC, this will expand when running xcodebuild to the value passed to the xcodebuild command (as in my original answer). After this, xcode will will gracefully use the passed KW_SPEC variable, xctool still has the skipped teste marked as failure issue.
Original answer
If you want KW_SPEC as an environment variable to xctool (or to any *nix tool), then you have to place it before the command, otherwise it will be considered a build setting:
KW_SPEC=NewAssessmentTests.m:12 xctool
-destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject test
-only SampleProject_Acceptance_Tests
This will however lead to another problem: xctool will report as errored the tests that don't run, and will report the test as failed, even if no tests have failed. xcodebuild doesn't have this problem as it either doesn't do unit tests discovery, or ignore tests that didn't run, a thing that xctool fails to do.
Hi I'm following examples from Test Driven iOS development and I've written a few unit tests with the new Xcode 5 and XCTest. My tests fail with the Xcode GUI client, but when I use xcodebuild the status code is 0.
xcodebuild -target TemperatureConverterTests build
and
echo $?
returns 0.
The tests are suppose to fail. Is the command to run the test cases correct?
xcodebuild command has changed with xcode 5. Here is a script to run Unit tests :
Don't forget that schemes must be shared, you've got to create your Tests Scheme, Xcode 5 didn't create it for you
xcodebuild -workspace MyApp.xcworkspace -scheme myApp-Tests -destination=build -configuration Debug -sdk iphonesimulator7.0 ONLY_ACTIVE_ARCH=YES clean build test
Hope this helps :)
If you don't want (as you rarely need) a separate (and annoying) "scheme" for your testing "target"... Use a variation on #Rémy Virin's answer...
xcodebuild -scheme YourAppOrLib -target YourAppOrLibTests
As pictured below.. this allows you to run the Unit tests for the target.... without cluttering the project with non-production schemes... In this case IndexedKeyMap is my YourAppOrLib.
I'm currently working on a script to build my iOS projects.
I am able to run the archive command using...
xcodebuild -workspace <workspace_name> -scheme <scheme_name> 'archive'
But what is the best way to check for errors? If there is a way to do it without parsing the logs, it would be great. However, if parsing the logs is the best way, what's the best string to look for?
I tested for return codes. xcodebuild returns 0 on success, but also when there are build errors.
Thanks in advance!
Using -project parameter, you can build your project. Goto your Project.xcodeproj folder on terminal. Then type,
xcodebuild -project projectname.xcodeproj
Check this xcodebuild reference .