I am using XCode7-UItest for testing my application.
I have added few logs in files for my UITestTarget. These test will be executed on device via my build machine's Xcode. I can view these logs on Xcode under "Show report navigator" option but I want to send these logs from build machine to other developers so that they can view those logs.
Is there a way to collect and export these UITest logs through Xcode ?
Only way I could find to do this by copy pasting the logs from XCode manually, and that is cumbersome process.
There doesn't seem to be a way to export the logs from xcode but as a workaround, In case you are running build/test with xcodebuild you can get the NSLog output in your test results for UI Tests from command line tool,
xcodebuild -workspace my.xcworkspace -scheme myscheme -configuration Debug -sdk iphonesimulator9.0 -destination 'platform=iOS Simulator,name=iPhone 6,OS=9.0' test
You can pass iOS device as destination specifier instead of simulator.
I also looked at xcode test logging for the UI tests, Xcode generates the formatted test results with some pre-defined keys as {UDID}_TestSummaries.plist under Logs/Test directory of app DerivedData.Though it won't show you the NSLog data but every Test result log that appears under Report Navigator is stored as key-val here.You can convert this .plist to Dictionary and load it into your code to export it any format you wish.
Hope this helps.
This blog test-logs is what answers it all.
Related
Here is how we are setup while facing the issue.
We have a project which has both UI and Unit tests.
Uses fastlane.
Has CI/CD implementation using Jenkins which runs on an aws ec2 instance.
Quite recently we moved to Xcode12 (I know!!! However, better late than never)
Problem statement:
After moving to xcode12, the jenkins pipeline is unable to generate the Coverage.profdata file which in turn will be used by Slather to derive code coverage metrics. Yes, did not happen when running xcode11.
We are using the xcodebuild command after pointing it to the appropriate command line tool version (in this case it is 12.4).
First we build the app using (after getting the simulator ID)
xcodebuild build-for-testing -workspace Example.xcworkspace -scheme ExampleUITests -derivedDataPath ./derivedData -destination 'platform=iOS Simulator,id=<simulator_id>'
Then we run the test using
xcodebuild test-without-building -workspace Example.xcworkspace -scheme ExampleUITests -enableCodeCoverage YES -destination 'platform=iOS Simulator,id=<simulator_id>' -derivedDataPath ./derivedData
All the UI tests run successfully but end up with the following statements when the test suite completes.
warning: <Path_to_derived_data>/Build/ProfileData/<Some_UUID>/<Filename>.profraw: Invalid instrumentation profile data (file header is corrupt)
error: No profiles could be merged.
P.S: This is not happening on my local machine. Only on the mac in the Jenkins ec2 instance.
We faced a similar issue. We were also using fastlane.
And the fix which worked for us was to ensure we delete any artifacts from the previous build. The situation you've described is quite similar to situation we faced. The first run for unit tests generates a few coverage artifacts, after which executing UI tests also generates coverage artifacts and then xcov/slather is unable to parse these artifacts.
Therefore, can you check if deleting the Logs/Test folder in derived data helps? If it doesn't try clearing derived data entirely and then check if that helps resolve the issue on CI.
See related threads on xcov
I'm currently writing UITests for an iPhone app and currently use NSLog to print when there is an unexpected situation in helper methods, this causes a false assertion at the test level so it's clear where the error occurs in the test. Then I just read the log to see what the issue was. Is there a way to print this to the test report so I can check all failures once all the tests run? Right now the only log results are the ones that the test generates automatically (such as "Tap 'myButton' Button" or "Find: Elements matching typeButton")
I'm happy to make any necessary clarifications.
First of all, you can check your
TestSummaries.plist file
which is generated in
/Users/username/Library/Developer/Xcode/DerivedData/project-name/Logs/Test/TestSummaries.plist
If you open the file in xcode, you will see all your test summaries but in little bit bored way. You have to expand all the nodes. But you can make an html report by using
xcpretty
just use this command in your terminal
xcodebuild test -project projectname -scheme schmefilename -destination 'platform=iOS Simulator,name=iPad 2' | xcpretty --report html --output outputfolderdestination/report.html
To install xcpretty, use the below command
gem install xcpretty
You can use XCUITestHTMLReport. It creates an Xcode like HTML report
You use it like this
$ xcodebuild test -workspace XCUITestHTMLReport.xcworkspace -scheme XCUITestHTMLReportSampleApp -destination 'platform=iOS Simulator,name=iPhone 7,OS=11.0' -resultBundlePath TestResults
$ ./xchtmlreport -r TestResults
Report successfully created at TestResults/index.html
i am facing a problem. I want to do a ui test for my application.
i have written six test class where my test cases are included.
I can run my test cases using
"command U" or the clicking in the run button.
But i don't want to do like this.
I want to run my test classes using code i mean using
xctest because after complete my execution, i have to do another task.
can anyone please give an example how i can run my test classes using xctest codding from a class file.
i dont want to use shell script or jenkins, i want it by using xctest framework.
Running tests with xcodebuild
The xcodebuild command-line tool drives tests just like the Xcode IDE. Run xcodebuild with the action test and specify different destinations with the -destination argument. For example, to test MyApp on the local OS X “My Mac 64 Bit,” specify that destination and architecture with this command:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=OS X,arch=x86_64'
If you have development-enabled devices plugged in, you can call them out by name or id. For example, if you have an iPod touch named “Development iPod touch” connected that you want to test your code on, you use the following command:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=iOS,name=Development iPod touch'
Tests can run in Simulator, too. Use the simulator to target different form factors, operating systems, and OS versions easily. Simulator destinations can be specified by name or id. For example:
> xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp -destination 'platform=Simulator,name=iPhone,OS=8.1'
Source
Edit your Xcode scheme.
Click the disclosure triangle next to "Test" to reveal more options.
Select "Post-actions"
Click '+' to add any actions to run after tests are complete.
As you can see, your choices are to run a script, or send a simple email. For more information on configuring actions, see https://michele.io/the-project-file-part-2/
I am investigating how to use XCTest to close our automation test gap.
From the developer document, I only see something like this:
xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp
https://developer.apple.com/library/tvos/documentation/DeveloperTools/Conceptual/testing_with_xcode/chapters/08-automation.html#//apple_ref/doc/uid/TP40014132-CH7-SW1
I would like to know if there is a way to run XCTest(UI test) against an archived build (.ipa)?
or
Can we just seperate the build and test so that we can build first and test against that build later?
Thanks
If you want to separate the build and test stages, you can use the xcodebuild flags build-for-testing (which builds a xctestrun file) and test-without-building (which runs it). You may want to build-without-testing to a generic device if you want the build-for-testing to be used on multiple devices.
Check out https://medium.com/xcblog/speed-up-ios-ci-using-test-without-building-xctestrun-and-fastlane-a982b0060676 for more details
I'm trying to build my XCode project via the terminal. I'd like use Xcode's Product > Build for > Profiling option. So far I have:
xcodebuild -project "MyGame.xcodeproj" -target "MyGameEditor - Mac" -destination 'name=My Mac 64-bit' -configuration Profile
The project builds but not for profiling. I get an error that configuration 'Profile' does not exist.
Any help is appreciated.
What you are trying to sepcify with -configuration is the build configuration, not the build action. Unless you have added extra configurations to your project, you only have "Debug" and "Release" configurations.
What Xcode does when you tell it to profile is it builds the configuration that your scheme specifies to use when profiling, launches the simulator, installs the app, then launches Instruments.
So you need to do a similar thing with two command line calls, one to xcodebuild, one to instruments.
Some helpful links that should help you figure out what you need to do:
http://blog.manbolo.com/2013/04/09/launching-uiautomation-tests-in-command-line
Can the UI Automation instrument be run from the command line?
It is also worth noting that rather than specifying a target and configuration, you should just specify a scheme which provides both and other optional additional features.