I have a project that has multiple different targets/schemes (~38 of them as of writing this question) and I am trying to get unit testing working properly across all of the different targets. I got things working with one target, and tried adding my testing target to all of the different schemes, but it looks like when tests run for each scheme they actually run on the same, original target.
Looking in the project file I see that there's a specific Host Application associated with my testing target, and in the build settings the Bundle Loader and Test Host point to that same Host Application.
Is there any way to override those values for each scheme to run the tests against the current scheme's build target? Or some other way to set up a single test target to run across multiple build targets?
If you run the tests from the command line, or from an CI tool, like Jenkins, you can instruct xcodebuild to use the build settings that you provide. A simple usage example would be:
xcodebuild -scheme SomeScheme test TEST_HOST=target
You can control almost (if not any) build setting from Xcode, like code coverage generation, build directory, derived data directory, code sign identity, etc.
You can select the scheme when you run tests with Xcode server.
Look at WWDC 2014 continues integration talk for a walk through on how to set it up
https://developer.apple.com/videos/play/wwdc2014-415/
It's using Xcode 6 but it's very similar process to Xcode 7
Also check this CI(continues integration) guideline from apple
https://developer.apple.com/library/ios/documentation/IDEs/Conceptual/xcode_guide-continuous_integration/adopt_continuous_integration.html
If anyone is wondering how to do that with UI tests (maybe it is working with unit tests, too), this is what I came up with:
First, we need to build the application which we are going to use to host our UI tests:
xcodebuild -scheme "<appSchemeName>" build -destination "<yourDestination>"
More info about destination parameter: https://mokacoding.com/blog/xcodebuild-destination-options/
Then, we need to run tests on our newly built application:
xcodebuild -scheme "<uiTestsSchemeName>" -destination "<yourDestination>" test TEST_TARGET_NAME="<yourNewlyBuiltAppTargetName>"
Destinations should match, since after the build .app is generated in your DerivedData folder, which will be used for UI tests hosting application
Related
why xcodebuild building the same application differently for scheme and target options?
xcodebuild -target uConnect build
Above commands builds with Release config in build folder inside
project.
xcodebuild -scheme uConnect build
Above builds with Debug config in Xcode’s DrivedData folder.
Wanted to understand what difference -target and -scheme options are making here?
As per my understanding it should always build the application in Release mode since we have selected Release at Project>>Info>>Configuration
use Release for command-line builds
Reference:
Screenshot of Scheme Details for reference
Try editing the scheme. You will see that it specifies the Debug configuration when building. It might look like this:
But, as you've said, when you build the target, you've specified the Release configuration.
A target describes a product that Xcode will build, like your app, the tests, an App Clip or an Extension. When using -target, it will use the configuration specified in the drop down at the bottom of your screenshot.
A scheme is a configuration that describes how a specific target will be built under different circumstances. If you go to the scheme editor, you'll see that you can choose which configuration to build when running, testing, profiling, analyzing, and archiving.
I am using xcodebuild commandline tool to generate my ios app(mainly c++ source code). The command is like
xcodebuild -project Application.xcodeproj
I want to ask whether can I separate the build phase and package phase, that is to say, first build the binary executable, then at sometime package the app later.
From Apple technical note, xcodebuild supports various build actions such as build, analyze, and archive that can be performed on your target or scheme. However, build is performed by default when no action is specified. So, when you want to package, specify action archive otherwise just leave it empty as shown in Apple Example in the document.
I am investigating how to use XCTest to close our automation test gap.
From the developer document, I only see something like this:
xcodebuild test -project MyAppProject.xcodeproj -scheme MyApp
https://developer.apple.com/library/tvos/documentation/DeveloperTools/Conceptual/testing_with_xcode/chapters/08-automation.html#//apple_ref/doc/uid/TP40014132-CH7-SW1
I would like to know if there is a way to run XCTest(UI test) against an archived build (.ipa)?
or
Can we just seperate the build and test so that we can build first and test against that build later?
Thanks
If you want to separate the build and test stages, you can use the xcodebuild flags build-for-testing (which builds a xctestrun file) and test-without-building (which runs it). You may want to build-without-testing to a generic device if you want the build-for-testing to be used on multiple devices.
Check out https://medium.com/xcblog/speed-up-ios-ci-using-test-without-building-xctestrun-and-fastlane-a982b0060676 for more details
I've set up jenkins on my local machine. Than I created a sample app with one failing and one succeeding unit test. When I issue the following command in the terminal
xcodebuild -scheme 'SampleWithTest' -sdk iphonesimulator7.0
-destination platform='iOS Simulator',OS=7.0,name='iPhone Retina (3.5-inch)' clean test
than it clean build the project, starts the simulator and makes the tests. This is how it should work.
When I run the same command via jenkins, it clean builds the project and ends with ** TEST SUCCEEDED ** without starting the simulator or print out the test results. Jenkins just fakes, that the tests was successful.
I test it on my local machine. Nothing headless, remote, source control, slave, code signing or whatever could make any trouble.
The Jenkins Xcode Plugin has not been updated to work with Xcode 5 and the XCTest framework. See this issue. Jenkins and the Xcode Plugin do not understand XCTest output.
Additionally, Xcode 5 changed how Xcode interacts with the simulator and devices when running tests. While there is something like a "headless mode" now for the simulator, access to that functionality isn't available to the Xcode plugin. This means that in some cases, it looks to the Xcode plugin as if the tests run and exit immediately, which is not accurate.
Xcode is known to be pretty tricky to setup with Jenkins, Travis or other non-Apple CI solutions.
Unit Tests support is even worse so you should try using xctool instead.
Related blog entry here.
In Xcode 5, I have a workspace with multiple projects and a scheme that should run all the test targets of the various projects. When I kick off the tests about 3 test project targets pass. So far, so good. But at the fourth test target I get an error message from Xcode telling me that the simulator can't be launched because it is already in use.
Is there any way to kill the simulator between each test target(!) in a single(!) scheme?
Or are there other solutions?
Edit: I have opened a rdar for this: http://openradar.appspot.com/15153136
Thanks!
I have come into this error whenever i try to run my functional tests using calabash. One way to do this maybe be to run a script in the after each build process to kill the simulator.
this guy has wrote a bash script that does this
http://cdrussell.blogspot.co.uk/2012/04/script-to-kill-iphone-simulator-if.html
Might be worth running this as a build phase for each test target