I have about 300 UI XCTest tests which i want run in every pull request.
I use some command, like it:
xcodebuild \
-project App.xcodeproj \
-scheme App \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 8,OS=15.0' \
test
But i want stopping after first failed (for save time in build server). Have any way to do it?
Update:
xcodebuild have't any flag for this.
Now i try get result stream, and check that one test is failed.
Add this lines to your script:
if test $? -eq 0
then
echo "xcodebuild succeeded"
else
echo "xcodebuild failed with " + $?
exit 1
fi
Related
Here is my travis.yml file:
osx_image: xcode10.2
language: objective-c
before_install:
- travis_wait 35; cd CalendarKitDemo; pod update
script:
- xcodebuild build -workspace CalendarKitDemo.xcworkspace -scheme "CalendarKitDemo" -sdk iphonesimulator | xcpretty
notifications:
email: false
The problem is that the build is always marked as "succeeded" even if the actual xcodebuild command exited with failure, for example, here:
"failing" job that succeeds
When I remove xcpretty, the job passes failure/success correctly, as with this example job.
How can I both use xcpretty and pass correct values to the Travis CI on the job success / failure?
Your issue is that bash by default uses the exist code from the last command, in xcpretty. So you get the exist code from xcpretty.
You can either go and set the pipefail in your environment (set -o pipefail).
pipefail will cause the script to exit with the first non-zero exit code.
E.g. in your Travis file
script:
- set -o pipefail
- xcodebuild build -workspace CalendarKitDemo.xcworkspace -scheme "CalendarKitDemo" -sdk iphonesimulator | xcpretty
If you want to be more explicit you can also get the exitcode from the first command (xcodebuild) bash exposes the exit codes of a pipeline in the PIPESTATUS array.
So e.g.
- xcodebuild build -workspace CalendarKitDemo.xcworkspace -scheme "CalendarKitDemo" -sdk iphonesimulator | xcpretty && exit ${PIPESTATUS[0]}"
I'm trying to build and test on device (iPhone) using command line:
xcodebuild \
-workspace MyApp.xcworkspace \
-scheme MyApp \
-destination 'generic/platform=iOS,id=iPhoneUUID' \
clean test
But I'm getting this error:
xcodebuild: error: Failed to build workspace MyApp with scheme MyApp.
Reason: A build only device cannot be used to run this target.
Any of you knows why I'm getting this error?
I'll really appreciate any help.
Cut off the generic part in your -destination value. Else Xcode won't try to build for your device, but for the generic build only device:
xcodebuild \
-workspace MyApp.xcworkspace \
-scheme MyApp \
-destination 'platform=iOS,id=iPhoneUUID' \
clean test
I am launching xcodebuild through the lldb debugger so I can alter its execution like:
lldb
# set debugging target
target create /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild
# launch process and stop at entry point
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test
# set a breakpoint
breakpoint set -F "+[IDELaunchParametersSnapshot launchParametersWithSchemeIdentifier:launcherIdentifier:debuggerIdentifier:launchStyle:runnableLocation:debugProcessAsUID:workingDirectory:commandLineArgs:environmentVariables:architecture:platformIdentifier:buildConfiguration:buildableProduct:deviceAppDataPackage:allowLocationSimulation:locationScenarioReference:showNonLocalizedStrings:language:region:routingCoverageFileReference:enableGPUFrameCaptureMode:enableGPUValidationMode:debugXPCServices:debugAppExtensions:internalIOSLaunchStyle:internalIOSSubstitutionApp:launchAutomaticallySubstyle:]"
break command add
po $rcx = (unsigned long)IDEDefaultLauncherIdentifier
po $r8 = (unsigned long)IDEDefaultDebuggerIdentifier
continue
DONE
# resume execution
continue
What I want to achieve is to attach xcpretty to:
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test
like
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test || xcpretty
but I can see that obviously that is not the way.
xcodebuild: error: Unknown build action '||'.
Any ideas if that is even possible and if so how ?
Try this
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test | xcpretty
instead of
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test || xcpretty
or you can go this way
process launch -s -- -sdk iphonesimulator -destination 'name=iPhone SE' test > output.txt
xcpretty < output.txt
The run-args in lldb's "process launch" do not support piping or redirection operators. If the -c option is set, lldb will get the shell to do shell expansion on the arguments. But that's all the interaction with the shell that lldb supports.
If you need to debug something that is running in a chain of execution, you will have to launch the process chain manually, and attach to the process in the chain after the fact. The way to do this is to start the attach in lldb before launching your process and use the -w option to process attach to tell lldb to attach to the process once it starts up. Then start the process. Unless your system is heavily loaded lldb will generally catch the process early in its startup, so for most debugging tasks you don't miss code you actually care about.
I have set up Travis for my repository and my script inside travis.yml contains two test commands:
xcodebuild -workspace "$WORKSPACE" -scheme "$SCHEME" -sdk "$SDK" -destination "$DESTINATION" -configuration Release ONLY_ACTIVE_ARCH=NO ENABLE_TESTABILITY=YES GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=YES GCC_GENERATE_TEST_COVERAGE_FILES=YES build test | xcpretty;
xcodebuild -workspace "$WORKSPACE" -scheme "$APP_SCHEME" -sdk "$SDK" -destination "$DESTINATION" -configuration Release ONLY_ACTIVE_ARCH=NO ENABLE_TESTABILITY=YES GCC_INSTRUMENT_PROGRAM_FLOW_ARCS=YES GCC_GENERATE_TEST_COVERAGE_FILES=YES build test | xcpretty;
I have noticed that if test suite from first command fails, but second succeeds, build is marked as successful, which is not right, as for me.
Use && instead of ; to separate your commands. That way, if the first test suite fails, the run will abort instead of running the second test suite.
I'm having some trouble with my cocoapods running Travis CI. Everything seems to install correctly, but the xcodebuild script fails
$ set -o pipefail && xcodebuild test -workspace Example/UIKitPlus.xcworkspace -scheme UIKitPlus-Example -sdk iphonesimulator ONLY_ACTIVE_ARCH=NO | xcpretty -c
xcodebuild: error: Failed to build workspace UIKitPlus with scheme UIKitPlus-Example.
Reason: The run destination iPad 2 is not valid for Testing the scheme 'UIKitPlus-Example'.
The command "set -o pipefail && xcodebuild test -workspace Example/UIKitPlus.xcworkspace -scheme UIKitPlus-Example -sdk iphonesimulator ONLY_ACTIVE_ARCH=NO | xcpretty -c" exited with 70.
https://travis-ci.org/JamieREvans/UIKitPlus/builds/54649639
I'm not sure why this is failing, because I can run my tests on the iPad 2 simulator, using that scheme, without any issues.
Is this a Travis issue or is my Travis script wrong?
I've managed to get it build by adding a destination to the xcodebuild command line, e.g. -destination "platform=iOS Simulator,name=iPhone 6"