When I run
xcodebuild -workspace ~/Documents/JudgecardXSwift/JudgecardXSwift.xcworkspace -scheme JudgecardXSwift -destination 'platform=iOS Simulator,name=iPhone 6' clean test
from command line, I can successfully run all of my tests.
Now I want to add a run script phase to my Xcode project so it will always run my tests each time I run the app. However, when I added the run script phase with the above xcodebuild command, my build always hangs:
and as you can see, it must be hanging on the run script phase I added, because it has just finished the build phase before it titled Run custom shell script 'Copy Pods Resources':
Is my method of adding a run script phase with the xcodebuild command the proper way to automatically run tests with every build? Why does it cause my build process to hang? Is it causing some kind of infinite loop with each xcodebuild command kicking off another through its run script phase?
Bit late here, so thought I would add my personal experience, but xcodebuild will kick off another build for you, importantly xcodebuild via command line as you have may not run your Build Phases as expected. I have only experienced silent failing copy commands, with those I could view the command in log files, but it did not actually copy the file.
So it may hit an infinite loop, more likely though is it's running through that build, hitting xcodebuild command, and then building as normal without some Build Phases executing properly and thus without running your tests.
In Xcode 9 this works. I've added a run script with:
echo "warning: 🚕🚕🚕 Started running myTests 🚕🚕🚕"
xcodebuild -sdk iphonesimulator -workspace myWorkspace.xcworkspace -scheme "myWorkspaceTests" test -destination 'platform=iOS Simulator,name=iPhone X,OS=11.2' CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NO
And if one of the tests fail, you're gonna get a build error.
Related
I am using Jenkins job for running current test cases for iOS app. using,
xcodebuild test ... | xcpretty
Although all tests are successful but Jenkins still marked build as failure.
How can I see the actual result?
Edit-1:
Even when i removed | xcpretty, still jobs marked as failure. Here are the full shell command,
xcodebuild test -workspace App.xcworkspace -scheme App -destination 'platform=iOS Simulator,name=iPhone 6, OS=10.2'
Finally the problem was solved by placing below line at the very begining of jenkins shell
#!/bin/sh
I am running the following command from command line to build/test my workspace. The idea is to port it eventually to a CI system. But what happens is that one of the runs succeed but all of the following ones fail. They fail in an odd manner in that the build completes as part of the command but then, I see the message which kicks off the simulator and I see the simulator up but the tests are never run on it. The command is either hung on it forever or it fails with no error message. I tried using xcrun simctl erase all before the command to make sure the simulator is clean. I also tried avoiding specifying derivedDatapath and OBJROOT hoping that would make the builds clean but so far, I am stuck here.
xcodebuild test -workspace <workspace-name> -scheme <scheme-name> -configuration Coverage -sdk iphonesimulator -destination platform='iOS Simulator',OS=9.0,name='iPhone 6'
The final error I see is /usr/bin/touch -c /home-folder/Library/Developer/Xcode/DerivedData/.../Build/Products/Coverage-iphonesimulator/Tests.xctest ** TEST FAILED **
The code runs smoothly from within XCode and the tests always pass. I believe this is some issue with the terminal app connecting to the simulator or the simulator being in a weird state after an earlier run but I could not figure out what could be causing the problem. Any suggestions would be great.
XCode Version: 7.0.1
I don't know if that is the issue, but shouldn't test be at the end of the line? Like this:
xcodebuild -workspace <workspace-name> -scheme <scheme-name> -configuration Coverage -sdk iphonesimulator -destination platform='iOS Simulator',OS=9.0,name='iPhone 6' test
Has anyone been able to successfully pass the KW_SPEC variable to xctool. I'm trying to run a single KIWI spec by using https://github.com/kiwi-bdd/Kiwi/wiki/Kiwi-FAQ#q-how-do-i-run-a-single-spec-describecontextit.
I can run all the tests successfully with xctool but it doesn't seem to pick up the KW_SPEC value. I've tried it in a lot of different places with the command line but no luck.
e.g.:
xctool -destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject KW_SPEC=NewAssessmentTests.m:12 test
-only SampleProject_Acceptance_Tests
Using Kiwi v2.3.1 and xctool 0.2.3
Cheers,
Mo
Update 11/03/2015
#OhadSchneider's comment made me realise that KW_SPEC worked for me because I had set it in the test configuration for my scheme (Edit scheme->Test->Arguments->Environment variables). Setting the variable from shell doesn't work as that variable applies only to the actual build, and not when executing the unit test target.
But there's a workaround to this, by modifying the test phase of your scheme and adding a KW_SPEC environment variable with the value $KW_SPEC, this will expand when running xcodebuild to the value passed to the xcodebuild command (as in my original answer). After this, xcode will will gracefully use the passed KW_SPEC variable, xctool still has the skipped teste marked as failure issue.
Original answer
If you want KW_SPEC as an environment variable to xctool (or to any *nix tool), then you have to place it before the command, otherwise it will be considered a build setting:
KW_SPEC=NewAssessmentTests.m:12 xctool
-destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject test
-only SampleProject_Acceptance_Tests
This will however lead to another problem: xctool will report as errored the tests that don't run, and will report the test as failed, even if no tests have failed. xcodebuild doesn't have this problem as it either doesn't do unit tests discovery, or ignore tests that didn't run, a thing that xctool fails to do.
Hi I'm following examples from Test Driven iOS development and I've written a few unit tests with the new Xcode 5 and XCTest. My tests fail with the Xcode GUI client, but when I use xcodebuild the status code is 0.
xcodebuild -target TemperatureConverterTests build
and
echo $?
returns 0.
The tests are suppose to fail. Is the command to run the test cases correct?
xcodebuild command has changed with xcode 5. Here is a script to run Unit tests :
Don't forget that schemes must be shared, you've got to create your Tests Scheme, Xcode 5 didn't create it for you
xcodebuild -workspace MyApp.xcworkspace -scheme myApp-Tests -destination=build -configuration Debug -sdk iphonesimulator7.0 ONLY_ACTIVE_ARCH=YES clean build test
Hope this helps :)
If you don't want (as you rarely need) a separate (and annoying) "scheme" for your testing "target"... Use a variation on #Rémy Virin's answer...
xcodebuild -scheme YourAppOrLib -target YourAppOrLibTests
As pictured below.. this allows you to run the Unit tests for the target.... without cluttering the project with non-production schemes... In this case IndexedKeyMap is my YourAppOrLib.
Question - short version:
Why won't my build phase scripts be executed when creating an IPA from the command line?
When I'm running xcodebuild to create an IPA the predefined build phase scripts does not get executed. Why is that?
Question - lengthy version:
I have a workspace with a scheme I want to create an IPA out of from command line.
This works fine except for one thing; I have two scripts in the build phases of the target that is used to put the correct app version (CFBundleShortVersionString) and the correct svn revision number (CFBundleVersion). These to scripts works fine when archiving from xcode but for some reason they do not get run when archiving from command line.
First of all why is that?
Here are the scripts that are working (if archiving form xCode)
When archiving and creating the IPA from the command line I do (the essentials)
# Building
xcodebuild ARCHS="armv7 armv7s" ONLY_ACTIVE_ARCH=NO -workspace MyWorkspace.xcworkspace/ -scheme MyScheme CONFIGURATION_BUILD_DIR=${PROJECT_BUILD_DIR} -configuration Release clean build
# Creating IPA
/usr/bin/xcrun -sdk iphoneos PackageApplication -v "${PROJECT_BUILD_DIR}/${APPLICATION_NAME}.app" -o "${IPA_OUTPUT_PATH}/${APPLICATION_NAME}.ipa"
It works and creates an IPA but none of the build phase scripts gets executed leaving both the revision number and version number untouched.
All suggestions are appreciated!
UPDATE DUE TO BDASH's ANSWER
Instead of making a clean build I make an install as
xcodebuild install ARCHS="armv7 armv7s" ONLY_ACTIVE_ARCH=NO -workspace MyWorkspace.xcworkspace/ -scheme MyScheme CONFIGURATION_BUILD_DIR=${PROJECT_BUILD_DIR} -configuration Release
The predefined script will IN FACT be executed (can be seen in the project version number) with no errors during the install. However the created IPA will have a size of ~300 bytes (instead of ~10MB) and cannot be installed on a device.
Building the app before installing it, i.e.
# Building
xcodebuild clean build ARCHS="armv7 armv7s" ONLY_ACTIVE_ARCH=NO -workspace MyWorkspace.xcworkspace/ -scheme MyScheme CONFIGURATION_BUILD_DIR=${PROJECT_BUILD_DIR} -configuration Release
# Installing
xcodebuild install ARCHS="armv7 armv7s" ONLY_ACTIVE_ARCH=NO -workspace MyWorkspace.xcworkspace/ -scheme MyScheme CONFIGURATION_BUILD_DIR=${PROJECT_BUILD_DIR} -configuration Release
and then creating the IPA will result in an IPA with executed version script and of correct size BUT it is not possible installing it on a device. Trying to put it on a device will give an error message saying
"The program "MyApp" was not installed on you iPhone device "My Device" because an unknown error has occurred."
You have "Run script only when installing" checked for at least one of the script phases. That phase won't be run when using the build action to xcodebuild, only if using the install action.