We have a Scheme (say PhoneRegression) with a couple of test targets in it (e.g. PhoneSearchTests, PhoneCheckoutTests) each with their own classes.
In one of our Jenkins jobs we only want to run the PhoneCheckoutFormValidationTests in PhoneCheckoutTests. Our configuration is the following:
xctool -scheme PhoneRegression \
-destination "platform=iOS Simulator,name=iPhone 6,OS=latest" \
-reporter junit:build/junit.xml -reporter plain \
test -only "PhoneCheckoutTests:PhoneCheckoutFormValidationTests" -freshInstall
We noticed this works. But it does run test methods that are disabled in the PhoneRegression scheme.
How can we tell xctool to adhere the scheme when specifying our test targets in -only?
Cross posted as an issue on the xctool github page
Update: the github issue has since been solved
Related
I am facing a strange issue:
I am trying to build using this xcode command
xcodebuild -project WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -destination ‘id={deviceID}' test
But every time I run this command I see an unusual prompt that shows up in the next line --
quote]
I am not sure what quote means and what I am expected to enter. I tried entering other device attributes but still it din help me .
Could some one please help me?
As far as I understand your provided information you are trying to start a test on a real device. Tests are executed on simulator only, which is why Xcode tells you it's not able to build your project.
Try something like this instead:
xcodebuild \
-project WebDriverAgent.xcodeproj \
-scheme WebDriverAgentRunner \
-sdk iphonesimulator \
-destination 'platform=iOS Simulator,name=iPhone 6,OS=8.1' \
test
As you can see this command tries to build your project on an iPhone 6 simulator with iOS 8.1 installed. You can define this to your liking (e.g. OS=13.0 or another iPhone/iPad).
You can give these reads a try, too, for more examples and a better understanding:
https://www.mokacoding.com/blog/running-tests-from-the-terminal/
https://www.appsdeveloperblog.com/run-xcode-unit-tests-from-the-command-line/
Is there a way to actually create UI Tests in Xcode8 via the terminal? I am aware that UI Tests are usually created via Xcode itself but i would like to do it via the terminal and automate the whole process. Thanks!
you can use xcodebuild command to run the test in Xcode.
like:
xcodebuild -workspace ${WORKSPACE_NAME} \
-scheme ${SCHEME_NAME} \
-destination "${DESTINATION}" \
-derivedDataPath "${OUT_PATH}" \
test
do not forget to set the scheme shared.
Has anyone been able to successfully pass the KW_SPEC variable to xctool. I'm trying to run a single KIWI spec by using https://github.com/kiwi-bdd/Kiwi/wiki/Kiwi-FAQ#q-how-do-i-run-a-single-spec-describecontextit.
I can run all the tests successfully with xctool but it doesn't seem to pick up the KW_SPEC value. I've tried it in a lot of different places with the command line but no luck.
e.g.:
xctool -destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject KW_SPEC=NewAssessmentTests.m:12 test
-only SampleProject_Acceptance_Tests
Using Kiwi v2.3.1 and xctool 0.2.3
Cheers,
Mo
Update 11/03/2015
#OhadSchneider's comment made me realise that KW_SPEC worked for me because I had set it in the test configuration for my scheme (Edit scheme->Test->Arguments->Environment variables). Setting the variable from shell doesn't work as that variable applies only to the actual build, and not when executing the unit test target.
But there's a workaround to this, by modifying the test phase of your scheme and adding a KW_SPEC environment variable with the value $KW_SPEC, this will expand when running xcodebuild to the value passed to the xcodebuild command (as in my original answer). After this, xcode will will gracefully use the passed KW_SPEC variable, xctool still has the skipped teste marked as failure issue.
Original answer
If you want KW_SPEC as an environment variable to xctool (or to any *nix tool), then you have to place it before the command, otherwise it will be considered a build setting:
KW_SPEC=NewAssessmentTests.m:12 xctool
-destination 'platform=iOS Simulator,name=iPad Retina,OS=latest'
-sdk iphonesimulator -workspace SampleProject.xcworkspace
-scheme SampleProject test
-only SampleProject_Acceptance_Tests
This will however lead to another problem: xctool will report as errored the tests that don't run, and will report the test as failed, even if no tests have failed. xcodebuild doesn't have this problem as it either doesn't do unit tests discovery, or ignore tests that didn't run, a thing that xctool fails to do.
I'm using xctool (v0.1.16) for an iOS project, which is configured as follows:
a workspace
two targets Project and ProjectTests
two schemes Project and ProjectTests
I have a .xctool-args file with the workspace and Project scheme configured.
I run xctool build and xctool -scheme ProjectTests -sdk iphonesimulator test successfully.
However, I run xctool clean and the following happens:
1st it runs xcodebuild clean Project successfully
2nd it runs xcodebuild build clean which fails with the error:
Scheme Tests is not currently configured for the clean action.
It is strange because I don't have a Tests scheme at all.
I run xctool clean -showTasks and have more details about the error. It runs and fails at
/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild \
-configuration Debug \
-workspace /var/folders/b7/472b_skj24gf7wmw6c3krp5c0000gn/T/xctool_temp_KMbH0e/Tests.xcworkspace \
-scheme Tests \
OBJROOT=/Users/paristote/Library/Developer/Xcode/DerivedData/Project-fmyqwebczmtplydrfhcmrdzezcft/Build/Intermediates \
SYMROOT=/Users/paristote/Library/Developer/Xcode/DerivedData/Project-fmyqwebczmtplydrfhcmrdzezcft/Build/Products \
SHARED_PRECOMPS_DIR=/Users/paristote/Library/Developer/Xcode/DerivedData/Project-fmyqwebczmtplydrfhcmrdzezcft/Build/Intermediates/PrecompiledHeaders \
-IDECustomDerivedDataLocation=/var/folders/b7/472b_skj24gf7wmw6c3krp5c0000gn/T/xctool_temp_KMbH0e/DerivedData \
clean
However, the same command executed separately finishes successfully.
The workspace at /var/folders/b7/472b_skj24gf7wmw6c3krp5c0000gn/T/xctool_temp_KMbH0e/Tests.xcworkspace simply references two projects, the Project xcodeproj and Pods xcodeproj. It does contain a Tests scheme with a build action.
Why does this command fail in the 1st place? Any help is appreciated.
Cheers.
I fixed this issue by checking the "shared" box for the main scheme:
Choose Product > Scheme > Manage Schemes.
Select the Shared option for the scheme to share, and click OK.
source: https://developer.apple.com/library/ios/recipes/xcode_help-scheme_editor/Articles/SchemeShare.html
I fixed it by
Unchecking auto create schemes in XCode scheme management
Removing and recreating the scheme(s) I wanted to build (in your case Project and ProjectTests)
I ran into this issue in xCode 7. I fixed this issue by doing the following:
- Choose Product > Scheme > Manage Schemes.
- Select the "Shared" option for the scheme, and click OK
Hi I'm following examples from Test Driven iOS development and I've written a few unit tests with the new Xcode 5 and XCTest. My tests fail with the Xcode GUI client, but when I use xcodebuild the status code is 0.
xcodebuild -target TemperatureConverterTests build
and
echo $?
returns 0.
The tests are suppose to fail. Is the command to run the test cases correct?
xcodebuild command has changed with xcode 5. Here is a script to run Unit tests :
Don't forget that schemes must be shared, you've got to create your Tests Scheme, Xcode 5 didn't create it for you
xcodebuild -workspace MyApp.xcworkspace -scheme myApp-Tests -destination=build -configuration Debug -sdk iphonesimulator7.0 ONLY_ACTIVE_ARCH=YES clean build test
Hope this helps :)
If you don't want (as you rarely need) a separate (and annoying) "scheme" for your testing "target"... Use a variation on #Rémy Virin's answer...
xcodebuild -scheme YourAppOrLib -target YourAppOrLibTests
As pictured below.. this allows you to run the Unit tests for the target.... without cluttering the project with non-production schemes... In this case IndexedKeyMap is my YourAppOrLib.