I am upgrading a Jenkins CI server from XCode 4.6 to XCode 5 and iOS SDK 7.
To run unit tests, I must now use xcodebuild test. I am piping the output of this command to a Ruby script, that generates test reports.
This does not work any longer after the upgrade. I think the problem lies in the fact that xcodebuild test launches a new process to actually run the tests. The last statement in my log is
/bin/sh -c /Volumes/Data/Jenkins/jobs/MyProject-Unit/workspace/Build/Intermediates/Redacted.build/Debug-iphonesimulator/UnitTests.build/Script-261C7EA912FD9A4B00F7A53E.sh
The output after that point can not be found in the Jenkins log, but when I run it on my own machine, I get the test results printed in the console.
What is happening here, and how can I get the test reports back ?
I'm on the same trail. I believe if you look into that script, it will turn around and call this script:
${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests
This is script that used to run unit tests. Now, in Xcode 5, it just tells you to remove the Run Script build phase that Xcode 4 automatically creates. If you run RunUnitTests from Xcode 5, it just says:
RunUnitTests is obsolete. To run unit tests for your target, use the
Test scheme action in the Xcode IDE and the test action in xcodebuild.
You can remove the Run Script build phase that invokes RunUnitTests
from your unit test bundle target.
I've gone ahead and done what it says (removed the Run Script phase) and I still get no output under Jenkins, perhaps because, as you surmise, it's a new process, and Jenkins sees the first process complete, and jumps out.
Update: More info here: How can I get Xcode 5 to run iOS unit tests on a Jenkins slave?
Related
We are setting up CI/CD for iOS project using Gitlab & fastlane tools on Mac Mini(M1 Chip).
When there is a new pull request created, pipeline starts and it executes below command success.
fastlane ios build
But it shows "no such module 'Alamofire'" error for below command.
fastlane ios test
Other observations on same build machine Mac Mini(M1 Chip):
It shows same errors "no such module 'Alamofire'" when try to build or run tests using Xcode(Uncheck Open using Rosetta).
Both build & run tests works fine with Xcode(Check Open using Rosetta).
We are looking for build the iOS project and run unit tests on simulator for new/update pull request. Build works fine but run unit tests fails during compilation.
Any help on this would be greatly appreciated.
You are using cocoapods to import Alamofire right?
Do you happen to have any post_install steps that are setting build_settings in your podfile?
Is it possible to make sure the tests are run whenever anyone archives an app in Xcode?
For example in this pre-action when i run Archive you could set so that the Test debug scheme is run. Is this possible in a hidden setting or with a script somehow?
I've set up jenkins on my local machine. Than I created a sample app with one failing and one succeeding unit test. When I issue the following command in the terminal
xcodebuild -scheme 'SampleWithTest' -sdk iphonesimulator7.0
-destination platform='iOS Simulator',OS=7.0,name='iPhone Retina (3.5-inch)' clean test
than it clean build the project, starts the simulator and makes the tests. This is how it should work.
When I run the same command via jenkins, it clean builds the project and ends with ** TEST SUCCEEDED ** without starting the simulator or print out the test results. Jenkins just fakes, that the tests was successful.
I test it on my local machine. Nothing headless, remote, source control, slave, code signing or whatever could make any trouble.
The Jenkins Xcode Plugin has not been updated to work with Xcode 5 and the XCTest framework. See this issue. Jenkins and the Xcode Plugin do not understand XCTest output.
Additionally, Xcode 5 changed how Xcode interacts with the simulator and devices when running tests. While there is something like a "headless mode" now for the simulator, access to that functionality isn't available to the Xcode plugin. This means that in some cases, it looks to the Xcode plugin as if the tests run and exit immediately, which is not accurate.
Xcode is known to be pretty tricky to setup with Jenkins, Travis or other non-Apple CI solutions.
Unit Tests support is even worse so you should try using xctool instead.
Related blog entry here.
I've been trying to find a way to run a single iOS unit test among multiple tests with the command line on an iOS device.
Here is what I've found thus far...
I can use xcodebuild test [options] to build & run every tests.
Found a tool called xctest in /Applications/Xcode.app/Contents/Developer/usr/bin. I've looked up in the man page and it is possible to run a single test. When I try using it, I get an error stating that I'm using x86_64 architecture and then a posix error pops after. This seems to indicate that it might be the xctest for Mac OS X apps.
Found another xctest in for the iPhone Simulator.. but I want to run it on a iOS device
I know that it is possible to select the tests I wish to run in a scheme, but this is problematic since I would need a very skillful & patient monkey to manually do this in a build system.
By the way, I am not using Apple's own Continuous Integration system with bots and using it is out of question because another build system is used for every other platform.
My question is: How can I run a single unit test (XCTest) from the command line with build automation in mind?
P.S: I don't have any code signing issues.
Try using xctool which can run tests on devices https://github.com/facebook/xctool
In Xcode 5, I have a workspace with multiple projects and a scheme that should run all the test targets of the various projects. When I kick off the tests about 3 test project targets pass. So far, so good. But at the fourth test target I get an error message from Xcode telling me that the simulator can't be launched because it is already in use.
Is there any way to kill the simulator between each test target(!) in a single(!) scheme?
Or are there other solutions?
Edit: I have opened a rdar for this: http://openradar.appspot.com/15153136
Thanks!
I have come into this error whenever i try to run my functional tests using calabash. One way to do this maybe be to run a script in the after each build process to kill the simulator.
this guy has wrote a bash script that does this
http://cdrussell.blogspot.co.uk/2012/04/script-to-kill-iphone-simulator-if.html
Might be worth running this as a build phase for each test target