I am attempting to run my Xcode unit tests using Jenkins. I can run them from Terminal on my build server using this command with out any issues:
xcodebuild test -scheme MyProject -destination OS=7.0,name="iPad"
CONFIGURATION_BUILD_DIR=/Users/dev/Jenkins/Home/workspace/MyProject/build
However when I try to add that command to my build script (buildscript.sh) file, it hangs when running the unit tests. I also tried adding a new Xcode task to Jenkins and running with these parameters:
XcodeSchema File: MyProject
Configuration: Debug
Custom xcode build arguments: test -destination OS=7.0,name="iPad" CONFIGURATION_BUILD_DIR=/Users/dev/Jenkins/Home/workspace/MyProject/build
It is hanging on a build file in the debug-iphone simulator folder.
** BUILD SUCCEEDED **
setenv SYSTEM_DEVELOPER_DIR /Applications/Xcode.app/Contents/Developer
.
.
.
/bin/sh -c /Users/dev/Library/Developer/Xcode/DerivedData/MyProject
-ggdmamhydtxxzadkbjuusdjmholn/Build/Intermediates/MyProject.build/Debug
-iphonesimulator/MyProjectApplicationTests.build
I took a look at the build file and the environment variable it is using is set correctly.
MyProjectApplicationTest.build
#!/bin/sh
# Run the unit tests in this test bundle.
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
# "${SYSTEM_DEVELOPER_DIR}/Tools/RunPlatformUnitTests"
# "${PROJECT_DIR}/XcodeScripts/RunPlatformUnitTests
The RunUnitTests file exists in that directory.
I don't know if Jenkins is having issues running the simulator. I spent most of the day yesterday racking my brain on this. If anyone has any ideas on what the issue may be, please let me know.
I was also facing the same problem and found that jenkins user is not correctly set up to run test case on simulator. Follow instruction given on below website:
https://content.pivotal.io/blog/ios-ci-with-jenkins
see the section "SETTING UP THE JENKINS USER".
hope it will work for you as well.
Related
I want to have a run script that looks at the number of previous warnings (probably during the compile phase), and generates an error which stops the run/buid if the number of warnings is too large.
I am working on an app that has a large amount of warnings that happen during compile. They do not stop the app from compiling, but I want to stop devs from adding more warnings as they add features.
I have looked everywhere, but so far have not found a way to use previous build output in the run script phase of the build. Is there any way of doing this? Some kind of env variable, or a way of monitoring the build output in a custom script as it happens?
I see a similar question here:
Is there a way to access Xcodes current build output from a build phase run script?
But it seems that the output is only available after the build is complete in that answer. Is there any other option that would allow the build to be failed before it finishes?
Any insight into the build system in Xcode would be appreciated! Cheers
Additional context:
All I need is for Xcode's default warnings to have a feature similar to SwiftLint's warning_threshold
https://stackoverflow.com/a/52256459/7623867
I have looked into that problem as well for quite some time (I wanted to use warnings for quality measurement). The only way I have found is to compile the whole project with xcodebuild, with a command similar to:
xcrun xcodebuild -project MyProject.xcodeproj/ -scheme MyScheme -destination "platform=iOS Simulator" build -quiet | tee xcodebuild.log
and then counting the warnings using AWK or some other tool.
Here is an example.
Summary
I'm trying to get a code coverage report from my project. However when I try to build and run tests, there is no CodeCoverage folder created in the derived data folder.
How am I trying to do this?
I am running xcodebuild as follows:
xcodebuild \
-workspace <some_workspace> \
-scheme <some_scheme> \
-destination <some_destination> \
-derivedDataPath <some_path> \
-enableCodeCoverage YES \
build test
What is the problem?
For my workspace/project it fails at the very end with a line:
xcodebuild: error: Failed to build workspace <some_workspace> with scheme <some_scheme>.
Reason: Could not determine bundle identifier for <some_test_target>'s TEST_HOST: "<some_path>/Build/Intermediates/CodeCoverage/Products/Debug-iphonesimulator/<some_product>.app"
At first it seemed directly linked to the TEST_HOST issue, but that's not the case here.
There is simply no CodeCoverage folder:
<some_path>/Build/Intermediates/CodeCoverage
What have I tried?
I tried the same with a clean new project, same running the same xcodebuild command, which succeeds. Within the Build/Intermediates/ folder exists the CodeCoverage folder.
However, in my own project, which is more complex, the Build/Intermediates/ folder contains a bunch of *.build folders (related to the app and the various frameworks, HockeySDK.build for example) and PrecompiledHeaders but no CodeCoverage folder.
When looking through the output of xcodebuild I never see a reference to the CodeCoverage folder for my project. However, for the test project, the first mention is at:
Write auxiliary files
/bin/mkdir -p /<some_path>/Build/Intermediates/CodeCoverage/Intermediates/<some_project>.build/Debug-iphonesimulator/<some_project>.build/Objects-normal/x86_64
For my project I see:
Write auxiliary files
/bin/mkdir -p /<some_path>/Build/Intermediates/<some_project>.build/Debug-iphonesimulator/<some_project>.build/Objects-normal/x86_64
Workaround
Something else that seemed to 'trick' it into working is to set the Host Application to None. After doing this it builds and tests, but tests fail due to the lack of host. However, this causes the CodeCoverage folder to be created, and if the Host Application is correctly set again, running the build and tests works fine, with a code coverage report produced.
You have to assure the package name is equal in all configurations. Xcode modifies it in case you are using unsupported characters. For instance Xcode replaces "-" with "_".
In all configurations, go to:
Project -> Build Settings -> Product Module
and set the exact same name without spaces.
I ran into this issue after upgrading to Xcode 8. When running the tests I get this error at run time:
/Users/<me>/work/<appname>/Build/Intermediates/<appname>.build/Debug-iphonesimulator/<appname>UnitTests.build/Script-231C35D610AC1F5000D830C2.sh: line 3: /Applications/Xcode.app/Contents/Developer/Tools/RunUnitTests: No such file or directory
The sh script in the error message is trying to access the RunUnitTests tool and fails. I assume this tool has been removed in Xcode 8. It seems that for my project Xcode tries to run tests in a way that is no longer supported. If I create a brand new project no such script is created and I can run tests.
Any idea what settings I need to update in my project to get the tests running again? I tried the 'Update to recommended settings' checklist but it doesn't solve this issue.
It seems I only needed to remove the Run Script phase that invoked that script.
Removing Run Script from the Build Phases didn't help me.
I simply copied /Applications/Xcode.app/Contents/Developer/Tools/RunUnitTests from my older Xcode into my XCode 8 package content. Give it right permissions:
RWXR-XR-X
And it worked for me.
I am trying to run tests automatically using Teamcity, but it seems that when the agent is compiling the project, it is not done correctly because when I run the command like for running test, I am getting the following error:
fatal error: 'Pushwoosh/PushNotificationManager.h' file not found
#import <Pushwoosh/PushNotificationManager.h>
[16:48:51][Step 2/2] ^
[16:48:51][Step 2/2] 1 error generated.
However, when I run manually the same commands I get no errors, but my test running. The application that I want to run was written in objective-c, and the command lines I am using are the following:
$ git clone REPOSITORY_URL (runs sucessfully)
$ pod install (runs sucessfully)
$ xctool -workspace Supermaxi.xcworkspace -scheme Supermaxi test -only FunctionalTests:TestCase000_Registration/test00_WrongEmail -sdk iphonesimulator (fails and I get the previous described error)
After being helped by I friend of mine, we found the solution to this issue in this link.
It turns out that the main problem was, as the link states, that CocoaPods uses symbolic links in it’s directory structure, and TeamCity can have trouble copying symbolic links in the VCS checkout to build agent, so I had to set the Version Control Settings of Teamcity as it is explained in the previous link.
Therefore, I have just changed the VCS checkout mode from Automatically on server to Automatically on agent (if supported by VCS roots). After that I run the agent again, and it worked properly.
I want to view the code coverage by running gcovr on generated gcda files. Jenkins seems to put the generated gcda files in Users/../Library/developer/Xcode/DerivedData/../../../i386. I expect them to be in Users/Shared/Jenkins/workspace/../build/example.build/Debug-iphonesimulator/example.build/Objects-normal/i386
When i run my IOS project with XCode locally it does generate gcda files and i can view my coverage.
Im running xcode 5 and all tests are created with XCTest.
I've set 'Generate Test Coverage Files=YES' and 'Instrument Program Flow=YES' for debug and release, basicly i have done everything according to this post
In Jenkins i use the xcode plugin to build. It has two build commands. The first one builds with the target "example" and configuration debug.
The second Xcode build command builds with the target "ExampleTests", configuration debug, arguments" test -destination OS=7.0,name=”iPhone Retina (4-inch)” and a scheme . In the output i can see the test being run and the simulator starts on the building machnine.
It seems i have missed something maybe in the project setting or havent set something right in the jenkins job. Maybe something like TEST_AFTER_BUILD=YES only then for XCode5.
As you recognised the .gcda files are put in the "wrong" directory.
Do the following:
Select your application target in your Xcode-Project and go to "Editor -> Add Build Phase -> Add Run Script Build Phase"
Paste this script into the script field:
echo "Creating derivedDataDirectory file"
echo "${OBJECT_FILE_DIR_normal}/${CURRENT_ARCH}" > ${PROJECT_DIR}/derivedDataDirectory
XCode setup
(This creates a file with the path to the derivedDataDirectory)
Go to your jenkins project, click on "Add build step" and select "Execute shell".
Paste this script into the "Command" field:
#CopyCodeCoverageFile
echo "Start copying code coverage Files"
projectname="[YOUR PROJECTNAME]"
source=$(cat ${WORKSPACE}/$projectname/derivedDataDirectory)
cp -a $source/. ${WORKSPACE}/$projectname/
#CodeCoverage
echo "Start CodeCoverage"
cd ${WORKSPACE}/$projectname
[YOUR PATH TO GCOVR]/gcovr -r /private/tmp/workspace/${JOB_NAME}/$projectname --xml > ${WORKSPACE}/$projectname/test-reports/coverage.xml
Insert [YOUR PROJECTNAME] and [YOUR PATH TO GCOVR]. If your PROJECTNAME or target has
spaces in it, this will cause trouble! Make sure all the paths are correct!
This works for me and if all your paths are correct it should work for you to.
Im running XCTests on Jenkins using XCode 5. This should be the same as your setup.
If you dont use gcovr to generate code coverage for cobertura you can delete the last step
in the script. If you are experiencing problems with gvovr try using gcovr 3.0 instead of
3.1!
Tell me if something is not working for you or you found a better solution!