We have a fairly large code base that takes a long time to clean build. Whenever we archive the build (Product->Archive) the archive process first cleans all, then builds.
This seems unnecessary and time-consuming, we would like to be able to create an archive without a clean build. Incremental builds should be fine.
Does anybody know how to disable the "clean all" step during the XCode archive process? Thank you so much, my searches on this have come up with nothing but advice on how to make a build faster (which is not useful advice for us).
Yes, it is possible.
As I suspected would be the case this can be done from the command-line. It took us a while to figure this out. Here is an excerpt from our TeamCity build scripts. Basically you generate a build (clean or incremental is your choice), then generate and .ipa from the build. Here is one option (developer identity and provision profile ID removed of course):
export CODESIGN_ALLOCATE="/Applications/Xcode.app/Contents/Developer/usr/bin/codesign_allocate"
xcodebuild -project <PROJECT NAME>.xcodeproj -target <PROJECT NAME> -configuration Release -sdk iphoneos -arch armv7 ONLY_ACTIVE_ARCH=NO CONTRIB_PATH=%system.agent.home.dir%/Contrib2 CODE_SIGN_IDENTITY="iPhone Developer: <DEV NAME> (ID)"
PROVISIONING_PROFILE=<PROFILE ID>
rm -rf Payload
mkdir Payload
cp -R build/Release-iphoneos/ Payload/
rm ~/<PROJECT NAME>.ipa
xcrun -sdk iphoneos PackageApplication -v Payload/<PROJECT NAME>.app -o ~/<PROJECT NAME>.ipa --sign "iPhone Developer: <DEV NAME> (ID)" --embed ~/Library/MobileDevice/Provisioning\ Profiles/<PROFILE ID>.mobileprovision
How about setting up a continuous integration server, so that every commit can trigger a process that results in an archived build. . . (hopefully after running tests, etc first), as well as publishing API docs, et al.
It will still take the same amount of time, but since its running in the background after every checkin of code, and the latest release candidate will always be available - you probably won't notice.
Otherwise, you run into a headache where you have to do your development in Release mode instead of debug, etc - its just not going to work.
Here's an example of an (OSX) project that includes a build script that can be run by a continuous integration server. I used Bamboo, but if you want something free there's also eg Jenkins:
https://github.com/jasperblues
In the above project, each successful build (triggered whenever someone commits code) publishes API docs and test coverage reports back up to the github page. . . for an iOS project you could also have it do archiving.
Incremental builds are not fine for Archive. The point of the clean is because sometimes you can get issues in incremental builds that a clean fixes. That's an acceptable problem to have during development, but Archive is intended for distribution builds, and distribution builds should not have this risk at all.
Not to mention, your normal build process is building a Debug build, and Archive is going to build a Release build, so you're going to need to rebuild most of the app anyway (anything that changed since your last Release build).
Related
I'm using Fastlane for my CI builds on Github Actions.
CI runs Fastlane's build_app function. The build_app internally uses xcodebuild archive to archive the project and then uses xcodebuild -exportArchive to extract IPA.
The problem is xcodebuild archive always ignores caches and builds all the modules from the source (even if DerivedData from previous builds are available). Is there a way to force either fastlane or xcodebuild archive to use caches from DerivedData. Or maybe there is some other way to achieve this.
Thanks
I recently checked out a fresh version of our iOS app from git and built from command line via xcodebuild. I then built a second time using the exact same command, while making no changes to files in the repo whatsoever (not even opening them).
I expected the second build to take no time at all, but it actually took longer than the first:
user$ time xcodebuild -sdk 'iphonesimulator' -scheme 'DemoApp' -configuration 'Debug' -target 'DemoApp' build > /dev/null
real 5m45.849s
user 0m15.270s
sys 0m5.640s
user$ time xcodebuild -sdk 'iphonesimulator' -scheme 'DemoApp' -configuration 'Debug' -target 'DemoApp' build > /dev/null
real 6m8.858s
user 0m12.904s
sys 0m4.198s
If I do builds in xcode with no changes, it builds and runs in a matter of seconds.
Here are things I've tried to get incremental builds on command line:
I tried adding -derivedDataPath ~/Library/Developer/Xcode/DerivedData, but got the same results.
I've added the -incremental flag to other Swift arguments.
I've turned off all shell scripts and ensured no files have changed between builds
Does xcodebuild not support incremental builds? Is there a way to figure out why this is happening?
I'm using the latest Xcode and the new Xcode 10 build system. Most of the code is in swift if that makes any difference.
Edit: changing back to the old build system builds incrementally via xcodebuild in under 30 seconds.
Update:
The way I'm testing this: I have a Jenkins job that will clone the repo and do a hard clean: running xcodebuild clean and deleting the build directory and and derived data directory by hand just in case. Then it does a build. Then it makes no changes and does a build again. The second build is the one I'm timing.
I'm trying to automate iOS building through CMake, up to and including authoring .ipas. I have my provisioning profile and certificates set up, build is successful, but when I execute cmake --build I can't get .xcarchives to generate, and my POST_BUILD command (using xcodebuild) to author the .ipa is dependent on that.
I've tried setting some archive options though CMake (setting cmake_archive_output_directory), but there's a flag or option to set that can produce xcarchives during build, I don't know it. Does anyone have experience with this?
Despite many attempts, a post-build command to xcodebuild archive never worked -- I'm still not sure why, either. What I ended up doing to work around this was have CMake generate a script.
So my create executable custom function does this:
It creates a script file if it doesn't already exist, then for each executable target, appends an xcodebuild command if it's not a duplicate:
xcodebuild -workspace ${pathToTargetWorkspace} -scheme ${targetScheme} -configuration ${config} -archivePath ${pathToGeneratedArchive}.xcarchive archive
Then after building is finished, you just run the script with the aggregation of archive commands and it archives all the targets.
I want to automate the iOS app building process that currently is totally manual on a desktop computer (Mac). A central build server Hudson/Jenkins, running on a Gnu/Linux machine, is already available inside the organization infrastructure.
Is it possible to build this iOS app with Jenkins, using tools like Jenkins distributed builds (and so configuring a Mac machine as a slave)?
Do you have any example, know any technique or alternative to do this?
Yes its possible. We are doing this and it works fine. You could configure a Xcode-slave within Jenkins by choosing the Xcode-plugin. There are tons of nice tutorials on the web.
Check out this one for starters.
I finally got my Jenkins Job up and running after toiling away for longer than I wanted using the only tutorials that I could find which all happened to be from 2015 or earlier unfortunately. So I posted a real quick checklist of stuff I did to successfully setup the Mac slave on a local Mac mini within my company. Similar to you #lifeisfoo my organization currently has a Jenkins instance running on an outside server.
The link for setting up a Mac slave as per 2018 is seen here.
Setting up jenkins slave on Mac OS
In terms of setting up a Jenkins Job after the node is successfully created and Launch Agent connects properly to your Mac slave from your Jenkins instance see below:
For the most part setting up a New Item a.k.a. New Job from the Manage Jenkins section is self explanatory
Most of the older tutorials are accurate in that you need some plugins Github, Keychains and Provisioning Profiles Management (still not 100% that you need this plugin though but still good to have and set it up) and others I downloaded and installed but never used including I did not use the Xcode plugin in the Build section of the Job setup.
Make up a project name, check Github project and add the url ending in .git/, restrict this project to the Node you created for the Mac slave, in Source Code Management we used Git and I added the url again (ending in .git without the "/" this time, created new credentials with my username and password, */main in the branches to build section, added Github hook trigger url for GITScm polling to my organization's Github account web hooks section
In Build I only used Execute Shell and cd into /Users/jenkins/path/to/file
Currently it is NOT a workspace so the commands are as follows in Execute Shell
xcrun xcodebuild -project ProjectName.xcodeproj \
-configuration Release \
-destination 'platform=iOS Simulator,name=iPhone 6s' \
-allowProvisioningUpdates \
CODE_SIGN_STYLE='Automatic'
PROVISIONING_PROFILE_SPECIFIER=${PROVISIONING_PROFILE}
CODE_SIGN_IDENTITY=${CODE_SIGNING_IDENTITY}
As of February 2018, I have changed it to workspace because of the need to include some cocoa pod SDKs hence I now use the below combination of "Execute Shell"s.
The below script is to run a pod install every time (because I added Pods/ to the gitignore file only pushing Podfile and Podfile.lock to the repository which Jenkins pulls from)
cd /Users/jenkins/Desktop/projectFileWhere.xcworkspaceExists
/usr/local/bin/pod install
The below is to run the project build in another "Execute Shell"
xcrun xcodebuild -workspace ProjectName.xcworkspace \
-scheme NameOfScheme \
-configuration Release \
-destination 'platform=iOS Simulator,name=iPhone 6s' \
-allowProvisioningUpdates \
CODE_SIGN_STYLE='Automatic'
PROVISIONING_PROFILE_SPECIFIER=${PROVISIONING_PROFILE}
CODE_SIGN_IDENTITY=${CODE_SIGNING_IDENTITY}
Note:
- I wasn't sure whether it was absolutely necessary but I made sure to clone the latest GitHub repo into my Mac slave machine into the same project directory where the .xcodeproj existed before.
- I also foolishly didn't realize at first that .xcworkspace was NOT getting pushed to my GitHub repo because build/*.xcworkspace was still in my gitignore file. Removed it, re-pushed to repo and cloned to Mac slave and it was all gravy baby
I saw many variations of CODE_SIGN_IDENTITY and some that specifically told me to add parameters like DEVELOPMENT_TEAM= and DISTRIBUTION_TEAM= but those weren't necessary.
Frankly speaking I don't know if -allowProvisioningUpdates nor CODE_SIGN_STYLE='Automatic' is necessary but its working for me.
I ended up going with the Execute Shell Build step and not the Xcode Plugin for Build parameters because I was able to more easily manipulate build parameters and test it against the errors I was getting in the console output of each build in my Jenkins instance. In the end there was also more and better documentation and help in trying to understand Xcodebuild with command line and that is essentially what Execute shell is doing hence I liked it better.
I wanted to write this post to point out what I finally did to make the job build and my issues in getting it to build were very easy to fix until this last one which was in that code signing couldn't find my provisioning profile for the certificate/team that I was using in Xcode's project file target and project:
I didn't use Fastlane or any other program. Just SSH connection into Mac slave.
Setup Xcode command line tools: xcode-select active developer directory error
1) In this Wiki I did what is stated in the paragraph directly under the title "Xcode project setup".
https://github.com/cyupa/JenkinsCI-iOS
2) You need to have both private and public key for a dev and distribution certificate on the Mac Slave in order to build through Jenkins to the Mac Slave. Need Provisioning Profiles on the Mac Slave for each of these accounts with the proper Bundle ID as well.
https://blog.noser.com/streamlining-ios-development-with-jenkins-and-wireless-app-distribution-2/
3) Move the keychains and certificates for both dev and distribution to Login or whatever other unique keychain you created but for each private key, right click (two finger tap) the private key and Get Info -> Access Control -> allow /usr/bin/codesign or all applications to access this item
4) In Build settings of the Xcode project on the Mac slave (make sure you've downloaded from your repo to the Mac slave and opened it once on the Mac Slave's Xcode app), change build settings to be automatic and/or just change it in General of the Target App and testing if you want.
5) The last piece of the puzzle to solve the problem that took me 2 days was to simply add my keychains to the security list so that Xcode could use the provisioning profiles for the certificates that it needed.
Not sure if what I suggested above regarding Access Control is necessary if you add the keychains to the security list but I kept it and its working for me. Maybe I'll experiment later and see what is actually needed and not needed but for now, this was the key. You may get "codesign failed with exit code 1" which is all the same thing. The terminal command to unlock is:
security list-keychains -s ~/Library/Keychains/{login, "any other keychains you want added to security list"}.keychain
Sooooo if you are seeing any build errors in Jenkins like.......
No profiles for 'com.organization.Target' were found: Xcode couldn't find any iOS App Development provisioning profiles matching 'com.organization.Target'
Code Signing Error: Code signing is required for product type 'Application' in SDK 'iOS 11.2'
.......then you know most likely its a keychain access thing and nothing else.
In hindsight I realize now that I saw this same advice in some other Stack posts and one or two other blog posts as linked below, but that wasn't my issue at the time so I totally forgot about it and now hate myself for not reading slower over these posts. Oh well. At least I'm getting a green circle now.
"User interaction is not allowed" trying to sign an OSX app using codesign
https://blog.noser.com/streamlining-ios-development-with-jenkins-and-wireless-app-distribution-2/
We have an iOS automated build system that is randomly failing the codesign for certain projects.
The build system uses xcodebuild (followed by xcrun) to build and package the application. We use this process across multiple apps. The build machines we run this on have multiple distribution signing certificates, and are running Xcode 5.1.1 (5B1008) and 10.9.4. We've been using the system for about two years now, and builds work fine. When they don't work, there's usually a clear xcodebuild error message. But right now we are getting random failures of the codesign call on xcodebuild for two of our apps.
The xcodebuild command is as follows:
xcodebuild -verbose -project ./src_dir/TechApp.xcodeproj ONLY_ACTIVE_ARCH=NO ARCHS="armv7 armv7s arm64" -scheme ${SCHEME} QA PROVISIONING_PROFILE=${PROFILE} CODE_SIGN_IDENTITY=${IDENTITY} CONFIGURATION_BUILD_DIR=./bld_dir/builds/QA build
And, after lots and lots of xcodebuild compile statements, it fails with the following:
CodeSign /Users/bldUser/buildScripts/archive_dir/1404933111/bld/bld_dir/builds/QA/Tech\ App.app
cd /Users/bldUser/buildScripts/archive_dir/1404933111/bld/src_dir
export CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate
export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin"
Using code signing identity "iPhone Distribution: Build Certificate" and provisioning profile "BuildWildcard" (C137C14D-630F-4D67-BD2B-2FD4AB4F2BA4)
/usr/bin/codesign --force --sign 637514541007B62BCEA94ACC41A98A0E1391C3D4 --resource-rules=/Users/bldUser/buildScripts/archive_dir/1404933111/bld/bld_dir/builds/QA/Tech\ App.app/ResourceRules.plist --entitlements /Users/bldUser/buildScripts/archive_dir/1404933111/bld/build/TechApp.build/Debug-iphoneos/TechApp\ QA.build/Tech\ App.xcent /Users/bldUser/buildScripts/archive_dir/1404933111/bld/bld_dir/builds/QA/Tech\ App.app
** BUILD FAILED **
The projects in which this are failing are big apps... and they have multiple dependent projects attached to them (and I think that it might be the dependent projects that are contributing to the random failure of the codesign, but I'm not sure).
The thing is that this randomly fails at this statement. If I run it again, it will work fine. So it's the randomness that confuses me. And there really is not any error message. It just says "BUILD FAILED". Some things I've tried to get this to work:
I've passed in additional parameters into xcodebuild OTHER_CODE_SIGN_FLAGS=--verbose=4 --continue --deep in order to see if any combination of these parameters "help." This was based on the problem described in this link, as it seemed to match the failure conditions for our particular projects that are failing. This post had me the most interested, as it seems like this issue may be related to the OS + build environment + the fact that the projects this is failing on have lots of sub-projects attached.
We currently have all our certificates in the login.keychain... but, as per this link (amongst others) there's a suggestion to run certificates off of a build-specific keychain. This is great, and a much cleaner way to do things-- but, after trying this, this had no impact for the issue above.
I thought maybe this might have to do with confusion on certificates-- so I've removed all iPhone Distribution certificates except the the one I need for these apps, and it still comes back with this error.
I've gone in to each of the targets for the projects, including the sub-projects, and put in code signing identity just to make sure there wasn't a possibility that there was some project or sub-project setting that would pick the wrong certificate. In addition, on the xcodebuild command above, I've put in the explicit scheme name and provisioning profile id-- so this is not an issue where it can't find the provisioning profile/certificate combination (or at least I don't think it is-- there's no indicator that it can't find the provisioning profile or certificate)
Given that it's random, I suspect it's either (1) a case where there is some unset code signing value, and it therefore randomly picks one; or (2) some issue with xcode or the os that was only recently introduced.
Any thoughts or ideas on what to try would be much appreciated.
One update:
This works just fine when run through Xcode. So this is something specific to xcodebuild for these projects. When compiled through Xcode: it works 100% of the time. When compiled through xcodebuild: it works like 25-50% of the time. So I'm really hoping there's something I"m missing on the xcodebuild command and this does not require an actual project change (since it works when compiled directly through Xcode).
Update on trojanfoe's question:
There may very well be an issue with spaces. This is the actual xcodebuild call on one of the projects:
xcodebuild -verbose -project /Users/bldUser/buildScripts/archive_dir/1404904225/bld/src_dir/MyTechApp.xcodeproj ONLY_ACTIVE_ARCH="NO" ARCHS="armv7 armv7s arm64" -scheme "TechApp Dev" PROVISIONING_PROFILE="" CODE_SIGN_IDENTITY="iPhone Distribution: Build Certificate" CONFIGURATION_BUILD_DIR="/Users/bldUser/buildScripts/archive_dir/1404897462/bld/bld_dir/builds/Dev" build
After running the command above, there is (lots) of build commands (there are lots and lots of warnings, but no errors). It finally stops at the following:
CodeSign /Users/bldUser/buildScripts/archive_dir/1404897462/bld/bld_dir/builds/Dev/Tech\ App.app
cd /Users/bldUser/buildScripts/archive_dir/1404904225/bld/src_dir
export CODESIGN_ALLOCATE=/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate
export PATH="/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin:/Applications/Xcode.app/Contents/Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/git/bin"
Using code signing identity "iPhone Distribution: Build Certificate" and provisioning profile "BuildWildcard" (C137C14D-630F-4D67-BD2B-2FD4AB4F2BA4)
/usr/bin/codesign --force --sign 637514541007B62BCEA94ACC41A98A0E1391C3D4 --resource-rules=/Users/bldUser/buildScripts/archive_dir/1404897462/bld/bld_dir/builds/Dev/Tech\ App.app/ResourceRules.plist --entitlements /Users/bldUser/buildScripts/archive_dir/1404904225/bld/build/MyTechApp.build/Debug-iphoneos/TechApp\ Dev.build/Tech\ App.xcent /Users/bldUser/buildScripts/archive_dir/1404897462/bld/bld_dir/builds/Dev/Tech\ App.app
** BUILD FAILED **
All of the "cd" commands and the "export" commands are run as part of the codesign command. In terms of the xcodebuild statement, we're leaving the provisioning profile blank so it picks up a default-- although even if I specify the specific provisioning profile, it will still fail (and from the code sign above, it IS picking up the correct provisioning profile). The above also specifies the scheme. This is normally run from a bash script where it will loop through an array of schemes and generate builds via xcodebuild for each scheme. As you can see above, there are spaces in the scheme and the project and the product name, so I would not be at all surprised if that were an issue in some way. I'm not sure why it would work one time and not another, but I'm going to try out removing spaces. The src_dir and the funky archive_dir numeric is a directory created by our build system, so the files are all checked out to the src_dir location.