I am currently testing Xcode Cloud as a member of Apple's private beta program and encountered an issue when trying to Archive / Build (any action) on the cloud for my project.
The project is a fairly basic SwiftUI app with CocoaPods dependencies. I have followed the steps to integrate CocoaPods into my project as described by Apple by simply committing my Pods directory to GitHub. However, I am getting a build error on the cloud for every attempted action:
Command PhaseScriptExecution failed with a nonzero exit code
Here is the ASC log for reference:
This is very strange because the same project builds and archives successfully on my local machine. I have used the same macOS and Xcode versions in the Workflow editor as my local version of Xcode.
How can I resolve this error?
Custom shell scripts restrictions
TL;DR
Apple has locked down the security of their hosted infrastructure by only enabling shell scripts to run as part of ci_post_clone.sh, ci_pre_xcodebuild.sh or ci_post_xcodebuild.sh in the ci_scripts folder. Your Pods project has a custom shell script outside of this folder that is not triggered by one of these CI scripts, so does not have running permissions.
The solution for this specific issue are:
(technically) Refactor your build script phase to inline the shell script file inside the run script.
(recommended, but not always possible) Use the Swift Package version of the CocoaPod if available.
(workaround) Downgrade the CocoaPod to a version without an external shell script.
Reference
From Customize your advanced Xcode Cloud workflows:
If your script doesn't appear to be running when you expect it to,
double-check that you've named it correctly and placed it in a
ci_scripts folder alongside your project.
...
Lastly, it should be
noted that in a test action, multiple environments are used to build
and run your tests. Only the environment that is used for building
your tests will have your source code cloned into it by default. The
environments that run your tests won't have source code cloned into
them. They'll only have the ci_scripts folder made available on
them. As a result, the post-clone script won't run in these
environments and your custom scripts and any of their dependencies,
such as other shell scripts and small tools, must be entirely
contained within the ci_scripts folder.
Build script phases ARE allowed to run user-defined code as part of the build process, however, we can only run inlined custom scripts here. As discussed, external shell scripts have restricted permissions. Running the external shell script file after moving to ci_scripts does NOT work. e.g. "${PODS_ROOT}/../ci_scripts/AppCenter-xcframeworks.sh".
Although not relevant here, note that the environment that tests your project won't have the source code cloned into them.
For tests or other environments to reference custom script files, we need to store additional scripts inside the ci_scripts folder to ensure the action has access to it. Apple only allows 3 scripts to run corresponding to 3 stages of a build:
After cloning the source code into the build environment.
Before running xcodebuild.
After running xcodebuild.
Additional shell scripts can can ONLY run here after delegating from the respective ci_post_clone.sh, ci_pre_xcodebuild.sh or ci_post_xcodebuild.sh files in the ci_scripts folder.
Solution 1
My issue was running an external shell script during the build process. Apple does allow Run Script Build Phases in Xcode Cloud workflows, but they have to be inlined. So, I had to do 4 steps to run a custom Pod shell script as part of a build phase:
Refactor your Build Phases -> Run Script Phase script to inline the shell script file.
Check the project builds locally after clearing DerivedData.
Commit your code to GitHub / VCS.
Trigger the workflow.
Solution 2 (recommended, but not always possible)
Add the Swift Package version of the CocoaPod as a dependency following Apple's documentation.
Solution 3 (workaround)
Downgrade your CocoaPod to a version without external shell scripts.
Notes
As you can tell from the amount of effort required to workaround custom shell script build phases with Xcode Cloud, I suggest raising an issue on the specific CocoaPod repository to migrate away from custom shell script files. These kinds of steps make using Xcode Cloud very painful.
As Xcode Cloud adoption grows it is entirely possible that individual CocoaPods no longer reference custom shell script files. I don't see Apple opening up their infrastructure to enable arbitrary shell script execution because this is a security risk, and to be honest, should have been prevented on other CI providers too.
I can also see how hassles like these to include legacy CocoaPods dependencies could accelerate more projects to migrate to SPM. SPM is already popular, and will likely become more popular as Apple ensures first-class integration.
Disclaimer: Xcode Cloud is in private beta so this issue may be resolved in future versions if shell script permissions are relaxed...
Related
Is it possible to run bentoml build without importing the services.py file during the process?
I'm trying to put the bento build and containarize steps in our CI/CD server. Our model depends on some OS packages installed and some python packages. I thought I could run bentoml build to package the model code and binaries that are present. I'd leave the dependencies especification to the contanairize step.
To my surprise the bentoml build process tried to import the service file during the packaging and the build failed since I didn't have the dependencies installed in my CI/CD machine.
Can I prevent this importing while building/packaging the model? Maybe I should ignore the bento containarize and create my bento container by hand and just execute the bentoml serve inside.
I feel that the need to install by hand the dependencies is doubling the effort to specify them in the bentofile.yaml and preventing the reproducibility of my environment.
This is not currently possible. The community is working on an environment management feature, such that an environment with the necessary dependencies will be automatically created during build.
Absolute beginner in DevOps here. I have a Gitlab repo that I would like to build and run its tests in the Gitlab pipeline CI.
So far, I'm only testing locally on my machine with a specific runner. There's a lot information out there and I'm starting to get lost with what to use and how to use it.
How would I go about creating a container with the tools that I need ? (VS compiler, cmake, git, etc...)
My application contains an SDK that only works on windows, so I'm not sure building on another platform would work at all, so how do I select a windows based container?
How would I use that container in the yml file in gitlab so that I can build my solution and run my tests?
Any specific documentation links or suggestions are welcomed and appreciated.
How would I go about creating a container with the tools that I need ? (VS compiler, cmake, git, etc...)
you can install those tools before the pipeline script runs. I usually do this in before_script.
If there's large-ish packages that need to be installed on every pipeline run, I'd recommend that you make yourown image, with all the required build dependencies, push it to GitLab and then just use it as your job image.
My application contains an SDK that only works on windows, so I'm not sure building on another platform would work at all, so how do I select a windows based container?
If you're using gitlab.com - Windows runners are currently in beta, but available for use.
SaaS runners on Windows are in beta and shouldn’t be used for production workloads.
During this beta period, the shared runner quota for CI/CD minutes applies for groups and projects in the same manner as Linux runners. This may change when the beta period ends, as discussed in this related issue.
If you're self-hosting - setup your own runner on Windows.
How would I use that container in the yml file in gitlab so that I can build my solution and run my tests?
This really depends on:
previous parts (you're using GL.com / self hosted)
how your application is built
what infrastructure you have access to
What I'm trying to say is that I feel like I can't give you a good answer without quite some more information
I have a problem. I need to build a job in my Jenkins server hosted by macmini (localhost) to automatic build a deploy for my mobile hybrid apps. That's apps was building with ionic2, and need a deploy for Android (apk) and for iOS (ipa).
But when i run a build from Jenkins, with this shell command
I get this error
that's not all... because i try to execute, from the jenkins folder, the npm i and the result was this:
When i try to build my application from other "folder" and not from jenkins, they works correctly. How i can solve?
Without more information hard to say; you can either just script the things you do when you deploy "manually", or you might want to try a CLI tool like https://www.bitrise.io/cli or https://fastlane.tools which can auto-scan your project and configure a suitable configuration which is then easier to tweak.
In case of Bitrise CLI the base config can be generated with bitrise init in the repo root, and you can also use a visual editor to modify your configuration: https://discuss.bitrise.io/t/how-to-experiment-with-bitrise-configs-locally-on-your-mac-linux/1751
After a lot of time, just find the solution. Jenkins have a own "tools management". So i need to install, into jenkins, a property version of nodeJS, Npm and all other tools i need to deploy the application.
So, first of all you need to install property plugin (in my case nodejs).
After this, going into Jenkins Management System and configure a NodeJS version. That's all
Using csx scripts in Azure Functions I can use the Project.json file to install nuget packages, but when I'm using fsx scripts the packages aren't installed (the log console never shows the Starting NuGet restore message). The only way I found is installing locally and uploading the dependencies. Am I missing something?
I think that the current execution model for F# in Azure functions does not support project.json. There is a work in progress PR to improve F# support that will enable this.
For now, I think there are two options:
Install the packages locally and upload them to Azure (as you are doing)
If you're deploying via git, then I think the deployment lets you run deployment script (in the same way in which Azure WebSites let you run a deployment script).
I have not tested the second approach with Azure functions, but I think it could work. For example, see the F# Snippets' deployment script which calls a build script that starts by using Paket to restore dependencies. This way, you need just paket.bootstrapper.exe and paket.dependencies with paket.lock to specify your NuGet dependencies.
When building an Hybrid iOS Cordova application (relying on Cordova CLI) I stumbled the need to build a custom plugin.
My workflow was the following:
I scaffolded my Cordova plugin outside my Hybrid Application
directory.
Code it blindly (in Vim) and push to GitHub.
Use cordova plugin rm and cordova plugin add to reinstall the plugin.
Test.
Repeat from step 2.
I deployed the plugin successfully in the product, however this process of coding Objective C blindly with Vim and doing a whole push to GitHub just to test it in my project feels awkward, however I couldn't find any information on the internet about a better workflow.
Ideally, this is the workflow I'd expect:
Scaffold my plugin inside my application directory (maybe in plugins/ or in my custom src/) and publish when I'm ready to do so (or not publish closed-source plugins at all).
Code the plugin using XCode.
Reinstall the plugin from the local filesystem, this totally avoiding a separate repo/push.
Test and iterate from step 2.
Is this workflow achievable?
You could try the following workflow:
have the plugin in a specific directory within the project
have a grunt task that watches for changes in the above directory that copies the relevant files to the plugins directory and runs cordova prepare right after the copy step is complete
edite and test...
Personally, I've used the same approach, with a twist:
Step 1:
I've coded the boilerplate part of the plugin using Sublime Text (vim works as well :) )
Step 2:
cordova plugin add --link ../path_to_my_plugin
Step 3:
I can add the platforms/android to Android Studio (you should be able to do the same with platforms/ios)
Step 4:
Any changes to the native part are reflected in the original sources (because of the --link parameter), any changes needed to the js part of the plugin I can edit directly in the app directory itself
Step 5:
Commit and push the files in your original plugin directory
If you need to reconfigure some part of the boilerplate (plugin.xml configurations), I've just:
cordova plugin remove <plugin-name>
And then resumed from step 2