I am using Xcode Cloud with my iOS project. There are a few workflows based on branches and tags. Commonly one of them is finishing with app publication on the Test Flight.
My goal is to make an automation that will be run on develop branch every time when I will make some push. This automation will arise when script will find #promote tag inside commit message.
My first approach was to use post build script in Xcode Cloud as shown below:
#!/bin/sh
# ci_post_xcodebuild.sh
# BeforeDaily
#
# Created by Piotrek on 28/08/2022.
#
cd "$CI_WORKSPACE"
MESSAGE=$(git log -1 --oneline --format=%s | sed 's/^.*: //')
if [[ "$MESSAGE" == *"#promote"* ]]; then
echo "Merging develop into stage"
git fetch --all
git checkout --track -b stage origin/stage
git merge develop
git push origin stage
git checkout develop
echo "Done"
else
echo "There is nothing to do here..."
fi
This approach was promising but Xcode Cloud does not have write permissions on the Github project and it is not possible to change it. The result of the script is here:
2022-08-28T21:37:40.713643324Z Merging develop into stage
2022-08-28T21:37:42.747978838Z From http://github.com/<username>/<project_name>
2022-08-28T21:37:42.748494378Z * [new branch] develop -> origin/develop
2022-08-28T21:37:42.749001513Z * [new branch] main -> origin/main
2022-08-28T21:37:42.749224260Z * [new branch] stage -> origin/stage
2022-08-28T21:37:42.749514956Z Switched to a new branch 'stage'
2022-08-28T21:37:42.749752757Z branch 'stage' set up to track 'origin/stage'.
2022-08-28T21:37:42.750006530Z fatal: refusing to merge unrelated histories
2022-08-28T21:37:42.852059197Z remote: Write access to repository not granted.
2022-08-28T21:37:42.852480361Z fatal: unable to access 'http://github.com/piotrekjeremicz/beforedaily-swiftui-app.git/': The requested URL returned error: 403
2022-08-28T21:37:42.852854256Z Switched to branch 'develop'
2022-08-28T21:37:42.853020173Z Done
So the question is:
Is there any method that could provide me automatic merge based on commit message content?
I could not find any other solution than make a clean clone and work on fresh git repository. Everything is done in Xcode Cloud Workflow Here is a final ci_post_xcbuild.sh that makes a merge to stage if some message on develop contains #promote string.
#!/bin/sh
cd "$CI_WORKSPACE"
MESSAGE=$(git log -1 --oneline --format=%s | sed 's/^.*: //')
if [[ "$CI_BRANCH" == develop && "$MESSAGE" == *"#promote"* ]]; then
echo "Automerge develop branch into stage"
mkdir tempclone
cd tempclone
git clone https://x-access-token:$GITHUB_TOKEN#github.com/github_account/github_repo_name.git
cd beforedaily-swiftui-app
git config --global user.email "xcodecloud#jeremicz.com"
git config --global user.name "Xcode Cloud"
git checkout --track -b stage origin/stage
git merge origin/develop
git push origin stage
cd ../../
rm -r tempclone
else
echo "There is nothing to do here..."
fi
This script works with Github Token as an environment value that could be setup in Workflow settings.
Related
I have 2 repos A and B. At the end of build A, I want to update a properties file in repo B with the build number of A
How can I checkout the master branch of repo B in repo A's Jenkinfile just for this stage(which is last) ?
After checkout, can I follow the steps
mentioned below to update the file or is there any better way to
achieve it?
Jenkinsfile of repo A:
stage('Update properties file in repo B') {
steps {
script {
// how do I checkout master branch ofrepo B here?
sh "git config --global user.email jenkins#abc.com"
sh "git config --global user.name Jenkins"
sh(script: 'echo "repoA_VERSION=$BUILD_NUMBER" > version.properties', returnStdout: true).trim()
git add .
git commit -m "Updated version.properties file with ${env.BUILD_NUMBER}"
}
}
}
You just need to call git step:
E.g.
git branch: 'your_branch', credentialsId: 'your_credentials', url: 'your_repo'
If you don't know identifier for your credentials, you can go to:
your_jenkins_server:8080/job/job_name/pipeline-syntax/
where job_name is any job in your server, and you will access to Pipeline Syntax, you can then configure your git checkout and generate command:
For number 2,
the code must be within sh command:
sh """git add .
git commit -m "Updated version.properties file with ${env.BUILD_NUMBER}"
git push
"""
A final recommendation, avoid using git add ., and add files individually or with a wildcard.
I've created a Jenkins Multibranch Pipeline with the GitHub Branch Source plugin. The Jenkinsfile essentially just calls a Cake Build script (build.ps1, build.cake) that contains all the build/deploy logic. This allows me to move to another CI service easily.
Unfortunately, I cannot seem to figure out how to add my Cake Build scripts as a trusted file so that PR's from forks will pull the files from the source repo instead. The Trust setting of the Discover pull requests from forks behavior seems to indicate that there can be other trusted files besides Jenkinsfile:
Nobody
Pull requests from forks will all be treated as untrusted. This means that where Jenkins requires a trusted file (e.g. Jenkinsfile) the contents of that file will be retrieved from the target branch on the origin repository and not from the pull request branch on the fork repository.
However, I cannot seem to find any documentation on adding other trusted files. The primary reason for this is to prevent a PR from a fork from accessing credentials from the Cake script. They wouldn't be able to change Jenkinsfile, but they could still change the Cake script to expose the credentials.
Is it actually possible to add other trusted files?
It seems like Jenkins does not support this. My solution is checking out the untrusted files manually from the base version instead. First getting the commit 's hash of the base version with:
def commit = sh(
script: 'git rev-parse HEAD',
returnStdout: true
).trim()
def base = sh(
script: "git rev-list --parents -n 1 ${commit}",
returnStdout: true
).trim().split('\\s+')[2]
git rev-list --parents -n 1 ${commit} will return the hash of current commit, which is a merge commit that was created by Jenkins; the latest commit of the PR and the latest commit of the target branch, separated by a space (e.g. 05e9322574ea03003f87dcbb44f172e6fa62581f b3f6ef892af9c645f490106757d7d05df3a26060 069ffd55ae36414a51b4de166aef86966f9447a8). Hence, we grab the hash of the latest commit of the target branch by trim().split('\\s+')[2].
Now we can do sh "git checkout ${base} FILE" on any file that we don't trust from the PR.
This does not works if the PR is already merged with the latest version of target branch. So what I did is something like this:
// revert untrusted files to the base version and backup it before we execute any untrusted code so the attacker
// don't have a chance to put a malicious content
def latest = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
sh "git checkout origin/${env.CHANGE_TARGET}"
def baseCompose = readFile('docker-compose.yml')
// switch back to latest commit
sh "git checkout ${latest}"
sh 'git clean -d -f -f -q -x'
I use continuous integration with Travis to run my unit tests on every commit. However, sometimes all I want to do is edit the README. Is there a way to skip Travis builds if all the changes are restricted to a whitelisted set of files?
There's no way to directly make Travis dynamically determine, based only on the type of file that has been changed, if it should run a build.
However, Travis will ignore any commit with [ci skip] or [skip ci] in the commit message.
Perhaps you could use a git hook (say prepare-commit-msg or similar) to append [ci-skip] to the commit message when only .md files have been modified.
In the git hook, you could detect this scenario with a command like git diff --exit-code --name-only -- . ':(exclude)*.md'.
In action:
$ git diff --name-only
README.md
$ git diff --exit-code --name-only -- . ':(exclude)*.md'
$ echo $?
0
If any non *.md files have been changed, the command will return 1, otherwise 0.
I setup a simple Jenkinsfile that just echo a few steps.
I setup a new repo on Bitbucket (git) and two branches called master and develop.
When I commit something to master then both branches checkout and build in jenkins. Same behaviour on the develop branch.
Is it possible to limit only master to build in Jenkins once there is a commit to master branch? Similar behaviour to develop?
I think you can use a temporary text file to save last successful build SHA ($LAST_SUCCESSFUL_BUILD_SHA) of each branch. Then, when having a new commit from the repo, we'll check which branch the commit comes from.
CURRENT_SHA=$(git rev-parse HEAD)
if [ $FORCE_REBUILD = true ] || [ $CURRENT_SHA != $LAST_SUCCESSFUL_BUILD_SHA ]; then
echo "New commits available OR it was forced to build."
else
echo "Already up-to-date. Skip build."
curl -v -X POST --data "description=no changes, skip." ${JENKINS_BUILD_URL}submitDescription --user <username>:<password>
curl -v -X POST ${BUILD_URL}stop --user <username>:<password>
echo "Waiting for abort to take effect :D"
fi
If the new commit comes from another branch, it's good to use Jenkins open API to skip the build.
I have applied this for my freestyle jobs but haven't tried with Jenkinsfile.
On the install step, Travis CI clones the repo, which looks similar to this:
git clone --depth=50 --branch=master https://github.com/user/repo.git user/repo
How can I customize / override this?
Background: I am using tag based deploys. The way Travis checks out tagged builds (--branch=<tagname>), the git repository is in a detached state without access to branches. However, for deployment I need to know on which branch I am. My solution is to do a "normal" clone and then switch to the tagged commit.
You can clone the repository again in the install step. That way you clone the repository twice, but it seems to work.
# .travis.yml
install:
- git clone https://github.com/$TRAVIS_REPO_SLUG.git $TRAVIS_REPO_SLUG
- cd $TRAVIS_REPO_SLUG
- git checkout -qf $TRAVIS_COMMIT
Per the Travis docs you can add the following to your .travis.yml to remove the --depth flag:
git:
depth: false
As --depth implies --single-branch, removing this flag means that all branches will be checked out, which isn't the default behaviour.
I found that in order to get access to your whole repo you need the following:
install:
- git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
- git fetch --unshallow --tags
This way you'll have access to remote branches and tags (e.g. can do checkout).
If you're on a tag but no longer want to be in a detached HEAD state you can create a new branch that points to the tag (according to this discussion):
install:
- git config remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
- git fetch --unshallow --tags
- git symbolic-ref --short HEAD || git checkout -b ${TRAVIS_BRANCH}-test $TRAVIS_BRANCH
Note: git symbolic-ref --short HEAD will fail if you're in a detached HEAD state.
The problem is not really that you are in a detached branch. It is that git does not allow you to fetch the tags: git fetch --tags will only fetch the branch spcified by --branch in the git clone command you gave.
I explain this in more details this answer.
To solve you issue (checking out a specific tag) you can call a script that looks like this, after the repo is cloned:
# Keep track of where Travis put us.
# We are on a detached head, and we need to be able to go back to it.
build_head=$(git rev-parse HEAD)
# fetch the tags
git config --replace-all remote.origin.fetch +refs/heads/*:refs/remotes/origin/*
git fetch --tags
# checkout the tagged commit
git checkout -qf <your tag>
# now do your stuff
# go back to where we were at the beginning
git checkout ${build_head}
Run this during your build to have access to origin tags / branches
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" 1>/dev/null
git fetch origin -q
After that you can run this command to find branches containing your commit
BRANCHES=`git branch -a --contains "$TRAVIS_TAG"`
I've created a script a loooong time ago to fetch the 'environment' branch where the tag was created for continuous deployment purpose.
It may inspire you: https://gist.github.com/rolebi/a0eb1f783b7f3a5f21a631c8da1582dc
Use it like that:
TARGET_ENV="`test $TRAVIS_TAG && bash scripts/get_branch_for_git_reference.sh $TRAVIS_TAG`"
Disable git clone and then clone the repository again in the install step. The repository will be cloned only once in this way. In this "normal" clone you will able to do what ever you want.
git:
clone: false
install:
- git clone https://github.com/$TRAVIS_REPO_SLUG.git $TRAVIS_REPO_SLUG
- cd $TRAVIS_REPO_SLUG
Or you could just query the remote. Add the following to .travis.yml:
env:
global:
# get all the branches referencing this commit
- REAL_BRANCH=$(git ls-remote origin | sed -n "\|$TRAVIS_COMMIT\s\+refs/heads/|{s///p}")
# or check if we are on a particular branch:
- IS_RELEASE=$(git ls-remote origin | grep "$TRAVIS_COMMIT\s\+refs/heads/release$"
(I am surprised that the git gurus hadn’t come up with this one already)
You can convert the already existing shallow clone to a full clone. To do so execute git fetch --unshallow (available since git version 1.8.3) during the install step.
# .travis.yml
install:
- git fetch --unshallow --tags
The --tags flag forces to fetch all tags even if they don't belong to the checked out branch. This is needed if your build also depends on tags from other branches.