I setup a simple Jenkinsfile that just echo a few steps.
I setup a new repo on Bitbucket (git) and two branches called master and develop.
When I commit something to master then both branches checkout and build in jenkins. Same behaviour on the develop branch.
Is it possible to limit only master to build in Jenkins once there is a commit to master branch? Similar behaviour to develop?
I think you can use a temporary text file to save last successful build SHA ($LAST_SUCCESSFUL_BUILD_SHA) of each branch. Then, when having a new commit from the repo, we'll check which branch the commit comes from.
CURRENT_SHA=$(git rev-parse HEAD)
if [ $FORCE_REBUILD = true ] || [ $CURRENT_SHA != $LAST_SUCCESSFUL_BUILD_SHA ]; then
echo "New commits available OR it was forced to build."
else
echo "Already up-to-date. Skip build."
curl -v -X POST --data "description=no changes, skip." ${JENKINS_BUILD_URL}submitDescription --user <username>:<password>
curl -v -X POST ${BUILD_URL}stop --user <username>:<password>
echo "Waiting for abort to take effect :D"
fi
If the new commit comes from another branch, it's good to use Jenkins open API to skip the build.
I have applied this for my freestyle jobs but haven't tried with Jenkinsfile.
Related
I am using Xcode Cloud with my iOS project. There are a few workflows based on branches and tags. Commonly one of them is finishing with app publication on the Test Flight.
My goal is to make an automation that will be run on develop branch every time when I will make some push. This automation will arise when script will find #promote tag inside commit message.
My first approach was to use post build script in Xcode Cloud as shown below:
#!/bin/sh
# ci_post_xcodebuild.sh
# BeforeDaily
#
# Created by Piotrek on 28/08/2022.
#
cd "$CI_WORKSPACE"
MESSAGE=$(git log -1 --oneline --format=%s | sed 's/^.*: //')
if [[ "$MESSAGE" == *"#promote"* ]]; then
echo "Merging develop into stage"
git fetch --all
git checkout --track -b stage origin/stage
git merge develop
git push origin stage
git checkout develop
echo "Done"
else
echo "There is nothing to do here..."
fi
This approach was promising but Xcode Cloud does not have write permissions on the Github project and it is not possible to change it. The result of the script is here:
2022-08-28T21:37:40.713643324Z Merging develop into stage
2022-08-28T21:37:42.747978838Z From http://github.com/<username>/<project_name>
2022-08-28T21:37:42.748494378Z * [new branch] develop -> origin/develop
2022-08-28T21:37:42.749001513Z * [new branch] main -> origin/main
2022-08-28T21:37:42.749224260Z * [new branch] stage -> origin/stage
2022-08-28T21:37:42.749514956Z Switched to a new branch 'stage'
2022-08-28T21:37:42.749752757Z branch 'stage' set up to track 'origin/stage'.
2022-08-28T21:37:42.750006530Z fatal: refusing to merge unrelated histories
2022-08-28T21:37:42.852059197Z remote: Write access to repository not granted.
2022-08-28T21:37:42.852480361Z fatal: unable to access 'http://github.com/piotrekjeremicz/beforedaily-swiftui-app.git/': The requested URL returned error: 403
2022-08-28T21:37:42.852854256Z Switched to branch 'develop'
2022-08-28T21:37:42.853020173Z Done
So the question is:
Is there any method that could provide me automatic merge based on commit message content?
I could not find any other solution than make a clean clone and work on fresh git repository. Everything is done in Xcode Cloud Workflow Here is a final ci_post_xcbuild.sh that makes a merge to stage if some message on develop contains #promote string.
#!/bin/sh
cd "$CI_WORKSPACE"
MESSAGE=$(git log -1 --oneline --format=%s | sed 's/^.*: //')
if [[ "$CI_BRANCH" == develop && "$MESSAGE" == *"#promote"* ]]; then
echo "Automerge develop branch into stage"
mkdir tempclone
cd tempclone
git clone https://x-access-token:$GITHUB_TOKEN#github.com/github_account/github_repo_name.git
cd beforedaily-swiftui-app
git config --global user.email "xcodecloud#jeremicz.com"
git config --global user.name "Xcode Cloud"
git checkout --track -b stage origin/stage
git merge origin/develop
git push origin stage
cd ../../
rm -r tempclone
else
echo "There is nothing to do here..."
fi
This script works with Github Token as an environment value that could be setup in Workflow settings.
Context
This question relates to multibranch pipelines where the behaviour merges the PR with the target branch revision (see screenshot of settings)
In this case, the merge may cause a new merge commit. So for a trigger with a given commit from a repository:
We actually get a different value of the GIT_COMMIT envvar:
If a tool (such as a build reporting tool) needs to use the GIT_COMMIT envvar to pass information onto a service, it cannot then be linked back to the actual commit from the project (this is a screenshot from Bitbucket but this would be the same for any repo hosting service):
Question
How, in a pipeline step, can I find the commit 709502c is the actual genesis of this build, when the GIT_COMMIT is set to 6781a3d1 (which is not an actual commit in the project)?
Maybe looking at git history can help?
REAL_GIT_COMMIT = sh (
script: "git rev-parse HEAD",
returnStdout: true,
).trim()
I've created a Jenkins Multibranch Pipeline with the GitHub Branch Source plugin. The Jenkinsfile essentially just calls a Cake Build script (build.ps1, build.cake) that contains all the build/deploy logic. This allows me to move to another CI service easily.
Unfortunately, I cannot seem to figure out how to add my Cake Build scripts as a trusted file so that PR's from forks will pull the files from the source repo instead. The Trust setting of the Discover pull requests from forks behavior seems to indicate that there can be other trusted files besides Jenkinsfile:
Nobody
Pull requests from forks will all be treated as untrusted. This means that where Jenkins requires a trusted file (e.g. Jenkinsfile) the contents of that file will be retrieved from the target branch on the origin repository and not from the pull request branch on the fork repository.
However, I cannot seem to find any documentation on adding other trusted files. The primary reason for this is to prevent a PR from a fork from accessing credentials from the Cake script. They wouldn't be able to change Jenkinsfile, but they could still change the Cake script to expose the credentials.
Is it actually possible to add other trusted files?
It seems like Jenkins does not support this. My solution is checking out the untrusted files manually from the base version instead. First getting the commit 's hash of the base version with:
def commit = sh(
script: 'git rev-parse HEAD',
returnStdout: true
).trim()
def base = sh(
script: "git rev-list --parents -n 1 ${commit}",
returnStdout: true
).trim().split('\\s+')[2]
git rev-list --parents -n 1 ${commit} will return the hash of current commit, which is a merge commit that was created by Jenkins; the latest commit of the PR and the latest commit of the target branch, separated by a space (e.g. 05e9322574ea03003f87dcbb44f172e6fa62581f b3f6ef892af9c645f490106757d7d05df3a26060 069ffd55ae36414a51b4de166aef86966f9447a8). Hence, we grab the hash of the latest commit of the target branch by trim().split('\\s+')[2].
Now we can do sh "git checkout ${base} FILE" on any file that we don't trust from the PR.
This does not works if the PR is already merged with the latest version of target branch. So what I did is something like this:
// revert untrusted files to the base version and backup it before we execute any untrusted code so the attacker
// don't have a chance to put a malicious content
def latest = sh(script: 'git rev-parse HEAD', returnStdout: true).trim()
sh "git checkout origin/${env.CHANGE_TARGET}"
def baseCompose = readFile('docker-compose.yml')
// switch back to latest commit
sh "git checkout ${latest}"
sh 'git clean -d -f -f -q -x'
The following code is an "Execute Shell" build step in Jenkins. The job pulls from a repo which contains a file ranger-policies/policies.json. What I'd like to do is update that file (with a curl command, in this case) and then commit the change to source control and update the remote repo. The job successfully pulls from the remote repo in the "Source Code Management" section of the job configuration page over SSH using SSH keys. However, when the job gets to the "git push origin master" line in the "Execute Shell" step, I get a Permission denied (publickey) error, as if those same SSH keys which allowed me to successfully pull the repo are not available in the "Execute Shell" step when I want to push.
curl -X GET --header "text/json" -H "Content-Type: text/json" -u user:pass "http://my-url.com/exportJson" > ranger-policies/policies.json
git add ranger-policies/policies.json
git commit -m "udpate policies.json with latest ranger policies `echo "$(date +'%Y-%m-%d')"`"
git push origin master
I ended up figuring out how to make it work. The solution involves using the SSH Agent plugin. Here's a step-by-step that describes how I did it, hopefully it helps someone else:
First, create a new pipeline job.
Then, as hinted at in this post from Jenkins' documentation, go to the home screen for your new pipeline job, and click on "Pipeline Syntax." Choose "git: Git" as the "Sample Step, and enter the git repo you want to push to in the "Repository URL" field. Then choose the corresponding valid SSH keys for that repo from the "Credentials dropdown." Everything should look like this:
Grab the value of "credentialsId", highlighted with red in the above screenshot. You'll need it later.
Install the "Workspace Cleanup Plugin" (https://wiki.jenkins.io/display/JENKINS/Workspace+Cleanup+Plugin, optional) and the "SSH Agent Plugin" (https://jenkins.io/doc/pipeline/steps/ssh-agent/, not optional, required for this process to work).
Now go back to your new pipeline job and hit "Configure," which will take you to the screen where you define the job. Drop the following code into the "Pipeline" section ("Definition" should be set to "Pipeline script"): https://gist.github.com/ScottNeaves/5cdce294296437043b24f0f3f0a8f1d8
Drop your "credentialsId" into the appropriate places in the above Jenkinsfile, and fix up the repo names to target the repo you want, and you should be good to go.
Relevant documentation:
https://jenkins.io/doc/pipeline/examples/#push-git-repo
https://gist.github.com/blaisep/eb8aa720b06eff4f095e4b64326961b5#file-jenkins-pipeline-git-cred-md
https://issues.jenkins-ci.org/browse/JENKINS-28335?focusedCommentId=269000&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-269000
As per this gist, you need to set the remote origin url as per:
git remote set-url origin git#github.com:username/your-repository.git
In some jenkins shell command I create new git branch as follow:
git checkout -b new_branch
...
run more commands
..
sometimes running some commands is not successfull and the job exits.
Now when I run the job again it says that the branch new_branch already exists.
Is there any option in jenkins to cleans up the locally created branches every time when the job starts or something like that?
2 ways to go about it :
Run "git branch -d new_branch" to delete your local branch
Clean your workspace before checking out