When I push an empty branch to my GitHub repo (perhaps by creating a branch through the GitHub web UI), CircleCI sees it and runs all the tests for the commit at the tip of that branch.
This makes sense, but I've discovered an unfortunate side effect: the CircleCI jobs for the branch clobber the GitHub statuses for master with inaccurate information.
Here's the sequence of events:
I push commit A to master. This runs my whole CircleCI workflow and posts information on test success / failure for A to GitHub using the GitHub Status API.
I create a new branch, mybranch, using the GitHub UI. Its tip is also at commit A.
CircleCI runs all the tests for mybranch. These are also on commit A, so CircleCI re-posts statuses to the GitHub status API for commit A.
This might be OK, except that I have an optimization: I early-out of tests that shouldn't be affected by a PR branch. So if the branch contains no TypeScript files in its diff from master, I won't run the TypeScript tests. Unfortunately, this means that step 3 overrides the GitHub status for the full run on commit A with one that early outs (since there are no commits, there's no diff).
Here's what the "skip" logic looks like:
skip-job-if-no-ts:
description: Skips the rest of the job if files under ts/ have not been modified
steps:
- when:
condition:
not:
or:
# Always run all tests if this is a commit to master branch
- equal: [master, << pipeline.git.branch >>]
steps:
- run:
name: skip-job-if-no-ts
command: |
# Skip the rest of the job if ts/ wasn't modified
TS_CHANGED=$(git diff $(git merge-base HEAD origin/master) --name-only ts/)
if [ ! -n "$TS_CHANGED" ]; then
circleci-agent step halt
fi
How can I prevent CircleCI from running my job on an empty branch? I found an old discussion of this issue on the CircleCI forums, but it was closed with no comments.
Related
Context
This question relates to multibranch pipelines where the behaviour merges the PR with the target branch revision (see screenshot of settings)
In this case, the merge may cause a new merge commit. So for a trigger with a given commit from a repository:
We actually get a different value of the GIT_COMMIT envvar:
If a tool (such as a build reporting tool) needs to use the GIT_COMMIT envvar to pass information onto a service, it cannot then be linked back to the actual commit from the project (this is a screenshot from Bitbucket but this would be the same for any repo hosting service):
Question
How, in a pipeline step, can I find the commit 709502c is the actual genesis of this build, when the GIT_COMMIT is set to 6781a3d1 (which is not an actual commit in the project)?
Maybe looking at git history can help?
REAL_GIT_COMMIT = sh (
script: "git rev-parse HEAD",
returnStdout: true,
).trim()
I am trying to create a base image for my repo that is optionally re-built when branches (merge requests) make changes to dependencies.
Let's say I have this pipeline configuration:
stages:
- Test
- Build
variables:
- image: main
Changes A:
stage: Test
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
changes:
- path/to/a
script:
- docker build -t a .
- docker push a
- echo 'image=a' > dotenv
artifacts:
reports:
dotenv: dotenv
Build:
stage: Build
image: $image
script:
- echo build from $image
Let's say I push to a new branch and the first commit changes /path/to/a, the docker image is build and pushed, the dotenv is updated and the Build job successfully uses image=a.
Now, let's say I push a new commit to the same branch. However, the new commit does not change /path/to/a so the Changes A job does not run. Now, the Build stage pulls the "wrong" default image=main while I would like it to still pull image=a since it builds on top of the previous commit.
Any ideas on how to deal with this?
Is there a way to make rules.changes refer to origin/main?
Any other ideas on how to achieve what I am trying to do?
Is there a way to make rules.changes refer to origin/main?
Yes, there is, since GitLab 15.3 (August 2022):
Improved behavior of CI/CD changes with new branches
Improved behavior of CI/CD changes with new branches
Configuring CI/CD jobs to run on pipelines when certain files are changed by using rules: changes is very useful with merge request pipelines.
It compares the source and target branches to see what has changed, and adds jobs as needed.
Unfortunately, changes does not work well with branch pipelines.
For example, if the pipeline runs for a new branch, changes has nothing to compare to and always returns true, so jobs might run unexpectedly.
In this release we’re adding compare_to to rules:changes for both jobs and workflow:rules, to improve the behavior in branch pipelines.
You can now configure your jobs to check for changes between the new branch and the defined comparison branch.
Jobs that use rules:changes:compare will work the way you expect, comparing against the branch you define.
This is useful for monorepos, where many independent jobs could be configured to run based on which component in the repo is being worked on.
See Documentation and Issue.
You can use it only as part of a job, and it must be combined with rules:changes:paths.
Example:
docker build:
script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
paths:
- Dockerfile
compare_to: 'refs/heads/branch1'
In this example, the docker build job is only included when the Dockerfile has changed relative to refs/heads/branch1 and the pipeline source is a merge request event.
There is a project setting, which defines how your MR pipelines are setup. This is only working for Merge requests and can be found in Settings -> Merge Requests under the section Merge options
each commit individually - nothing checked
this means, each commit is treated on its own, and changes checks are done against the triggering commit on it's own
Enabled merged results pipeline
This will merge your MR with the target branch before running the CI Jobs. This also will evaluate all your changes, and take a look at all of them within the MR and not commit wise.
Merge trains
This is a whole different chapter, and for this usecase not relevant. But for completeness i have to mention it see https://gitlab.com/help/ci/pipelines/merge_trains.md
What you are looking for is Option 2 - Merged pipeline results. but as i said, this will only work in Merge Request pipelines and not general pipelines. So you would also need to adapt your rules to something like:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes:
- path/to/a
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
I have an issue with gitlab, gitlab-ci and fastlane. In fastlane, I am using :
def increment_build_and_commit
incrementBuildNumber()
commit_version_bump(message: 'build number bump [skip ci]')
push_to_git_remote(
remote: 'origin_rw',
remote_branch: ENV['CI_COMMIT_BRANCH']
)
end
So I want to increase build number and commit. So it does the job with:
incrementBuildNumber(),
commit_version_bump(message: 'build number bump [skip ci]')
But when running, it fails at push_to_git_remote
[!] Exit status of command 'git push origin_rw HEAD:HEAD --tags
--set-upstream' was 1 instead of 0. error: The destination you provided is not a full refname (i.e., starting with "refs/"). We
tried to guess what you meant by:
Looking for a ref that matches 'HEAD' on the remote side.
Checking if the being pushed ('HEAD') is a ref in "refs/{heads,tags}/". If so we add a corresponding
refs/{heads,tags}/ prefix on the remote side. Neither worked, so we
gave up. You must fully qualify the ref. hint: The part of the
refspec is a commit object. hint: Did you mean to create a new branch
by pushing to hint: 'HEAD:refs/heads/HEAD'? error: failed to push
some refs to '**********.git'
Some of you have any clues what to do and how to solve this problem? As I am going to be a google machine on that, but unfortunetly can't find the solution.
Thanks in advance!!
This happens because gitlab-ci checkouts git commit instead of git branch and you enter detached HEAD state.
There are few ways to get around this:
1) Use CI provided value. For GitLab I would recommend using CI_PIPELINE_IID. Downside is that you cannot control it and if you migrate your project to other repository it will start from 1 again.
2) Checkout git branch before making commit. Downside is that if other developer pushes to the same branch, your pipeline will continue with wrong source code.
3) Commit version number bump manually.
I am trying to set up Chromatic in Travis.
I saw Chromatic document recommends using this script in Travis
if [[ $TRAVIS_EVENT_TYPE != 'pull_request' || $TRAVIS_PULL_REQUEST_SLUG != $TRAVIS_REPO_SLUG ]];
then
npm run chromatic
fi
with explanation
Travis offers two type of builds for commits on pull requests: so
called pr and push builds. It only makes sense to run Chromatic once
per PR, so we suggest disabling Chromatic on pr builds for internal
PRs (i.e. PRs that aren’t from forks). You should make sure that you
have push builds turned on, and add the following code: [[THE CODE ABOVE]]
For external PRs (PRs from forks of your repo), the above code will
ensure Chromatic does run on the pr build, because Travis does not
trigger push builds in such cases.
Note: We recommend running Chromatic on push builds as pr builds can't
always run and fall out of the normal git ancestry. For instance, if
you change the base branch of a PR, you may find that you need to
re-approve changes as some history may be lost.
Chromatic does work with Travis pr builds however!
Then I read the Travis document about TRAVIS_PULL_REQUEST_SLUG and TRAVIS_REPO_SLUG.
TRAVIS_PULL_REQUEST_SLUG:
if the current job is a pull request, the slug (in the form
owner_name/repo_name) of the repository from which the PR originated.
if the current job is a push build, this variable is empty ("").
TRAVIS_REPO_SLUG:
The slug (in form: owner_name/repo_name) of the repository currently being built.
So my understanding is when $TRAVIS_PULL_REQUEST_SLUG != $TRAVIS_REPO_SLUG, it is a push build, then why it still needs $TRAVIS_EVENT_TYPE != 'pull_request'?
Is there any difference between them?
Gert from Chroma here.
We recommend to run chromatic only when a build does not originate from a pull request (i.e. a push build), OR when it is a pull request build originating from a fork, in which case the PR slug differs from the repo slug.
Based on the quoted Travis docs, in case of a push build, TRAVIS_PULL_REQUEST_SLUG is empty, in case of a PR build from a fork, it will reference the fork's repo owner. In either case it will be different from TRAVIS_REPO_SLUG. So you're correct that the left-hand side of this condition is superfluous. Feel free to omit it.
Gitflow workflow states that Hotfix branches will be merged to both Develop and Master branches.
Since engineers forgot more than once to merge their hotfix to Develop, I want to mandate that the merge to develop is done first.
How can I configure Bitbucket to block merges of commits, that do not already exist in Develop, to master?
I don't know about BitBucket, but if you're able to install your own hooks, it's easy enough to script whatever you want in a pre-recive or update hook.
The following is an (untested) example, there are perhaps more efficient git commands to accomplish this, I'm not sure.
#!/bin/bash
# ... << Code to set up your hook variables here >>> ...
if [ "$(git branch Develop --contains $new_rev | wc -l)" -eq 0 ]; then
echo "ERROR: You must commit to Develop first and then merge"
exit 1
fi
exit 0