To automate our CI process, I need run the Bitbucket pipelines only when the title not starts with "Draft" or "WIP". Atlassian has only this features https://support.atlassian.com/bitbucket-cloud/docs/use-glob-patterns-on-the-pipelines-yaml-file/.
I tried with the regex ^(?!Draft:|WIP:).+ like this:
pipelines:
pull-requests:
'^(?!Draft:|WIP:).+':
- step:
name: Tests
but the pipeline not start under any circumstances (with or withour Draft:/WIP:). Any suggestions?
Note the PR pattern you define in the pipelines is matched against the source branch, not the PR title. Precisely, I used to feature an empty pipeline for PRs from wip/* branches, e.g.
pipelines:
pull-requests:
wip/*:
- step:
name: Pass
script:
- exit 0
"**":
- step:
name: Tests
# ...
But this workflow requires you to work on wip/* branches and changing their source branch later on. This is somewhat cumbersome and developers just did not opt-in.
This works, though.
I am little confused here regarding PR being triggered against main branch?
All branches:
(I know this will trigger pull request from any branch to any branch)
pipelines:
pull-requests:
'**':
Main branch:
(Does this trigger pull request if created from feature/pe-1234 to main?)
pipelines:
pull-requests:
'main':
I want to know what happens if I mention only main. It is not clear in documentation or may be I didn't get it right
The branch name / glob pattern in the pull-request pipeline definition is the source branch that should trigger that pipeline, not the target branch.
E.g. if you were following git-flow instead of github-flow, it would make sense to override the pipeline run by the PR from main to a release/whatever branch so that it simply passes, or does an integration test, but does not perform the usual tests, linting, coverage and whatnot.
pipelines:
pull-requests:
'**': # triggers if no other specific pipeline was triggered
- parallel:
- step: *linting-step
- step: *testing-step
main: # triggers from main to anywhere else
- step:
name: Pass
script:
- exit 0
If following github-flow, you will probably never make a PR from main to anywhere else, so you can safely skip this definition. Only if you wanted PRs from feature/AAA-NNNN branches to trigger a special pipeline besides the testing workflow, you can write an alternate pipeline like
pipelines:
pull-requests:
'**': # triggers if no other specific pipeline was triggered
- parallel:
- step: *linting-step
- step: *testing-step
feature/*: # triggers from feature/* to anywhere else (including to main)
- parallel:
- step: *linting-step
- step: *testing-step
- step: *maybe-hook-issue-tracker-step # ?
so that the simpler default '**' pipeline will not run. But it will run irrespective of the target branch, usually main but not necessarily.
I have a monorepo project that is deployed to 3 environments - testing, staging and production. Deploys to testing come from the next branch, while staging and production from the master branch. Testing deploys should run automatically on every commit to next (but I'm also fine with having to trigger them manually), but deploys from the master branch should be triggered manually. In addition, every deploy may consist of a client push and server push (depending on the files changed). The commands to deploy to each of the hosts are exactly the same, the only thing changing is the host itself and the environment variables.
Therefore I have 2 questions:
Can I make Bitbucket prompt me the deployment target when I manually trigger the pipeline, thus basically letting me choose the set of the env variables to inject into the set sequence of commands? I've seen a screenshot for this in a tutorial, but I lost it and can't find it since.
Can I have parallel sequences of commands? I'd like the server and the client push to run simultaneously, but both of them have different steps. Or do I need to merge those into the same step with multiple scripts to achieve that?
Thank you for your help.
The answer to both of your questions is 'Yes'.
The feature that makes it possible is called custom pipelines. Here is a neat doc that demonstrates how to use them.
There is a parallel keyword which you can use to define parallel steps. Check out this doc for details.
If I'm not misinterpreting the description of your setup, your final pipeline should look very similar to this:
pipelines:
custom:
deploy-to-staging-or-prod: # As you say the steps are the same, only variable values will define the destination.
- variables: # List variable names under here, and Bitbucket will prompt you to supply their values.
- name: VAR1
- name: VAR2
- parallel:
- step:
- ./deploy-client.sh
- step:
- ./deploy-server.sh
branches:
next:
- step:
script:
- ./deploy-to-testing.sh
UPD
If you need to use Deployments instead of providing each variable separately, use can utilise manual type of trigger:
definitions:
steps:
- step: &RunTests
script:
- ./run-tests.sh
- step: &DeployFromMaster
script:
- ./deploy-from-master.sh
pipelines:
branches:
next:
- step:
script:
- ./deploy-to-testing.sh
master:
- step: *RunTests
- parallel:
- step:
<<: *DeployFromMaster
deployment: staging
trigger: manual
- step:
<<: *DeployFromMaster
deployment: production
trigger: manual
Key docs for understanding this pipeline is still this one and this one for yaml anchors. Keep in mind that I introduced a 'RunTests' step on purpose, as
Since a pipeline is triggered on a commit, you can't make the first step manual.
It will act as a stopper for the deploy step which can only be manual due to your requirements.
I am setting up Github Actions for a project repository.
The workflow consists of the following steps:
Building a docker image
Pushing the image to a container registry
Rollout a Kubernetes deployment.
However, I have two different Kubernetes deployments: one for development, and one for production. Hence, I have also two Github Action workflows.
The Github Action workflow for development is triggered everytime that a commit is pushed:
on:
push:
branches:
- master
But I don't want that for my production workflow. I would need a manual trigger, like a Send to production button. I didn't see anything close to that in the docs.
Is there a way to trigger a workflow manually in Github Actions?
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?
Is there a way to trigger a workflow manually in Github Actions?
You might consider, from July2020:
GitHub Actions: Manual triggers with workflow_dispatch
(Note: or multiple workflows, through the new Composite Run Steps, August 2020)
You can now create workflows that are manually triggered with the new workflow_dispatch event.
You will then see a 'Run workflow' button on the Actions tab, enabling you to easily trigger a run.
You can choose which branch the workflow is run on.
philippe adds in the comments:
One thing that's not mentioned in the documentation: the workflow must exist on the default branch for the "Run workflow" button to appear.
Once you add it there, you can continue developing the action on its own branch and the changes will take effect when run using the button
The documentation goes on:
In addition, you can optionally specify inputs, which GitHub will present as form elements in the UI. Workflow dispatch inputs are specified with the same format as action inputs.
For example:
on:
workflow_dispatch:
inputs:
logLevel:
description: 'Log level'
required: true
default: 'warning'
tags:
description: 'Test scenario tags'
The triggered workflow receives the inputs in the github.event context.
For example:
jobs:
printInputs:
runs-on: ubuntu-latest
steps:
- run: |
echo "Log level: ${{ github.event.inputs.logLevel }}"
echo "Tags: ${{ github.event.inputs.tags }}"
shim adds in the comments:
You can add workflow_dispatch to a workflow that also has other triggers (like on push and / or schedule)
For instance:
on:
workflow_dispatch:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
EDITED :
Great tweet explaining the use of workflow dispatch :
https://twitter.com/github/status/1321859709075394563?s=19
Is there a way to trigger a workflow manually in Github Actions?
I've got a little hack to do so...
With the watch event, you can manually trigger an action by star or unstar the repo. The code for the event in your workflow is :
on:
watch
types: [started]
I know it's weird but it works! Nevertheless, it's not the best way if it's a public repo with potential stars.
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?
In Github Actions I mean, you can do multiple workflows / jobs and filter by targeted branches or events. You can combine multiple events for example trigger a workflow for push and with a cron on midnight.
Update: For a slash command style "ChatOps" solution see slash-command-dispatch action. This can allow you to trigger workflows with slash commands (e.g. /deploy) from issue and pull request comments.
Here is a basic example for a deploy slash command. REPO_ACCESS_TOKEN is a repo scoped Personal Access Token
name: Slash Command Dispatch
on:
issue_comment:
types: [created]
jobs:
slashCommandDispatch:
runs-on: ubuntu-latest
steps:
- name: Slash Command Dispatch
uses: peter-evans/slash-command-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
commands: deploy
The command can be processed in this workflow.
name: Deploy Command
on:
repository_dispatch:
types: [deploy-command]
There are many more options and different setups. See slash-command-dispatch for full usage instructions.
Original Answer:
A repository_dispatch workflow can be manually triggered by a call to the GitHub API as follows.
on:
repository_dispatch:
types: [production-deploy]
[username] is a GitHub username
[token] is a repo scoped Personal Access Token
[repository] is the name of the repository the workflow resides in.
curl -XPOST -u "[username]:[token]" \
-H "Accept: application/vnd.github.everest-preview+json" \
-H "Content-Type: application/json" \
https://api.github.com/repos/[username]/[repository]/dispatches \
--data '{"event_type": "production-deploy"}'
Another way to resolve this with the current Github Action offering is to create a production branch from master when a deploy is needed & trigger deploy action on the production branch. The production branch is essentially a mirror of the master.
on:
push:
branches:
- master
Dev builds/push can happen whenever there is a commit to the master.
on:
push:
branches:
- production
At some point in the release schedule, you can raise the PR to the production branch. This will take care of the prod build/deploy.
Although Sarah's post was the closest and simplest answer to the original question, it is somewhat hacky so we eventually ended up by creating a dev branch to use the following triggers:
Development workflow: triggered when a push is made on the dev branch:
on:
push:
branches:
- dev
Production workflow: triggered when a pull request / merge is made from dev to master:
on:
pull_request:
branches:
- master
Edited for more detail/explanation.
One thing that you can do is call to repository_dispatch. You can view the GitHub documentation for using a repository_dispatch here.
For example, if you have a GitHub Actions workflow that looks like this:
on:
repository_dispatch:
types: [run_tests]
name: Run tests
jobs:
test:
name: Run your tests
runs-on: ubuntu-latest
steps:
- run: |
echo "I just ran all your tests!"
You can create a repository dispatch event by following the steps that are explained on the GitHub v3 API Documentation.
First, create a personal access token (PAT) on GitHub for authentication.
Then, you can run curl like so:
curl \
-H "Authorization: token $YOUR_PAT" \
--request POST \
--data '{"event_type": "run_tests"}' \
https://api.github.com/repos/$USER/$REPOSITORY/dispatches
At the same time, I also wanted to share a small project that I've been working on with a buddy that solves this exact problem.
https://www.actionspanel.app/
ActionsPanel uses this same repository_dispatch API but does so with a GitHub App token so that you don't need to worry about managing your own PAT. This also makes it much easier to trigger your actions across teams with multiple people.
Based on user requests and feedback, we've built in features to specify which branch to send the repository_dispatch to, and we've even built in a way to inject parameters when you want to execute the action.
You configure your buttons with a declarative yaml file that you leave in the repo, and ActionsPanel will read that file and dynamically create your UI for you to trigger your actions.
What GitHub cryptic documentation fails to clarify is that you can have multiple workflow files under .github/workflows, each with its own trigger. For instance, I've a workflow that builds and runs tests on every push and pull request, and another that is triggered manually to publish the artifact.
(ci.yml)
name: CI Pipeline
on: [push, pull_request]
---
(publish.yml)
name: Publish
on:
workflow_dispatch:
I am able to generate build trigger url and able to call build operation via Gitlab Web hook.
But the build operation is calling in each commit irrespective of any branch. But I want to trigger build operation for a specific branch commit. Means want to execute build only if any code pushed to a specific branch.
In Gitlab yaml you can specify each job to trigger on certain branches or excluding branches
https://docs.gitlab.com/ee/ci/yaml/#only-and-except
job_name:
script:
- rake spec
- coverage
stage: test
only:
- master
tags:
- ruby
- postgres
allow_failure: true
The above yaml would only execute on master