Manual workflow triggers in Github Actions - docker

I am setting up Github Actions for a project repository.
The workflow consists of the following steps:
Building a docker image
Pushing the image to a container registry
Rollout a Kubernetes deployment.
However, I have two different Kubernetes deployments: one for development, and one for production. Hence, I have also two Github Action workflows.
The Github Action workflow for development is triggered everytime that a commit is pushed:
on:
push:
branches:
- master
But I don't want that for my production workflow. I would need a manual trigger, like a Send to production button. I didn't see anything close to that in the docs.
Is there a way to trigger a workflow manually in Github Actions?
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?

Is there a way to trigger a workflow manually in Github Actions?
You might consider, from July2020:
GitHub Actions: Manual triggers with workflow_dispatch
(Note: or multiple workflows, through the new Composite Run Steps, August 2020)
You can now create workflows that are manually triggered with the new workflow_dispatch event.
You will then see a 'Run workflow' button on the Actions tab, enabling you to easily trigger a run.
You can choose which branch the workflow is run on.
philippe adds in the comments:
One thing that's not mentioned in the documentation: the workflow must exist on the default branch for the "Run workflow" button to appear.
Once you add it there, you can continue developing the action on its own branch and the changes will take effect when run using the button
The documentation goes on:
In addition, you can optionally specify inputs, which GitHub will present as form elements in the UI. Workflow dispatch inputs are specified with the same format as action inputs.
For example:
on:
workflow_dispatch:
inputs:
logLevel:
description: 'Log level'
required: true
default: 'warning'
tags:
description: 'Test scenario tags'
The triggered workflow receives the inputs in the github.event context.
For example:
jobs:
printInputs:
runs-on: ubuntu-latest
steps:
- run: |
echo "Log level: ${{ github.event.inputs.logLevel }}"
echo "Tags: ${{ github.event.inputs.tags }}"
shim adds in the comments:
You can add workflow_dispatch to a workflow that also has other triggers (like on push and / or schedule)
For instance:
on:
workflow_dispatch:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]

EDITED :
Great tweet explaining the use of workflow dispatch :
https://twitter.com/github/status/1321859709075394563?s=19
Is there a way to trigger a workflow manually in Github Actions?
I've got a little hack to do so...
With the watch event, you can manually trigger an action by star or unstar the repo. The code for the event in your workflow is :
on:
watch
types: [started]
I know it's weird but it works! Nevertheless, it's not the best way if it's a public repo with potential stars.
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?
In Github Actions I mean, you can do multiple workflows / jobs and filter by targeted branches or events. You can combine multiple events for example trigger a workflow for push and with a cron on midnight.

Update: For a slash command style "ChatOps" solution see slash-command-dispatch action. This can allow you to trigger workflows with slash commands (e.g. /deploy) from issue and pull request comments.
Here is a basic example for a deploy slash command. REPO_ACCESS_TOKEN is a repo scoped Personal Access Token
name: Slash Command Dispatch
on:
issue_comment:
types: [created]
jobs:
slashCommandDispatch:
runs-on: ubuntu-latest
steps:
- name: Slash Command Dispatch
uses: peter-evans/slash-command-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
commands: deploy
The command can be processed in this workflow.
name: Deploy Command
on:
repository_dispatch:
types: [deploy-command]
There are many more options and different setups. See slash-command-dispatch for full usage instructions.
Original Answer:
A repository_dispatch workflow can be manually triggered by a call to the GitHub API as follows.
on:
repository_dispatch:
types: [production-deploy]
[username] is a GitHub username
[token] is a repo scoped Personal Access Token
[repository] is the name of the repository the workflow resides in.
curl -XPOST -u "[username]:[token]" \
-H "Accept: application/vnd.github.everest-preview+json" \
-H "Content-Type: application/json" \
https://api.github.com/repos/[username]/[repository]/dispatches \
--data '{"event_type": "production-deploy"}'

Another way to resolve this with the current Github Action offering is to create a production branch from master when a deploy is needed & trigger deploy action on the production branch. The production branch is essentially a mirror of the master.
on:
push:
branches:
- master
Dev builds/push can happen whenever there is a commit to the master.
on:
push:
branches:
- production
At some point in the release schedule, you can raise the PR to the production branch. This will take care of the prod build/deploy.

Although Sarah's post was the closest and simplest answer to the original question, it is somewhat hacky so we eventually ended up by creating a dev branch to use the following triggers:
Development workflow: triggered when a push is made on the dev branch:
on:
push:
branches:
- dev
Production workflow: triggered when a pull request / merge is made from dev to master:
on:
pull_request:
branches:
- master

Edited for more detail/explanation.
One thing that you can do is call to repository_dispatch. You can view the GitHub documentation for using a repository_dispatch here.
For example, if you have a GitHub Actions workflow that looks like this:
on:
repository_dispatch:
types: [run_tests]
name: Run tests
jobs:
test:
name: Run your tests
runs-on: ubuntu-latest
steps:
- run: |
echo "I just ran all your tests!"
You can create a repository dispatch event by following the steps that are explained on the GitHub v3 API Documentation.
First, create a personal access token (PAT) on GitHub for authentication.
Then, you can run curl like so:
curl \
-H "Authorization: token $YOUR_PAT" \
--request POST \
--data '{"event_type": "run_tests"}' \
https://api.github.com/repos/$USER/$REPOSITORY/dispatches
At the same time, I also wanted to share a small project that I've been working on with a buddy that solves this exact problem.
https://www.actionspanel.app/
ActionsPanel uses this same repository_dispatch API but does so with a GitHub App token so that you don't need to worry about managing your own PAT. This also makes it much easier to trigger your actions across teams with multiple people.
Based on user requests and feedback, we've built in features to specify which branch to send the repository_dispatch to, and we've even built in a way to inject parameters when you want to execute the action.
You configure your buttons with a declarative yaml file that you leave in the repo, and ActionsPanel will read that file and dynamically create your UI for you to trigger your actions.

What GitHub cryptic documentation fails to clarify is that you can have multiple workflow files under .github/workflows, each with its own trigger. For instance, I've a workflow that builds and runs tests on every push and pull request, and another that is triggered manually to publish the artifact.
(ci.yml)
name: CI Pipeline
on: [push, pull_request]
---
(publish.yml)
name: Publish
on:
workflow_dispatch:

Related

Avoid trigger Bitbucket pipeline when the title starts with Draft or WIP

To automate our CI process, I need run the Bitbucket pipelines only when the title not starts with "Draft" or "WIP". Atlassian has only this features https://support.atlassian.com/bitbucket-cloud/docs/use-glob-patterns-on-the-pipelines-yaml-file/.
I tried with the regex ^(?!Draft:|WIP:).+ like this:
pipelines:
pull-requests:
'^(?!Draft:|WIP:).+':
- step:
name: Tests
but the pipeline not start under any circumstances (with or withour Draft:/WIP:). Any suggestions?
Note the PR pattern you define in the pipelines is matched against the source branch, not the PR title. Precisely, I used to feature an empty pipeline for PRs from wip/* branches, e.g.
pipelines:
pull-requests:
wip/*:
- step:
name: Pass
script:
- exit 0
"**":
- step:
name: Tests
# ...
But this workflow requires you to work on wip/* branches and changing their source branch later on. This is somewhat cumbersome and developers just did not opt-in.
This works, though.

Skip CircleCI Jobs on empty branches

When I push an empty branch to my GitHub repo (perhaps by creating a branch through the GitHub web UI), CircleCI sees it and runs all the tests for the commit at the tip of that branch.
This makes sense, but I've discovered an unfortunate side effect: the CircleCI jobs for the branch clobber the GitHub statuses for master with inaccurate information.
Here's the sequence of events:
I push commit A to master. This runs my whole CircleCI workflow and posts information on test success / failure for A to GitHub using the GitHub Status API.
I create a new branch, mybranch, using the GitHub UI. Its tip is also at commit A.
CircleCI runs all the tests for mybranch. These are also on commit A, so CircleCI re-posts statuses to the GitHub status API for commit A.
This might be OK, except that I have an optimization: I early-out of tests that shouldn't be affected by a PR branch. So if the branch contains no TypeScript files in its diff from master, I won't run the TypeScript tests. Unfortunately, this means that step 3 overrides the GitHub status for the full run on commit A with one that early outs (since there are no commits, there's no diff).
Here's what the "skip" logic looks like:
skip-job-if-no-ts:
description: Skips the rest of the job if files under ts/ have not been modified
steps:
- when:
condition:
not:
or:
# Always run all tests if this is a commit to master branch
- equal: [master, << pipeline.git.branch >>]
steps:
- run:
name: skip-job-if-no-ts
command: |
# Skip the rest of the job if ts/ wasn't modified
TS_CHANGED=$(git diff $(git merge-base HEAD origin/master) --name-only ts/)
if [ ! -n "$TS_CHANGED" ]; then
circleci-agent step halt
fi
How can I prevent CircleCI from running my job on an empty branch? I found an old discussion of this issue on the CircleCI forums, but it was closed with no comments.

Gitlab CI: How do I make `rules.changes` to compare changed file to main branch?

I am trying to create a base image for my repo that is optionally re-built when branches (merge requests) make changes to dependencies.
Let's say I have this pipeline configuration:
stages:
- Test
- Build
variables:
- image: main
Changes A:
stage: Test
rules:
- if: '$CI_PIPELINE_SOURCE == "push"'
changes:
- path/to/a
script:
- docker build -t a .
- docker push a
- echo 'image=a' > dotenv
artifacts:
reports:
dotenv: dotenv
Build:
stage: Build
image: $image
script:
- echo build from $image
Let's say I push to a new branch and the first commit changes /path/to/a, the docker image is build and pushed, the dotenv is updated and the Build job successfully uses image=a.
Now, let's say I push a new commit to the same branch. However, the new commit does not change /path/to/a so the Changes A job does not run. Now, the Build stage pulls the "wrong" default image=main while I would like it to still pull image=a since it builds on top of the previous commit.
Any ideas on how to deal with this?
Is there a way to make rules.changes refer to origin/main?
Any other ideas on how to achieve what I am trying to do?
Is there a way to make rules.changes refer to origin/main?
Yes, there is, since GitLab 15.3 (August 2022):
Improved behavior of CI/CD changes with new branches
Improved behavior of CI/CD changes with new branches
Configuring CI/CD jobs to run on pipelines when certain files are changed by using rules: changes is very useful with merge request pipelines.
It compares the source and target branches to see what has changed, and adds jobs as needed.
Unfortunately, changes does not work well with branch pipelines.
For example, if the pipeline runs for a new branch, changes has nothing to compare to and always returns true, so jobs might run unexpectedly.
In this release we’re adding compare_to to rules:changes for both jobs and workflow:rules, to improve the behavior in branch pipelines.
You can now configure your jobs to check for changes between the new branch and the defined comparison branch.
Jobs that use rules:changes:compare will work the way you expect, comparing against the branch you define.
This is useful for monorepos, where many independent jobs could be configured to run based on which component in the repo is being worked on.
See Documentation and Issue.
You can use it only as part of a job, and it must be combined with rules:changes:paths.
Example:
docker build:
script: docker build -t my-image:$CI_COMMIT_REF_SLUG .
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
paths:
- Dockerfile
compare_to: 'refs/heads/branch1'
In this example, the docker build job is only included when the Dockerfile has changed relative to refs/heads/branch1 and the pipeline source is a merge request event.
There is a project setting, which defines how your MR pipelines are setup. This is only working for Merge requests and can be found in Settings -> Merge Requests under the section Merge options
each commit individually - nothing checked
this means, each commit is treated on its own, and changes checks are done against the triggering commit on it's own
Enabled merged results pipeline
This will merge your MR with the target branch before running the CI Jobs. This also will evaluate all your changes, and take a look at all of them within the MR and not commit wise.
Merge trains
This is a whole different chapter, and for this usecase not relevant. But for completeness i have to mention it see https://gitlab.com/help/ci/pipelines/merge_trains.md
What you are looking for is Option 2 - Merged pipeline results. but as i said, this will only work in Merge Request pipelines and not general pipelines. So you would also need to adapt your rules to something like:
rules:
- if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
changes:
- path/to/a
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Trigger Github Action after CircleCI workflow or job

I would like to set up github actions in order to perform security scans on docker images. However, the build and push of the image I would like to scan is made by CircleCI. Hence I need to make sure that the images have been properly built and pushed on my repo before performing the security scan.
I know there exists an action to trigger a CircleCI job, but is there any way to trigger the execution of my Github action from Circle CI?
Thank you for the answers or any workaround you could provide :)
You could use a repository dispatch event from you CircleCI pipeline to start a repository workflow (through CURL or through a script).
In that case, your Github repository workflow file will need a repository_dispatch event to trigger it, to perform your security scan job.
Dispatch Request Example
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/<USERNAME>/<REPOSITORY-NAME>/dispatches \
-d '{"event_type":"event_type"}'
Workflow Trigger Example
on: [repository_dispatch]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- name: security scan
run:|
...

How to notify Github after merge job is finished? [duplicate]

Is there a way to show the Jenkins build status on my project's GitHub Readme.md?
I use Jenkins to run continuous integration builds. After each commit it ensures that everything compiles, as well as executes unit and integration tests, before finally producing documentation and release bundles.
There's still a risk of inadvertently committing something that breaks the build. It would be good for users visiting the GitHub project page to know the current master is in that state.
Ok, here's how you can set up Jenkins to set GitHub build statuses. This assumes you've already got Jenkins with the GitHub plugin configured to do builds on every push.
Go to GitHub, log in, go to Settings, Developer Settings, Personal access tokens and click on Generate new token.
Check repo:status (I'm not sure this is necessary, but I did it, and it worked for me).
Generate the token, copy it.
Make sure the GitHub user you're going to use is a repository collaborator (for private repos) or is a member of a team with push and pull access (for organization repos) to the repositories you want to build.
Go to your Jenkins server, log in.
Manage Jenkins → Configure System
Under GitHub Web Hook select Let Jenkins auto-manage hook URLs, then specify your GitHub username and the OAuth token you got in step 3.
Verify that it works with the Test Credential button. Save the settings.
Find the Jenkins job and add Set build status on GitHub commit to the post-build steps
That's it. Now do a test build and go to GitHub repository to see if it worked. Click on Branches in the main repository page to see build statuses.
You should see green checkmarks:
In the meanwhile the UI of Jenkins and GitHub has changed a bit and it took me a while to figure out how to configure Jenkins now correctly. The explanation here is based on Jenkins version 2.121.1.
I also assume that you have already configured your Jenkins Job be triggered by a webhook or by polling.
Those are the steps that I have taken to get it working:
Configure Github: Create Personal Access Token with OAuth Scope repo:status
Configure Jenkins: Configure System and add the OAuth Secret as a GitHub Server - use Secret Text as an authentication method to put the OAuth Secret in there.
Configure your Jenkins Job: Add Set GitHub commit status as Post-build action. Set the Status Result to One of the default messages and statuses.
Check your result on GitHub: Check if you get the build status and build execution duration on your GitHub commit.
Configure Github
Configure Jenkins
Configure Jenkins Job
Result
You will now see the status for your commits and branches:
What I did is quite simple:
Install the Hudson Post Task Plugin
Create a Personal Access Token here : https://github.com/settings/tokens
Add a Post Task Plugin that always put success
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"success\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has succeeded!\"
}"
Add a Post Task Plugin that will put failure if "marked build as failure"
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"failure\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build has failed!\"
}"
You can also add a call to pending at the beginning of tests
curl -XPOST -H "Authorization: token OAUTH TOKEN" https://api.github.com/repos/:organization/:repos/statuses/$(git rev-parse HEAD) -d "{
\"state\": \"pending\",
\"target_url\": \"${BUILD_URL}\",
\"description\": \"The build is pending!\"
}"
This plugin should work: https://wiki.jenkins-ci.org/display/JENKINS/Embeddable+Build+Status+Plugin
You should be able to embed badges like this into your README.md file:
The Commit Status API allows you to see the "Repo Statuses API".
And since April 26th 2013, you now can see the build status on your GitHub repo branch page:
That means it is another way, by visiting the GitHub project page, to see those statuses instead of having only Jenkins.
Starting April 30th, 2013, the API endpoint for commit statuses has been extended to allow branch and tag names, as well as commit SHAs.
There's also the plugin Embeddable Build Status that will give you a badge URL that you can post in your README.md file, and it looks like this:
If you have the GitHub plugin installed on your Jenkins, you can do it in the Post build actions like this:
Add the below line in your README.md file and change both URLs according to your Jenkins project.
[![Build Status](https://jenkins../..project/lastBuild/buildStatus)](https://jenkins../..project/lastBuild/)
In regards with setting up Jenkins and GitHub's protected branch. I'm using Jenkins 2.6, and these are the steps I did to make it work:
On your repository's GitHub webpage:
Navigate to Settings > Branches.
Under Protect branches, click on
the Choose a branch drown down menu and select the branch you want
to set as a Protected branch.
Enable the options as needed.
On the Jenkins Server:
(Make sure you have the Git and GitHub Plugin installed)
Navigate to Manage Jenkins > Configure System.
Under GitHub, set the API URL to https://api.github.com. Though this is the default value.
Select your generated token for the Credentials. If you haven't generated a token yet, click on Advanced... then on Additional actions, you can convert your login and password to token and use it as your credential.
Also, make sure the GitHub account that your Jenkins is using is a collaborator for the repository. I've set it with write permission level.
Hope this helps.
I followed the directions from Alex and it worked.
But, for GitHub Enterprise you need to modify the API URL when adding the server to Jenkins.
For example, if your company is creditcard.com, your URL would be
https://github.creditcard.com/api/v3/
Edit:
I'm no longer using this approach, please use one of the other answers.
Update: what I ended up doing, for our specific case: (above answers were great - thanks!)
Because our build server is not on the internet, we have a script to publish the build status to the gh-pages branch in github.
Start of build stamps failing
End of build stamps success
Project runs after main project to publish results -> build-status, API docs, test reports and test coverage.
GitHub caches images, so we created .htaccess file, that instructs a short cache timeout for the build-status image.
Put this in the directory with the build-status image:
ExpiresByType image/png "access plus 2 minutes"
Here's the build script. The target that publishes to gh-pages is '--publish.site.dry.run'
With less than 400 lines of config, we have:
Compile checks
unit & integration tests
Test Reports
Code Coverage Reports
API Docs
Publishing to Github
. . and this script can be run in or outside of Jenkins, so that:
Developers can run this script before commit, reducing the chance of a broken build that impacts others.
A failure is easy to reproduce locally.
The Results:
Project main page has the build status, updated after each build, along with latest API Docs, test results and test coverage.
Jently updates your GitHub commit status (as described by vonc), but unfortunately they have yet to implement the Repo Status API.
For pipeline projects, you can use the post section like so:
void setBuildStatus(String message, String state) {
step([
$class: "GitHubCommitStatusSetter",
reposSource: [$class: "ManuallyEnteredRepositorySource", url: "https://github.com/my-user/my-repo"],
contextSource: [$class: "ManuallyEnteredCommitContextSource", context: "ci/jenkins/build-status"],
errorHandlers: [[$class: "ChangingBuildStatusErrorHandler", result: "UNSTABLE"]],
statusResultSource: [ $class: "ConditionalStatusResultSource", results: [[$class: "AnyBuildResult", message: message, state: state]] ]
]);
}
pipeline {
agent any
triggers {
githubPush()
}
stages {
stage('Hello') {
steps {
echo 'Hello World'
}
}
}
post {
success {
setBuildStatus("Build succeeded", "SUCCESS");
}
failure {
setBuildStatus("Build failed", "FAILURE");
}
}
}
Just change "https://github.com/my-user/my-repo" to meet your GitHub repository.
Reference: How to set GitHub commit status with Jenkinsfile NOT using a pull request builder
I am adding to this answer, and also to this one. They have answered the question, but they didn't give us a proper intuition about the problem
So, here are my intuitions:
We need to add status to our GitHub commit. This status is based upon the results of our Jenkins build.
To do this, we need to give Jenkins access to the GitHub API, not to the repo. We do this through OAuth, and we can create the token going to the GitHub settings → Developer settings → Personal access tokens. Then for a public GitHub repository just check repo:status, and for a private repository, check the whole repository section
After creating your access token you need to add it to your Jenkins server:
Copy and paste the access token to the GitHub plugin section settings, under your Jenkins configurations
Create a GitHub server. It defaults to api.github.com. And add the access token as a secret text credentials.
The last step is to add a post build settings when you create your pipeline.
Resources:
Here is a section of a YouTube video in which he goes over the above steps.
Read about OAuth

Resources