I would like to set up github actions in order to perform security scans on docker images. However, the build and push of the image I would like to scan is made by CircleCI. Hence I need to make sure that the images have been properly built and pushed on my repo before performing the security scan.
I know there exists an action to trigger a CircleCI job, but is there any way to trigger the execution of my Github action from Circle CI?
Thank you for the answers or any workaround you could provide :)
You could use a repository dispatch event from you CircleCI pipeline to start a repository workflow (through CURL or through a script).
In that case, your Github repository workflow file will need a repository_dispatch event to trigger it, to perform your security scan job.
Dispatch Request Example
curl \
-X POST \
-H "Accept: application/vnd.github.v3+json" \
https://api.github.com/repos/<USERNAME>/<REPOSITORY-NAME>/dispatches \
-d '{"event_type":"event_type"}'
Workflow Trigger Example
on: [repository_dispatch]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- name: security scan
run:|
...
Related
Can you please help, I have the following scenario and I went through many videos, blogs but could not find anything matching with my use-case
Requirement:
To write a CI\CD pipeline in GitLab, which can facilitate the following stages in this order
- verify # unit test, sonarqube, pages
- build # package
- publish # copy artifact in repository
- deploy # Deploy artifact on runtime in an test environment
- integration # run postman\integration tests
All other stages are fine and working but for the deploy stage, because of a few restrictions I have to submit an existing Jenkins job using Jenkin remote API with the following script but the problem that script returns an asynchronous response and start the Jenkins job and deploy stage completes and it moves to next stage (integration).
Run Jenkins Job:
image: maven:3-jdk-8
tags:
- java
environment: development
stage: deploy
script:
- artifact_no=$(grep -m1 '<version>' pom.xml | grep -oP '(?<=>).*(?=<)')
- curl -X POST http://myhost:8081/job/fpp/view/categorized/job/fpp_PREP_party/build --user mkumar:1121053c6b6d19bf0b3c1d6ab604f22867 --data-urlencode json="{\"parameter\":[{\"name\":\"app_version\",\"value\":\"$artifact_no\"}]}"
Note: Using GitLab CE edition and Jenkins CI project service is not available.
I am looking for a possible way of triggering the Jenkins job from the pipeline and only on successful completion of the Jenkins job my integration stage starts executing.
Thanks for the help!
Retrieving the status of a Jenkins job that is triggered programmatically through the remote access API is notorious for not being quite convoluted.
Normally you would expect to receive in the response header, under the Location attribute, a url that you can poll to get the status of your request, but unfortunately there are some in-between steps to reach that point. You can find a guide in this post. You may also have a look in this older post.
Once you have the url, you can pool and parse the status job and either sh "exit 1" or sh "exit 0" in your script to force the job that is invoking the external job to fail or succeed, depending on how you want to assert the result of the remote job
I am setting up Github Actions for a project repository.
The workflow consists of the following steps:
Building a docker image
Pushing the image to a container registry
Rollout a Kubernetes deployment.
However, I have two different Kubernetes deployments: one for development, and one for production. Hence, I have also two Github Action workflows.
The Github Action workflow for development is triggered everytime that a commit is pushed:
on:
push:
branches:
- master
But I don't want that for my production workflow. I would need a manual trigger, like a Send to production button. I didn't see anything close to that in the docs.
Is there a way to trigger a workflow manually in Github Actions?
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?
Is there a way to trigger a workflow manually in Github Actions?
You might consider, from July2020:
GitHub Actions: Manual triggers with workflow_dispatch
(Note: or multiple workflows, through the new Composite Run Steps, August 2020)
You can now create workflows that are manually triggered with the new workflow_dispatch event.
You will then see a 'Run workflow' button on the Actions tab, enabling you to easily trigger a run.
You can choose which branch the workflow is run on.
philippe adds in the comments:
One thing that's not mentioned in the documentation: the workflow must exist on the default branch for the "Run workflow" button to appear.
Once you add it there, you can continue developing the action on its own branch and the changes will take effect when run using the button
The documentation goes on:
In addition, you can optionally specify inputs, which GitHub will present as form elements in the UI. Workflow dispatch inputs are specified with the same format as action inputs.
For example:
on:
workflow_dispatch:
inputs:
logLevel:
description: 'Log level'
required: true
default: 'warning'
tags:
description: 'Test scenario tags'
The triggered workflow receives the inputs in the github.event context.
For example:
jobs:
printInputs:
runs-on: ubuntu-latest
steps:
- run: |
echo "Log level: ${{ github.event.inputs.logLevel }}"
echo "Tags: ${{ github.event.inputs.tags }}"
shim adds in the comments:
You can add workflow_dispatch to a workflow that also has other triggers (like on push and / or schedule)
For instance:
on:
workflow_dispatch:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
EDITED :
Great tweet explaining the use of workflow dispatch :
https://twitter.com/github/status/1321859709075394563?s=19
Is there a way to trigger a workflow manually in Github Actions?
I've got a little hack to do so...
With the watch event, you can manually trigger an action by star or unstar the repo. The code for the event in your workflow is :
on:
watch
types: [started]
I know it's weird but it works! Nevertheless, it's not the best way if it's a public repo with potential stars.
How can I split my development and my production workflows to achieve what I want, either on Github Actions, Docker or Kubernetes?
In Github Actions I mean, you can do multiple workflows / jobs and filter by targeted branches or events. You can combine multiple events for example trigger a workflow for push and with a cron on midnight.
Update: For a slash command style "ChatOps" solution see slash-command-dispatch action. This can allow you to trigger workflows with slash commands (e.g. /deploy) from issue and pull request comments.
Here is a basic example for a deploy slash command. REPO_ACCESS_TOKEN is a repo scoped Personal Access Token
name: Slash Command Dispatch
on:
issue_comment:
types: [created]
jobs:
slashCommandDispatch:
runs-on: ubuntu-latest
steps:
- name: Slash Command Dispatch
uses: peter-evans/slash-command-dispatch#v1
with:
token: ${{ secrets.REPO_ACCESS_TOKEN }}
commands: deploy
The command can be processed in this workflow.
name: Deploy Command
on:
repository_dispatch:
types: [deploy-command]
There are many more options and different setups. See slash-command-dispatch for full usage instructions.
Original Answer:
A repository_dispatch workflow can be manually triggered by a call to the GitHub API as follows.
on:
repository_dispatch:
types: [production-deploy]
[username] is a GitHub username
[token] is a repo scoped Personal Access Token
[repository] is the name of the repository the workflow resides in.
curl -XPOST -u "[username]:[token]" \
-H "Accept: application/vnd.github.everest-preview+json" \
-H "Content-Type: application/json" \
https://api.github.com/repos/[username]/[repository]/dispatches \
--data '{"event_type": "production-deploy"}'
Another way to resolve this with the current Github Action offering is to create a production branch from master when a deploy is needed & trigger deploy action on the production branch. The production branch is essentially a mirror of the master.
on:
push:
branches:
- master
Dev builds/push can happen whenever there is a commit to the master.
on:
push:
branches:
- production
At some point in the release schedule, you can raise the PR to the production branch. This will take care of the prod build/deploy.
Although Sarah's post was the closest and simplest answer to the original question, it is somewhat hacky so we eventually ended up by creating a dev branch to use the following triggers:
Development workflow: triggered when a push is made on the dev branch:
on:
push:
branches:
- dev
Production workflow: triggered when a pull request / merge is made from dev to master:
on:
pull_request:
branches:
- master
Edited for more detail/explanation.
One thing that you can do is call to repository_dispatch. You can view the GitHub documentation for using a repository_dispatch here.
For example, if you have a GitHub Actions workflow that looks like this:
on:
repository_dispatch:
types: [run_tests]
name: Run tests
jobs:
test:
name: Run your tests
runs-on: ubuntu-latest
steps:
- run: |
echo "I just ran all your tests!"
You can create a repository dispatch event by following the steps that are explained on the GitHub v3 API Documentation.
First, create a personal access token (PAT) on GitHub for authentication.
Then, you can run curl like so:
curl \
-H "Authorization: token $YOUR_PAT" \
--request POST \
--data '{"event_type": "run_tests"}' \
https://api.github.com/repos/$USER/$REPOSITORY/dispatches
At the same time, I also wanted to share a small project that I've been working on with a buddy that solves this exact problem.
https://www.actionspanel.app/
ActionsPanel uses this same repository_dispatch API but does so with a GitHub App token so that you don't need to worry about managing your own PAT. This also makes it much easier to trigger your actions across teams with multiple people.
Based on user requests and feedback, we've built in features to specify which branch to send the repository_dispatch to, and we've even built in a way to inject parameters when you want to execute the action.
You configure your buttons with a declarative yaml file that you leave in the repo, and ActionsPanel will read that file and dynamically create your UI for you to trigger your actions.
What GitHub cryptic documentation fails to clarify is that you can have multiple workflow files under .github/workflows, each with its own trigger. For instance, I've a workflow that builds and runs tests on every push and pull request, and another that is triggered manually to publish the artifact.
(ci.yml)
name: CI Pipeline
on: [push, pull_request]
---
(publish.yml)
name: Publish
on:
workflow_dispatch:
I have a CI pipeline in Bitbucket which is building, testing and deploying an application.
The thing is that after the deploy I want to run selenium tests.
Selenium tests are in an another repository in Bitbucket and they have their own pipeline.
Is there a trigger step in the Bitbucket pipeline to trigger a pipeline when a previous one has finished?
I do not want to do a fake push to the test repository to trigger those tests.
The most "correct" way I can think of doing this is to use the Bitbucket REST API to manually trigger a pipeline on the other repository, after your deployment completes.
There are several examples of how to create a pipeline here: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Bworkspace%7D/%7Brepo_slug%7D/pipelines/#post
Copy + pasting the first example. How to trigger a pipeline for the latest commit on master:
$ curl -X POST -is -u username:password \
-H 'Content-Type: application/json' \
https://api.bitbucket.org/2.0/repositories/jeroendr/meat-demo2/pipelines/ \
-d '
{
"target": {
"ref_type": "branch",
"type": "pipeline_ref_target",
"ref_name": "master"
}
}'
according to their official documentation there is no "easy way" to do that, cause the job are isolated in scope of one repository, yet you can achieve your task in following way:
create docker image with minimum required setup for execution of your tests inside
upload to docker hub (or some other repo if you have such)
use docker image in last step of you pipeline after deploy to execute tests
Try out official component Bitbucket pipeline trigger: https://bitbucket.org/product/features/pipelines/integrations?p=atlassian/trigger-pipeline
You can run in after deploy step
script:
- pipe: atlassian/trigger-pipeline:4.1.7
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
REPOSITORY: 'your-awesome-repo'
ACCOUNT: 'teams-in-space'
#BigGinDaHouse I did something more or less like you say.
My step is built on top of docker image with headless chrome, npm and git.
I did follow the steps below:
I have set private key for the remote repo in the original repo. Encoded base 64. documentation. The public key is being set into the remote repo in SSH Access option in bitbucket menu.
In the pipeline step I am decoding it and setting it to a file. I am also changing its permission to be 400.
I am adding this Key inside the docker image. ssh-add
Then I am able to do a git clone followed by npm install and npm test
NOTE: The entry.sh is because I am starting the headless browser.
- step:
image: kimy82/headless-selenium-npm-git
script:
- echo $key_in_env_variable_in_bitbucket | base64 --decode > priv_key
- chmod 400 ./priv_key
- eval `ssh-agent -s`
- ssh-agent $(ssh-add priv_key; git clone git#bitbucket.org:project.git)
- cd project
- nohup bash /usr/bin/entry.sh >> out.log &
- npm install
- npm test
Top answers (this and this) are correct, they work.
Just adding that we found out (after a LOT of trial and error) that the user executing the pipeline must have WRITE permissions on the repo where the pipeline is invoked (even though his app password permissions were set to "WRITE" for repos and pipelines...)
Also, this works for executing pipelines in Bitbucket's cloud or on-premise, through local runners.
(Answering as I am lacking reputation for commenting)
I have a requirement where i need to trigger the build jobs remotely using curl command. I am unable to pass the branch/tag name as a parameter to trigger the build.
I used the below command :
& $CURLEXE -k -X POST $dst_job_url --user username:token --data-urlencode json='{"parameters": [{"name":"branch","branch":"branches"}]}'
If i run the above command it was triggers the build for the trunk ( default ).
You omitted the URL, so it's hard to be certain. Jenkins has two urls for building: "build" and "buildWithParameters". If you're not using buildWithParameters, switching to it will probably help.
See:
How to trigger Jenkins builds remotely and to pass parameters
Is there a way to specify a hook in the single repository?
Now we have specified the hook in the "/etc/mercurial/hgrc" file, but every time it builds twice, and it builds for each commit in each repository.
So we want to specify a build per repository.
This is how we implemented the hook:
[hooks]
changegroup = curl --silent http://jenkins:8080/job/ourProject/build
It's on a Ubuntu server.
Select the Poll SCM option under Build Triggers.
Make sure that schedule form is empty.
You should be creating in the .hg directory, /home/user/mercurial/.hg/hgrc and add hooks as:
[hooks]
commit.jenkins = wget -q http://localhost:8080/mercurial/notifyCommit?url=file:///home/user/mercurial > /dev/null
incoming.jenkins = wget -q http://localhost:8080/mercurial/notifyCommit?url=file:///home/user/mercurial > /dev/null
You should make sure that
Your Jenkins project doesn't poll
You use the proper notifyCommit URLs for your Mercurial hooks: https://wiki.jenkins-ci.org/display/JENKINS/Mercurial+Plugin
Ok, I found what I looked for (I'm the bounty; my case is Mercurial with a specific branch).
In the main/origin repository, place a hook with your desired build script. Pregroupchange is to maintain the incoming changes. I have a rhodecode installed on the main repository and itself has its own hooks.
In this way, I still trigger Jenkins and still have the changes afther the trigger for rhodecode push notifications and others.
[hooks]
pregroupchange = /path/to/script.extention
In the script, place your desired actions, also a trigger for Jenkins. Don't forget to enable in Jenkins:Job:Configure:Build Triggers:checkbox Trigger builds remotely + put here your desired_token (for my case: Mercurial).
Because you can't trigger only to a specific branch in Mercurial, I found the branch name in this way. Also, to trigger from a remote script, you need to give in Jenkins read permission for anonymous overall, or create a specific user with credentials and put them into the trigger URL.
Bash example:
#!/bin/bash
BRANCH_NAME=`hg tip --template "{branch}"`
if [ $BRANCH_NAME = "branch_name" ]; then
curl --silent http://jenkins_domain:port/path/to/job?token=desired_token
fi
For the original question:
In this way you only execute one build, for a desired branch. Hooks are meant only for the main repository in case you work with multiple clones and multiple developers. You may have your local hooks, but don't trigger Jenkins from you local, for every developer. Trigger Jenkins only from the main repository when a push came (commit, incoming, and groupchange). Your local hooks are for other things, like email, logs, configuration, etc.