I have a Github Action which consist to tag, build and deploy a docker image.
When there is a pull request I just do the build job with this file: build.yml
# This is a basic workflow to help you get started with Actions
name: Build
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
pull_request:
branches: [ master ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
docker:
# The type of runner that the job will run on
runs-on: ubuntu-latest
env:
DOCKERHUB_REPOSITORY: ${{ secrets.DOCKERHUB_REPOSITORY }}
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
- name: docker build
run: | # Tag the image with the commit hash
docker build -t $DOCKERHUB_REPOSITORY .
docker tag $DOCKERHUB_REPOSITORY:latest $DOCKERHUB_REPOSITORY:$(git log -1 --pretty=%h)
And for the deployment I have to build and deploy with this file: deploy.yml
# This is a basic workflow to help you get started with Actions
name: Deploy
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ master ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
docker:
# The type of runner that the job will run on
runs-on: ubuntu-latest
env:
DOCKERHUB_REPOSITORY: ${{ secrets.DOCKERHUB_REPOSITORY }}
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v2
- name: docker login
env:
DOCKERHUB_USER: ${{ secrets.DOCKERHUB_USER }}
DOCKERHUB_ACCESS_TOKEN: ${{ secrets.DOCKERHUB_ACCESS_TOKEN }}
run: |
docker login -u $DOCKERHUB_USER -p $DOCKERHUB_ACCESS_TOKEN
- name: docker build
run: | # Tag the image with the commit hash
docker build -t $DOCKERHUB_REPOSITORY .
docker tag $DOCKERHUB_REPOSITORY:latest $DOCKERHUB_REPOSITORY:$(git log -1 --pretty=%h)
- name: docker push
run: |
docker push $DOCKERHUB_REPOSITORY
For me there is a repetition with the build section but I did not found how can I use a dependency in jobs on different files. It is not working.
How can I tell github action that the deploy section depends on the build ?, with 2 different files.
link: https://docs.github.com/en/free-pro-team#latest/actions/learn-github-actions/managing-complex-workflows#creating-dependent-jobs
You can merge them into one CI/CD pipeline. This should be triggered by both pushes and pull_requests in master. This has several advantages;
If the build fails, the pipeline is automatically aborted
(to be absolutely sure, you can add if: ${{ success() }}!
No duplicate steps, docker build is only defined once.
Steps can still only be performed on a push or pull_request by using conditions:
if: ${{ github.event_name == 'push' }} // OR
if: ${{ github.event_name == 'pull_request' }}
Fewer pipelines to maintain!
Related
Im building a simple CI/CD workflow with github actions. The workflow starts with running all unit tests. When the unit tests were successfull, a docker image gets build and uploaded to docker hub. I have a few environment variables that need to be set, in order run the tests and also to run the docker container.
Thats currently the only way I get it to work:
name: Deploy to Linode k8s Cluster Workflow
env:
REGISTRY: "abc"
IMAGE_NAME: "defg"
DO_POSTGRESQL_URL: ${{ secrets.DO_POSTGRESQL_URL }}
DO_POSTGRESQL_USER: ${{ secrets.DO_POSTGRESQL_USER }}
DO_POSTGRESQL_PASS: ${{ secrets.DO_POSTGRESQL_PASS }}
# ... and more
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
environment: test
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
- name: Setup Java
uses: actions/setup-java#v3
with:
java-version: 18
distribution: temurin
cache: gradle
- name: Setup Gradle
uses: gradle/gradle-build-action#v2
- name: Execute Gradle build
run: ./gradlew build --scan
- name: Login to Docker Hub
uses: docker/login-action#v2.1.0
with:
# Username used to log against the Docker registry
username: ${{ secrets.DOCKER_HUB_USERNAME }}
# Password or personal access token used to log against the Docker registry
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Build and push Docker images
uses: docker/build-push-action#v4.0.0
with:
context: .
# because two jar files get generated (...-SNAPSHOT-plain.jar and ...-SNAPSHOT.jar) only ...-SNAPSHOT.jar is needed
build-args: JAR_FILE=build/libs/*SNAPSHOT.jar, DO_POSTGRESQL_URL=$DO_POSTGRESQL_URL #,... and so on
push: true
tags: abc/defg:latest
# Linode deployment here
But imagine im having multiple workflow files, then I have to add the environment variables on multiple different places. Is there a clever way to solve this problem. So that I only have to define the env variables once?
I would like to achieve the following CI pipeline with GitHub Actions workflow.
Pull Request is merged -> GitHub action is being triggered -> docker image is being built with semantic version incremented by one or the version is based on GitHub tag - if it is possible to somehow tag the pull request merge.
How to achieve that or is the better approach?
I have tried with secrets but no to avail. How to implement semantic versioning in GitHub Actions workflow?
name: Docker Image CI
on:
push:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Build the Docker image
run: docker build . --file Dockerfile --tag my-image-name:${{github.ref_name}}
${{github.ref_name}} pulls the tag for you or run git command like git describe --abbrev=0 in the previous step to get the latest tag and append it to image name and use it like:
- name: Get Tag
id: vars
run: echo ::set-output name=tag::${git describe --abbrev=0}
- name: Build the Docker image
run: docker build . --file Dockerfile --tag my-image-name:${{ steps.vars.outputs.tag }}
You can use a lot of semver actions on the market place.
For example, I have tried using this - Semver action
This will bump your repo version, and you can use a git bash command to get that bumped version on the next job.
So combining with docker build, you can do something like:
jobs:
update-semver:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: haya14busa/action-update-semver#v1
id: version
with:
major_version_tag_only: true # (optional, default is "false")
Build:
name: Build Image
needs: [update-semver]
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Build image
run: |
tag_v=$(git describe --tags $(git rev-list --tags --max-count=1))
tag=$(echo $tag_v | sed 's/v//')
docker build -t my_image_${tag} .
I have link a github repo with my docker hub account. When there is a push to github master branch, a new image will be built in docker hub. But the image only has LATEST tag. How can I make the version increased automatically?
Ideally, I'd like it follow the sversion 1.0.0. And increase for every push 1.0.1, 1.0.2 1.0.3 etc.
Is there a way to make it follow this pattern?
You could associate a GitHub Action workflow to your repository, like docker/metadata-action
GitHub Action to extract metadata (tags, labels) for Docker. This action is particularly useful if used with Docker Build Push action.
You can see it used here. Warning: the tag name (as generated by the GitHub Action) will contain the branch name as well.
I was having the same problem, solved with this GitHub Action Code:
Create a secret called MAJOR to save your mayor version
Create a secret called MINOR to save your minor version
You will need a token to update you repo secrets, so... create a secret called REPO_ACCESS_TOKEN to grant your action dose his work.
name: Docker Image CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
-
name: Build the Docker image
run: docker build . --file src/MasterReport.UI/Dockerfile --tag eriksongm/master-report:${{ secrets.MAJOR }}.${{ secrets.MINOR }}
-
name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Push to DockerHub
run: docker push eriksongm/master-report:${{ secrets.MAJOR }}.${{ secrets.MINOR }}
-
name: Update Minor version
uses: hmanzur/actions-set-secret#v2.0.0
with:
name: 'MINOR'
value: $((${{ secrets.MINOR }}+1))
repository: EriksonGM/MasterReport
token: ${{ secrets.REPO_ACCESS_TOKEN }}
This was my final code, as you can see, I have a last step just to update the minor version, only if all the other jobs run ok.
I'm trying to build and push a Docker image in GitHub Actions.
In the YAML file I have other steps as well, which work fine. But when I tried to build a Docker image, the GitHub Action fails. The error is:
Invalid workflow file
The workflow is not valid. Job package depends on unknown job test.
I have a YAML extension installed in VS Code and it shows no errors related to indentation. If I remove the snippet of Docker build part (after arg command), action test runs successfully.
The GitHub Action error doesn't describe the reason of action fail properly so that I could debug.
name: Netlify workflow
on:
push:
branches: [master, develop]
pull_request:
branches: [master, develop]
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
node: [10.x, 12.x]
steps:
- name: Setup node
uses: actions/setup-node#v1
with:
node-version: ${{matrix.node}}
- name: Checkout
uses: actions/checkout#v2
- name: Setup cache
uses: actions/cache#v1
with:
path: ~/.npm
key: ${{runner.os}}-modules-${{hashFiles('**/package-lock.json') }}
restore-keys: |
${{runner.os}}-modules-
${{runner.os}}-
- name: Install
run: npm ci
- name: Lint
run: npm run lint
- name: Build
run: npm run build
- name: Deploy
uses: netlify/actions/cli#master
env:
NETLIFY_SITE_ID: ${{secrets.NETLIFY_SITE_ID}}
NETLIFY_AUTH_TOKEN: ${{secrets.NETLIFY_AUTH_TOKEN}}
with:
args: deploy --dir=build --prod
package:
runs-on: ubuntu-latest
needs: test
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Build docker image
run: docker builder build -t dockerHubUsername/repoName:latest .
- name: Login to docker hub
run: docker login --username ${{ secrets.DOCKER_USERNAME }} --password ${{ secrets.DOCKER_PASSWORD }}
- name: Push docker image to docker hub
run: docker push dockerHubUsername/repoName:latest
The jobs map in a GitHub Workflow, per jobs.<job_id>, is a map where:
The key job_id is a string and its value is a map of the job's
configuration data.
Stripping all of the other content out of the YAML to focus on that map:
jobs:
build:
# ...
package:
# ...
At the top level, two jobs have been defined, with the IDs build and package. Now let's look at some of the content of those jobs:
jobs:
build:
runs-on: ubuntu-latest
# ...
package:
runs-on: ubuntu-latest
needs: test
# ...
Per job.<job_id>.needs, the needs configuration:
Identifies any jobs that must complete successfully before this job
will run. It can be a string or array of strings.
Although it's not stated explicitly, the example shows that the jobs are identified by their IDs, so it needs to be a string or array of strings corresponding with defined job IDs.
Here we've said that, to run the job with ID package, it "needs" the job with ID test to have successfully completed. The ID of the only other job we've defined is build, though, hence the error:
Job package depends on unknown job test.
// ^~~~~~~ ^~~~~~~~~~ ^~~~
// job_id "needs" job_id
Given that you have only two jobs and likely do want the second to depend on the first, you either need to:
Rename the build job to test; or
Change the dependency to needs: build.
Either way, the two IDs need to correspond for this to be a semantically valid workflow (even though it's already syntactically valid YAML). An alternative would be to remove the dependency entirely, by deleting the needs: test line, although then build and package would be run in parallel (workers permitting).
I have this code at .github/workflows/main.yaml
# .github/workflows/main.yaml
name: CI Workflow
on: [push]
jobs:
rspec-job:
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
container:
image: I-stucked-here
volumes:
- /vendor/bundle
steps:
- code omitted for brevity
The main idea of this job is to run all steps in container mode. Not in Linux host mode.
Under the same repository, I have a public Docker image named ruby-rimy-2.6.3. Since it's not publicly hosted on DockerHub, I can't find a way to programmatically authenticate myself to GitHub Packages/Registry.
I did try with different syntax (see code below) but it didn't work.
# .github/workflows/main.yaml
name: CI Workflow
on: [push]
jobs:
rspec-job:
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
container:
image: docker://docker.pkg.github.com/zulhfreelancer/rimy/ruby-rimy-2.6.3:latest
volumes:
- /vendor/bundle
steps:
- code omitted for brevity
From the docs, GitHub says the GITHUB_TOKEN is available when the job is running. How do I use this GITHUB_TOKEN environment variable to run something like docker login on top of that
container: section so that the job is able to pull the image?
Using GitHub Personal Token is not an option for me because that repository is just my experiment repository before applying the same thing to my GitHub organization. I don't want to put my personal token under my organization's repository environment variables/secrets — that will simply exposes my personal token to my co-workers.
You do not need to use the container instruction to run tests in a container.
The GitHub Actions host comes with docker and docker-compose installed. The way I do it, is have a docker-compose.yml in my repository, which includes a "service" that runs tests. Then, your workflow needs to do docker login and simply run the docker-compose run test command.
Note that the beauty of this approach, is that your tests are executed exactly the same on your own machine and on the CI machine. Same exact steps.
Something along these lines:
name: Test
on:
pull_request:
push: { branches: master }
jobs:
test:
name: Run test suite
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- name: Checkout code
uses: actions/checkout#v2
- name: Docker login
run: echo ${GITHUB_TOKEN} | docker login -u ${GITHUB_ACTOR} --password-stdin docker.pkg.github.com
- name: Build docker images
run: docker-compose build
- name: Run tests
run: docker-compose run test
I am doing the same with DockerHub, with great ease and success.
Of course, if you do not want to use docker-compose, you can still use any normal docker run ... commands after you login properly in the login step.
I am not sure that docker login command will work as is, see these for a deeper discussion:
https://github.com/actions/starter-workflows/issues/66
https://github.community/t5/GitHub-Actions/Github-Actions-Docker-login/td-p/29852/page/2