Is it possible for multiple branches in a github repo to share entries in the gha cache? I have a CI workflow in a github repo. This workflow builds a Docker image and pushes it to the gha cache using build-push-action. When I push commits to an existing branch/PR, build-push-action successfully loads the Docker image from the cache (good). However, when I push a new branch, CI builds the Docker image from scratch, then caches the image.
Since the Dockerfile is the same in both branches, I would like the second branch to load the original image from the cache rather than rebuilding it. It seems as if cache entries are keyed by branch name. How can I share cache entries among multiple branches?
Here is an example workflow file which demonstrates my issue:
name: My CI
on:
workflow_dispatch:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
name: Build Docker Image
steps:
- uses: docker/setup-buildx-action#v1
- uses: docker/build-push-action#v2
with:
load: true
cache-from: type=gha
cache-to: type=gha,mode=max
Related
Im building a simple CI/CD workflow with github actions. The workflow starts with running all unit tests. When the unit tests were successfull, a docker image gets build and uploaded to docker hub. I have a few environment variables that need to be set, in order run the tests and also to run the docker container.
Thats currently the only way I get it to work:
name: Deploy to Linode k8s Cluster Workflow
env:
REGISTRY: "abc"
IMAGE_NAME: "defg"
DO_POSTGRESQL_URL: ${{ secrets.DO_POSTGRESQL_URL }}
DO_POSTGRESQL_USER: ${{ secrets.DO_POSTGRESQL_USER }}
DO_POSTGRESQL_PASS: ${{ secrets.DO_POSTGRESQL_PASS }}
# ... and more
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the "master" branch
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
environment: test
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout#v3
- name: Setup Java
uses: actions/setup-java#v3
with:
java-version: 18
distribution: temurin
cache: gradle
- name: Setup Gradle
uses: gradle/gradle-build-action#v2
- name: Execute Gradle build
run: ./gradlew build --scan
- name: Login to Docker Hub
uses: docker/login-action#v2.1.0
with:
# Username used to log against the Docker registry
username: ${{ secrets.DOCKER_HUB_USERNAME }}
# Password or personal access token used to log against the Docker registry
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
- name: Build and push Docker images
uses: docker/build-push-action#v4.0.0
with:
context: .
# because two jar files get generated (...-SNAPSHOT-plain.jar and ...-SNAPSHOT.jar) only ...-SNAPSHOT.jar is needed
build-args: JAR_FILE=build/libs/*SNAPSHOT.jar, DO_POSTGRESQL_URL=$DO_POSTGRESQL_URL #,... and so on
push: true
tags: abc/defg:latest
# Linode deployment here
But imagine im having multiple workflow files, then I have to add the environment variables on multiple different places. Is there a clever way to solve this problem. So that I only have to define the env variables once?
I would like to achieve the following CI pipeline with GitHub Actions workflow.
Pull Request is merged -> GitHub action is being triggered -> docker image is being built with semantic version incremented by one or the version is based on GitHub tag - if it is possible to somehow tag the pull request merge.
How to achieve that or is the better approach?
I have tried with secrets but no to avail. How to implement semantic versioning in GitHub Actions workflow?
name: Docker Image CI
on:
push:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Build the Docker image
run: docker build . --file Dockerfile --tag my-image-name:${{github.ref_name}}
${{github.ref_name}} pulls the tag for you or run git command like git describe --abbrev=0 in the previous step to get the latest tag and append it to image name and use it like:
- name: Get Tag
id: vars
run: echo ::set-output name=tag::${git describe --abbrev=0}
- name: Build the Docker image
run: docker build . --file Dockerfile --tag my-image-name:${{ steps.vars.outputs.tag }}
You can use a lot of semver actions on the market place.
For example, I have tried using this - Semver action
This will bump your repo version, and you can use a git bash command to get that bumped version on the next job.
So combining with docker build, you can do something like:
jobs:
update-semver:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: haya14busa/action-update-semver#v1
id: version
with:
major_version_tag_only: true # (optional, default is "false")
Build:
name: Build Image
needs: [update-semver]
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout#v2
- name: Build image
run: |
tag_v=$(git describe --tags $(git rev-list --tags --max-count=1))
tag=$(echo $tag_v | sed 's/v//')
docker build -t my_image_${tag} .
I have an action.yml file and a Dockerfile in repository A like this:
Repository A
action.yml
[...]
runs:
using: 'docker'
image: 'Dockerfile'
[...]
Dockerfile
FROM hello-world
# do something
Now in repository B, I use the action and the Dockerfile:
Repository B
.github/workflows/hello.yml
name: hello
on:
push:
branches:
- master
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: username/repoa#master
My point of using the Dockerfile in repository A is, among other things:
that the GitHub action log is not cluttered with preparation steps which have no relation to the action in repository B
so that it only builds when username/repoa is updated, not when username/repob is updated, because the Dockerfile didn't change.
However in practice, GitHub will happily rebuild the Dockerfile everytime I commit to username/repob and clutter the GitHub action logs with it.
How can I tell GitHub to only build the Dockerfile when repository A is updated and keep it out of the action logs?
Github Workflows create a new machine to run a job, this means that when Github Actions build your image, you need to store this image in a registry or something and use this stored image to create another step.
You need to improve your action.yml to build and store the image and create another action2.yml to say what to do with your container.
I would like to run my CI on a Docker image. How should I write my .github/workflow/main.yml?
name: CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
name: build
runs:
using: 'docker'
image: '.devcontainer/Dockerfile'
steps:
- uses: actions/checkout#v2
- name: Build
run: make
I get the error:
The workflow is not valid. .github/workflows/main.yml
(Line: 11, Col: 5): Unexpected value 'runs'
I managed to make it work but with an ugly workaround:
build:
name: Build Project
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout#v1
- name: Build docker images
run: >
docker build . -t foobar
-f .devcontainer/Dockerfile
- name: Build exam
run: >
docker run -v
$GITHUB_WORKSPACE:/srv
-w/srv foobar make
Side question: where can I find the documentation about this? All I found is how to write actions.
If you want to use a container to run your actions, you can use something like this:
jobs:
build:
runs-on: ubuntu-latest
container:
image: docker://{host}/{image}:{tag}
steps:
...
Here is an example.
If you want more details about the jobs.<job_id>.container and its sub-fields, you can check the official documentation.
Note that you can also use docker images at the step level: Example.
I am reposting my answer to another question, in order to be sure to find it while Googling it.
The best solution is to build, publish and re-use a Docker image based on your Dockerfile.
I would advise to create a custom build-and-publish-docker.yml action following the Github documentation: Publishing Docker images.
Assuming your repository is public, you should be able to automatically upload your image to ghcr.io without any required configuration. As an alternative, it's also possible to publish the image to Docker Hub.
Once your image is built and published (based on the on event of the action previously created, which can be triggered manually also), you just need to update your main.yml action so it uses the custom Docker image. Again, here is a pretty good documentation page about the container option: Running jobs in a container.
As an example, I'm sharing what I used in a personal repository:
Dockerfile: the Docker image to be built on CI
docker.yml: the action to build the Docker image
lint.yml: the action using the built Docker image
I have link a github repo with my docker hub account. When there is a push to github master branch, a new image will be built in docker hub. But the image only has LATEST tag. How can I make the version increased automatically?
Ideally, I'd like it follow the sversion 1.0.0. And increase for every push 1.0.1, 1.0.2 1.0.3 etc.
Is there a way to make it follow this pattern?
You could associate a GitHub Action workflow to your repository, like docker/metadata-action
GitHub Action to extract metadata (tags, labels) for Docker. This action is particularly useful if used with Docker Build Push action.
You can see it used here. Warning: the tag name (as generated by the GitHub Action) will contain the branch name as well.
I was having the same problem, solved with this GitHub Action Code:
Create a secret called MAJOR to save your mayor version
Create a secret called MINOR to save your minor version
You will need a token to update you repo secrets, so... create a secret called REPO_ACCESS_TOKEN to grant your action dose his work.
name: Docker Image CI
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
-
name: Build the Docker image
run: docker build . --file src/MasterReport.UI/Dockerfile --tag eriksongm/master-report:${{ secrets.MAJOR }}.${{ secrets.MINOR }}
-
name: Login to DockerHub
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
-
name: Push to DockerHub
run: docker push eriksongm/master-report:${{ secrets.MAJOR }}.${{ secrets.MINOR }}
-
name: Update Minor version
uses: hmanzur/actions-set-secret#v2.0.0
with:
name: 'MINOR'
value: $((${{ secrets.MINOR }}+1))
repository: EriksonGM/MasterReport
token: ${{ secrets.REPO_ACCESS_TOKEN }}
This was my final code, as you can see, I have a last step just to update the minor version, only if all the other jobs run ok.