I would like to semantic versioning my docker images which are built and pushed to GitHub Container Registry by the GitHub Action.
I found a satisfying solution here: https://stackoverflow.com/a/69059228/12877180
According to the solution I reproduced the following YAML.
name: Docker CI
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
env:
REGISTRY: ghcr.io
jobs:
build-push:
# needs: build-test
name: Buid and push Docker image to GitHub Container registry
runs-on: ubuntu-latest
permissions:
packages: write
contents: read
steps:
- name: Checkout the repository
uses: actions/checkout#v2
- name: Login to GitHub Container registry
uses: docker/login-action#v1
env:
USERNAME: ${{ github.actor }}
PASSWORD: ${{ secrets.GITHUB_TOKEN }}
with:
registry: ${{ env.REGISTRY }}
username: ${{ env.USERNAME }}
password: ${{ env.PASSWORD }}
- name: Get lowercase repository name
run: |
echo "IMAGE=${REPOSITORY,,}">>${GITHUB_ENV}
env:
REPOSITORY: ${{ env.REGISTRY }}/${{ github.repository }}
- name: Build and export the image to Docker
uses: docker/build-push-action#v2
with:
context: .
file: ./docker/Dockerfile
target: final
push: true
tags: |
${{ env.IMAGE }}:${{ secrets.MAJOR }}.${{ secrets.MINOR }}
build-args: |
ENVIRONMENT=production
- name: Update Patch version
uses: hmanzur/actions-set-secret#v2.0.0
with:
name: 'MINOR'
value: $((${{ secrets.MINOR }} + 1))
repository: ${{ github.repository }}
token: ${{ secrets.GH_PAT }}
Unfortunately this does not work.
The initial value of the MINOR secret is 0. If the build-push job is executed very first time, the docker image is perfectly pushed to the GHCR with the ghcr.io/my-org/my-repo:0.0 syntax.
The purpose of the build-push job is then increment the MINOR secret by 1.
If the action job build-push is executed again after new event, I get error while trying to build docker image using the incremented tag.
/usr/bin/docker buildx build --build-arg ENVIRONMENT=production --tag ghcr.io/my-org/my-repo:***.*** --target final --iidfile /tmp/docker-build-push-HgjJR7/iidfile --metadata-file /tmp/docker-build-push-HgjJR7/metadata-file --file ./docker/Dockerfile --push .
error: invalid tag "ghcr.io/my-org/my-repo:***.***": invalid reference format
Error: buildx failed with: error: invalid tag "ghcr.io/my-org/my-repo:***.***": invalid reference format
You need to increment the version in a bash command like this:
- name: Autoincrement a new patch version
run: |
echo "NEW_PATCH_VERSION=$((${{ env.PATCH_VERSION }}+1))" >> $GITHUB_ENV
- name: Update patch version
uses: hmanzur/actions-set-secret#v2.0.0
with:
name: 'PATCH_VERSION'
value: ${{ env.NEW_PATCH_VERSION }}
repository: ${{ github.repository }}
token: ${{ secrets.REPO_ACCESS_TOKEN }}
Related
I'm trying to troubleshoot a Github actions issue, and it looks like Github and/or Docker is doing a cross-platform build for me when they shouldn't.
Using buildx, but specifying only linux/amd64 platform for the builder.
Not setting up Qemu explicitly.
Asking for linux/arm64,linux/ppc64le platforms for the image.
... and it builds! How?
I even added docker buildx inspect command, and it prints what I expect, e.g. only Intel platforms are listed.
Here's my test repo: https://github.com/haimgel/test-docker-build
Here's my action file from there:
name: TEST
on:
push:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#v4
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- id: builder-github
uses: docker/setup-buildx-action#v2
with:
platforms: linux/amd64
- run: |
docker buildx ls
docker buildx inspect --bootstrap
- uses: docker/login-action#v2
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- uses: docker/build-push-action#v3
with:
context: .
platforms: linux/arm64,linux/ppc64le
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
I'm using the below workflow code (found in the github documentation) to build and publish a docker image to the Github Container Registry.
name: Create and publish a Docker image
on:
push:
branches: ['release']
pull_request:
branches: ['release']
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout repository
uses: actions/checkout#v3
- name: Log in to the Container registry
uses: docker/login-action#f054a8b539a109f9f41c372932f1ae047eff08c9
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push Docker image
uses: docker/build-push-action#ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
This works and I now see a public docker image under "Packages" on the github repo. When I click on the image, I am directed to a github page with more information about the image (official docs here):
"Install from the command line:"
docker pull ghcr.io/OWNER/IMAGE_NAME:pr-75
And its Digest sha: sha256:04ea7757e34c4fae527bbe6fb56eb984f54543f2313775572f0817d696ecf48a
I want to add a new job to the same workflow, that pulls the image to a virtual machine using ssh.
deploy:
- name: Deploy to Digital Ocean droplet via SSH action
uses: appleboy/ssh-action#v0.1.4
with:
host: ${{ secrets.DO_HOST }}
username: root
key: ${{ secrets.DO_PRIVATE_SSHKEY }}
port: 22
script: |
docker pull ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
This fails with:
err: invalid reference format: repository name must be lowercase (lowercasing it is not enough, read on)
Of course I cannot hard-code docker pull ghcr.io/OWNER/IMAGE_NAME:pr-75 or the Digest sha, because each new branch will increment in its PR number, so the pr-75 tag will change.
How can I deploy the image that was just published? Seems I can either use the tag value or the sha and how can I retrieve those values in real time?
There are two jobs in the above workflow:
"build-and-push-image"
"deploy"
The first one uses the docker/metadata-action to retrieve the tag name ghcr.io/OWNER/IMAGE_NAME:pr-75 which is used in the next step to name the image when docker/build-push-action is used.
I have simply used the docker/metadata-action again in the second job:
deploy:
needs: build-and-push-image
runs-on: ubuntu-latest
steps:
- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action#69f6fc9d46f2f8bf0d5491e4aabe0bb8c6a4678a
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Deploy to Digital Ocean droplet via SSH action
uses: appleboy/ssh-action#v0.1.4
with:
host: ${{ secrets.DO_HOST }}
username: root
key: ${{ secrets.DO_PRIVATE_SSHKEY }}
port: 22
script: |
docker pull ${{ steps.meta.outputs.tags }}
I am trying to stop my github CI from failing completely in case the build of a multi-arch docker images is successful for at least on architecture such that the successful builds of the those architectures are still pushed to docker hub. What I do so far:
name: 'build images'
on:
push:
branches:
- master
tags:
- '*'
schedule:
- cron: '0 4 1 * *'
jobs:
docker:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Prepare
id: prep
run: |
DOCKER_IMAGE=${{ secrets.DOCKER_USERNAME }}/${GITHUB_REPOSITORY#*/}
VERSION=latest
# If this is git tag, use the tag name as a docker tag
if [[ $GITHUB_REF == refs/tags/* ]]; then
VERSION="${GITHUB_REF#refs/tags/v}"
fi
TAGS="${DOCKER_IMAGE}:${VERSION}"
# If the VERSION looks like a version number, assume that
# this is the most recent version of the image and also
# tag it 'latest'.
if [[ $VERSION =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
TAGS="$TAGS,${DOCKER_IMAGE}:latest"
fi
# Set output parameters.
echo ::set-output name=tags::${TAGS}
echo ::set-output name=docker_image::${DOCKER_IMAGE}
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
with:
platforms: all
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Inspect builder
run: |
echo "Name: ${{ steps.buildx.outputs.name }}"
echo "Endpoint: ${{ steps.buildx.outputs.endpoint }}"
echo "Status: ${{ steps.buildx.outputs.status }}"
echo "Flags: ${{ steps.buildx.outputs.flags }}"
echo "Platforms: ${{ steps.buildx.outputs.platforms }}"
- name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action#v1
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build
uses: docker/build-push-action#v2
with:
builder: ${{ steps.buildx.outputs.name }}
context: .
file: ./Dockerfile
platforms: linux/amd64,linux/arm64,linux/arm/v7
push: true
tags: ${{ steps.prep.outputs.tags }}
- name: Sync
uses: ms-jpq/sync-dockerhub-readme#v1
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
repository: xx/yy
readme: "./README.md"
What I also did is: create this CI for each architecture individually with an own architecture tag but that way, I do not have a "multi-arch" tag..
Summary:
Github Actions allows using Docker containers to run jobs, but it doesn't seem to allow providing a dynamic value for this container image (using environment variables).
This works (not the desired solution):
jobs:
pytest-test:
container:
image: ghcr.io/ashrafgt/test:latest ...
This does not work (the desired solution):
jobs:
pytest-test:
container:
# env variables defined at the start of the workflow
image: ${{ env.REGISTRY_NAME }}/test:${{ env.IMAGE_TAG }}
...
Giving this error:
Invalid workflow file : .github/workflows/workflow.yaml
The workflow is not valid. Unrecognized named-value: 'env'. Located at position 1 within expression: env.REGISTRY_NAME
Are there any ways to do this besides doing a run: docker run ...?
Full Example:
In this example, I try to build and push a Docker image (tagged with the current commit SHA) then use the same image to run unit tests:
name: Main CI Pipeline
on: [push]
env:
REGISTRY_NAME: ghcr.io/${{ github.repository_owner }}
REGISTRY_USERNAME: ${{ github.actor }}
REGISTRY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
IMAGE_TAG: ${{ github.sha }}
jobs:
docker-build:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout#v2
- uses: docker/setup-buildx-action#v1
- uses: docker/login-action#v1
with:
registry: ${{ env.REGISTRY_NAME }}
username: ${{ env.REGISTRY_USERNAME }}
password: ${{ env.REGISTRY_PASSWORD }}
- uses: docker/build-push-action#v2
with:
tags: ${{ env.REGISTRY_NAME }}/test:${{ env.IMAGE_TAG }}
push: true
pytest-test:
needs: docker-build
runs-on: ubuntu-latest
permissions:
contents: read
packages: read
container:
image: ${{ env.REGISTRY_NAME }}/test:${{ env.IMAGE_TAG }}
steps:
- uses: actions/checkout#v2
- run: pytest
Please find the full repository here.
The full error message is:
Invalid workflow file : .github/workflows/workflow.yaml#L38
The workflow is not valid. .github/workflows/workflow.yaml (Line: 38, Col: 14): Unrecognized named-value: 'env'. Located at position 1 within expression: env.REGISTRY_NAME
Please find the Github Actions run here.
Fix Attempts:
Using only container instead of container.image:
jobs:
pytest-test:
container: ${{ env.REGISTRY_NAME }}/test:${{ env.IMAGE_TAG }}
...
Using the docker:// syntax for a single step:
jobs:
pytest-test:
steps:
- uses: docker://${{ env.REGISTRY_NAME }}/test:${{ env.IMAGE_TAG }}
entrypoint: pytest
...
Both fix attempts failed with the same error as the original syntax.
I'm trying to deploy my repository to a private Docker registry on every new release and everything works except for the version tagging. No matter what I've tried ${{ github.event.release.tag_name }} is always '', which cancels the workflow since docker tags can't be empty.
on:
release:
types: [published]
jobs:
push_to_registry:
name: Push Docker image to Docker Registry
runs-on: ubuntu-latest
steps:
-
name: Check out the repo
uses: actions/checkout#v2
-
name: Set up QEMU
uses: docker/setup-qemu-action#v1
-
name: Setup Docker Buildx
uses: docker/setup-buildx-action#v1
-
name: Log in to Docker Registry
uses: docker/login-action#v1
with:
registry: ${{ secrets.DOCKER_REGISTRY }}
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: check tag
if: ${{ github.event.release.tag_name }} == ''
run: |
echo Epic fail
exit 1
-
name: Build and Push to Docker Registry
id: docker_build
uses: docker/build-push-action#v2
with:
push: true
tags: ${{ secrets.DOCKER_REGISTRY }}/repos:latest, ${{ secrets.DOCKER_REGISTRY }}/$repos:${{ github.event.release.tag_name }}
-
name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
The repository this is running on is private so that might have something to do with it but I haven't been able to figure out what.
Any suggestions?
I think the problem is that github.event.release.tag_name is only available if the tagging itself triggered the build (but I'm not sure)
I made it work using a separate action: https://github.com/dawidd6/action-get-tag
Here's my usage:
steps:
- uses: actions/checkout#v2
- name: Get git tag
id: tag
uses: dawidd6/action-get-tag#v1
- uses: docker/build-push-action#v2
with:
context: ./
file: ./Dockerfile
push: true
tags: locustio/locust:${{ steps.tag.outputs.tag }}