Provide arguments to docker container in composite GitHub action - docker

I'm trying to pass some dynamically created arguments within a composite GitHub Action.
The documentation however is lacking examples on how to pass arguments in this case to the docker container.
https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsuses
See here the thing I'm trying to achieve.
runs:
using: 'composite'
steps:
- name: compose arguments
id: compose-args
shell: bash
run: |
encoded_github="$(echo '${{ inputs.github_context }}' | base64)"
encoded_runner="$(echo '${{ inputs.runner_context }}' | base64)"
args=('${{ inputs.command }}')
args+=('${{ inputs.subcommand }}')
args+=('--github-context')
args+=("${encoded_github}")
args+=('--runner-context')
args+=("${encoded_runner}")
args+=('${{ inputs.arguments }}')
echo "::set-output name=provenance_args::$(echo "[$(printf "\"%s\"," ${args[*]})]" | sed 's/,]$/]/')"
- name: Debug arguments
shell: bash
run: |
echo Running slsa-provenance with following arguments
echo ${{ steps.compose-args.outputs.provenance_args }}
- uses: 'docker://ghcr.io/philips-labs/slsa-provenance:v0.5.0-draft'
with:
args: ${{ fromJSON(steps.compose-args.outputs.provenance_args) }}
fromJSON is giving me a JSON object from the composed array of bash arguments. I made the assumption this uses: 'docker://…part should receive it's arguments in the same way a docker based action would receive.
e.g.:
runs:
using: 'docker'
image: 'docker://ghcr.io/philips-labs/slsa-provenance:v0.4.0'
args:
- "generate"
- '${{ inputs.subcommand }}'
- "-artifact_path"
- '${{ inputs.artifact_path }}'
- "-output_path"
- '${{ inputs.output_path }}'
- "-github_context"
- '${{ inputs.github_context }}'
- "-runner_context"
- '${{ inputs.runner_context }}'
- "-tag_name"
- '${{ inputs.tag_name }}'
Unfortunately I'm getting the following error in the GitHub actions workflow.
The template is not valid. philips-labs/slsa-provenance-action/v0.5.0-draft/action.yaml (Line: 47, Col: 15): A sequence was not expected
See here the workflow. https://github.com/philips-labs/slsa-provenance-action/runs/4618706311?check_suite_focus=true
How can I resolve this error?
Is it resolvable with current approach?
Is this a missing feature?
What would be an alternative?

Take a look at this GitHub Action https://github.com/mr-smithers-excellent/docker-build-push
it has buildArgs as an input so it can be a solution for your case
for instance:
steps:
- uses: actions/checkout#v2
name: Check out the code
- uses: mr-smithers-excellent/docker-build-push#v5
name: Build & push Docker image
with:
image: repo/image
tags: v1, latest
registry: registry-url.io
dockerfile: ./your/path/Dockerfile
buildArgs: Test=true

I think you can just run via bash command:
- name: pull image
shell: bash
run: |
docker pull ghcr.io/philips-labs/slsa-provenance:v0.4.0
- name: run container
shell: bash
run: |
docker run --rm -i \
--workdir /github/workspace \
-v "/var/run/docker.sock":"/var/run/docker.sock" \
-v ${{ runner.temp }}/_github_home:"/github/home" \
-v ${{ github.workflow }}:"/github/workflow" \
-v ${{ runner.temp }}/_runner_file_commands:"/github/file_commands" \
-v ${{ github.workspace }}:"/github/workspace" \
ghcr.io/philips-labs/slsa-provenance:v0.4.0 \ # image
generate \ # start pass the args here, after the image
${{ inputs.subcommand }} \
-artifact_path ${{ inputs.artifact_path }} \
-output_path ${{ inputs.output_path }} \
-github_context ${{ inputs.github_context }} \
-runner_context ${{ inputs.runner_context }} \
-tag_name ${{ inputs.tag_name }}

Related

No such image: my-image:latest while pushing container with Github Actions

I am modifying my docker-publish file to build a docker image so it can work with Arm64. The previous version was working fine with x86 architecture, but now I need to make it work for Arm 64 so I just changed the way the docker builds the images.
The build process works fine but somehow the git push stopped working and I am getting the error
Error response from daemon: No such image: myimage-arm64:latest
This is my docker-publish.yml
name: Docker
on:
push:
# Publish `master` as Docker `latest` image.
branches:
- master
# Publish `v1.2.3` tags as releases.
tags:
- v*
# Run tests for any PRs.
pull_request:
env:
IMAGE_NAME: myimage-arm64
jobs:
# Push image to GitHub Packages.
# See also https://docs.docker.com/docker-hub/builds/
push:
runs-on: ubuntu-latest
if: github.event_name == 'push'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout#v2
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Prepare multiarch docker
run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
- name: Builder create
run: docker buildx create --use
- name: Log into registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
- name: Build image
run: |
docker buildx build \
--tag $IMAGE_NAME \
--file Dockerfile \
--platform linux/arm64 .
- name: Push image
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
# VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# I changed this so it takes the version from a file on my project
VERSION=$(cat version)
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
###
The two previous echo print the correct stuff
I get the error in these last two lines
###
docker tag $IMAGE_NAME $IMAGE_ID:$VERSION
docker push $IMAGE_ID:$VERSION
Any help? The push phase was working fine previously and I haven't touched it to make it work with arm64
EDIT 1:
I modified the procedure following the answers but still it does not work (error: tag is needed when pushing to register)
- name: Log into registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
- name: Builder create
run: docker buildx create --use
- name: Build image
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
VERSION=$(cat version)
echo TAG=$IMAGE_ID:$VERSION
docker buildx build --push \
--tag $IMAGE_ID:$VERSION \
--file Dockerfile \
--platform linux/arm64 .
Precisely, the logs are these ones:
Run IMAGE_ID=docker.pkg.github.com/GiamBoscaro/portfolio-website/$IMAGE_NAME
TAG=docker.pkg.github.com/UserName/RepoName/ImageName:1.2.0
#1 [internal] booting buildkit
#1 sha256:bfa0dddd89a9c970aa189079c1d31d17f7a75edd434bb19ad90432b27b266e3a
#1 pulling image moby/buildkit:buildx-stable-1
#1 pulling image moby/buildkit:buildx-stable-1 0.4s done
#1 creating container buildx_buildkit_intelligent_volhard0
#1 creating container buildx_buildkit_intelligent_volhard0 0.9s done
#1 DONE 1.3s
error: tag is needed when pushing to registry
Error: Process completed with exit code 1.
EDIT 2: Finally fixed the issue. Even if it's not the best way, here's the code that works. I switched over to the new container registry and moved the docker login in the same job of docker buildx:
jobs:
push:
runs-on: ubuntu-latest
if: github.event_name == 'push'
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout#v2
- name: Set up QEMU
uses: docker/setup-qemu-action#v1
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action#v1
- name: Prepare multiarch docker
run: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
- name: Builder create
run: docker buildx create --use
- name: Build image
run: |
IMAGE_ID=ghcr.io/${{ github.actor }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
VERSION=$(cat version)
echo TAG=$IMAGE_ID:$VERSION
echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker buildx build --push \
--tag $IMAGE_ID:$VERSION \
--file Dockerfile.arm \
--platform linux/arm64 .
Buildx runs builds within a separate container, not directly in your docker engine. And the output of buildx does not store the resulting image in the local docker engine. This doesn't work when you get into multi-platform images anyway, so you typically push directly to the registry. It's much more efficient to avoid moving layers around that didn't change in the registry, and allows you to manage multi-platform images (everything loaded into the docker engine is dereferenced to a single platform).
If you really want to save the output to the local docker engine, you can use --load in the buildx command. However, the preferred option is to use the build-push-action that builds your tag directly and pushes it in one step. This would mean reordering your steps to determine the versions and other variables first, and then run the build against that. You can see an example of this in my own project which was assembled from various other docker examples out there.
Here's a quick untested attempt to make that change:
- name: Prepare
id: prep
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
# VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# I changed this so it takes the version from a file on my project
VERSION=$(cat version)
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
echo ::set-output name=version::${VERSION}
echo ::set-output name=docker_tag::${IMAGE_ID}:${VERSION}
- name: Build and push
uses: docker/build-push-action#v2
with:
context: .
file: Dockerfile
platforms: linux/arm64
push: true
tags: ${{ steps.prep.outputs.docker_tag }}
From the updated question, this is the entire command being run:
docker buildx build --push
The next command to run would be:
--tag $IMAGE_ID:$VERSION ...
I'm sure you're saying "Wait, what? There's a trailing slash, that's a multi-line command!" But there's also whitespace after that slash, so instead of escaping a linefeed, you've escaped a space character. Docker treats that space as the one arg and will attempt to build with the context being a directory named . To fix, remove the trailing whitespace after the backslash.

Issues building Docker images with GitHub Actions

So, I have been trying to move to GitHub Actions for CI/CD. While I was writing a workflow, I faced some issues in building the image. The error is "Process completed with exit code 1". And I am unable to figure out what's wrong with my file. I mean, I got most of it from docs.
The step which is causing the error is "build" - docker build -t gcr.io/${{ secrets.GCP_PROJECT_ID }}/github-action-test:latest.
name: cloud-run-deploy
on:
push:
branches:
- develop
jobs:
build-and-deploy:
name: Cloud Run Deployment
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#master
- name: Setup GCP Service Account
uses: GoogleCloudPlatform/github-actions/setup-gcloud#master
with:
version: "latest"
service_account_email: ${{ secrets.GCP_SA_EMAIL }}
service_account_key: ${{ secrets.GCP_SA_KEY }}
export_default_credentials: true
- name: Configure Docker
run: |
gcloud auth configure-docker
- name: Build
run: |
docker build -t gcr.io/${{ secrets.GCP_PROJECT_ID }}/github-action-test:latest .
- name: Push
run: |
docker push gcr.io/${{ secrets.GCP_PROJECT_ID }}/github-action-test:latest
- name: Deploy
run: |
gcloud run deploy github-action-test \
--region europe-west1 \
--image gcr.io/${{ secrets.GCP_PROJECT_ID }}/github-action-test \
--platform managed \
--allow-unauthenticated \
--project ${{ secrets.GCP_PROJECT_ID }}

How to pull image hosted as github package in a digitalocean k8s cluster? Error: Faille dto pull image .. no basic auth credentials

Im trying to run a cron job in a digital ocean kubernetes cluster. The image is hosted as a github package. I am getting an authenticatin error when the image is being pulled. I also get the same error when trying to pull directly from docker in the command line. Is that the same problem? Or do I need to auth in 2 differnet places?
docker publish github
source: https://github.com/actions/starter-workflows/blob/aa9d3bc6cc46ac11a53ca196e504d4f901a8de8d/ci/docker-publish.yml
name: Docker
on:
push:
# Publish `master` as Docker `latest` image.
branches:
- master
# Publish `v1.2.3` tags as releases.
tags:
- v*
# Run tests for any PRs.
pull_request:
env:
# TODO: Change variable to your image's name.
IMAGE_NAME: image
jobs:
# Run tests.
# See also https://docs.docker.com/docker-hub/builds/automated-testing/
test:
ru fi
# Push image to GitHub Packages.
# See also https://docs.docker.com/docker-hub/builds/
push:
# Ensure test job passes before pushing image.
needs: test
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout#v2
- name: Build image
run: docker build . --file Dockerfile --tag $IMAGE_NAME
- name: Log into registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
- name: Push image
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
docker tag $IMAGE_NAME $IMAGE_ID:$VERSION
docker push $IMAGE_ID:$VERSION
docker build . --file Dockerfile
fi
# Push image to GitHub Packages.
# See also https://docs.docker.com/docker-hub/builds/
push:
# Ensure test job passes before pushing image.
needs: test
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout#v2
- name: Build image
run: docker build . --file Dockerfile --tag $IMAGE_NAME
- name: Log into registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
- name: Push image
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
docker tag $IMAGE_NAME $IMAGE_ID:$VERSION
docker push $IMAGE_ID:$VERSION
cron service
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: test1
spec:
schedule: "*/15 * * * *"
jobTemplate:
spec:
template:
spec:
containers:
- name: data
image: docker.pkg.github.com/lambda-capital/k8s-test/image:latest
restartPolicy: OnFailure
You can't pull anonymously from docker.pkg.github.com, you'll have to create a dockercfg with some GitHub access token and use it as imagePullSecret.
If you want to publicly host your image on GitHub you can use the newer ghcr.io container registry, which provides anonymous pulling capabilities for public images.

GitHub Actions: How to get contents of VERSION file into environment variable?

In my Docker project's repo, I have a VERSION file that contains nothing more than the version number.
1.2.3
In Travis, I'm able to cat the file to an environment variable, and use that to tag my build before pushing to Docker Hub.
---
env:
global:
- USER=username
- REPO=my_great_project
- VERSION=$(cat VERSION)
What is the equivalent of that in GitHub Actions? I tried this, but it's not working.
name: Test
on:
...
...
env:
USER: username
REPO: my_great_project
jobs:
build_ubuntu:
name: Build Ubuntu
runs-on: ubuntu-latest
env:
BASE: ubuntu
steps:
- name: Check out the codebase
uses: actions/checkout#v2
- name: Build the image
run: |
VERSION=$(cat VERSION)
docker build --file ${BASE}/Dockerfile --tag ${USER}/${REPO}:${VERSION} .
build_alpine:
name: Build Alpine
runs-on: ubuntu-latest
env:
BASE: alpine
...
...
...
I've also tried this, which doesn't work.
- name: Build the image
run: |
echo "VERSION=$(cat ./VERSION)" >> $GITHUB_ENV
docker build --file ${BASE}/Dockerfile --tag ${USER}/${REPO}:${VERSION} .
I went down the road that Benjamin W. was talking about with having VERSION in my environment vs just in that specific step.
This worked for me to set the variable in one step, then use it in separate steps.
- name: Set variables
run: |
VER=$(cat VERSION)
echo "VERSION=$VER" >> $GITHUB_ENV
- name: Build Docker Image
uses: docker/build-push-action#v2
with:
context: .
file: ${{ env.BASE_DIR }}/Dockerfile
load: true
tags: |
${{ env.USER }}/${{ env.REPO }}:${{ env.VERSION }}
${{ env.USER }}/${{ env.REPO }}:latest
As I want to re-use ENV_VAR between jobs, this is how I do it. I wish I could find a way to minimize this code.
In this example, I use VARs from my Dockerfile. But it will work from any file.
pre_build:
runs-on: ubuntu-20.04
steps:
...
-
name: Save variables to disk
run: |
cat $(echo ${{ env.DOCKERFILE }}) | grep DOCKERHUB_USER= | head -n 1 | grep -o '".*"' | sed 's/"//g' > ~/varz/DOCKERHUB_USER
cat $(echo ${{ env.DOCKERFILE }}) | grep GITHUB_ORG= | head -n 1 | grep -o '".*"' | sed 's/"//g' > ~/varz/GITHUB_ORG
cat $(echo ${{ env.DOCKERFILE }}) | grep GITHUB_REGISTRY= | head -n 1 | grep -o '".*"' | sed 's/"//g' > ~/varz/GITHUB_REGISTRY
echo "$(cat ~/varz/DOCKERHUB_USER)/$(cat ~/varz/APP_NAME)" > ~/varz/DKR_PREFIX
-
name: Set ALL variables for this job | à la sauce GitHub Actions
run: |
echo "VERSION_HASH_DATE=$(cat ~/varz/VERSION_HASH_DATE)" >> $GITHUB_ENV
echo "VERSION_HASH_ONLY=$(cat ~/varz/VERSION_HASH_ONLY)" >> $GITHUB_ENV
echo "VERSION_CI=$(cat ~/varz/VERSION_CI)" >> $GITHUB_ENV
echo "VERSION_BRANCH=$(cat ~/varz/VERSION_BRANCH)" >> $GITHUB_ENV
-
name: Show variables
run: |
echo "${{ env.VERSION_HASH_DATE }} < VERSION_HASH_DATE"
echo "${{ env.VERSION_HASH_ONLY }} < VERSION_HASH_ONLY"
echo "${{ env.VERSION_CI }} < VERSION_CI"
echo "${{ env.VERSION_BRANCH }} < VERSION_BRANCH"
-
name: Upload variables as artifact
uses: actions/upload-artifact#master
with:
name: variables_on_disk
path: ~/varz
test_build:
needs: [pre_build]
runs-on: ubuntu-20.04
steps:
...
-
name: Job preparation | Download variables from artifact
uses: actions/download-artifact#master
with:
name: variables_on_disk
path: ~/varz
-
name: Job preparation | Set variables for this job | à la sauce GitHub Actions
run: |
echo "VERSION_HASH_DATE=$(cat ~/varz/VERSION_HASH_DATE)" >> $GITHUB_ENV
echo "VERSION_HASH_ONLY=$(cat ~/varz/VERSION_HASH_ONLY)" >> $GITHUB_ENV
echo "VERSION_BRANCH=$(cat ~/varz/VERSION_BRANCH)" >> $GITHUB_ENV
echo "BRANCH_NAME=$(cat ~/varz/BRANCH_NAME)" >> $GITHUB_ENV

Deploy docker image from Github registry to Digital Ocean droplet using Github Actions

My aim is to have continuous deployment of a nodejs-express application to my digital ocean droplet.
I have a dockerized nodejs application, I am building an image and pushing the image into github packages registry. Now I want deploy this image into digital ocean droplet using github actions.
How can I do that?
My current github action file is the following:
name: Docker
on:
push:
# Publish `master` as Docker `latest` image.
branches:
- master
# Publish `v1.2.3` tags as releases.
tags:
- v*
# Run tests for any PRs.
pull_request:
env:
# TODO: Change variable to your image's name.
IMAGE_NAME: image
jobs:
# Run tests.
# See also https://docs.docker.com/docker-hub/builds/automated-testing/
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Run tests
run: |
if [ -f docker-compose.test.yml ]; then
docker-compose --file docker-compose.test.yml build
docker-compose --file docker-compose.test.yml run sut
else
docker build . --file Dockerfile
fi
# Push image to GitHub Packages.
# See also https://docs.docker.com/docker-hub/builds/
push:
# Ensure test job passes before pushing image.
needs: test
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout#v2
- name: Build image
run: docker build . --file Dockerfile --tag $IMAGE_NAME
- name: Log into registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin
- name: Push image
run: |
IMAGE_ID=docker.pkg.github.com/${{ github.repository }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
docker tag $IMAGE_NAME $IMAGE_ID:$VERSION
docker push $IMAGE_ID:$VERSION
For SCP should the following code be fine? Please correct me if I am not using the correct file name variable here
copy:
# Ensure test job passes before pushing image.
needs: push
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Copy file via scp
uses: appleboy/scp-action#master
env:
HOST: ${{ secrets.HOST }}
USERNAME: ${{ secrets.USERNAME }}
PORT: ${{ secrets.PORT }}
KEY: ${{ secrets.SSHKEY }}
with:
source: "$IMAGE_NAME"
target: "/usr/images/"
``

Resources