I am trying to use GitHub workflow to build an ASP.NET 6 project using Dockerfile then push the image to a private Azure Registry using docker.
Here is my .yml file
name: Docker Image CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Login To Azure Container Registr
uses: Azure/docker-login#v1
with:
login-server: ${{ secrets.ACR_HOST }}
username: ${{ secrets.ACR_USERNAME }}
password: ${{ secrets.ACR_PASSWWORD }}
- name: Build And Push Docker Images
uses: docker/build-push-action#v3.1.1
with:
push: true
file: ./Dockerfile
tags: companyname/projectname:${{ github.run_number }}
In the above, the Dockerfile is located in the root of my project's code.
However, the the build runs I get the following error
Error: buildx failed with: error: denied: requested access to the resource is denied
In the Secrets > Action section in my repository settings, I added ACR_HOST, ACR_USERNAME and ACR_PASSWORD secrets.
When viewing the logs, this issue seems to happen after this line in the logs
pushing companyname/projectname:2 with docker:
How can I solve this issue?
UPDATED
I changed the .yml script to the following
name: Docker Image CI
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Login To Azure Container Registr
uses: Azure/docker-login#v1
with:
login-server: mycontainer.azurecr.io
username: "The admin username"
password: "The admin password"
- run: cat ${{ env.DOCKER_CONFIG }}/config.json
- name: Build And Push Docker Images
uses: docker/build-push-action#v3.1.1
with:
push: true
file: ./Dockerfile
tags: companyname/projectname:${{ github.run_number }}
The added step (i.e., cat ${{ env.DOCKER_CONFIG }}/config.json) displayed a json string that look like this
{"auths":{"mycontainer.azurecr.io":{"auth":"BASE64 string with the admin username:password as expected"}}}
The base64 string was formatted like this username:password
I am assuming that the step Azure/docker-login#v1 has no issue and stages the token for docker/build-push-action#v3.1.1 correctly.
If I set the push flag to false in the docker/build-push-action#v3.1.1 step, the workflow runs with no issue. So from what I can tell, the issue is when the step docker/build-push-action#v3.1.1 tries to push the created image to the Azure registry.
I use my local machine to login using the same credentials and all worked with no issue docker login mycontainer.azurecr.io
Additionally, the login request from my local machine is logged into Azure portal. However, I do not see the request when I run the workflow.
I think that main issue is that the step docker/build-push-action#v3.1.1 does not attempt to login before it pushes the image.
I followed the instructions here and it worked.
Related
I'm trying to pull docker image from ECR and deploy it on ec2 instance. However it's throwing an error like
docker pull $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
======END======
err: invalid reference format
2022/11/03 15:31:54 Process exited with status 1
My yml file is:
name: Docker Image CI
on:
push:
branches: [ "main" ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials#v1
with:
aws-access-key-id: ${{ secrets.TF_USER_AWS_KEY }}
aws-secret-access-key: ${{ secrets.TF_USER_AWS_SECRET }}
aws-region: us-east-1
- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1
- name: Build, tag, and push image to Amazon ECR
env:
ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
ECR_REPOSITORY: githubactions
IMAGE_TAG: githubactions_image
run: |
docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
- name: Docker pull & run from github
uses: appleboy/ssh-action#master
with:
host: ec2-3-86-102-151.compute-1.amazonaws.com
username: ec2-user
key: ${{ secrets.ACTIONS_PRIVATE_KEY }}
envs: GITHUB_SHA
script: |
docker pull $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
I spent a lot of time and I can't really understand what's wrong. Any idea really appreciated.
Your issue is with the Env vars.
You are confusing the github server to the ec2 server.
The env var you mention in the github yml do no exist on the remote machine(ec2).
The cmd -> docker pull $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG failing cause the remote mahicne(ec2) do not know about your github env vars.
from the docs here you can see that there is an easy built-in way to pass it to the ec2 instance
- name: pass environment
uses: appleboy/ssh-action#master
+ env:
+ FOO: "BAR"
+ BAR: "FOO"
+ SHA: ${{ github.sha }}
with:
host: ${{ secrets.HOST }}
username: ${{ secrets.USERNAME }}
key: ${{ secrets.KEY }}
port: ${{ secrets.PORT }}
+ envs: FOO,BAR,SHA
script: |
echo "I am $FOO"
echo "I am $BAR"
echo "sha: $SHA"
hope it helps and goodluck
I would first add an ls and echo:
run: |
pwd
ls -arth
echo "docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG ."
docker ...
That way, I would check if I am in the right folder (with a Dockerfile in it), and if all variables are indeed valued.
If you see <aregistry>/<arepo>: (meaning no tag), the final ':' might be enough to trigger the error message.
That or:
the name use the wrong hyphen '-' as in this issue.
the image name has an invalid character like a \
the image name uses the wrong syntax
I have two GitHub actions workflow set up right now. One to publish an image to a jfrog registry, and another to promote the image with a new tag to the jfrog artifactory.
I am trying to use the github.event.number in the push workflow but for some reason you can't get the PR number if it isn't a pull_request being made... hence I get the error:
"Error response from daemon: manifest for (company jfrog artifactory url) - not found: manifest unknown: The named manifest is not known to the registry.
Anyone know any work arounds to this?
I successfully got the PR number from a PUSH event by using this implementation:
name: Get PR Number on PUSH event
on: [push, pull_request]
jobs:
push:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'push' }}
steps:
- uses: actions/checkout#v2.3.4
with:
fetch-depth: 0
- name: Get Pull Request Number
id: pr
run: echo "::set-output name=pull_request_number::$(gh pr view --json number -q .number || echo "")"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- run: echo ${{ steps.pr.outputs.pull_request_number }}
pull-request:
runs-on: ubuntu-latest
if: ${{ github.event_name == 'pull_request' }}
steps:
- run: echo ${{ github.event.number }}
I also let the pull_request job to show how to get it from this event as well (if you want to compare when you realize a push to an already opened PR).
Part of the solution was shared here but you also needed to add the actions/checkout to the job steps otherwise the gh cli didn't recognised the repo.
You can check the 2 workflow runs here:
push event: https://github.com/GuillaumeFalourd/poc-github-actions/runs/4317001027?check_suite_focus=true
pull request event: https://github.com/GuillaumeFalourd/poc-github-actions/actions/runs/1501093093
I am trying to create a Github-Actions workflow - pytorch_error.yml to automatically push Docker images to Docker Hub using Github Actions -
# This is a basic workflow to help you get started with Actions
name: Building and pushing Docker images to Docker hub
# Controls when the workflow will run
on:
workflow_dispatch:
branches: [main]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build_push_pytorch_docker_image:
name: Build and push apex-pytorch-image image to Docker Hub
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- name: Checkout Github repo
uses: actions/checkout#v2
- name: Log into Docker Hub
uses: docker/login-action#f054a8b539a109f9f41c372932f1ae047eff08c9
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASSWORD }}
- name: Get Metadata (tags,labels) for Docker images
id: meta_pytorch
uses: docker/metadata-action#98669ae865ea3cffbcbaa878cf57c20bbf1c6c38
with:
images: kusur/apex-pytorch-image
- name: Build and push Docker image to Docker Hub
uses: docker/build-push-action#ad44023a93711e3deb337508980b4b5e9bcdc5dc
with:
context: .
file: ./dockerfile-pytorch
push: true
tags: ${{ steps.meta_pytorch.ouputs.tags }}
labels: ${{ steps.meta_pytorch.outputs.labels }}
Whenever I execute this code, I get the following error -
error: tag is needed when pushing to registry
Error: buildx call failed with: error: tag is needed when pushing to registry
While looking at the logs, I see that the tag is being generated at the previous step i.e "Get Metadata (tags,labels) for Docker images" -
with:
images: ***/apex-pytorch-image
github-token: ***
Context info
eventName: workflow_dispatch
sha: 046137ce5ae09aac18ba44083cd061ac3a37e48a
ref: refs/heads/main
workflow: Building and pushing Docker images to Docker hub
action: meta_pytorch
actor: ***
runNumber: 2
runId: 1090239471
Processing tags input
type=schedule,pattern=nightly,enable=true,priority=1000
type=ref,event=branch,enable=true,priority=600
type=ref,event=tag,enable=true,priority=600
type=ref,event=pr,prefix=pr-,enable=true,priority=600
Processing flavor input
latest=auto
prefix=
suffix=
Docker image version
main
Docker tags
***/apex-pytorch-image:main
Docker labels
org.opencontainers.image.title=learning-audio-processing
org.opencontainers.image.description=Learning Audio Processing
org.opencontainers.image.url=https://github.com/***/learning-audio-processing
org.opencontainers.image.source=https://github.com/***/learning-audio-processing
org.opencontainers.image.version=main
org.opencontainers.image.created=2021-08-02T12:39:20.636Z
org.opencontainers.image.revision=046137ce5ae09aac18ba44083cd061ac3a37e48a
org.opencontainers.image.licenses=Unlicense
JSON output
{
"tags": [
"***/apex-pytorch-image:main"
],
"labels": {
"org.opencontainers.image.title": "learning-audio-processing",
"org.opencontainers.image.description": "Learning Audio Processing",
"org.opencontainers.image.url": "https://github.com/***/learning-audio-processing",
"org.opencontainers.image.source": "https://github.com/***/learning-audio-processing",
"org.opencontainers.image.version": "main",
"org.opencontainers.image.created": "2021-08-02T12:39:20.636Z",
"org.opencontainers.image.revision": "046137ce5ae09aac18ba44083cd061ac3a37e48a",
"org.opencontainers.image.licenses": "Unlicense"
}
}
Bake definition file
{
"target": {
"docker-metadata-action": {
"tags": [
"***/apex-pytorch-image:main"
],
"labels": {
"org.opencontainers.image.title": "learning-audio-processing",
"org.opencontainers.image.description": "Learning Audio Processing",
"org.opencontainers.image.url": "https://github.com/***/learning-audio-processing",
"org.opencontainers.image.source": "https://github.com/***/learning-audio-processing",
"org.opencontainers.image.version": "main",
"org.opencontainers.image.created": "2021-08-02T12:39:20.636Z",
"org.opencontainers.image.revision": "046137ce5ae09aac18ba44083cd061ac3a37e48a",
"org.opencontainers.image.licenses": "Unlicense"
},
"args": {
"DOCKER_META_IMAGES": "***/apex-pytorch-image",
"DOCKER_META_VERSION": "main"
}
}
}
}
but it is not being read by the build-push-action. This code is copied from Publishing Docker Images. Another file created from this reference is pytorch_image.yml and this code executes without any issue but the code in question is breaking again and again. I am not able to make out any difference between pytorch_image.yml and pytorch_error.yml Any help?
Coming from this issue
I am using GitHub Actions for Gradle project with this given steps:
name: Java CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Set up JDK 13
uses: actions/setup-java#v1
with:
java-version: 13
- run: ./gradlew bootJar
- name: Login to Github regestry
run: docker login docker.pkg.github.com -u xxxxx -p xxxxx
- name: Build the Docker image
run: docker build . -t docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.$GITHUB_REF
- name: Push the image to github
run: docker push docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.$GITHUB_REF
At the last step I get this error:
The push refers to repository
[docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.refs/heads/master]
3aad04996f8f: Preparing
77cae8ab23bf: Preparing
error parsing HTTP 404 response body: invalid character 'p' after top-level value:
"404 page not found\n"
actually I was using the wrong environment variable to tag my images.
I used $GITHUB_REF what I should use $GITHUB_SHA
I am trying to write a custom github-action that runs some commands in a docker container but allows the user to select which docker container they are run in (i.e. so I can run the same build instructions across different versions of the runtime environment)
My gut instinct was to have my .github/actions/main/action.yml file as
name: 'Docker container command execution'
inputs:
dockerfile:
default: Dockerfile_r_latest
runs:
using: 'docker'
image: '${{ inputs.dockerfile }}'
args:
- /scripts/commands.sh
However this errors with:
##[error](Line: 7, Col: 10): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.dockerfile
Any help would be appreciated !
File References
My .github/workflow/build_and_test.yml file is:
name: Test Package
on:
[push, pull_request]
jobs:
R_latest:
name: Test on latest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
name: Checkout project
- uses: ./.github/actions/main
name: Build and test
with:
dockerfile: Dockerfile_r_latest
And my Dockerfile .github/actions/main/Dockerfile_r_latest is:
FROM rocker/verse:latest
ADD scripts /scripts
ENTRYPOINT [ "bash", "-c" ]
Interesting approach! I'm not sure if it's possible to use expressions in the image field of the action metadata. I would guess that the only fields that can take expressions instead of hardcoded strings are the args for the image so that the inputs can be passed.
For reference this is the args section of the action.yml metadata.
https://help.github.com/en/articles/metadata-syntax-for-github-actions#args
I think there are other ways to achieve what you want to do. Have you tried using the jobs.<job_id>.container syntax? That allows you to specify an image that the steps of a job will run in. It will require that you publish the image to a public repository, though. So take care not to include any secrets.
For example, if you published your image to Docker Hub at gowerc/r-latest your workflow might look something like this:
name: Test Package
on:
[push, pull_request]
jobs:
R_latest:
name: Test on latest
runs-on: ubuntu-latest
container: gowerc/r-latest
steps:
- uses: actions/checkout#master
name: Checkout project
- name: Build and test
run: ./scripts/commands.sh
ref: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idcontainer
Alternatively, you can also specify your image at the step level with uses. You could then pass a command via args to execute your script.
name: my workflow
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Check container
uses: docker://alpine:3.8
with:
args: /bin/sh -c "cat /etc/alpine-release"
ref: https://help.github.com/en/github/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#example-using-a-docker-hub-action
In addition to #peterevans answer, I would add there's a 3rd option where you can use a simple docker run command and pass any env that you have defined.
That helped to solve 3 things :
Reuse a custom docker image being build within the steps for testing actions. It seems not possible to do so with uses as it first tries to pull that image that doesn't exist yet in a Setup job step that occurs before any steps of the job.
This specific image can also be stored in a private docker registry
Be able to use a variable for the docker image
My workflow looks like this :
name: Build-Test-Push
on:
push:
branches:
- master
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
ECR_REGISTRY: ${{ secrets.AWS_ECR_REGISTRY }}
ECR_REPOSITORY: myproject/myimage
IMAGE_TAG: ${{ github.sha }}
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checking out
uses: actions/checkout#v2
with:
ref: master
- name: Login to AWS ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1
- name: Build
run: |
docker pull $ECR_REGISTRY/$ECR_REPOSITORY || true
docker build . -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -t $ECR_REGISTRY/$ECR_REPOSITORY:latest
- name: Test
run: |
docker run $ECR_REGISTRY/$ECR_REPOSITORY:latest /bin/bash -c "make test"
- name: Push
run: |
docker push $ECR_REGISTRY/$ECR_REPOSITORY
Here is another approach. The Docker image to use is passed to a cibuild shell script that takes care of pulling the right image.
GitHub workflow file:
name: 'GH Actions CI'
on:
push:
branches: ['*master', '*0.[0-9]?.x']
pull_request:
# The branches below must be a subset of the branches above
branches: ['*master', '*0.[0-9]?.x']
jobs:
build:
name: Build
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
include:
- FROM: 'ubuntu:focal'
- FROM: 'ubuntu:bionic'
- FROM: 'ubuntu:xenial'
- FROM: 'debian:buster'
- FROM: 'debian:stretch'
- FROM: 'opensuse/leap'
- FROM: 'fedora:33'
- FROM: 'fedora:32'
- FROM: 'centos:8'
steps:
- name: Checkout repository
uses: actions/checkout#v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
- name: Run CI
env:
FROM: ${{ matrix.FROM }}
run: script/cibuild
Bash script script/cibuild:
#!/bin/bash
set -e
docker run --name my-docker-container $FROM script/custom-script.sh
docker cp my-docker-container:/usr/src/my-workdir/my-outputs .
docker rm my-docker-container
echo "cibuild Done!"
Put your custom commands in script/custom-script.sh.