I'm trying to build and push image with drone.io's plugins/docker, but it seems cannot find my repo name.
Here is the last log about the build step.
---> Running in afca20280587
Removing intermediate container afca20280587
---> cb05c781a4c4
Successfully built cb05c781a4c4
Successfully tagged caa418f0605dc7a6b2bc84faebabac55a09a373b:latest
+ /usr/local/bin/docker tag caa418f0605dc7a6b2bc84faebabac55a09a373b :latest
Error parsing reference: ":latest" is not a valid repository/tag: invalid reference format
time="2019-01-02T02:05:18Z" level=fatal msg="exit status 1"
The sixth line should be
+ /usr/local/bin/docker tag caa418f0605dc7a6b2bc84faebabac55a09a373b registry.cn-beijing.aliyuncs.com/xxx/xxx_xxx:latest
But now it didn't find my repo name.
It's drone/drone:1.0.0-rc.3, and here is my .drone.yml
kind: pipeline
name: default
steps:
- name: build
image: python:3.6
commands:
- pip install -r requirements.txt
- python -m pytest app.py
when:
branch: master
event:
- push
- pull_request
- name: publish
image: plugins/docker
registry: registry.cn-beijing.aliyuncs.com
repo: xxx/xxx_xxx
tags: [ latest ]
username:
- from_secret: ali_username
password:
- from_secret: ali_password
Is there something wrong? Thanks for any tip!
When you define the repository you need to use the fully qualified image name:
- repo: xxx/xxx_xxx
+ repo: registry.cn-beijing.aliyuncs.com/xxx/xxx_xxx
In addition, all of the plugin settings need to be declared inside the settings block [1] like this:
- name: publish
image: plugins/docker
settings:
registry: registry.cn-beijing.aliyuncs.com
repo: registry.cn-beijing.aliyuncs.com/xxx/xxx_xxx
username:
- from_secret: ali_username
password:
- from_secret: ali_password
[1] http://plugins.drone.io/drone-plugins/drone-docker/
Related
I'm trying to setup github workflow for building image and pushing it to the registry using redhat-actions actions:
workflow.yaml
name: build-maven-runner
on:
workflow_dispatch:
jobs:
build-test-push:
outputs:
image-url: ${{ steps.push-to-artifactory.outputs.registry-path }}
image-digest: ${{ steps.push-to-artifactory.outputs.digest }}
name: build-job
env:
runner_memorylimit: 2Gi
runner_cpulimit: 2
runs-on: [ linux ]
steps:
- name: Clone
uses: actions/checkout#v2
- name: Pre-Login
# podman-login: requires docker config repo auths
# Error: TypeError: Cannot set property 'some.repo.com' of undefined
mkdir /home/runner/.docker/
cat <<EOT >> /home/runner/.docker/config.json
{
"auths": {
"some.repo.com": {}
}
}
EOT
- name: Login
uses: redhat-actions/podman-login#v1
with:
registry: some.repo.com
username: ${{ secrets.USERNAME }}
password: ${{ secrets.PASSWORD }}
auth_file_path: /tmp/podman-run-1000/containers/auth.json
- name: Build
id: build-image
uses: redhat-actions/buildah-build#v2
with:
image: some-image
tags: latest
containerfiles: ./config/Dockerfile
tls-verify: false
- name: Push
id: push-to-artifactory
uses: redhat-actions/push-to-registry#v2
with:
image: ${{ steps.build-image.outputs.image }}
tags: latest
registry: some.other.repo.com/project
username: ${{ secrets.USERNAME }}
password: ${{ secrets.PASSWORD }}
tls-verify: false
./config/Dockerfile
FROM .../openshift/origin-cli:4.10
USER root
RUN sudo yum update -y
RUN sudo yum install -y maven
RUN maven -version
RUN oc version
But Build step failing resulting in:
/usr/bin/buildah version
Version: 1.22.3
Go Version: go1.15.2
Image Spec: 1.0.1-dev
Runtime Spec: 1.0.2-dev
CNI Spec: 0.4.0
libcni Version:
image Version: 5.15.2
Git Commit:
Built: Thu Jan 1 00:00:00 1970
OS/Arch: linux/amd64
Overriding storage mount_program with "fuse-overlayfs" in environment
Performing build from Containerfile
/usr/bin/buildah bud -f /runner/_work/some-project/some-project/config/Dockerfile --format docker --tls-verify=false -t some-image:latest /runner/_work/some-project/some-project
chown /home/runner/.local/share/containers/storage/overlay/l: operation not permitted
time="2022-12-12T16:13:52Z" level=warning msg="failed to shutdown storage: \"chown /home/runner/.local/share/containers/storage/overlay/l: operation not permitted\""
time="2022-12-12T16:13:52Z" level=error msg="exit status 125"
Error: Error: buildah exited with code 125
I'm fairly out of ideas at this point.. I was thinking if it has to do with storage.conf like mentioned here, but even overriding storage.conf still having same error. Originally this how storage.conf looks like:
[storage]
driver = "overlay"
runroot = "/run/containers/storage"
graphroot = "/var/lib/containers/storage"
[storage.options]
additionalimagestores = [
]
[storage.options.overlay]
mountopt = "nodev,metacopy=on"
[storage.options.thinpool]
Does the problem lies deeper like in Dockerfile image ```openshif/origin-cli?
Any help would be appreciated
I ran into this issue today because I was doing some tests locally, typically your CICD should give the correct permissions to your containers (or the workers running your jobs). I fixed this issue by adding the --privileged flag while running my container, I do not recommend using that mode in production unless you are really sure what you are doing. Perhaps not exactly your issue but dropping it here in case it helps someone else.
I have this drone.yml which is running fine and generating a docker image. I am building the docker image with 2 tags.
SHA of commit in GitHub.
latest
What I want to do is to append the date & time of image creation in the format of YYYY-MM-DD_HH-MM to the 1st tag (SHA of commit in GitHub).
---
name: api-graph-base
kind: pipeline
type: docker
steps:
- name: push-azure-nonprod
when:
event:
- push
- pull_request
branch:
- master
image: plugins/docker
settings:
username: abc
password:
from_secret: xyz
repo: nonprodazure/graph-base-api
build_args:
- LOG_LEVEL=debug
registry: nonprodregistry.azurecir.io
custom_dns: [100.100.100.100, 99.99.99.99]
tags:
- '${DRONE_BUILD_FINISHED}-${DRONE_COMMIT}'
- latest
dockerfile: Dockerfile
Image tags in drone build logs:
/usr/local/bin/docker tag c692df4346acada797d9541135f37124b15d1183 nonprodazure/graph-base-api:1600986079-c692df4346acada797d9541135f37124b15d1183
You can see in logs that the UNIX timestamp(1600986079) appended in the docker image tag name. How can I change the value of ${DRONE_BUILD_FINISHED} - Unix timestamp to DateTime human-readable string format (YYYY-MM-DD_HH-MM)?
This is what I needed to do to get date/time in a human-readable format and not UNIX format.
---
name: api-graph-base
kind: pipeline
type: docker
steps:
- name: send-tags-to-tags-file
image: node
when:
event:
- push
- pull_request
commands:
- echo -n "$(date +'%Y-%m-%d_%H-%M')_${DRONE_COMMIT}, latest" > .tags
- name: push-azure-nonprod
when:
event:
- push
branch:
- master
image: plugins/docker
settings:
username: abc
password:
from_secret: xyz
repo: nonprodazure/graph-base-api
build_args:
- LOG_LEVEL=debug
registry: nonprodregistry.azurecir.io
custom_dns: [100.100.100.100, 99.99.99.99]
dockerfile: Dockerfile
Below 2 tags created for the docker image. The second tag name contains the required date-time format now.
Solution:
Remove tags in the existing step.
Added a new step to generate tags in a .tags file.
If you need to share data between sibling processes (pipeline steps) you need to share this information by writing to and reading from disk. The docker plugin will automatically read tags from a .tags file. One cannot create an environment variable in one process and read that environment variable in a sibling process.
References:
using custom generated tags for docker images.
plugins/docker configuration multiple tags using .tags file
could somebody help me, please?
I try to build and publish my image to a private docker registry:
kind: pipeline
name: default
steps:
- name: docker
image: plugins/docker
settings:
username: ****
password: ****
repo: https://*****.com:5000/myfirstimage
registry: https://*****.com:5000
tags:
- latest
But got the next error:
Error parsing reference: "https://*****.com:5000/myfirstimage:latest" is not a valid repository/tag: invalid reference format
15921 time="2020-10-18T17:52:20Z" level=fatal msg="exit status 1"
But, when I try to push manually, all is ok.
What am I doing wrong? Will be grateful for any help.
It is not common to include the scheme in the address to docker registry and repo.
Try to write those addresses without https:// as in
repo: <registry-hostname>.com:5000/myrepo/myfirstimage
registry: <registry-hostname>.com:5000
Coming from this issue
I am using GitHub Actions for Gradle project with this given steps:
name: Java CI
on: [push]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Set up JDK 13
uses: actions/setup-java#v1
with:
java-version: 13
- run: ./gradlew bootJar
- name: Login to Github regestry
run: docker login docker.pkg.github.com -u xxxxx -p xxxxx
- name: Build the Docker image
run: docker build . -t docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.$GITHUB_REF
- name: Push the image to github
run: docker push docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.$GITHUB_REF
At the last step I get this error:
The push refers to repository
[docker.pkg.github.com/sulimanlab/realtime-chat/realtimechat-snapshot-0.refs/heads/master]
3aad04996f8f: Preparing
77cae8ab23bf: Preparing
error parsing HTTP 404 response body: invalid character 'p' after top-level value:
"404 page not found\n"
actually I was using the wrong environment variable to tag my images.
I used $GITHUB_REF what I should use $GITHUB_SHA
I am trying to write a custom github-action that runs some commands in a docker container but allows the user to select which docker container they are run in (i.e. so I can run the same build instructions across different versions of the runtime environment)
My gut instinct was to have my .github/actions/main/action.yml file as
name: 'Docker container command execution'
inputs:
dockerfile:
default: Dockerfile_r_latest
runs:
using: 'docker'
image: '${{ inputs.dockerfile }}'
args:
- /scripts/commands.sh
However this errors with:
##[error](Line: 7, Col: 10): Unrecognized named-value: 'inputs'. Located at position 1 within expression: inputs.dockerfile
Any help would be appreciated !
File References
My .github/workflow/build_and_test.yml file is:
name: Test Package
on:
[push, pull_request]
jobs:
R_latest:
name: Test on latest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
name: Checkout project
- uses: ./.github/actions/main
name: Build and test
with:
dockerfile: Dockerfile_r_latest
And my Dockerfile .github/actions/main/Dockerfile_r_latest is:
FROM rocker/verse:latest
ADD scripts /scripts
ENTRYPOINT [ "bash", "-c" ]
Interesting approach! I'm not sure if it's possible to use expressions in the image field of the action metadata. I would guess that the only fields that can take expressions instead of hardcoded strings are the args for the image so that the inputs can be passed.
For reference this is the args section of the action.yml metadata.
https://help.github.com/en/articles/metadata-syntax-for-github-actions#args
I think there are other ways to achieve what you want to do. Have you tried using the jobs.<job_id>.container syntax? That allows you to specify an image that the steps of a job will run in. It will require that you publish the image to a public repository, though. So take care not to include any secrets.
For example, if you published your image to Docker Hub at gowerc/r-latest your workflow might look something like this:
name: Test Package
on:
[push, pull_request]
jobs:
R_latest:
name: Test on latest
runs-on: ubuntu-latest
container: gowerc/r-latest
steps:
- uses: actions/checkout#master
name: Checkout project
- name: Build and test
run: ./scripts/commands.sh
ref: https://help.github.com/en/articles/workflow-syntax-for-github-actions#jobsjob_idcontainer
Alternatively, you can also specify your image at the step level with uses. You could then pass a command via args to execute your script.
name: my workflow
on: push
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#master
- name: Check container
uses: docker://alpine:3.8
with:
args: /bin/sh -c "cat /etc/alpine-release"
ref: https://help.github.com/en/github/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#example-using-a-docker-hub-action
In addition to #peterevans answer, I would add there's a 3rd option where you can use a simple docker run command and pass any env that you have defined.
That helped to solve 3 things :
Reuse a custom docker image being build within the steps for testing actions. It seems not possible to do so with uses as it first tries to pull that image that doesn't exist yet in a Setup job step that occurs before any steps of the job.
This specific image can also be stored in a private docker registry
Be able to use a variable for the docker image
My workflow looks like this :
name: Build-Test-Push
on:
push:
branches:
- master
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
ECR_REGISTRY: ${{ secrets.AWS_ECR_REGISTRY }}
ECR_REPOSITORY: myproject/myimage
IMAGE_TAG: ${{ github.sha }}
jobs:
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checking out
uses: actions/checkout#v2
with:
ref: master
- name: Login to AWS ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login#v1
- name: Build
run: |
docker pull $ECR_REGISTRY/$ECR_REPOSITORY || true
docker build . -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG -t $ECR_REGISTRY/$ECR_REPOSITORY:latest
- name: Test
run: |
docker run $ECR_REGISTRY/$ECR_REPOSITORY:latest /bin/bash -c "make test"
- name: Push
run: |
docker push $ECR_REGISTRY/$ECR_REPOSITORY
Here is another approach. The Docker image to use is passed to a cibuild shell script that takes care of pulling the right image.
GitHub workflow file:
name: 'GH Actions CI'
on:
push:
branches: ['*master', '*0.[0-9]?.x']
pull_request:
# The branches below must be a subset of the branches above
branches: ['*master', '*0.[0-9]?.x']
jobs:
build:
name: Build
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
include:
- FROM: 'ubuntu:focal'
- FROM: 'ubuntu:bionic'
- FROM: 'ubuntu:xenial'
- FROM: 'debian:buster'
- FROM: 'debian:stretch'
- FROM: 'opensuse/leap'
- FROM: 'fedora:33'
- FROM: 'fedora:32'
- FROM: 'centos:8'
steps:
- name: Checkout repository
uses: actions/checkout#v2
with:
# We must fetch at least the immediate parents so that if this is
# a pull request then we can checkout the head.
fetch-depth: 2
# If this run was triggered by a pull request event, then checkout
# the head of the pull request instead of the merge commit.
- run: git checkout HEAD^2
if: ${{ github.event_name == 'pull_request' }}
- name: Run CI
env:
FROM: ${{ matrix.FROM }}
run: script/cibuild
Bash script script/cibuild:
#!/bin/bash
set -e
docker run --name my-docker-container $FROM script/custom-script.sh
docker cp my-docker-container:/usr/src/my-workdir/my-outputs .
docker rm my-docker-container
echo "cibuild Done!"
Put your custom commands in script/custom-script.sh.