I'm trying to build image for platform ppc64le via Docker Buildx and Buildkit on our enterprise Travis CI instance.
.travis.yml:
os: linux
dist: bionic
language: shell
branches:
only:
- master
before_install:
- set -e
# Configure environment so changes are picked up when the Docker daemon is restarted after upgrading
- echo '{"experimental":true}' | sudo tee /etc/docker/daemon.json
- export DOCKER_CLI_EXPERIMENTAL=enabled
- sudo rm -rf /var/lib/apt/lists/*
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
- sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) edge"
- sudo apt-get update
- sudo apt-get -y -o Dpkg::Options::="--force-confnew" install docker-ce
- mkdir -vp ~/.docker/cli-plugins/
- curl --silent -L "https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64" > ~/.docker/cli-plugins/docker-buildx
- chmod a+x ~/.docker/cli-plugins/docker-buildx
jobs:
include:
- stage: build and push docker image
script:
- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
- sudo docker buildx build --platform linux/ppc64le --tag myimage:ppc64le -f src/main/docker/Dockerfile.ppc64 --push .
Build will fail on error:
$ sudo docker buildx build --platform linux/ppc64le --tag myimage:ppc64le -f src/main/docker/Dockerfile.ppc64 --push .
unknown flag: --platform
See 'docker --help'.
Usage: docker [OPTIONS] COMMAND
Looks like Buildx extension is not enabled, but Docker info will show that experimental_cli is enabled.
Any ideas about how to enable buildx on Travis?
I'm having trouble myself but TravisCI official documentation states you need to install buildx plugin. Here: https://www.docker.com/blog/multi-arch-build-what-about-travis/
From what I can see you are missing this in before_install
- mkdir -vp ~/.docker/cli-plugins/
- curl --silent -L "https://github.com/docker/buildx/releases/download/v0.3.0/buildx-v0.3.0.linux-amd64" > ~/.docker/cli-plugins/docker-buildx
- chmod a+x ~/.docker/cli-plugins/docker-buildx
Related
I have this .gitlab-ci.yml file wanting to automate the docker image building, basically I'm using the one from the template:
docker-build:
image: my_image_build_with_docker_inside_inprivate_repo
stage: build
services:
- docker:dind
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
# Default branch leaves tag empty (= latest tag)
# All other branches are tagged with the escaped branch name (commit ref slug)
script:
- |
if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
tag=""
echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
else
tag=":$CI_COMMIT_REF_SLUG"
echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
fi
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
- docker push "$CI_REGISTRY_IMAGE${tag}"
# Run this job in a branch where a Dockerfile exists
rules:
- if: $CI_COMMIT_BRANCH
exists:
- Dockerfile
#$CI_REGISTRY_IMAGE = my_image_build_with_docker_inside_inprivate_repo
When I run it get this error:
$ docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGIST
/bin/bash: line 132: docker: command not found
Cleaning up project directory and file based variables
ERROR: Job failed: exit code 1
With : RUN apt-get install -y docker-compose , I get everything needed for docker to run in the debian image. But I still get the error of docker command not found as above. What are the other steps needed to run docker daemon from your custom image?
Update: From the private repository, pushing images with docker installed is not allowed either. Looks like I have to use something called kaniko. Any good resource for this?
You can try installing docker in the Dockerfile to build your custom image.
You can follow the steps defined in the official docs https://docs.docker.com/engine/install/debian/ which would look like something like this in your Dockerfile
RUN apt-get install -y \
ca-certificates \
curl \
gnupg \
lsb-release
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
RUN echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null
RUN apt-get update -y
RUN apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin
Ive created a Dockerfile that is based off jenkins/jenkins:lts-jdk11
Im trying to install docker + docker compose so that jenkins will have access to this when i create my pipeline for CD/CI.
Here is my Dockerfile:
FROM jenkins/jenkins:lts-jdk11 AS jenkins
WORKDIR /home/jenkins
RUN chown -R 1000:1000 /var/jenkins_home
USER root
# Install aws cli version 2
RUN apt-get update && apt-get install -y unzip curl vim bash sudo
RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
RUN unzip awscliv2.zip
RUN ./aws/install
#Install docker cli command
RUN sudo apt-get update
RUN sudo apt install -y apt-transport-https ca-certificates curl gnupg lsb-release
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
RUN echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
RUN sudo apt-get update
RUN sudo apt-get install -y docker-ce docker-ce-cli containerd.io
##Install docker compose
RUN mkdir -p /usr/local/lib/docker/cli-plugins
RUN curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o /usr/local/lib/docker/cli-plugins/docker-compose
RUN chmod +x /usr/local/lib/docker/cli-plugins/docker-compose
RUN sudo usermod -a -G docker jenkins
The docker commands work well within the container but as soon as i start to build an image it displays this error:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
If i try to start the docker service with service docker start i get the following error:
mkdir: cannot create directory ‘cpuset’: Read-only file system
Im not sure how to solve this one.
TIA
container does not use an init system. The Docker service cannot be started because of this.
As I understand, a GitHub action can also be executed by a (Linux) Docker image to perform a certain task.
For example this GitHub action Azure/static-web-apps-deploy#v0.0.1-preview uses Docker image mcr.microsoft.com/appsvc/staticappsclient:stable to deploy the project to an Azure Static Web App.
Is there a way to call/execute/run this Docker image in a YAML DevOps Pipeline?
Container jobs in Azure devops pipeline is probably what you are looking for.
When you specify a container in your pipeline, the agent will first fetch and start the container. Then, each step of the job will run inside the container. So you can set the Docker image as the container. See below example:
To run a job in a container:
pool:
vmImage: 'ubuntu-18.04'
container: mcr.microsoft.com/appsvc/staticappsclient:stable
steps:
- script: printenv
To run a certain task in a container:
resources:
containers:
- container: staticappsclient
image: mcr.microsoft.com/appsvc/staticappsclient:stable
steps:
- task: SampleTask#1
target: host
- task: AnotherTask#1
target: staticappsclient # this task will run in the container
Update:
To run sudo command inside a container without sudo preinstalled. You can checkout below steps:
1, Name the container by defining the --name parameter in Option options: "--name ci-container -v /usr/bin/docker:/tmp/docker:ro"
2, Add a script task on top of your yaml pipeline to install the sudo
- script: |
/tmp/docker exec -t -u 0 ci-container \
sh -c "apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confold" -y install sudo"
3, Run the scripts to Install dotnet 3.1 in the following script task: See below full yaml example:
resources:
containers:
- container: staticappsclient
image: mcr.microsoft.com/appsvc/staticappsclient:stable
options: "--name ci-container -v /usr/bin/docker:/tmp/docker:ro"
container: staticappsclient
steps:
- script: |
/tmp/docker exec -t -u 0 ci-container \
sh -c "apt-get update && DEBIAN_FRONTEND=noninteractive apt-get -o Dpkg::Options::="--force-confold" -y install sudo"
- script: |
wget -O - https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg
sudo mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
wget https://packages.microsoft.com/config/debian/9/prod.list
sudo mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
sudo chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
sudo chown root:root /etc/apt/sources.list.d/microsoft-prod.list
sudo apt-get install -y apt-transport-https && \
sudo apt-get update && \
sudo apt-get install -y dotnet-sdk-3.1
See this thread for more information.
I am trying to push a Docker image to Google Cloud Registry via the GitLab ci pipeline.
The image builds but when its time to push to registry i get the following error.
denied: Token exchange failed for project 'nice-column-247216'. Caller
does not have permission 'storage.buckets.get'. To configure
permissions, follow instructions at:
https://cloud.google.com/container-registry/docs/access-control
.gitlab-ci.yml
stages:
- security
- quality
- test
- build
- deploy
image: node:10.16.0
services:
- mongo
- docker:dind
.before_script_template: &before_docker_script
before_script:
- apt-get update
- apt-get -y install apt-transport-https ca-certificates curl gnupg2 software-properties-common
- curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
- add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
- apt-get update
- apt-get -y install docker-ce
- curl -L "https://github.com/docker/compose/releases/download/1.23.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
- chmod +x /usr/local/bin/docker-compose
- echo "$SERVICE_ACCOUNT_KEY" > key.json
- docker login -u _json_key --password-stdin https://eu.gcr.io < key.json
build:
stage: build
<<: *before_docker_script
variables:
DOCKER_IMAGE_TAG: 'eu.gcr.io/nice-column-247216/my-application'
script:
- docker build --cache-from "${DOCKER_IMAGE_TAG}" -t "${DOCKER_IMAGE_TAG}" .
- docker push ${DOCKER_IMAGE_TAG}
As you can see I am logging in to Docker via the json key. You can see in the below image the permissions this token has, both Storage Admin and Storage Object Viewer.
I am having difficulties with enabling docker for build job. This is how gitlab ci config file looks like:
image: docker:latest
services:
- docker:dind
stages:
- build
build:
image: java:8
stage: build
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com/...
- sbt server/docker:publish
And here is the output from job:
gitlab-ci-multi-runner 1.3.2 (0323456)
Using Docker executor with image java:8 ...
Pulling docker image docker:dind ...
Starting service docker:dind ...
Waiting for services to be up and running...
Pulling docker image java:8 ...
Running on runner-30dcea4b-project-1408237-concurrent-0 via runner-30dcea4b-machine-1470340415-c2bbfc45-digital-ocean-4gb...
Cloning repository...
Cloning into '/builds/.../...'...
Checking out 9ba87ff0 as master...
$ docker info
/bin/bash: line 42: docker: command not found
ERROR: Build failed: exit code 1
Any clues why docker is not found?
After few days of struggling, I came up with following setup:
image: gitlab/dind
stages:
- test
- build
before_script:
- echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections
- apt-get update
- apt-get install -y curl
- apt-get install -y software-properties-common python-software-properties
- add-apt-repository -y ppa:webupd8team/java
- apt-get update
- apt-get install -y oracle-java8-installer
- rm -rf /var/lib/apt/lists/*
- rm -rf /var/cache/oracle-jdk8-installer
- apt-get update -yqq
- apt-get install apt-transport-https -yqq
- echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
- apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 642AC823
- apt-get update -yqq
- apt-get install sbt -yqq
- sbt sbt-version
test:
stage: test
script:
- sbt scalastyle && sbt test:scalastyle
- sbt clean coverage test coverageReport
build:
stage: build
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com/...
- sbt server/docker:publish
It has docker (mind gitlab/dind image), java and sbt. Now I can push to gitlab registry from sbt docker plugin.
docker info command is running inside java:8 based container which will not have docker installed/available in it.