I was following the Microsoft [Doc-][1] to run the ADO build agent in Azure Container registry.
When I tried to build the Docker image by the docker file, getting lot of network related issues.
FROM ubuntu:18.04
# To make it easier for build and release pipelines to run apt-get,
# configure apt to not require confirmation (assume the -y argument by default)
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates \
curl \
jq \
git \
iputils-ping \
libcurl4 \
libicu60 \
libunwind8 \
netcat \
libssl1.0 \
&& rm -rf /var/lib/apt/lists/*
RUN curl -LsS https://aka.ms/InstallAzureCLIDeb | bash \
&& rm -rf /var/lib/apt/lists/*
# Can be 'linux-x64', 'linux-arm64', 'linux-arm', 'rhel.6-x64'.
ENV TARGETARCH=linux-x64
WORKDIR /azp
COPY ./start.sh .
RUN chmod +x start.sh
ENTRYPOINT ["./start.sh"]
So I could see there are apt-get update and install commands which are trying for the internet connectivity and getting failed.
---> Running in 456779f71a59
Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
Connection failed [IP: 91.189.91.38 80]
Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Connection failed [IP: 91.189.91.39 80]
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Connection failed [IP: 91.189.91.38 80]
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Connection failed [IP: 91.189.91.39 80]
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease Connection failed [IP: 91.189.91.39 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease Connection failed [IP: 91.189.91.38 80]
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease Connection failed [IP: 91.189.91.39 80]
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease Connection failed [IP: 91.189.91.38 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
Package ca-certificates is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'ca-certificates' has no installation candidate
E: Unable to locate package curl
E: Unable to locate package jq
E: Unable to locate package git
E: Unable to locate package iputils-ping
E: Unable to locate package libcurl4
E: Unable to locate package libicu60
E: Unable to locate package libunwind8
E: Unable to locate package netcat
E: Unable to locate package libssl1.0
E: Couldn't find any package by glob 'libssl1.0'
E: Couldn't find any package by regex 'libssl1.0'
So looking for a way to create image in offline and will ned to have below list of softwares coupled in it.
Azure CLI
Git 2.30.1
Python2
Python3
OpenJDK 11
OpenJDK 1.8.0
Docker 19.03.13
Helm 3.5.2
Kubectl
NodeJS 10
Maven 3.6.3
jq 1.6
Azure CLI DevOps extension
Configuration for Maven and NPM needs to be done manually.
MAVEN: "~/.m2/settings.xml"
is there any way to do this or Any Microsoft official image available in dockerhub for the selfhosted agent ?
Related
I have a Dockerfile
FROM mariadb:10.1.21
COPY mariadb.cnf /etc/mysql/conf.d
RUN apt-get update \
&& apt-get install --assume-yes mariadb-backup-10.1 \
&& rm -rf /var/lib/apt/lists/*
when I'm running docker build . the apt-get update command throws errors:
Get:8 https://repo.percona.com jessie/main amd64 Packages [22.1 kB]
Get:9 http://deb.debian.org jessie/main amd64 Packages [9098 kB]
W: GPG error: http://security.debian.org jessie/updates InRelease: The following signatures were invalid: KEYEXPIRED 1668892417 The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AA8E81B4331F7F50
W: GPG error: http://deb.debian.org jessie-updates InRelease: The following signatures were invalid: KEYEXPIRED 1668891673
W: GPG error: http://deb.debian.org jessie Release: The following signatures were invalid: KEYEXPIRED 1668891673
W: Failed to fetch http://ftp.osuosl.org/pub/mariadb/repo/10.1/debian/dists/jessie/main/binary-amd64/Packages 404 Not Found [IP: 64.50.236.52 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
I know how to update the key-list following this tutorial on my local machine. But how do I update the keys for the Dockerfile? I ran apt-key list inside the docker container and got a bunch of expired keys. Or do I just update the mariadb image version at the top?
I have the following Dockerfile
FROM jupyter/scipy-notebook
#RUN apt-get update && apt-get install -y curl
RUN pip install mlflow
RUN pip install sklearn
RUN apt-get update && apt-get install -y curl
When I do
docker build -t mycontainer .
I get
Step 4/4 : RUN apt-get update && apt-get install -y curl
---> Running in 5defd9816a22
Reading package lists...
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
The command '/bin/bash -o pipefail -c apt-get update && apt-get install -y curl' returned a non-zero code: 100
I suspect it is something related with not running as a root?
So how can I install curl in my container from the Dockerfile?
EDIT: I applied the answer that I was given.
Unfortunately it did not work
Step 4/7 : USER root
---> Running in aa6a1b7a023f
Removing intermediate container aa6a1b7a023f
---> 59e87b9598b2
Step 5/7 : RUN apt-get update && apt-get install -y curl
---> Running in 4440ce61ebc6
Err:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu focal InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-updates/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-backports/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/InRelease Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package curl
The command '/bin/bash -o pipefail -c apt-get update && apt-get install -y curl' returned a non-zero code: 100
EDIT2: I tried it in a different network setup and it worked.
You're seeing a "permission denied" error when apt attempts to create /var/lib/apt/lists/partial. This is because your process isn't running as root; the jupyter/scipy-notebook image is configured to run as non-root user (it runs as user jovyan).
You can change the user under which commands run in your Dockerfile with the USER directive, like this:
FROM jupyter/scipy-notebook
RUN pip install mlflow
RUN pip install sklearn
USER root
RUN apt-get update && apt-get install -y curl
USER jovyan
Note that I've made sure to reset the user back to jovyan after running the apt-get command.
Try to install it with root as it done in official jupyter/scipy-notebook Dockerfile and set it back to $NB_UID: (see user declaration in official base docker image)
USER root
# Install packages you need:
RUN apt-get update && apt-get install -y curl
# Switch back to jovyan to avoid accidental container runs as root
USER $NB_UID
Here is the hierarchy of base docker images used by jupyter/scipy-notebook:
jupyter/base-notebook -> jupyter/minimal-notebook -> jupyter/scipy-notebook
I am trying to build the following dockerfile
FROM php:8.0-apache
RUN apt-get update
# because apparently composer can shell out to unzip? Who knew...
RUN apt-get install wget unzip zip -y
but it failed with the following error
[root#dev svc]# docker build -t wsb -f Dockerfile .
Sending build context to Docker daemon 17.62 MB
Step 1/11 : FROM php:8.0-apache
---> 97f22a92e1d1
Step 2/11 : RUN apt-get update
---> Using cache
---> ddba6cf13bac
Step 3/11 : RUN apt-get install wget unzip zip -y
---> Running in 6fb3e2f37401
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package wget
E: Unable to locate package unzip
E: Unable to locate package zip
The command '/bin/sh -c apt-get install wget unzip zip -y' returned a non-zero code: 100
Existing contents of sources.list file
# deb http://snapshot.debian.org/archive/debian/20210511T000000Z buster main
deb http://deb.debian.org/debian buster main
# deb http://snapshot.debian.org/archive/debian-security/20210511T000000Z buster/updates main
deb http://security.debian.org/debian-security buster/updates main
# deb http://snapshot.debian.org/archive/debian/20210511T000000Z buster-updates main
deb http://deb.debian.org/debian buster-updates main
I am not sure why it's unable to find. Should I add any URL to /etc/apt/sources.list to get this working?
EDIT
Sending build context to Docker daemon 17.62 MB
Step 1/9 : FROM php:8.0-apache
---> 97f22a92e1d1
Step 2/9 : RUN apt-get update && apt-get install wget unzip zip -y
---> Running in 374b1060dfb3
Err:1 http://security.debian.org/debian-security buster/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:2 http://deb.debian.org/debian buster InRelease
Temporary failure resolving 'deb.debian.org'
Err:3 http://deb.debian.org/debian buster-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists...
W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
You've cached a stale update so apt is trying to install these packages from versions that are no longer in the repositories. This is why Docker's best practices mention to not separate these steps:
FROM php:8.0-apache
RUN apt-get update \
&& apt-get install wget unzip zip -y
From the second issue you are now seeing, you have a networking issue within your containers. There are lots of possible causes, including the host not being connected to the network, and networking changing after the docker engine was started. You'd need to provide a lot more debugging (can you reach these nodes from the host, is DNS working, is there a proxy or firewall on the network, etc).
I have Dockerfile that I have used many times without an issue. Now I need to add some packages to it (ssmtp and sendmail) and when I add them the build fails with:
Sending build context to Docker daemon 645.3 MB
Sending build context to Docker daemon
Step 0 : FROM debian:jessie
---> 736e5442e772
Step 1 : MAINTAINER Larry Martell <larry.martell#foo.com>
---> Using cache
---> bd272aa26940
Step 2 : ENV HOME /opt/django/CAPgraph/
---> Using cache
---> 1c540ed91808
Step 3 : RUN echo "deb http://http.debian.net/debian jessie-backports main" >> /etc/apt/sources.list
---> Using cache
---> 8788d48e625d
Step 4 : RUN (apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential git python python-dev python-setuptools nginx sqlite3 supervisor mysql-server libmysqlclient-dev vim cron unzip software-properties-common python-software-properties openjdk-8-jre xvfb wkhtmltopdf ssmtp sendmail)
---> Running in 8986bca93fdb
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
Get:2 http://security.debian.org jessie/updates/main amd64 Packages [436 kB]
Get:3 http://http.debian.net jessie-backports InRelease [166 kB]
Get:4 http://httpredir.debian.org jessie-updates InRelease [145 kB]
Get:5 http://http.debian.net jessie-backports/main amd64 Packages [1031 kB]
Get:6 http://httpredir.debian.org jessie-updates/main amd64 Packages [17.6 kB]
Ign http://httpredir.debian.org jessie InRelease
Get:7 http://httpredir.debian.org jessie Release.gpg [2373 B]
Get:8 http://httpredir.debian.org jessie Release [148 kB]
Get:9 http://httpredir.debian.org jessie/main amd64 Packages [9049 kB]
Fetched 11.1 MB in 9s (1211 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
sendmail : Depends: sendmail-bin but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c (apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential git python python-dev python-setuptools nginx sqlite3 supervisor mysql-server libmysqlclient-dev vim cron unzip software-properties-common python-software-properties openjdk-8-jre xvfb wkhtmltopdf ssmtp sendmail)' returned a non-zero code: 100
If I add those packages to the list I then complains about others. What does this 'held broken packages' message mean and how do I fix it?
Here is the first part of my Dockerfile:
FROM debian:jessie ENV HOME /opt/django/CAPgraph/
RUN echo "deb http://http.debian.net/debian jessie-backports main" >> /etc/apt/sources.list
RUN (apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential git python python-dev python-setuptools nginx sqlite3 supervisor mysql-server libmysqlclient-dev vim cron unzip software-properties-common python-software-properties openjdk-8-jre xvfb wkhtmltopdf sendmail ssmtp)
I tried adding sendmail-bin and then it failed with:
The following packages have unmet dependencies:
sendmail-bin : Conflicts: mail-transport-agent
ssmtp : Conflicts: mail-transport-agent
E: Unable to correct problems, you have held broken packages.
I then added mail-transport-agent and it failed with:
Package mail-transport-agent is a virtual package provided by:
opensmtpd 5.7.3p2-1~bpo8+1
ssmtp 2.64-8
sendmail-bin 8.14.4-8+deb8u1
qmail-run 2.0.2+nmu1
postfix 2.11.3-1
nullmailer 1:1.13-1+deb8u1
msmtp-mta 1.4.32-2
masqmail 0.2.30-1
lsb-invalid-mta 4.1+Debian13+nmu1
exim4-daemon-light 4.84.2-2+deb8u3
exim4-daemon-heavy 4.84.2-2+deb8u3
esmtp-run 1.2-12
dma 0.9-1
courier-mta 0.73.1-1.6
citadel-mta 8.24-1+b3
E: Package 'mail-transport-agent' has no installation candidate
Debian is setup to only allow one mail transport agent, and your install command is trying to include two, ssmtp and sendmail/sendmail-bin. Since they conflict with each other, you'll need to remove one of these from your install command.
If the sendmail dependency is so your Python app can send email via the sendmail binary, just install ssmtp and configure it to use an external MTA.
Trying to run sendmail in a Docker container is not recommended
I'm just following the getting started of docker and I'm a bit lost.
I've create a mydockerbuild folder in my document and a dockerfile with this code (from the tutorial)
FROM docker/whalesay:latest
RUN apt-get -y update && apt-get install -y fortunes
CMD /usr/games/fortune -a | cowsay
Then following the tutorial I run this command:
docker build -t docker-whale .
I got the following error:
Sending build context to Docker daemon 2.048 kB
Step 0 : FROM docker/whalesay:latest
---> fb434121fc77
Step 1 : RUN apt-get -y update && apt-get install -y fortunes
---> Running in dafe01cfcd2b
Err http://archive.ubuntu.com trusty InRelease
Err http://archive.ubuntu.com trusty-updates InRelease
Err http://archive.ubuntu.com trusty-security InRelease
Err http://archive.ubuntu.com trusty Release.gpg
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com trusty-updates Release.gpg
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com trusty-security Release.gpg
Could not resolve 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty/InRelease
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/InRelease
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/InRelease
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg Could not resolve 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg Could not resolve 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg Could not resolve 'archive.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
fortune-mod fortunes-min librecode0
Suggested packages:
x11-utils bsdmainutils
The following NEW packages will be installed:
fortune-mod fortunes fortunes-min librecode0
0 upgraded, 4 newly installed, 0 to remove and 3 not upgraded.
Need to get 1961 kB of archives.
After this operation, 4817 kB of additional disk space will be used.
Err http://archive.ubuntu.com/ubuntu/ trusty/main librecode0 amd64 3.6-21
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com/ubuntu/ trusty/universe fortune-mod amd64 1:1.99.1-7
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com/ubuntu/ trusty/universe fortunes-min all 1:1.99.1-7
Could not resolve 'archive.ubuntu.com'
Err http://archive.ubuntu.com/ubuntu/ trusty/universe fortunes all 1:1.99.1-7
Could not resolve 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/r/recode/librecode0_3.6-21_amd64.deb Could not resolve 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/f/fortune-mod/fortune-mod_1.99.1-7_amd64.deb Could not resolve 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/f/fortune-mod/fortunes-min_1.99.1-7_all.deb Could not resolve 'archive.ubuntu.com'
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/f/fortune-mod/fortunes_1.99.1-7_all.deb Could not resolve 'archive.ubuntu.com'
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get -y update && apt-get install -y fortunes' returned a non-zero code: 100
I think the error come from apt-get update but when I do it on my computer I got no problem.
I'm running ubuntu 14.04
you might try to adjust you update command as per the error message:
Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
So I found the problem, the error is a connectivity issue within the container.
I have found other topic about this problem for people not using a personnal wifi network.
So I just run the command using 3G from my phone and it works.