While building a docker image, it's possible to set the custom apt mirror by overwriting the /etc/apt/sources.list, e.g.
FROM ubuntu:focal
RUN echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal main restricted universe multiverse" > /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal-updates main restricted universe multiverse" >> /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal-security main restricted universe multiverse" >> /etc/apt/sources.list
...
If the base image is a variable, e.g. FROM ${DISTRO}, the sources.list should be adjusted based on the ubuntu release.
I tried $(lsb_release -cs) like below:
RUN echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs) main restricted universe multiverse" > /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs)-updates main restricted universe multiverse" >> /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs)-security main restricted universe multiverse" >> /etc/apt/sources.list
But it says lsb_release: not found.
The workaround is to install the package before running it.
RUN apt-get update && apt-get install -y lsb-release
However, the install of lsb-release package could be very slow in some areas.
So the question is, is there a proper way to set the apt source mirror before using apt?
Just use sed to edit the existing sources.list files in-place. The following snippet will modify the sources file in-place without regards to the exact distribution (and is safe to run on other Debian-based distributions, too, but will not modify the files there).
It changes all references to http://archive.ubuntu.com/ubuntu to mirror://mirrors.ubuntu.com/mirrors.txt (which tells the built-in apt client to automatically use a local mirror when fetching from the network) in /etc/apt/sources.list
sed -i -e 's/http:\/\/archive\.ubuntu\.com\/ubuntu\//mirror:\/\/mirrors\.ubuntu\.com\/mirrors\.txt/' /etc/apt/sources.list
For use in a Dockerfile, simply RUN this - preferably as early in the Dockerfile as possible as so:
RUN sed -i -e 's/http:\/\/archive\.ubuntu\.com\/ubuntu\//mirror:\/\/mirrors\.ubuntu\.com\/mirrors\.txt/' /etc/apt/sources.list
The lsb-release package is not included in the minimal Ubuntu image, but you could make use of /etc/lsb-release or /etc/os-release file instead (the second one is in common use, refer to this answer for comparison).
For Dockerfile, just change $(lsb_release -cs) to $(. /etc/os-release && echo $VERSION_CODENAME), you won't waste time in updating and installing packages.
In the dockerfile reflect the .. /etc/os-release && echo $VERSION_CODENAME for the lsb-release
Related
I am trying to install some azxure tools in a docker container and following the instructions here:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=v3%2Clinux%2Ccsharp%2Cportal%2Cbash&view=form-recog-3.0.0
I setup my docker file as follows:
FROM --platform=linux/amd64 python:3.9-slim AS build
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
build-essential \
curl \
gpg
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-$(lsb_release -cs)-prod $(lsb_release -cs) main"
RUN sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/debian/$(lsb_release -rs | cut -d'.' -f 1)/prod $(lsb_release -cs) main" > /etc/apt/sources.list.d/dotnetdev.list'
RUN apt-get update
Now on the last update statement, I am getting the error:
> [ 7/11] RUN apt-get update:
#10 0.835 E: Malformed entry 1 in list file /etc/apt/sources.list.d/dotnetdev.list (Component)
#10 0.835 E: The list of sources could not be read.
I am also not sure how to debug this. I would like to see the output of the generated list file but am unable to echo it during the build process.
EDIT
What was needed was a change in the base image. Changing it to:
FROM --platform=linux/amd64 python:3.9-slim-bullseye AS build
works. I think the older one was based on Alpine.
I'm running a command that worked until yesterday, and works locally on my local Docker, to install gcsfuse version 0.28.1:
E: Version '0.28.1' for 'gcsfuse' was not found
I tried it on the google cloud console too, and got the same error there.
Any suggestions or pointers?
Here's the original command:
export GCSFUSE_REPO=gcsfuse-lsb_release -c -s
&& echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
&& apt-get update && apt-get install -y gcsfuse=0.28.1 \
I tried the below commands on Google Cloud Shell as per this document reference and I was able to install GCSFuse successfully.
Add the gcsfuse distribution URL as a package source and import its public key using :
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Update the list of packages available and install gcsfuse using :
sudo apt-get update
sudo apt-get install gcsfuse
(Ubuntu before wily only) Add yourself to the fuse group, then log out and back in using :
sudo usermod -a -G fuse $USER
exit
I also found a line in this document which says “The following instructions set up apt-get to see updates to gcsfuse, and are supported for the bionic, artful, zesty, yakkety, xenial, and trusty releases of Ubuntu, and the jessie and stretch releases of Debian. (Run lsb_release -c to find your release codename.) Users of older releases should follow the instructions for other distributions below.
So you should also try and find the release codename first. If you are a user of an older release then these commands might not work for you. For that, follow the instructions/commands of installing older distributions, which is clearly differentiated and specified in this document link
I am not sure if it was your formatting but it looks like it was missing the release variable in your commands. Please try with the commands below:
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s` && \
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update -y && \
apt-get install -y gcsfuse=0.28.1 -V
If it does not work, here is the repo with all gcsfuse releases:
https://github.com/GoogleCloudPlatform/gcsfuse/releases/tag/v0.28.1
I tried to install Docker on Kubuntu 19.10.
I used these steps to install: https://docs.docker.com/install/linux/docker-ce/ubuntu/
During step 2, after added repository and key I got this message
Package containerd.io is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'containerd.io' has no installation candidate
It's not the cleanest way to do it I guess, but I had the same problem as you and I changed:
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
to
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
bionic \
stable"
because
$(lsb_release -cs)
returns "eoan" (the name of the latest ubuntu version, currently 19.10) and it seems to cause problems.
Replacing this with "bionic" (ubuntu 18.10) worked perfectly for me, and I still have no problems 4 months later.
Don't forget to open "/etc/apt/sources.list" and comment-out/delete the line:
deb-src [arch=amd64] https://download.docker.com/linux/ubuntu eoan stable
to avoid conflicts.
(You added this with the first command you entered in this process.)
Edit the file : /etc/apt/sources.list
Look for repo entry :
deb-src [arch=amd64] https://download.docker.com/linux/ubuntu eoan stable
Changing it to:
deb-src [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable
resolved the issue for me
The official docker documentation recommend to use
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
for Ubuntu 19.10 which seems that doesn't contain 'containerd.io' package yet.
Remove the previously added repository using add-apt-repository --remove :
$ sudo add-apt-repository --remove \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
and add the one used for bionic (18.10) Ubuntu release:
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
bionic \
stable"
After this, update and install docker:
$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io
If the from #vavalm did not work either, check that you have installed ubuntu amd64. There is a chance that you have installed the Ubuntu version i384 instead
Whilst the particular base linux docker image we use currently escapes me, the problem I have is I need to include the Postgresql Client.
apt-get update allowed me to install it, but I discovered it was only version 9, and I needed 10 to match my Postgres DB.
If the docker is running and I used exec bash to access it I can run the following commands to install version 10...
echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get -y install postgresql-client-10
So I then set about adding these to my Dockerfile so it would be included automatically, the result being...
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN apt-get update
RUN apt-get -y install postgresql-client-10
When Visual Studio (Mac 2019) builds the docker, it completely ignores this. When I built it from the command line, the output suggested it had done as it should, what Docker Cloud builds it I get the error...
Step 17/24 : COPY --from=publish /app .
---> 091e71bd17c0
Step 18/24 : RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
---> Running in 8582e2dba910
/bin/sh: can't create /etc/apt/sources.list.d/pgdg.list: nonexistent directory
Removing intermediate container 8582e2dba910
The command '/bin/sh -c echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' returned a non-zero code: 1
Note the 3rd line up about can't create pgdg.list
So how should I go about doing this?
Your approach with the additional Dockerfile statements is sound. However, it's unclear how you're extending your previous container image to include these.
Usually you'd have something of the form:
FROM my-original-image:its-version
RUN ...
RUN ...
RUN apt-get update
RUN apt-get -y install postgresql-client-10
This approach would add postgresql-client-10 to whatever's installed in my-original-image:its-version.
An alternative approach is to start from the base from which my-original-image:its-version uses and install postgresql-client-10 instead of the previous version. Your choice.
If my-original-image were created using:
FROM some-base
RUN apt update && apt install -y postgresql-client-9
You could:
FROM some-base
RUN apt update && apt install -y postgresql-client-10
NB You'd have to duplicate the installation of other dependencies too.
Update
See instructions for Dockerizing Postgres here. The Postgres PGP key has possibly been dropped from your image and is causing errors.
FROM debian:stretch-slim
RUN apt-get update && apt-get install -y \
gnupg2 \
wget \
&& apt-key adv \
--keyserver hkp://p80.pool.sks-keyservers.net:80 \
--recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8 \
&& echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" \
>> /etc/apt/sources.list.d/pgdg.list \
&& wget \
--quiet \
--output-document - \
https://www.postgresql.org/media/keys/ACCC4CF8.asc \
| apt-key add - \
&& apt-get update \
&& apt-get install -y \
postgresql-client-10 \
&& rm -rf /var/lib/apt/lists/*
I am building an image from a Dockerfile that needs to install the package rti-connext-dds-5.3.1. (It's one of the dependencies when building ROS2 on Linux).
The problem with that package is that it displays a license agreement that must be scrolled-down and then accepted by entering "yes" on the prompt. I cannot seem to set up the Dockerfile commands to auto-scroll and/or auto-accept this license agreement:
Pressing the Enter or Space key does not scroll the license down, it just displays blank lines. Pressing any other key/s just prints it out to the console. At this point, the build is stuck, and it can't proceed.
Here is the Dockerfile:
FROM ubuntu:bionic
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y apt-utils debconf-utils gnupg2 lsb-release && \
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 421C365BD9FF1F717815A3895523BAEEB01FA116 && \
echo "deb http://packages.ros.org/ros2/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros2-latest.list && \
apt-get update && \
apt-get install -y rti-connext-dds-5.3.1
WORKDIR /home
I already tried:
Setting DEBIAN_FRONTEND=noninteractive different ways based on the answers from Is it possible to answer dialog questions when installing under docker?
Setting DEBIAN_FRONTEND=teletype based on How to accept license agreement during docker build?
Using debconf-set-selections based on apt-get install without debconf prompt
# echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections && \
# echo "rti-connext-dds-5.3.1 rti-connext-dds-5.3.1/license string y" | debconf-set-selections && \
# echo "rti-connext-dds-5.3.1 rti-connext-dds-5.3.1/license string yes" | debconf-set-selections && \
Piping the yes command (this is worse, as I can't even abort with Ctrl+C)
# apt-get install -y -q rti-connext-dds-5.3.1
# yes "yes" | apt-get install -y -q rti-connext-dds-5.3.1
How do I auto-scroll and/or auto-accept the license during installation?
You can use the env variable "RTI_NC_LICENSE_ACCEPTED=yes". Your dockerfile will look something like this:
FROM ubuntu:bionic
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y apt-utils debconf-utils gnupg2 lsb-release && \
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 421C365BD9FF1F717815A3895523BAEEB01FA116 && \
echo "deb http://packages.ros.org/ros2/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros2-latest.list && \
apt-get update
RUN RTI_NC_LICENSE_ACCEPTED=yes apt-get install rti-connext-dds-5.3.1
WORKDIR /home
This is a valid answer but def not the best one. I'm currently using the trial version of DDS which does NOT allow you to automatically accept the license. So my work around was to do the following:
First I installed the expect command which allows you to write scripts that will interact and say hit enter 21 times and then the letter y etc.
Additionally there is a command called autoexpect that will generate a script for you. So my steps are:
Launch a basic container and copy over rti_connext_dds-6.0.1-eval-x64Linux4gcc7.3.0.run - or mount the file locally
Run autoexpect ./rti_connext_dds-6.0.1-eval-x64Linux4gcc7.3.0.run which will generate script1.exp.
Now build an actual container that copies over the install file as well as the expect script and do the following:
ARG RTI_INSTALL_FILE=rti_connext_dds-6.0.1-eval-x64Linux4gcc7.3.0.run
RUN chmod +x /rti/${RTI_INSTALL_FILE} && expect /rti/script.exp
The best path - of course - is to do what was suggested above - however - to know more about docker expect is a great little hack