I am trying to install some azxure tools in a docker container and following the instructions here:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-run-local?tabs=v3%2Clinux%2Ccsharp%2Cportal%2Cbash&view=form-recog-3.0.0
I setup my docker file as follows:
FROM --platform=linux/amd64 python:3.9-slim AS build
RUN apt-get update \
&& apt-get install --no-install-recommends -y \
build-essential \
curl \
gpg
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-$(lsb_release -cs)-prod $(lsb_release -cs) main"
RUN sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/debian/$(lsb_release -rs | cut -d'.' -f 1)/prod $(lsb_release -cs) main" > /etc/apt/sources.list.d/dotnetdev.list'
RUN apt-get update
Now on the last update statement, I am getting the error:
> [ 7/11] RUN apt-get update:
#10 0.835 E: Malformed entry 1 in list file /etc/apt/sources.list.d/dotnetdev.list (Component)
#10 0.835 E: The list of sources could not be read.
I am also not sure how to debug this. I would like to see the output of the generated list file but am unable to echo it during the build process.
EDIT
What was needed was a change in the base image. Changing it to:
FROM --platform=linux/amd64 python:3.9-slim-bullseye AS build
works. I think the older one was based on Alpine.
Related
I'm trying to install fuse to my docker image that's inside Google Kubernetes Engine.
Here is my Dockerfile:
FROM --platform=amd64 ubuntu:22.10
# Use baseimage-docker's init system.
CMD ["/sbin/my_init"]
# Install.
EXPOSE 80
RUN ls -la
RUN \
sed -i 's/# \(.*multiverse$\)/\1/g' /etc/apt/sources.list && \
apt-get update && \
apt-get -y upgrade && \
apt-get install -y build-essential && \
apt-get install -y gcc && \
apt-get install -y software-properties-common && \
apt-get install -y cmake && \
apt-get install -y make && \
apt-get install -y clang && \
apt-get install -y mesa-common-dev && \
apt-get install -y git && \
apt-get install -y xorg-dev && \
apt-get install -y nasm && \
apt-get install -y byobu curl git htop man unzip vim wget && \
rm -rf /var/lib/apt/lists/*
# RUN apt-get install -y gobjc++
#RUN apt-get install -y gnupg lsb-release wget
#RUN export DOCKER_DEFAULT_PLATFORM=linux/amd64
#RUN lsb_release -c -s > /tmp/lsb_release
#RUN GCSFUSE_REPO=$(cat /tmp/lsb_release); echo "deb http://packages.cloud.google.com/apt gcsfuse-$GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
#RUN wget -O - https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
RUN apt-get install --yes --no-install-recommends ca-certificates curl gpg gpg-agent
RUN echo "deb http://packages.cloud.google.com/apt gcsfuse-buster main" | tee /etc/apt/sources.list.d/gcsfuse.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
RUN apt-get install -y gcsfuse
But when i try to build this image i get this following error:
Step 7/23 : RUN echo "deb http://packages.cloud.google.com/apt gcsfuse-buster main" | tee /etc/apt/sources.list.d/gcsfuse.list
---> Running in c027599dc506
deb http://packages.cloud.google.com/apt gcsfuse-buster main
Removing intermediate container c027599dc506
---> 732f7fb73280
Step 8/23 : RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
---> Running in 47ad78e4351e
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
100 2537 100 2537 0 0 48417 0 --:--:-- --:--:-- --:--:-- 48788
OK
Removing intermediate container 47ad78e4351e
---> 90819264fc33
Step 9/23 : RUN apt-get install -y gcsfuse
---> Running in 8805b3fcaae8
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package gcsfuse
The command '/bin/sh -c apt-get install -y gcsfuse' returned a non-zero code: 100
I'm trying to run this on an ubuntu image inside my docker container to access the Google Cloud Storage inside my container.
I just want to download gcsfuse inside my container, I've tried many things which all ended up with an error but this error seems the most plausible one so I'm asking this one, but if there is a better way to download fuse I could also try that.
I tried the solutions here to no avail:
Github Issue
Stackoverflow Question
I've also tried to implement These two instalations which resulted in different error messages.
Edit: When i try doing it like this:
RUN export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
RUN echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" \
| tee /etc/apt/sources.list.d/gcsfuse.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg \
| apt-key add -
RUN apt-get install -y gcsfuse
I get this error:
Step 5/30 : RUN export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
---> Running in 94ee52e0b35f
Removing intermediate container 94ee52e0b35f
---> fa5a33fd2305
Step 6/30 : RUN echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
---> Running in 785cfe4c4d4c
deb http://packages.cloud.google.com/apt main
Removing intermediate container 785cfe4c4d4c
---> f4aaed9a03ae
Step 7/30 : RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
---> Running in 8ccfdfab4681
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 2537 100 2537 0 0 24924 0 --:--:-- --:--:-- --:--:-- 25118
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
OK
Removing intermediate container 8ccfdfab4681
---> 9a856aa4bd1a
Step 8/30 : RUN apt-get install -y gcsfuse
---> Running in e4e0445ae72f
E: Malformed entry 1 in list file /etc/apt/sources.list.d/gcsfuse.list (Component)
E: The list of sources could not be read.
The command '/bin/sh -c apt-get install -y gcsfuse' returned a non-zero code: 100
The error is useful:
apt-get install -y gcsfuse ... returned a non-zero error code
You want 0 exit status which means the command succeeeded.
You can then confirm|debug this by trying to install in a container:
docker run \
--interactive --tty --rm \
ubuntu:22.10 \
apt install gcsfuse
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package gcsfuse
Googling gcsfuse provides documentation for install on Linux
You need to install gcsfuse's URL and its public key so that you can apt install it.
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" \
| sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg \
| sudo apt-key add -
Update
FROM --platform=amd64 ubuntu:20.04
RUN apt update && \
DEBIAN_FRONTEND=noninteractive \
TZ=Americas/Los_Angeles \
apt install -y curl lsb-core
RUN export GCSFUSE_REPO=gcsfuse-$(lsb_release -c -s) && \
echo ${GCSFUSE_REPO} && \
( echo "deb http://packages.cloud.google.com/apt ${GCSFUSE_REPO} main" \
| tee /etc/apt/sources.list.d/gcsfuse.list ) && \
more /etc/apt/sources.list.d/gcsfuse.list && \
( curl https://packages.cloud.google.com/apt/doc/apt-key.gpg \
| apt-key add - )
RUN apt update && apt -y install gcsfuse
ENTRYPOINT ["gcsfuse"]
CMD ["--help"]
If you use the following in your dockerfile, it will solve your issues with installing gcsfuse to your gke container :
ENV GCSFUSE_REPO gcsfuse-stretch
ENV GOOGLE_APPLICATIONS_CREDENTIALS=test-serviceaccount.json
ENV GCS_BUCKET: "my-bucket"
ENV GCS_BUCKET_FOLDER: "shared-data"
USER root
# Add google repositories for gcsfuse and google cloud sdk
RUN apt-get update -y && apt-get install -y --no-install-recommends apt-transport-https ca-certificates curl gnupg
RUN echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
RUN echo "deb https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
# Install gcsfuse and google cloud sdk
RUN apt-get update -y && apt-get install -y gcsfuse google-cloud-sdk \
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
I'm running a command that worked until yesterday, and works locally on my local Docker, to install gcsfuse version 0.28.1:
E: Version '0.28.1' for 'gcsfuse' was not found
I tried it on the google cloud console too, and got the same error there.
Any suggestions or pointers?
Here's the original command:
export GCSFUSE_REPO=gcsfuse-lsb_release -c -s
&& echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
&& curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
&& apt-get update && apt-get install -y gcsfuse=0.28.1 \
I tried the below commands on Google Cloud Shell as per this document reference and I was able to install GCSFuse successfully.
Add the gcsfuse distribution URL as a package source and import its public key using :
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s`
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Update the list of packages available and install gcsfuse using :
sudo apt-get update
sudo apt-get install gcsfuse
(Ubuntu before wily only) Add yourself to the fuse group, then log out and back in using :
sudo usermod -a -G fuse $USER
exit
I also found a line in this document which says “The following instructions set up apt-get to see updates to gcsfuse, and are supported for the bionic, artful, zesty, yakkety, xenial, and trusty releases of Ubuntu, and the jessie and stretch releases of Debian. (Run lsb_release -c to find your release codename.) Users of older releases should follow the instructions for other distributions below.
So you should also try and find the release codename first. If you are a user of an older release then these commands might not work for you. For that, follow the instructions/commands of installing older distributions, which is clearly differentiated and specified in this document link
I am not sure if it was your formatting but it looks like it was missing the release variable in your commands. Please try with the commands below:
export GCSFUSE_REPO=gcsfuse-`lsb_release -c -s` && \
echo "deb http://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list && \
curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
apt-get update -y && \
apt-get install -y gcsfuse=0.28.1 -V
If it does not work, here is the repo with all gcsfuse releases:
https://github.com/GoogleCloudPlatform/gcsfuse/releases/tag/v0.28.1
I get an error when I do docker-compose build.
I am using apple siilcon and I was able to build it with intel without any problems.
Does anyone know if something is wrong?
Contents of docker
RUN apt-get update && apt-get install -y unzip && \
CHROME_DRIVER_VERSION=`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE` && \
wget -N http://chromedriver.storage.googleapis.com/$CHROME_DRIVER_VERSION/chromedriver_linux64.zip -P ~/ && \
unzip ~/chromedriver_linux64.zip -d ~/ && \
rm ~/chromedriver_linux64.zip && \
chown root:root ~/chromedriver && \
chmod 755 ~/chromedriver && \
mv ~/chromedriver /usr/bin/chromedriver && \
sh -c 'wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -' && \
sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
RUN apt-get update
RUN apt-get install -y google-chrome-stable
Error
=> ERROR [ 5/15] RUN apt-get install -y google-chrome-stable 3.2s
------
> [ 5/15] RUN apt-get install -y google-chrome-stable:
#8 1.090 Reading package lists...
#8 2.038 Building dependency tree...
#8 2.154 Reading state information...
#8 2.214 E: Unable to locate package google-chrome-stable
You can add the platform flag to the FROM statement in your Dockerfile. This will ensure the docker container builds the correct architecture every time.
I used this for Debian Linux and Chrome headless:
FROM --platform=linux/amd64 python:3.9
While building a docker image, it's possible to set the custom apt mirror by overwriting the /etc/apt/sources.list, e.g.
FROM ubuntu:focal
RUN echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal main restricted universe multiverse" > /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal-updates main restricted universe multiverse" >> /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt focal-security main restricted universe multiverse" >> /etc/apt/sources.list
...
If the base image is a variable, e.g. FROM ${DISTRO}, the sources.list should be adjusted based on the ubuntu release.
I tried $(lsb_release -cs) like below:
RUN echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs) main restricted universe multiverse" > /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs)-updates main restricted universe multiverse" >> /etc/apt/sources.list && \
echo "deb mirror://mirrors.ubuntu.com/mirrors.txt $(lsb_release -cs)-security main restricted universe multiverse" >> /etc/apt/sources.list
But it says lsb_release: not found.
The workaround is to install the package before running it.
RUN apt-get update && apt-get install -y lsb-release
However, the install of lsb-release package could be very slow in some areas.
So the question is, is there a proper way to set the apt source mirror before using apt?
Just use sed to edit the existing sources.list files in-place. The following snippet will modify the sources file in-place without regards to the exact distribution (and is safe to run on other Debian-based distributions, too, but will not modify the files there).
It changes all references to http://archive.ubuntu.com/ubuntu to mirror://mirrors.ubuntu.com/mirrors.txt (which tells the built-in apt client to automatically use a local mirror when fetching from the network) in /etc/apt/sources.list
sed -i -e 's/http:\/\/archive\.ubuntu\.com\/ubuntu\//mirror:\/\/mirrors\.ubuntu\.com\/mirrors\.txt/' /etc/apt/sources.list
For use in a Dockerfile, simply RUN this - preferably as early in the Dockerfile as possible as so:
RUN sed -i -e 's/http:\/\/archive\.ubuntu\.com\/ubuntu\//mirror:\/\/mirrors\.ubuntu\.com\/mirrors\.txt/' /etc/apt/sources.list
The lsb-release package is not included in the minimal Ubuntu image, but you could make use of /etc/lsb-release or /etc/os-release file instead (the second one is in common use, refer to this answer for comparison).
For Dockerfile, just change $(lsb_release -cs) to $(. /etc/os-release && echo $VERSION_CODENAME), you won't waste time in updating and installing packages.
In the dockerfile reflect the .. /etc/os-release && echo $VERSION_CODENAME for the lsb-release
Whilst the particular base linux docker image we use currently escapes me, the problem I have is I need to include the Postgresql Client.
apt-get update allowed me to install it, but I discovered it was only version 9, and I needed 10 to match my Postgres DB.
If the docker is running and I used exec bash to access it I can run the following commands to install version 10...
echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
apt-get update
apt-get -y install postgresql-client-10
So I then set about adding these to my Dockerfile so it would be included automatically, the result being...
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN apt-get update
RUN apt-get -y install postgresql-client-10
When Visual Studio (Mac 2019) builds the docker, it completely ignores this. When I built it from the command line, the output suggested it had done as it should, what Docker Cloud builds it I get the error...
Step 17/24 : COPY --from=publish /app .
---> 091e71bd17c0
Step 18/24 : RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list
---> Running in 8582e2dba910
/bin/sh: can't create /etc/apt/sources.list.d/pgdg.list: nonexistent directory
Removing intermediate container 8582e2dba910
The command '/bin/sh -c echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' returned a non-zero code: 1
Note the 3rd line up about can't create pgdg.list
So how should I go about doing this?
Your approach with the additional Dockerfile statements is sound. However, it's unclear how you're extending your previous container image to include these.
Usually you'd have something of the form:
FROM my-original-image:its-version
RUN ...
RUN ...
RUN apt-get update
RUN apt-get -y install postgresql-client-10
This approach would add postgresql-client-10 to whatever's installed in my-original-image:its-version.
An alternative approach is to start from the base from which my-original-image:its-version uses and install postgresql-client-10 instead of the previous version. Your choice.
If my-original-image were created using:
FROM some-base
RUN apt update && apt install -y postgresql-client-9
You could:
FROM some-base
RUN apt update && apt install -y postgresql-client-10
NB You'd have to duplicate the installation of other dependencies too.
Update
See instructions for Dockerizing Postgres here. The Postgres PGP key has possibly been dropped from your image and is causing errors.
FROM debian:stretch-slim
RUN apt-get update && apt-get install -y \
gnupg2 \
wget \
&& apt-key adv \
--keyserver hkp://p80.pool.sks-keyservers.net:80 \
--recv-keys B97B0AFCAA1A47F044F244A07FCC7D46ACCC4CF8 \
&& echo "deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main" \
>> /etc/apt/sources.list.d/pgdg.list \
&& wget \
--quiet \
--output-document - \
https://www.postgresql.org/media/keys/ACCC4CF8.asc \
| apt-key add - \
&& apt-get update \
&& apt-get install -y \
postgresql-client-10 \
&& rm -rf /var/lib/apt/lists/*