I was trying to install zeppelin as shown in this dockerfile, but with a different version of spark and hadoop (see this dockerfile in my repo for more details). However, I am getting the following error log when running apt-get update:
Step 5/10 : RUN apt-get update
---> Running in 1ee84513acc7
Ign http://ftp.debian.org jessie-backports InRelease
Ign http://ftp.debian.org jessie-backports Release.gpg
Ign http://deb.debian.org jessie InRelease
Ign http://ftp.debian.org jessie-backports Release
Get:1 http://security.debian.org jessie/updates InRelease [44.9 kB]
Err http://ftp.debian.org jessie-backports/main amd64 Packages
Get:2 http://deb.debian.org jessie-updates InRelease [16.3 kB]
Err http://ftp.debian.org jessie-backports/main amd64 Packages
Err http://ftp.debian.org jessie-backports/main amd64 Packages
Get:3 http://deb.debian.org jessie Release.gpg [1652 B]
Err http://ftp.debian.org jessie-backports/main amd64 Packages
Get:4 http://deb.debian.org jessie-updates/main amd64 Packages [20 B]
Get:5 http://security.debian.org jessie/updates/main amd64 Packages [992 kB]
Err http://ftp.debian.org jessie-backports/main amd64 Packages
404 Not Found
Get:6 http://deb.debian.org jessie Release [77.3 kB]
Get:7 http://deb.debian.org jessie/main amd64 Packages [9098 kB]
W: Size of file /var/lib/apt/lists/deb.debian.org_debian_dists_jessie-updates_main_binary-amd64_Packages.gz is not what the server reported 20 23046
W: Failed to fetch http://ftp.debian.org/debian/dists/jessie-backports/main/binary-amd64/Packages 404 Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
Fetched 10.2 MB in 1s (7375 kB/s)
The command '/bin/sh -c apt-get update' returned a non-zero code: 100
ERROR: Service 'zeppelin' failed to build
It seems like this is somehow related to the jessie package. I read this and this article and I tried including
RUN echo "deb [check-valid-until=no] http://cdn-fastly.deb.debian.org/debian jessie main" > /etc/apt/sources.list.d/jessie.list
RUN echo "deb [check-valid-until=no] http://archive.debian.org/debian jessie-backports main" > /etc/apt/sources.list.d/jessie-backports.list
RUN sed -i '/deb http:\/\/deb.debian.org\/debian jessie-updates main/d' /etc/apt/sources.list
RUN apt-get -o Acquire::Check-Valid-Until=false update
script before the apt-get update command in the dockerfile of my repo, but still without success.
Related
i am trying to build a docker from the tensorflow dockerhub, using tensorflow version 2.4.3, i did a simple pull and docker file:
$ docker pull tensorflow/tensorflow:2.4.2-gpu
And here is the dockerfile:
FROM tensorflow/tensorflow:2.4.3-gpu
ADD . /path
WORKDIR /path
RUN apt-get -y update
RUN apt-get -y install sudo
However i get errors on the apt-get update line:
$ docker build -t tensorflow_gpu_build .
...
Step 4/10 : RUN apt-get -y update
---> Running in 7c67f26124cb
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease [1581 B]
Hit:3 http://archive.ubuntu.com/ubuntu bionic InRelease
Get:4 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1573 kB]
Get:6 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB]
Err:2 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease
The following signatures couldn''t be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [3145 kB]
Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1371 kB]
Ign:9 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 InRelease
Hit:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1804/x86_64 Release
Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1412 kB]
Get:12 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [22.9 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [2351 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3568 kB]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [30.8 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [64.0 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [20.5 kB]
Reading package lists...
W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease: The following signatures couldn''t be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC
E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64 InRelease' is no longer signed.
I am unsure of where the problem is coming from, the same dockerfile worked fine on the latest tensorfow docker version, could it be normal behaviour because the tensorflow version is a little old?
It's possible that the key included in the image has been revoked by Nvidia in the meantime.
So, you have three possibilities:
You can check for a valid pub key for that repository, copy it to the image via Dockerfile and add it with this command:
apt-key add /path/to/key
You can add an "[ allow-insecure=yes ]" to the respective line in the /etc/apt/sources.list as elaborated here: https://unix.stackexchange.com/questions/198000/bypass-gpg-signature-checks-only-for-a-single-repository
You could try a newer image.
I'm following these instructions to install Docker: https://docs.docker.com/engine/install/debian/
I've successfully created /etc/apt/sources.list.d/docker.list:
deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian stretch stable
However, when I run sudo apt-get update, I get this error about not being able to fetch https://download.docker.com/linux/debian/dists/stretch/InRelease:
Hit:1 http://security.debian.org/debian-security stretch/updates InRelease
Ign:4 http://debian.mirror.iweb.ca/debian stretch InRelease
Hit:5 http://ftp.debian.org/debian stretch-backports InRelease
Hit:6 http://deb.debian.org/debian oldstable InRelease
Hit:7 http://debian.mirror.iweb.ca/debian stretch-updates InRelease
Hit:8 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu xenial InRelease
Hit:9 http://debian.mirror.iweb.ca/debian stretch Release
Ign:11 http://archive.debian.org/debian jessie InRelease
Hit:12 http://archive.debian.org/debian jessie Release
Reading package lists... Done
E: The method driver /usr/lib/apt/methods/https could not be found.
N: Is the package apt-transport-https installed?
E: Failed to fetch https://download.docker.com/linux/debian/dists/stretch/InRelease
W: Some index files failed to download. They have been ignored, or old ones used instead.
Has anyone run into this issue?
I'm tring to build an image from a Dockerfile, and get stuck when "RUN apt-get install ...", i want to know the reason. May be due to GFW, I don't know.
Another thing confused me is that the output log printed when run in build is different from the log printed when run in a container(i.e. run in "docker container run -it image_name /bin/bash").
Please help me.
I have tried some keywords like "apt-get install stuck", but i didn't find a similar question.
The Dockerfile:
FROM nvidia/cuda:9.0-devel-ubuntu16.04
MAINTAINER avanetten
ENV CUDNN_VERSION 7.3.0.29
LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"
RUN apt-get update && apt-get install -y --no-install-recommends \
apt-utils \
libcudnn7=$CUDNN_VERSION-1+cuda9.0 \
libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && \
apt-mark hold libcudnn7 && \
rm -rf /var/lib/apt/lists/*
I run this:
sudo docker build --no-cache -t testimage .
and i stuck here
Sending build context to Docker daemon 2.043GB
Step 1/37 : FROM nvidia/cuda:9.0-devel-ubuntu16.04
---> 2a64416134d8
Step 2/37 : MAINTAINER avanetten
---> Running in 0a0f4e3bb485
Removing intermediate container 0a0f4e3bb485
---> 23c2cb4ab1ff
Step 3/37 : ENV CUDNN_VERSION 7.3.0.29
---> Running in 41b6be7dc176
Removing intermediate container 41b6be7dc176
---> 65f16199d658
Step 4/37 : LABEL com.nvidia.cudnn.version="${CUDNN_VERSION}"
---> Running in 0f5e7aece401
Removing intermediate container 0f5e7aece401
---> c41f04004970
Step 5/37 : RUN apt-get update && apt-get install -y --no-install-recommends apt-utils libcudnn7=$CUDNN_VERSION-1+cuda9.0 libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && apt-mark hold libcudnn7 && rm -rf /var/lib/apt/lists/*
---> Running in 3a67681d2578
Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:2 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
Ign:3 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 InRelease
Ign:4 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 InRelease
Get:5 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Release [170 B]
Get:6 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Release [170 B]
Get:7 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [940 kB]
Get:8 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Release.gpg [170 B]
Get:9 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Get:10 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Release.gpg [170 B]
Get:11 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Packages [231 kB]
Get:12 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Packages [54.1 kB]
Get:13 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:14 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [12.7 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1558 kB]
Get:16 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [582 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]
Get:18 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [6114 B]
Get:19 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9827 kB]
Get:20 http://archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [176 kB]
Get:21 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [1322 kB]
Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [13.1 kB]
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [986 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [19.1 kB]
Get:25 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7942 B]
Get:26 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8807 B]
Fetched 16.3 MB in 28s (570 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
libapt-inst2.0
The following NEW packages will be installed:
apt-utils libapt-inst2.0 libcudnn7 libcudnn7-dev
0 upgraded, 4 newly installed, 0 to remove and 13 not upgraded.
Need to get 242 MB of archives.
After this operation, 598 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libapt-inst2.0 amd64 1.2.32 [55.8 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-utils amd64 1.2.32 [196 kB]
Get:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 libcudnn7 7.3.0.29-1+cuda9.0 [126 MB]
^C
I can not see progress or rate from the log, then I delete the RUN part and finished image build and tried
docker container run --rm -it testimage /bin/bash
, and when the container is running, I run this:
apt-get update && apt-get install -y --no-install-recommends \
apt-utils \
libcudnn7=$CUDNN_VERSION-1+cuda9.0 \
libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && \
apt-mark hold libcudnn7 && \
rm -rf /var/lib/apt/lists/*
, it works,the log is
root#74ce912f6318:/# apt-get update && apt-get install -y --no-install-recommends \
> apt-utils \
> libcudnn7=$CUDNN_VERSION-1+cuda9.0 \
> libcudnn7-dev=$CUDNN_VERSION-1+cuda9.0 && \
> apt-mark hold libcudnn7 && \
> rm -rf /var/lib/apt/lists/*
Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [109 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]
Get:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [109 kB]
Ign:4 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 InRelease
Ign:5 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 InRelease
Get:6 http://archive.ubuntu.com/ubuntu xenial-backports InRelease [107 kB]
Get:7 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Release [170 B]
Get:8 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1558 kB]
Get:9 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Release [170 B]
Get:10 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Release.gpg [170 B]
Get:11 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Release.gpg [170 B]
Get:12 https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 Packages [231 kB]
Get:13 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]
Get:14 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 Packages [54.1 kB]
Get:15 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9827 kB]
Get:16 http://archive.ubuntu.com/ubuntu xenial/multiverse amd64 Packages [176 kB]
Get:17 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [1322 kB]
Get:18 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages [940 kB]
Get:19 http://security.ubuntu.com/ubuntu xenial-security/restricted amd64 Packages [12.7 kB]
Get:20 http://security.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [582 kB]
Get:21 http://security.ubuntu.com/ubuntu xenial-security/multiverse amd64 Packages [6114 B]
Ign:22 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages
Get:23 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [986 kB]
Get:24 http://archive.ubuntu.com/ubuntu xenial-updates/multiverse amd64 Packages [19.1 kB]
Get:25 http://archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages [7942 B]
Get:26 http://archive.ubuntu.com/ubuntu xenial-backports/universe amd64 Packages [8807 B]
Get:22 http://archive.ubuntu.com/ubuntu xenial-updates/restricted amd64 Packages [13.1 kB]
Fetched 16.3 MB in 39s (409 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libapt-inst2.0
The following NEW packages will be installed:
apt-utils libapt-inst2.0 libcudnn7 libcudnn7-dev
0 upgraded, 4 newly installed, 0 to remove and 13 not upgraded.
Need to get 242 MB of archives.
After this operation, 598 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 libapt-inst2.0 amd64 1.2.32 [55.8 kB]
Get:2 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 apt-utils amd64 1.2.32 [196 kB]
Get:3 https://developer.download.nvidia.com/compute/machine-learning/repos/ubuntu1604/x86_64 libcudnn7 7.3.0.29-1+cuda9.0 [126 MB]
31% [3 libcudnn7 63.8 MB/170 B 37523776%] 8995 kB/s 19s^C
This time, I can see the progress(i.e. 31%), I stop it myself with ctrl-c.
I have two expectations, first is to make the Dockerfile work, second is to see the progress. Thanks.
I want to build a docker image based on php:5.6.32-apache.
The content involved in this within my Dockerfile is:
FROM php:5.6.32-apache
....
RUN apt-get update -y
The console output from the built process is:
Step 6/10 : RUN apt-get update -y
---> Running in 8781a7a005ad
Get:1 http://security.debian.org jessie/updates InRelease [44.9 kB]
Ign http://deb.debian.org jessie InRelease
Get:2 http://deb.debian.org jessie-updates InRelease [7340 B]
Get:3 http://deb.debian.org jessie Release.gpg [2420 B]
Get:4 http://deb.debian.org jessie Release [148 kB]
Get:5 http://security.debian.org jessie/updates/main amd64 Packages [846 kB]
Get:6 http://deb.debian.org jessie/main amd64 Packages [9098 kB]
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/InRelease Unable to find expected entry 'main/binary-amd64/Packages' in Release file (Wrong sources.list entry or malformed file)
E: Some index files failed to download. They have been ignored, or old ones used instead.
Fetched 10.1 MB in 17s (565 kB/s)
ERROR: Service 'php' failed to build: The command '/bin/sh -c apt-get update -y' returned a non-zero code: 100
You can find the original php:5.6.32-apache Dockerfile here:
https://github.com/docker-library/php/blob/ec6b6e60b42c283d48775596f722f5250c1f5ca5/5.6/jessie/apache/Dockerfile
Duplicate. Please find solution at:
Build Error. Failed to fetch Build Error. Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages
When I run docker on local machine with following Dockerfile:
FROM python:3
WORKDIR /app
COPY . /app
RUN apt-get update && apt-get install -y libhunspell-1.3-0
RUN pip install -r requirements.txt
EXPOSE 9876
CMD ["python","flask_compose.py"]
It got libhunspell from following repository (jessie):
Get:1 http://security.debian.org jessie/updates InRelease [94.4 kB]
Ign http://deb.debian.org jessie InRelease
Get:2 http://deb.debian.org jessie-updates InRelease [145 kB]
Get:3 http://security.debian.org jessie/updates/main amd64 Packages [623 kB]
Get:4 http://deb.debian.org jessie Release.gpg [2434 B]
Get:5 http://deb.debian.org jessie Release [148 kB]
Get:6 http://deb.debian.org jessie-updates/main amd64 Packages [23.0 kB]
Get:7 http://deb.debian.org jessie/main amd64 Packages [9064 kB]
And everything is fine (image build and run successfully).
Unfortunately, when I run docker on gitlab repository with the same Dockerfile it calls update apt-get from another repository (stretch):
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2434 B]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [440 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [12.1 kB]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [9530 kB]
Due to this inconsistency gitlab CI/CD throws following exception:
E: Unable to locate package libhunspell-1.3-0
E: Couldn't find any package by glob 'libhunspell-1.3-0'
E: Couldn't find any package by regex 'libhunspell-1.3-0'
The command '/bin/sh -c apt-get install -y libhunspell-1.3-0' returned a non-zero code: 100
ERROR: Job failed: exit code 100
Does anyone knows how change my Dockerfile that local and gitlab uses the same package repository?
There current tag for python:3 points to a Debian stretch based image. To update your local environment, run a docker pull python:3 or run your build with the --pull option.
You can also select a more specific tag to force your desired base image used by python. See the docker hub page for python to see all the possible tags, e.g. python:3-jessie.
From gitlab official forum I found a solution which fixed my problem.
1) define a setup.sh file and following command:
echo "deb http://pkg.adfinis-sygroup.ch/debian/ jessie main non-free contrib" > /etc/apt/sources.list
echo "deb http://security.debian.org/ jessie/updates main" >> /etc/apt/sources.list
echo "deb http://pkg.adfinis-sygroup.ch/debian/ jessie-updates main contrib non-free" >> /etc/apt/sources.list
apt-get update
apt-get install --no-install-recommends -y libhunspell-1.3-0
2) In Dockerfile add following commands:
From python:3
ADD setup.sh /opt/
RUN /bin/bash /opt/setup.sh
PS: Although my hack works very fine, I prefer #BMitch solution.
--------------------------------------------------------------------------------------------------------
By extending #Bmitch solution I decrease image size from ~600 to ~150 with following Dockerfile:
FROM python:3-slim-jessie
WORKDIR .
# hunspell deps
RUN apt-get update && apt-get install --no-install-recommends -y libtool libc6-dev gcc g++ build-essential libhunspell-1.3-0
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . .
CMD ["python","flask_compose.py"]