Cannot retrieve latest version of Debian package inside Docker container - docker

Inside a docker container (created from a node:9 image) I am trying to install stress-ng package using apt-get. However, for some reason an old version of the package is retrieved everytime I try to install it. For installing the package I use the commands:
root#7e7a683bf288:/usr/src/app# apt-get update
root#7e7a683bf288:/usr/src/app# apt-get install stress-ng
I get the following version:
root#7e7a683bf288:/usr/src/app# stress-ng --version
stress-ng, version 0.01.32
However, I would like to get the latest version if possible, 0.09.42-1 (https://packages.ubuntu.com/cosmic/stress-ng). I have tried some of the solutions in similar questions but was not able to get this to work.
Additional info:
root#7e7a683bf288:/usr/src/app# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root#7e7a683bf288:/usr/src/app# cat /etc/apt/sources.list
deb http://deb.debian.org/debian jessie main
deb http://security.debian.org/debian-security jessie/updates main
deb http://deb.debian.org/debian jessie-updates main
root#7e7a683bf288:/usr/src/app# add-apt-repository
bash: add-apt-repository: command not found

You have to add the unstable repo to your sources.list. When I did that, I still couldn't install stress-ng as it stated:
root#096865e3637f:/# apt-get install stress-ng
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libc6-dev : Breaks: binutils (< 2.26) but 2.25-5+deb8u1 is to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.
So before installing I had to remove binutils.
Maybe that is an option for you.
The complete Dockerfile looks like:
FROM node:9
RUN echo "deb http://http.us.debian.org/debian unstable main non-free contrib" >> /etc/apt/sources.list && \
echo "deb-src http://http.us.debian.org/debian unstable main non-free contrib" >> /etc/apt/sources.list && \
apt-get remove binutils -y && \
apt-get update && \
apt-get install stress-ng -y
CMD stress-ng --version
stress-ng --version:
stress-ng, version 0.09.50 💻🔥
So, it is not 0.09.42, but the latest (unstable) version - as requested.

Related

How do I build an Ubuntu/Ruby image that also contains foreman? [duplicate]

My team uses Docker (with ubuntu:14.04 base image) for local development and we often have to rebuild some or all of our images. But we often get failures downloading packages with apt-get install, even immediately after running apt-get -y update. For instance, today I see
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2 amd64 2.9.1+dfsg1-3ubuntu4.7
404 Not Found [IP: 91.189.88.161 80]
Err http://archive.ubuntu.com/ubuntu/ trusty-security/main libxml2-dev amd64 2.9.1+dfsg1-3ubuntu4.7
404 Not Found [IP: 91.189.88.161 80]
Fetched 84.7 MB in 1min 6s (1281 kB/s)
Unable to correct missing packages.
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb 404 Not Found [IP: 91.189.88.161 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/libx/libxml2/libxml2-dev_2.9.1+dfsg1-3ubuntu4.7_amd64.deb 404 Not Found [IP: 91.189.88.161 80]
E: Aborting install.
Apparently the specific version of a particular package has been deleted from the archive and replaced with a slightly differently named patch version. For instance, the above error is looking for libxml2_2.9.1+dfsg1-3ubuntu4.7_amd64.deb but the version on the server is libxml2_2.9.1+dfsg1-3ubuntu4.8_amd64.deb.
Often this is solvable by removing the base image (docker rmi ubuntu:14.04) and rebuilding; the newly downloaded ubuntu image has the correct patch number and finds the right archive file. But even this doesn't always work -- probably due to a delay between a new minor upgrade to Ubuntu's dependency db and the deployment of that new ubuntu:14.04 image onto Docker Hub.
We've tried using apt-get flags --fix-missing and --fix-broken and those don't consistently work either.
Any other ideas?
apt-get install fails with Not Found error because package removed from repository is a similar problem but the accepted answer is unacceptable because it's not possible to be automated. Our daily development process, including automatic build and deploy, is all scripted and using Docker and it's not practical to hack around inside a Dockerfile every time a particular archive goes missing (then remove the hack after a few hours or days).
In response to #prateek05, here's the /etc/apt/sources.list from the official ubuntu:14.04 docker image:
root#72daa1942714:/# cat /etc/apt/sources.list
# See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
# newer versions of the distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty main restricted
## Major bug fix updates produced after the final release of the
## distribution.
deb http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates main restricted
## Uncomment the following two lines to add software from the 'universe'
## repository.
## N.B. software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu
## team. Also, please note that software in universe WILL NOT receive any
## review or updates from the Ubuntu security team.
deb http://archive.ubuntu.com/ubuntu/ trusty universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty universe
deb http://archive.ubuntu.com/ubuntu/ trusty-updates universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-updates universe
## N.B. software from this repository may not have been tested as
## extensively as that contained in the main release, although it includes
## newer versions of some applications which may provide useful features.
## Also, please note that software in backports WILL NOT receive any review
## or updates from the Ubuntu security team.
# deb http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-backports main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security main restricted
deb http://archive.ubuntu.com/ubuntu/ trusty-security universe
deb-src http://archive.ubuntu.com/ubuntu/ trusty-security universe
# deb http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
# deb-src http://archive.ubuntu.com/ubuntu/ trusty-security multiverse
You have stated that your Dockerfile contains RUN apt-get -y update as its own RUN instruction. However, due to build caching, if all changes to the Dockerfile occur later in the file, when docker build is run, Docker will reuse the intermediate image created the last time RUN apt-get -y update executed instead of running the command again, and so any recently-added or -edited apt-get install lines will be using old data, leading to the errors you've observed.
There are two ways to fix this:
Pass the --no-cache option to docker build, forcing every statement in the Dockerfile to be run every time the image is built.
Rewrite the Dockerfile to combine the apt-get commands in a single RUN instruction: RUN apt-get update && apt-get install foo bar .... This way, whenever the list of packages to install is edited, docker build will be forced to re-execute the entire RUN instruction and thus rerun apt-get update before installing.
The Dockerfile best practices page actually has an entire section on apt-get commands in Dockerfiles. I suggest you read it.
The issue could be potentially with the ubuntu sources
Check /etc/apt/sources.list
If you see deb http://archive.ubuntu.com/ubuntu main universe restricted multiverse , that could be the potential issue.
Fix that by replacing it with deb http://archive.ubuntu.com/ubuntu/ trusty main universe restricted multiverse
Alternatively it could be the mirror itself doesn't respond. archive.ubuntu.com
Name: archive.ubuntu.com
Address: 91.189.88.152
Name: archive.ubuntu.com
Address: 91.189.88.161
Name: archive.ubuntu.com
Address: 91.189.88.149
Replace archive.ubuntu.com with a more trusted mirror say us.archive.ubuntu.com
Name: us.archive.ubuntu.com
Address: 91.189.91.23
Name: us.archive.ubuntu.com
Address: 91.189.91.26
(edit by the oiriginal asker):
Thanks, prateek05! My Dockerfile now starts:
FROM ubuntu:14.04
RUN sed -i'' 's/archive\.ubuntu\.com/us\.archive\.ubuntu\.com/' /etc/apt/sources.list
RUN apt-get -y update
and it seems to be working. But since this is a sporadic issue, only time will tell...
In my case the problem was caused by the parent image since it had not cleared the apt cache properly.
I solve the problem including cleaning commands before the first apt update ...
RUN apt clean && \
rm -rf /var/lib/apt/lists/* && \
apt update && \
...
Hope this helps
Found something that works!
So I found this:
https://www.mail-archive.com/ubuntu-bugs#lists.ubuntu.com/msg5682159.html
The solution is to create a pinning_file with the contents
# Written by ubuntu-advantage-tools
Package: *
Pin: release o=UbuntuESM, n=trusty
Pin-Priority: never
Then add
COPY pinning_file /etc/apt/preferences.d/ubuntu-esm-infra-trusty
To you Dockerfile before you run the sudo apt-get -y update
I was able to fix this error only after adding some extra arguments to apt-get to deal with http issues:
sudo apt-get \
-o Acquire::BrokenProxy="true" \
-o Acquire::http::No-Cache="true" \
-o Acquire::http::Pipeline-Depth="0" install \
ignition
What worked for me ...
I had the statements in two lines, but when i merged to one it worked (don't know if it was cache related ...)
it was
RUN apt-get -y update && apt-get upgrade -y
# Install tools && libraries
RUN apt-get -y install --fix-missing apt-utils iputils-ping nano wget dialog \
build-essential git zip \
And then i changed to
RUN apt-get -y update && apt-get upgrade -y \
&& apt-get -y install --fix-missing apt-utils nano wget \
git zip \
mysql-client \
Using FTP sources works 100% of the time.
RUN echo \
'deb ftp://ftp.us.debian.org/debian/ jessie main\n \
deb ftp://ftp.us.debian.org/debian/ jessie-updates main\n \
deb http://security.debian.org jessie/updates main\n' \
> /etc/apt/sources.list

GPG error in Ubuntu 21.04 after second apt-get update during Docker build

Getting error while building the following Docker file
FROM ubuntu:21.04
RUN apt-get update && \
apt-get install --no-install-recommends -y curl=7.\* && \
apt-get install --no-install-recommends -y unzip=6.\* &&\
rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
mkdir -p /usr/share/man/man1 && \
apt-get install --no-install-recommends -y maven=3.6.3-5 && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
The error occurs when the second apt-get update runs.
The error is as follows :-
E: The repository 'http://security.ubuntu.com/ubuntu hirsute-security InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu hirsute InRelease: gpgv, gpgv2 or gpgv1 required for verification, but neither seems installed
E: The repository 'http://archive.ubuntu.com/ubuntu hirsute InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu hirsute-updates InRelease: gpgv, gpgv2 or gpgv1 required for verification, but neither seems installed
E: The repository 'http://archive.ubuntu.com/ubuntu hirsute-updates InRelease' is not signed.
W: GPG error: http://archive.ubuntu.com/ubuntu hirsute-backports InRelease: gpgv, gpgv2 or gpgv1 required for verification, but neither seems installed
E: The repository 'http://archive.ubuntu.com/ubuntu hirsute-backports InRelease' is not signed.
Any kind of help would be appreciated.
That's a bug in the docker / seccomp / glibc interaction: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1916485
I've run your docker file and get the same error. Playing around with various ways to disable the verification also produced no good results. Neither did removing the version constraints and just installing the latest versions of the tools. The only solution I could find was to downgrade ubuntu to 20.04, but there is no 3.6.3-5 version of maven for that version of the OS, only 3.6.3-1 (afaik).
The closest I could get working is quite different from your desired image:
FROM ubuntu:20.04
RUN apt update && \
apt install --no-install-recommends -y curl=7.\* unzip=6.\* maven=3.6.3-1 && \
apt clean && \
rm -rf /var/lib/apt/lists/* && \
mkdir -p /usr/share/man/man1
Also note how I use apt rather than apt-get and I only do a single run (which makes a simpler image by having only a single layer) and only a single apt update and chain the things I want to install into a single apt install rather than separate ones. This is just quicker and easier.
However, if you want a maven build box, perhaps you'd be better advised using one of the prebuilt maven images from docker hub that are themselves based on openjdk images. For java the underlying linux distro rarely matters and the openjdk images are pretty well respected:
from maven:3.6.3-jdk-11
run apt update && apt install -y curl unzip && apt clean
This bug does not occur if using a newer version of Docker (tested with 20.10). If using an older version of Docker, I recommend switching to a previous version of the ubuntu image. I tested ubuntu:20.10 with Docker 19.03 and it worked just fine. This is discussed here: https://bugs.launchpad.net/cloud-images/+bug/1928218
Update Docker version to the latest to solve this issue.
For ubuntu users follow these steps:
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
For others please refer this link: https://docs.docker.com/engine/install/
I ran into this problem when I was running the Ubuntu 21.04 image under Rootless Docker, but the apt-get update command worked fine under the system Docker (invoked via sudo). Since my need was just for a manual test of an environment setup script, I just ran under the system Docker but, depending on your application, that might not be secure.
Substituting apt-get with apt has worked for me.

error in docker build - Err:1 http://deb.debian.org/debian stretch/main amd64 unzip amd64 6.0-21+deb9u1 404 Not Found

I am doing a docker build on my MacBook Pro and it always keeps failing with following error:
Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
zip
The following NEW packages will be installed:
unzip
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 170 kB of archives.
After this operation, 547 kB of additional disk space will be used.
Err:1 http://deb.debian.org/debian stretch/main amd64 unzip amd64 6.0-21+deb9u1
404 Not Found
E: Failed to fetch http://deb.debian.org/debian/pool/main/u/unzip/unzip_6.0-21+deb9u1_amd64.deb 404 Not Found
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get install unzip' returned a non-zero code: 100
docker version:
Docker version 19.03.8, build afacb8b
MacOS: Mojave 10.14.6
Dockerfile snippet:
FROM debian:latest
RUN apt-get update
RUN apt-get install -y ca-certificates
RUN apt-get install unzip
The build works fine in our travis CI which is using docker-ce=17.09.0~ce-0~ubuntu
Any suggestions on how to debug it further? Initially we thought it may be a temporary issue on debian side but the problem has persisted so likely an issue with my environment.
Combine the three RUN lines you show into a single command:
FROM debian:latest
RUN apt-get update \
&& apt-get install -y \
ca-certificates \
unzip
There's a combination of two things that leads to that 404 error. On the one hand, Docker will cache individual Dockerfile steps: it sees that, starting from debian:latest, it's already RUN apt-get update, so it uses the version of that command from yesterday. On the other hand, Debian updates their repositories fairly frequently with very minor updates (see the +deb9u1 part of that version number) and when they do they delete the previous version from their repositories. This combination means you can be in a sequence where you're using a cached version of the apt-get update index, but the package version it mentions doesn't exist any more.
Combining these lines together like this means Docker will always run both apt-get update and apt-get install together; if you add a package to the list it will re-run the update step before trying to download things. That avoids this problem, at the cost of a little extra download time when the package list changes.

Unable to install apt-transport-https on Raspberry Pi 4

I want to install docker on raspberry, i used the script on docker:https://docs.docker.com/engine/install/debian/#install-using-the-convenience-script
I run the script, then i met this issue:
sudo sh get-docker.sh
# Executing docker install script, commit: 442e66405c304fa92af8aadaa1d9b31bf4b0ad94
+ sh -c apt-get update -qq >/dev/null
+ sh -c DEBIAN_FRONTEND=noninteractive apt-get -y install -qq apt-transport-https ca-certificates curl >/dev/null
E: Essential packages were removed and -y was used without --allow-remove-essential.
then i install this package alone, ca-certificates, curl is ok, but apt-transport-https met question agina:
sudo apt-get install apt-transport-https
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
python-apt-common python3-apt python3-debconf
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
libapt-pkg4.12
The following packages will be REMOVED:
apt apt-listchanges apt-utils tasksel tasksel-data
The following NEW packages will be installed:
apt-transport-https libapt-pkg4.12
WARNING: The following essential packages will be removed.
This should NOT be done unless you know exactly what you are doing!
apt
0 upgraded, 2 newly installed, 5 to remove and 0 not upgraded.
Need to get 847 kB of archives.
After this operation, 3,112 kB disk space will be freed.
You are about to do something potentially harmful.
To continue type in the phrase 'Yes, do as I say!'
?]

Cannot install Docker on Debian Jessie

Despite trying both the official installation mechanism using the new apt repo described here, as well as the curl -fsSL https://get.docker.com/ | sh route, I still get E: Unable to locate package docker-engine from APT when I try to apt-get install docker-engine.
My versions are:
$ uname -a
Linux blah 4.5.5-x86_64-linode69 #3 SMP Fri May 20 15:25:13 EDT 2016 x86_64 GNU/Linux
$ lsb_release -c
Codename: jessie
$ cat /etc/debian_version
8.5
$ cat /etc/apt/sources.list
deb http://ftp.uk.debian.org/debian/ stable main contrib non-free
deb-src http://ftp.uk.debian.org/debian/ stable main
deb http://security.debian.org/ stable/updates main
deb-src http://security.debian.org/ stable/updates main
deb http://http.debian.net/debian wheezy-backports main
The only file in my /etc/apt/sources.list.d is docker.list which contains:
deb https://apt.dockerproject.org/repo debian-jessie main
apt-cache policy docker-engine doesn't find it either:
apt-cache policy docker-engine
N: Unable to locate package docker-engine
How might I resolve this?
Edit your sources.list and change the following line from:
deb http://http.debian.net/debian wheezy-backports main
to
deb http://ftp.debian.org/debian jessie-backports main
Update and install docker:
apt-get update
apt-get install docker.io
Edit
To install a specific version of docker-engine download the .deb package from here, e,g the latest one is docker-engine_1.9.1-0~jessie_amd64.deb:
wget https://apt.dockerproject.org/repo/pool/main/d/docker-engine/docker-engine_1.9.1-0~jessie_amd64.deb
sudo apt-get update
dpkg -i docker-engine_1.9.1-0~jessie_amd64.deb
Maybe you will get an error , to fix it run:
apt-get -f install
dpkg -i docker-engine_1.9.1-0~jessie_amd64.deb
Your dpkg architecture is probably using 32bit. You can check this using:
dpkg --print-architecture
Fix it by adding amd64 as a foreign architecture:
dpkg --add-architecture amd64
dpkg --print-foreign-architectures
Update your package lists and check for docker-engine:
apt-get update
apt-cache policy docker-engine
Source: https://wiki.debian.org/Multiarch/HOWTO
Login as root user
$ sudo su
Create this file if it does not exist:
# vi /etc/apt/sources.list.d/backports.list
Add this as content of your backports.list
deb http://http.debian.net/debian jessie-backports main
Now perform your apt-get update
# apt-get update
Install the CA certificates
# apt-get install apt-transport-https ca-certificates
Add the new GPG key
# apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
Now open /etc/apt/sources.list.d/docker.list (or create when it does not exist)
# vi /etc/apt/sources.list.d/docker.list
Add as content:
deb https://apt.dockerproject.org/repo debian-jessie main
Perform again your update:
# apt-get update
Verify that APT is pulling from the right repository.
# apt-cache policy docker-engine
Update again
# sudo apt-get update
Install Docker:
# sudo apt-get install docker-engine
Start the docker daemon.
# sudo service docker start
Verify docker is installed correctly.
# sudo docker run hello-world
Hi guys I faced the same problem and recently found a script automated the docker installation process in debian 8. You could see the snippet here (https://gist.github.com/frgomes/a6f889583860f5b330c06c8b46fa0f42). Credit goes to the original script creator.
I add this on line 4 to removed older versions of Docker if it were existed:
sudo apt-get remove docker docker-engine
and few line on line 7:
sudo apt-get install apt-transport-https ca-certificates curl gnupg2 software-properties-common -y
Then as superuser:
# chmod +x ./install-docker.sh
# sudo ./install-docker.sh
And you get latest docker instead of v 1.5-1:
# docker --version
Docker version 17.05.0-ce, build 89658be

Resources