How can I install curl inside my container? - docker

I have the following Dockerfile
FROM jupyter/scipy-notebook
#RUN apt-get update && apt-get install -y curl
RUN pip install mlflow
RUN pip install sklearn
RUN apt-get update && apt-get install -y curl
When I do
docker build -t mycontainer .
I get
Step 4/4 : RUN apt-get update && apt-get install -y curl
---> Running in 5defd9816a22
Reading package lists...
E: List directory /var/lib/apt/lists/partial is missing. - Acquire (13: Permission denied)
The command '/bin/bash -o pipefail -c apt-get update && apt-get install -y curl' returned a non-zero code: 100
I suspect it is something related with not running as a root?
So how can I install curl in my container from the Dockerfile?
EDIT: I applied the answer that I was given.
Unfortunately it did not work
Step 4/7 : USER root
---> Running in aa6a1b7a023f
Removing intermediate container aa6a1b7a023f
---> 59e87b9598b2
Step 5/7 : RUN apt-get update && apt-get install -y curl
---> Running in 4440ce61ebc6
Err:1 http://security.ubuntu.com/ubuntu focal-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu focal InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-updates/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/focal-backports/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/focal-security/InRelease Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package curl
The command '/bin/bash -o pipefail -c apt-get update && apt-get install -y curl' returned a non-zero code: 100
EDIT2: I tried it in a different network setup and it worked.

You're seeing a "permission denied" error when apt attempts to create /var/lib/apt/lists/partial. This is because your process isn't running as root; the jupyter/scipy-notebook image is configured to run as non-root user (it runs as user jovyan).
You can change the user under which commands run in your Dockerfile with the USER directive, like this:
FROM jupyter/scipy-notebook
RUN pip install mlflow
RUN pip install sklearn
USER root
RUN apt-get update && apt-get install -y curl
USER jovyan
Note that I've made sure to reset the user back to jovyan after running the apt-get command.

Try to install it with root as it done in official jupyter/scipy-notebook Dockerfile and set it back to $NB_UID: (see user declaration in official base docker image)
USER root
# Install packages you need:
RUN apt-get update && apt-get install -y curl
# Switch back to jovyan to avoid accidental container runs as root
USER $NB_UID
Here is the hierarchy of base docker images used by jupyter/scipy-notebook:
jupyter/base-notebook -> jupyter/minimal-notebook -> jupyter/scipy-notebook

Related

Temporary failure Archlinux Docker

I use a custom dockerfile for my container and i got this error when i try to up my container with docker-compose up -d
Step 4/14 : RUN apt-get update && apt-get install -y libzip-dev wget curl nodejs npm
---> Running in 6007160b4c25
Err:1 http://deb.debian.org/debian bullseye InRelease
Temporary failure resolving 'deb.debian.org'
i test with a dns in /etc/docker/daemon.json
{ "dns" : ["8.8.8.8", "8.8.4.4"]}
but that doesn't work

Unable to install Jenkins on Ubuntu 20.04

I am trying to install Jenkins on my Ubuntu EC2 instance and I performed the following steps to install but couldn't install it.
$sudo apt update
$sudo apt install openjdk-8-jdk
$wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | sudo apt-key add -
$sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
$sudo apt update <--------- (Here I am getting below error)
root#ip-172-31-44-187:~# sudo apt update
Ign:1 https://pkg.jenkins.io/debian-stable binary/ InRelease
Err:2 https://pkg.jenkins.io/debian-stable binary/ Release
Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in
the certificate verification. [IP: 151.101.154.133 443]
Hit:3 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu focal InRelease
Get:4 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB]
Get:5 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]
Get:6 http://ap-south-1.ec2.archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB]
Reading package lists... Done
E: The repository 'http://pkg.jenkins.io/debian-stable binary/ Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
Instead of upgrading every package with apt-get upgrade, I used:
sudo apt install ca-certificates
And then:
sudo apt-get update worked just fine.
Yeah , I had same problem with this from yesterday , I think this is after yesterday's new update in jenkins 2.303.2 Lts .
Just do ,
apt upgrade ,
apt update,
apt get install jenkins -y .
It worked for me .
I was facing same issue when I tried to install jenkins in AWS ec2 instance (Ubuntu 20.04). Below steps helped me.
Update Ubuntu packages and all installed applications
sudo apt-get update -y
sudo apt-get upgrade -y
Next, Install JDK
sudo apt install openjdk-11-jdk -y
Verify Java version
java -version
Add gpg key for jenkins installation
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
Add the repository address to our /etc/apt/sources.list.d file
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
e> /etc/apt/sources.list.d/jenkins.list'
Update our package list again
sudo apt-get update -y
Install Jenkins
sudo apt-get install jenkins -y
It worked like charm!
I had the same problem with adding Jenkins repository on Ubuntu 18.04
add-apt-repository 'deb https://pkg.jenkins.io/debian-stable binary/'
Ign:5 https://pkg.jenkins.io/debian-stable binary/ InRelease Err:6 https://pkg.jenkins.io/debian-stable binary/ Release Certificate verification failed: The certificate is NOT trusted. The certificate chain uses expired certificate. Could not handshake: Error in the certificate verification. [IP: 199.232.66.133 443] Hit:7 http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic InRelease Reading package lists... Done E: The repository 'https://pkg.jenkins.io/debian-stable binary/ Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. N: See apt-secure(8) manpage for repository creation and user configuration details.
For fixing this issue you need to install/update ca-certificates
sudo apt install ca-certificates
After that, you can successfully add the Jenkins repository
I would like to correct the first answer provided. You need to run apt install Jenkins -y instead of apt get install jenkins -y. Running the below commands will fix your error. If you are not using root ensure that you add sudo before of all the below commands.
apt upgrade
apt update
apt install jenkins -y
You will find out your jenkins is started using the below command.
service jenkins status
As it might help some, none of the above solutions worked for me, but it was a silly mistake! *Please* read ALL the outputs. In my case, I'd missed an error a few lines above which indicated that I didn't have "curl" installed (!) on my Debian server. And so copy pasting the key installation lines from the Jenkins manual (which uses curl) didn't succeed, so all those unsecure errors in result.
Instead of upgrading every package with the apt-get upgrade, I used: sudo apt install ca-certificates
And then: sudo apt-get update worked just fine
Please Follow these commands
sudo apt-get update
sudo apt install openjdk-8-jdk
wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
sudo sh -c 'echo deb https://pkg.jenkins.io/debian-stable binary/ > \
e> /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins -y

E: Unable to locate package wget in docker

I am trying to build the following dockerfile
FROM php:8.0-apache
RUN apt-get update
# because apparently composer can shell out to unzip? Who knew...
RUN apt-get install wget unzip zip -y
but it failed with the following error
[root#dev svc]# docker build -t wsb -f Dockerfile .
Sending build context to Docker daemon 17.62 MB
Step 1/11 : FROM php:8.0-apache
---> 97f22a92e1d1
Step 2/11 : RUN apt-get update
---> Using cache
---> ddba6cf13bac
Step 3/11 : RUN apt-get install wget unzip zip -y
---> Running in 6fb3e2f37401
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package wget
E: Unable to locate package unzip
E: Unable to locate package zip
The command '/bin/sh -c apt-get install wget unzip zip -y' returned a non-zero code: 100
Existing contents of sources.list file
# deb http://snapshot.debian.org/archive/debian/20210511T000000Z buster main
deb http://deb.debian.org/debian buster main
# deb http://snapshot.debian.org/archive/debian-security/20210511T000000Z buster/updates main
deb http://security.debian.org/debian-security buster/updates main
# deb http://snapshot.debian.org/archive/debian/20210511T000000Z buster-updates main
deb http://deb.debian.org/debian buster-updates main
I am not sure why it's unable to find. Should I add any URL to /etc/apt/sources.list to get this working?
EDIT
Sending build context to Docker daemon 17.62 MB
Step 1/9 : FROM php:8.0-apache
---> 97f22a92e1d1
Step 2/9 : RUN apt-get update && apt-get install wget unzip zip -y
---> Running in 374b1060dfb3
Err:1 http://security.debian.org/debian-security buster/updates InRelease
Temporary failure resolving 'security.debian.org'
Err:2 http://deb.debian.org/debian buster InRelease
Temporary failure resolving 'deb.debian.org'
Err:3 http://deb.debian.org/debian buster-updates InRelease
Temporary failure resolving 'deb.debian.org'
Reading package lists...
W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease Temporary failure resolving 'deb.debian.org'
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease Temporary failure resolving 'security.debian.org'
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease Temporary failure resolving 'deb.debian.org'
W: Some index files failed to download. They have been ignored, or old ones used instead.
You've cached a stale update so apt is trying to install these packages from versions that are no longer in the repositories. This is why Docker's best practices mention to not separate these steps:
FROM php:8.0-apache
RUN apt-get update \
&& apt-get install wget unzip zip -y
From the second issue you are now seeing, you have a networking issue within your containers. There are lots of possible causes, including the host not being connected to the network, and networking changing after the docker engine was started. You'd need to provide a lot more debugging (can you reach these nodes from the host, is DNS working, is there a proxy or firewall on the network, etc).

docker-compose build: Retry a step in case it fails for x times?

Due to to partly bad internet in our company we are having issues with building our containers constantly. Lets assume we have our php-fpm container which requires a lot of additional software and has about 60 build steps.
Now when doing docker-compose build --no-cache php-fpm the process quite often quits when e.g. doing apt-get update or installing a certain library.
Output is like this:
Step 29/56 : RUN apt-get update && apt-get -y --no-install-recommends install wkhtmltopdf
---> Running in 7ec583e71349
[...]
Err:1 http://deb.debian.org/debian buster/main amd64 libdouble-conversion1 amd64 3.1.0-3
Could not resolve 'deb.debian.org'
Err:2 http://deb.debian.org/debian buster/main amd64 libpcre2-16-0 amd64 10.32-5
Could not resolve 'deb.debian.org'
[.....]
ERROR: Service 'php-fpm' failed to build: The command '/bin/sh -c apt-get update && apt-get -y --no-install-recommends install wkhtmltopdf' returned a non-zero code: 100
So I wonder if there is a possibility to tell docker to try a step or even certain steps for like three times before quitting the build process?

How to fix 'python: not found' error when using PostCreateCommand in vscode remote docker extension

I'm using the Visual Studio Code Remote - Containers extension with a customized DockerFile. It is based on https://github.com/microsoft/vscode-dev-containers/blob/master/containers/python-3/.devcontainer/Dockerfile but uses a different base image and doesn't try to pip install from requirements.txt.
When I build the container in vscode, with PostCreateCommand set to "python --version", the following errors appears in the dev containers terminal output:
Run: docker exec -w /workspaces/media-classifier dd5e552b4f113ecf74504cc6d3aed3ca1727b4a172645515392c4632b7c45b81 /bin/sh -c python --version
/bin/sh: 1: python: not found
postCreateCommand "python --version" failed.
I've tried using the same setting value for PostCreateCommand (python --version) using both the standard python3 container and the python3 anaconda container and they both successfully output the python version.
I've also tried setting PostCreateCommand to the following values, which all produce the same 'not found' error:
pip --version
conda --version
When the container has started, I'm successfully able to use python, pip and conda so they are definitely installed.
Dockerfile
FROM microsoft/cntk:2.6-cpu-python3.5
# Configure apt and install packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends apt-utils 2>&1 \
#
# Verify git, process tools, lsb-release (common in install instructions for CLIs) installed
&& apt-get -y install git procps lsb-release \
# Clean up
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
devcontainer.json
{
"name": "CNTK Python3.5",
"context": "..",
"dockerFile": "Dockerfile",
// Uncomment the next line if you want to publish any ports.
// "appPort": [],
// Uncomment the next line to run commands after the container is created.
"postCreateCommand": "python --version",
"extensions": [
"ms-python.python",
"neuron.neuron-ipe"
],
"settings": {
"python.pythonPath": "/opt/conda/bin/python",
"python.linting.pylintEnabled": true,
"python.linting.enabled": true
}
}
I'm expecting PostCreateCommand to execute successfully and output the python version installed in whichever anaconda environment is active at the time.
you are trying to run python when python3 is installed
try running
python3 --version
Since python3 is not a direct replacement for python v2, Debian based systems have not setup an automatic alias to point python to the python3 binaries. The recommended solution is to point all python v3 commands to python3. The workaround for applications that can't be easily updated is to setup an alias to point python to python3 anyway.
FROM microsoft/cntk:2.6-cpu-python3.5
# Configure apt and install packages
RUN apt-get update \
&& apt-get -y install --no-install-recommends apt-utils 2>&1 \
#
# Verify git, process tools, lsb-release (common in install instructions for CLIs) installed
&& apt-get -y install \
git \
lsb-release \
procps \
python-is-python3 \
# Clean up
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
For more on this, see https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3
From the comments, here's a similar example with a different base image, also fixed by installing python-is-python3:
$ docker run -it --rm --entrypoint bash mcr.microsoft.com/vscode/devcontainers/dotnet:0.202.1-6.0
Unable to find image 'mcr.microsoft.com/vscode/devcontainers/dotnet:0.202.1-6.0' locally
0.202.1-6.0: Pulling from vscode/devcontainers/dotnet
e5ae68f74026: Pull complete
a74667493539: Pull complete
3a0bffe13264: Pull complete
913bac4f4fc9: Pull complete
d2ea5cb43486: Pull complete
1414be57e953: Pull complete
868d7bfddf03: Pull complete
63ab446ca68f: Pull complete
1259b98fc625: Pull complete
802a0f1d31f7: Pull complete
094ecb532868: Pull complete
f643f7ed0620: Pull complete
Digest: sha256:d3bfb3e7c9ecfcb4472b59272e2f8857807667c0bd83fb1da935d28e9087e733
Status: Downloaded newer image for mcr.microsoft.com/vscode/devcontainers/dotnet:0.202.1-6.0
root ➜ / $ type python
bash: type: python: not found
root ➜ / $ type python3
python3 is /usr/bin/python3
root ➜ / $ apt-get update
Get:1 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Get:4 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]
Get:5 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [102 kB]
Get:6 http://deb.debian.org/debian bullseye/contrib amd64 Packages [50.5 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 Packages [8183 kB]
Get:8 https://dl.yarnpkg.com/debian stable/main all Packages [10.5 kB]
Get:9 https://dl.yarnpkg.com/debian stable/main amd64 Packages [10.5 kB]
Get:10 http://deb.debian.org/debian bullseye/non-free amd64 Packages [93.8 kB]
Get:11 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
Fetched 8670 kB in 3s (3246 kB/s)
Reading package lists... Done
root ➜ / $ apt-get install -y python-is-python3
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
python-is-python3
0 upgraded, 1 newly installed, 0 to remove and 13 not upgraded.
Need to get 2800 B of archives.
After this operation, 13.3 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 python-is-python3 all 3.9.2-1 [2800 B]
Fetched 2800 B in 0s (67.0 kB/s)
Selecting previously unselected package python-is-python3.
(Reading database ... 22711 files and directories currently installed.)
Preparing to unpack .../python-is-python3_3.9.2-1_all.deb ...
Unpacking python-is-python3 (3.9.2-1) ...
Setting up python-is-python3 (3.9.2-1) ...
Processing triggers for man-db (2.9.4-2) ...
root ➜ / $ type python
python is /usr/bin/python

Resources