Docker build pip install fails - docker

Question

I have the following dockerfile:
FROM ubuntu:20.04
RUN apt-get update -qq \
&& apt-get install -y --no-install-recommends \
wget \
build-essential \
curl \
&& apt-get autoremove -y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN wget --no-check-certificate -O miniconda.sh https://repo.anaconda.com/miniconda/Miniconda3-py37_4.8.3-Linux-x86_64.sh \
&& mkdir /root/.conda \
&& bash miniconda.sh -b \
&& rm -f miniconda.sh
ENV PATH="/root/miniconda3/bin:${PATH}"
WORKDIR /home
#COPY download_model.sh download_model.sh
#RUN /bin/bash /home/download_model.sh
RUN conda config --set ssl_verify false
#RUN curl -I https://repo.anaconda.com/pkgs/main/linux-64/current_repodata.json -k
RUN conda install pytorch==1.4.0 torchvision==0.5.0 cpuonly -c pytorch
COPY requirements.txt requirements.txt
RUN pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -r requirements.txt
#RUN pip install -r requirements.txt
COPY m39v1.tar ./
RUN tar -xvf m39v1.tar && rm -f m39v1.tar
COPY src src/
COPY test_bin test_bin/
COPY run_paraphraser.py run_paraphraser.py
ENTRYPOINT ["python", "run_paraphraser.py"]
I'm having problems with running pip install, more specifically i've had to replace this command RUN pip install -r requirements.txt with RUN pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -r requirements.txt
Otherwise i was getting this wierd ssl certificate error:
#12 8.692 Could not fetch URL https://pypi.org/simple/config/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/config/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))) - skipping
#12 8.694 ERROR: Could not find a version that satisfies the requirement config (from versions: none)
#12 8.695 ERROR: No matching distribution found for config
#12 9.195 Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))) - skipping
#12 ERROR: executor failed running [/bin/sh -c pip install -r requirements.txt config --global http.sslVerify false]: exit code:
I'm on a company laptop, however the problem arised only when trying to reach the website from docker, i noticed the problem when i added this line RUN curl -I https://pypi.org/simple/pip/ to the dockerfile to test connection to pip:
#10 [ 7/14] RUN curl -I https://pypi.org/simple/pip/
#10 sha256:95fa072f708fe705a1d5a9f7af20972255a078ce1df2cf90ad52a4d9a844ec40
#10 0.378 % Total % Received % Xferd Average Speed Time Time Time Current
#10 0.378 Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
#10 0.412 curl: (77) error setting certificate verify locations:
#10 0.412 CAfile: /etc/ssl/certs/ca-certificates.crt
#10 0.412 CApath: /etc/ssl/certs
#10 ERROR: executor failed running [/bin/sh -c curl -I https://pypi.org/simple/pip/]: exit code: 77
As i've mentioned replacing pip install with this trusted version seemed to work initially RUN pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -r requirements.txt
However when building the dockerfile i ended up with another error which i can't really wrap my head around:
#13 [ 9/14] RUN pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -r requirements.txt
#13 sha256:1e3540de7554d95d43c31a02a3079a9d8c5a8104255b93ca63d4dd006e1df31e
#13 0.881 Collecting sentencepiece==0.1.86
#13 1.057 Downloading sentencepiece-0.1.86-cp37-cp37m-manylinux1_x86_64.whl (1.0 MB)
#13 1.368 Collecting fairseq==0.9.0
#13 3.330 Downloading fairseq-0.9.0.tar.gz (306 kB)
#13 12.30 ERROR: Command errored out with exit status 1:
#13 12.30 command: /root/miniconda3/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_2im9ux1/fairseq/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_2im9ux1/fairseq/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-_2im9ux1/fairseq/pip-egg-info
#13 12.30 cwd: /tmp/pip-install-_2im9ux1/fairseq/
#13 12.30 Complete output (39 lines):
#13 12.30 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))': /simple/cython/
#13 12.30 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))': /simple/cython/
#13 12.30 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))': /simple/cython/
#13 12.30 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))': /simple/cython/
#13 12.30 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])"))': /simple/cython/
#13 12.30 ERROR: Could not find a version that satisfies the requirement cython (from versions: none)
#13 12.30 ERROR: No matching distribution found for cython
#13 12.30 /root/miniconda3/lib/python3.7/site-packages/setuptools/installer.py:30: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
#13 12.30 SetuptoolsDeprecationWarning,
#13 12.30 Traceback (most recent call last):
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/installer.py", line 82, in fetch_build_egg
#13 12.30 subprocess.check_call(cmd)
#13 12.30 File "/root/miniconda3/lib/python3.7/subprocess.py", line 363, in check_call
#13 12.30 raise CalledProcessError(retcode, cmd)
#13 12.30 subprocess.CalledProcessError: Command '['/root/miniconda3/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp91n1bbee', '--quiet', 'cython']' returned non-zero exit status 1.
#13 12.30
#13 12.30 The above exception was the direct cause of the following exception:
#13 12.30
#13 12.30 Traceback (most recent call last):
#13 12.30 File "<string>", line 1, in <module>
#13 12.30 File "/tmp/pip-install-_2im9ux1/fairseq/setup.py", line 161, in <module>
#13 12.30 zip_safe=False,
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 86, in setup
#13 12.30 _install_setup_requires(attrs)
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/__init__.py", line 80, in _install_setup_requires
#13 12.30 dist.fetch_build_eggs(dist.setup_requires)
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/dist.py", line 878, in fetch_build_eggs
#13 12.30 replace_conflicting=True,
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 791, in resolve
#13 12.30 replace_conflicting=replace_conflicting
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1075, in best_match
#13 12.30 return self.obtain(req, installer)
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1087, in obtain
#13 12.30 return installer(requirement)
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/dist.py", line 945, in fetch_build_egg
#13 12.30 return fetch_build_egg(self, req)
#13 12.30 File "/root/miniconda3/lib/python3.7/site-packages/setuptools/installer.py", line 84, in fetch_build_egg
#13 12.30 raise DistutilsError(str(e)) from e
#13 12.30 distutils.errors.DistutilsError: Command '['/root/miniconda3/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp91n1bbee', '--quiet', 'cython']' returned non-zero exit status 1.
#13 12.30 ----------------------------------------
#13 12.31 ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
#13 ERROR: executor failed running [/bin/sh -c pip install --trusted-host pypi.org --trusted-host pypi.python.org --trusted-host files.pythonhosted.org -r requirements.txt]: exit code: 1
i've also tried to update certificates by adding:
RUN apt-get install ca-certificates -y
RUN update-ca-certificates
with no success. It could be important to know that i've also had the same issue with conda install section: RUN conda install pytorch==1.4.0 torchvision==0.5.0 cpuonly -c pytorch
and i've managed to find a workaround by disabling the certificate verification: RUN conda config --set ssl_verify false
What should i do? thank you!