Dockerfile: Python.h: No such file or directory - docker

Question

I have a simple dockerfile that I am using to run containers on AWS, I'm hitting an issue though when installing s3fs, which is strange since I've used this snippet in previous dockerfiles without issue.
Is it something with the distribution?
Error:
multidict/_multidict.c:1:10: fatal error: Python.h: No such file or directory
Dockerfile:
FROM amazonlinux:latest
RUN yum -y install which unzip aws-cli \
&& yum install -y python3-pip python3 python3-setuptools \
&& yum install -y tar.x86_64 \
&& DEBIAN_FRONTEND=noninteractive yum install -y ksh
RUN pip3 install boto3 \
&& yum install -y gcc \
&& pip3 install s3fs
Here is the output log:
Installing collected packages: fsspec, docutils, botocore, typing-extensions, aioitertools, wrapt, attrs, chardet, multidict, async-timeout, idna, yarl, aiohttp, aiobotocore, s3fs
Found existing installation: botocore 1.19.11
Uninstalling botocore-1.19.11:
Successfully uninstalled botocore-1.19.11
Running setup.py install for wrapt: started
Running setup.py install for wrapt: finished with status 'done'
Running setup.py install for multidict: started
Running setup.py install for multidict: finished with status 'error'
Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-l35mgnnl/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-7d6e5wlf-record/install-record.txt --single-version-externally-managed --compile:
**********************
* Accellerated build *
**********************
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/multidict
copying multidict/_abc.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/__init__.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict_base.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict_py.py -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_compat.py -> build/lib.linux-x86_64-3.7/multidict
running egg_info
writing multidict.egg-info/PKG-INFO
writing dependency_links to multidict.egg-info/dependency_links.txt
writing top-level names to multidict.egg-info/top_level.txt
reading manifest file 'multidict.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files found matching 'multidict/_multidict.html'
warning: no previously-included files found matching 'multidict/*.so'
warning: no previously-included files found matching 'multidict/*.pyd'
warning: no previously-included files found matching 'multidict/*.pyd'
no previously-included directories found matching 'docs/_build'
writing manifest file 'multidict.egg-info/SOURCES.txt'
copying multidict/__init__.pyi -> build/lib.linux-x86_64-3.7/multidict
copying multidict/_multidict.c -> build/lib.linux-x86_64-3.7/multidict
copying multidict/py.typed -> build/lib.linux-x86_64-3.7/multidict
creating build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/defs.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/dict.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/istr.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/iter.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/pair_list.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
copying multidict/_multilib/views.h -> build/lib.linux-x86_64-3.7/multidict/_multilib
running build_ext
building 'multidict._multidict' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/multidict
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.7m -c multidict/_multidict.c -o build/temp.linux-x86_64-3.7/multidict/_multidict.o -O2 -std=c99 -Wall -Wsign-compare -Wconversion -fno-strict-aliasing -pedantic
multidict/_multidict.c:1:10: fatal error: Python.h: No such file or directory
#include "Python.h"
^~~~~~~~~~
compilation terminated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-l35mgnnl/multidict/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-7d6e5wlf-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-l35mgnnl/multidict/
The command '/bin/sh -c pip3 install boto3 && yum install -y gcc && pip3 install s3fs' returned a non-zero code: 1
Any help is much appreciated!

Answer 1

You should add python3-devel package that contains absent headers.
FROM amazonlinux:latest
RUN yum -y install which unzip aws-cli \
&& yum install -y python3-pip python3 python3-setuptools \
&& yum install -y python3-devel.x86_64 \
&& yum install -y tar.x86_64 \
&& DEBIAN_FRONTEND=noninteractive yum install -y ksh
RUN pip3 install boto3 \
&& yum install -y gcc \
&& pip3 install s3fs