I would like to install aws-cli for below images but I received below error. I tried with apk, apt but none of then did not work. Can you please help how should I update my dockerfile?
I do not want to change my base image, I need to use maven:3.6.3-openjdk-14.
sh: apt-get: command not found
FROM maven:3.6.3-openjdk-14
RUN apt-get update \
&& apt-get install -y vim jq unzip curl \
&& apt-get upgrade -y \
#install aws 2
RUN curl --silent --show-error --fail "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip awscliv2.zip && \
./aws/install && \
rm -rf awscliv2.zip
Docker image maven:3.6.3-openjdk-14 is based on Oracle Linux which uses rpm to manage packages, so apt-get is not available.
docker run -i -t maven:3.6.3-openjdk-14 -- cat /etc/os-release
Related
I got a K8S+DinD issue:
launch Kubernetes cluster
start a main docker image and a DinD image inside this cluster
when running a job requesting GPU, got error could not select device driver "nvidia" with capabilities: [[gpu]]
Full error
http://localhost:2375/v1.40/containers/long-hash-string/start: Internal Server Error ("could not select device driver "nvidia" with capabilities: [[gpu]]")
exec to the DinD image inside of K8S pod, nvidia-smi is not available.
Some debugging and it seems it's due to the DinD is missing the Nvidia-docker-toolkit, I had the same error when I ran the same job directly on my local laptop docker, I fixed the same error by installing nvidia-docker2 sudo apt-get install -y nvidia-docker2.
I'm thinking maybe I can try to install nvidia-docker2 to the DinD 19.03 (docker:19.03-dind), but not sure how to do it? By multiple stage docker build?
Thank you very much!
update:
pod spec:
spec:
containers:
- name: dind-daemon
image: docker:19.03-dind
I got it working myself.
Referring to
https://github.com/NVIDIA/nvidia-docker/issues/375
https://github.com/Henderake/dind-nvidia-docker
First, I modified the ubuntu-dind image (https://github.com/billyteves/ubuntu-dind) to install nvidia-docker (i.e. added the instructions in the nvidia-docker site to the Dockerfile) and changed it to be based on nvidia/cuda:9.2-runtime-ubuntu16.04.
Then I created a pod with two containers, a frontend ubuntu container and the a privileged docker daemon container as a sidecar. The sidecar's image is the modified one I mentioned above.
But since this post is 3 year ago from now, I did spent quite some time to match up the dependencies versions, repo migration over 3 years, etc.
My modified version of Dockerfile to build it
ARG CUDA_IMAGE=nvidia/cuda:11.0.3-runtime-ubuntu20.04
FROM ${CUDA_IMAGE}
ARG DOCKER_CE_VERSION=5:18.09.1~3-0~ubuntu-xenial
RUN apt-get update -q && \
apt-get install -yq \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable" && \
apt-get update -q && apt-get install -yq docker-ce docker-ce-cli containerd.io
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN set -eux; \
apt-get update -q && \
apt-get install -yq \
btrfs-progs \
e2fsprogs \
iptables \
xfsprogs \
xz-utils \
# pigz: https://github.com/moby/moby/pull/35697 (faster gzip implementation)
pigz \
# zfs \
wget
# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -x \
&& addgroup --system dockremap \
&& adduser --system -ingroup dockremap dockremap \
&& echo 'dockremap:165536:65536' >> /etc/subuid \
&& echo 'dockremap:165536:65536' >> /etc/subgid
# https://github.com/docker/docker/tree/master/hack/dind
ENV DIND_COMMIT 37498f009d8bf25fbb6199e8ccd34bed84f2874b
RUN set -eux; \
wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
chmod +x /usr/local/bin/dind
##### Install nvidia docker #####
# Add the package repositories
RUN curl -fsSL https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add --no-tty -
RUN distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && \
echo $distribution && \
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
tee /etc/apt/sources.list.d/nvidia-docker.list
RUN apt-get update -qq --fix-missing
RUN apt-get install -yq nvidia-docker2
RUN sed -i '2i \ \ \ \ "default-runtime": "nvidia",' /etc/docker/daemon.json
RUN mkdir -p /usr/local/bin/
COPY dockerd-entrypoint.sh /usr/local/bin/
RUN chmod 777 /usr/local/bin/dockerd-entrypoint.sh
RUN ln -s /usr/local/bin/dockerd-entrypoint.sh /
VOLUME /var/lib/docker
EXPOSE 2375
ENTRYPOINT ["dockerd-entrypoint.sh"]
#ENTRYPOINT ["/bin/sh", "/shared/dockerd-entrypoint.sh"]
CMD []
When I use exec to login into the Docker-in-Docker container, I can successfully run nvidia-smi (which previously return not found error then cannot run any GPU resource related docker run)
Welcome to pull my image at brandsight/dind:nvidia-docker
I am relatively new to docker. I have an application which I want to containerize.
Below is is my docker file:
FROM ubuntu:16.04
## ENV Variables
ENV PYTHON_VERSION="3.6.5"
# Update and Install packages
RUN apt-get update -y \
&& apt-get install -y \
curl \
wget \
tar
# Install Python 3.6.5
RUN wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz \
&& tar -xvf Python-${PYTHON_VERSION}.tar.xz \
&& cd Python-${PYTHON_VERSION} \
&& ./configure \
&& make altinstall \
&& cd / \
&& rm -rf Python-${PYTHON_VERSION}
# Install Google Cloud SDK
# Downloading gcloud package
RUN curl https://dl.google.com/dl/cloudsdk/release/google-cloud-sdk.tar.gz > /tmp/google-cloud-sdk.tar.gz
# Installing the package
RUN mkdir -p /usr/local/gcloud \
&& tar -C /usr/local/gcloud -xvf /tmp/google-cloud-sdk.tar.gz \
&& /usr/local/gcloud/google-cloud-sdk/install.sh
# Adding the package path to local
ENV PATH $PATH:/usr/local/gcloud/google-cloud-sdk/bin
I am trying to install python3.6.5 version but I am receiving the following error.
020-01-09 17:26:13 (107 KB/s) - 'Python-3.6.5.tar.xz' saved [17049912/17049912]
tar (child): xz: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
The command '/bin/sh -c wget https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tar.xz && tar -xvf Python-${PYTHON_VERSION}.tar.xz && cd Python-${PYTHON_VERSION} && ./configure && make altinstall && cd / && rm -rf Python-${PYTHON_VERSION}' returned a non-zero code: 2
Decompressing an .xz file requires the xz binary which under ubuntu is provided by the package xz-utils So You have to instal xz-utils on your image prior to decompressing an .xz file.
You can add this to your previous apt-get install run:
# Update and Install packages
RUN apt-get update -y \
&& apt-get install -y \
curl \
wget \
tar \
xz-utils
This should fix the following call to tar in the next RUN expression
Instead of trying to install Python, just start with a base image that has Python preinstalled, e.g. python:3.6-buster. This image is based on Debian Buster, which was released in 2019. Since Ubuntu is based on Debian, everything will be pretty similar, and since it's from 2019 (as opposed to Ubuntu 16.04, which is from 2016) you'll get more up-to-date software.
See https://pythonspeed.com/articles/base-image-python-docker-images/ for longer discussion.
I have been experimenting to create a docker image with python3.6 based on amazonlinux.
So far, I have not been very successful. I use
docker run -it amazonlinux
to start an interactive docker terminal. Inside the terminal, I run "yum install python36" and see the following error message. Note that I copied this step was from an old amazonlinux based Dockerfile. This Dockerfile used to work. So I suspend the error I see below is due to amazon updated their docker linux image
bash-4.2# yum install python36
Loaded plugins: ovl, priorities
amzn2-core | 2.4 kB 00:00:00
No package python36 available.
Error: Nothing to do
I have tried to add a python3.6 repo by following this post
https://janikarhunen.fi/how-to-install-python-3-6-1-on-centos-7 however, it still gives the same error when I run
yum install python36u
Is there any way to add python3.6 to amazonlinux base layer? Thanks in advance.
There is now a far easier answer to this question thanks to aws 'extras'. Now this will work:
amazon-linux-extras install python3
You can check this Dockerfile based on amazon Linux and having python version is PYTHON_VERSION=3.6.4.
Or you can work with your existing one like
ARG PYTHON_VERSION=3.6.4
ARG BOTO3_VERSION=1.6.3
ARG BOTOCORE_VERSION=1.9.3
ARG APPUSER=app
RUN yum -y update &&\
yum install -y shadow-utils findutils gcc sqlite-devel zlib-devel \
bzip2-devel openssl-devel readline-devel libffi-devel && \
groupadd ${APPUSER} && useradd ${APPUSER} -g ${APPUSER} && \
cd /usr/local/src && \
curl -O https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz && \
tar -xzf Python-${PYTHON_VERSION}.tgz && \
cd Python-${PYTHON_VERSION} && \
./configure --enable-optimizations && make && make altinstall && \
rm -rf /usr/local/src/Python-${PYTHON_VERSION}* && \
yum remove -y shadow-utils audit-libs libcap-ng && yum -y autoremove && \
yum clean all
But better to clone the repo and make your own image form that.
I too had similiar issue for docker.
yum install docker
Loaded plugins: ovl, priorities
amzn2-core | 3.7 kB 00:00:00
No package docker available.
Error: Nothing to do
instead yum I used amazon-linux-extras, it worked
amazon-linux-extras install docker
I installed Elasticsearch in my image based on ubuntu:16.04.
And start the service using
RUN service elasticsearch start
but, it was not started.
If I go into the container and run it, it starts.
I want to run the service and dump the index when I create the image, below is a part of my Dockerfile.
How do I start Elasticsearch in the Dockerfile?
#install OpenJDK-8
RUN apt-get update && apt-get install -y openjdk-8-jdk && apt-get install -y ant && apt-get clean
RUN apt-get update && apt-get install -y ca-certificates-java && apt-get clean
RUN update-ca-certificates -f
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
#download ES
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
RUN apt-get install -y apt-transport-https
RUN echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list
RUN apt-get update && apt-get install -y elasticsearch
RUN service elasticsearch start
The RUN command executes only during the build phase. It stops after the build is completed. You should use CMD (or ENTRYPOINT) instead:
CMD service elasticsearch start && /bin/bash
It's better wrapping the starting command in your own file and then only execute the file:
CMD /start_elastic.sh
I don't know why not take official oss image, but, this Docker file based on Debian work:
FROM java:8-jre
ENV ES_NAME=elasticsearch \
ELASTICSEARCH_VERSION=6.6.1
ENV ELASTICSEARCH_URL=https://artifacts.elastic.co/downloads/$ES_NAME/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz
RUN apt-get update && apt-get install -y --assume-yes openssl bash curl wget \
&& mkdir -p /opt \
&& echo '[i] Start create elasticsearch' \
&& wget -T 15 -O /tmp/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz $ELASTICSEARCH_URL \
&& tar -xzf /tmp/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz -C /opt/ \
&& ln -s /opt/$ES_NAME-$ELASTICSEARCH_VERSION /opt/$ES_NAME \
&& useradd elastic \
&& mkdir -p /var/lib/elasticsearch /opt/$ES_NAME/plugins /opt/$ES_NAME/config/scripts \
&& chown -R elastic /opt/$ES_NAME-$ELASTICSEARCH_VERSION/
ENV PATH=/opt/elasticsearch/bin:$PATH
USER elastic
CMD [ "/bin/sh", "-c", "/opt/elasticsearch/bin/elasticsearch --E cluster.name=test --E network.host=0 $ELASTIC_CMD_OPTIONS" ]
I believe most of the commands you'll be able to use on Ubuntu.
Don't forget to run sudo sysctl -w vm.max_map_count=262144 on your host
i need a container running with java installed in it and i want to expose the port 8090.
Here is the Docker file i have written to achieve this.
FROM ubuntu:16.04
ENV DEBIAN_FRONTEND noninteractive
ENV VERSION 8
ENV UPDATE 152
ENV BUILD 16
ENV SIG aa0333dd3019491ca4f6ddbe78cdb6d0
ENV JAVA_HOME /usr/lib/jvm/java-${VERSION}-oracle
# install jre
RUN apt-get update -qq && \
apt-get upgrade -qqy --no-install-recommends && \
apt-get install curl unzip bzip2 -qqy && \
mkdir -p "${JAVA_HOME}" && \
curl --silent --location --insecure --junk-session-cookies --retry 3 \
--header "Cookie: oraclelicense=accept-securebackup-cookie;" \
http://download.oracle.com/otn-pub/java/jdk/"${VERSION}"u"${UPDATE}"-b"${BUILD}"/"${SIG}"/jre-"${VERSION}"u"${UPDATE}"-linux-x64.tar.gz \
| tar -xzC "${JAVA_HOME}" --strip-components=1 && \
apt-get remove --purge --auto-remove -y curl unzip bzip2 && \
apt-get autoclean && apt-get --purge -y autoremove && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN update-alternatives --install "/usr/bin/java" "java" "${JAVA_HOME}/bin/java" 1 && \
update-alternatives --install "/usr/bin/javaws" "javaws" "${JAVA_HOME}/bin/javaws" 1 && \
update-alternatives --set java "${JAVA_HOME}/bin/java" && \
update-alternatives --set javaws "${JAVA_HOME}/bin/javaws"
EXPOSE 8090
Using the Dockerfile i was able to successfully build the image and i have pushed to my account in hub.docker.com
but when i run try to run the container using the following command , The Container was not running.
i broked my head analyzing the root cause more than 2 hours i was not able to find the problem.
I know i am missing something silly, can anyone have a look and point the mistake i am doing?
Thanks in advance
Your Dockerfile is missing an ENTRYPOINT or CMD instruction. They define what command is run when starting the container.
Reference CMD
Reference ENTRYPOINT
You didn't define a ENTRYPOINT or CMD.
Some useful links:
What is the difference between CMD and ENTRYPOINT in a Dockerfile?
https://www.ctl.io/developers/blog/post/dockerfile-entrypoint-vs-cmd/
https://docs.docker.com/engine/reference/builder/#known-issues-run