elastic search is successfully running on docker container. but i'm not able access in browser. i mapped ports correctly. but the problem is in docker container. in container elasticsearch is mapped with localhost
127.0.0.1:9200
Dokcerfile
FROM ubuntu:16.04
MAINTAINER Rajesh Gurram
RUN apt-get update && \
apt-get install -y net-tools curl wget gnupg
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:webupd8team/java && \
apt-get update && \
echo oracle-java7-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get install -y oracle-java8-installer && apt-get clean
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle
RUN apt-get install apt-transport-https
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add - && \
echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list && \
apt update && apt install -y elasticsearch
RUN sed -i 's/#network.host: 192.168.0.1/network.host: 0.0.0.0/g' /etc/elasticsearch/elasticsearch.yml
EXPOSE 9200 9300
Run Below command on Host machine it will resolve the issue
$ sysctl -w vm.max_map_count=262144
If you want to use docker to get an instance of Elasticsearch, you can read the following Guide:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
You can also use docker images directly from elastic, if ubuntu is not a necessary base image:
https://www.docker.elastic.co/
If you want to upgrade to an ELK Stack later on, I recommend a docker volume for persistency purposes.
Related
I got a K8S+DinD issue:
launch Kubernetes cluster
start a main docker image and a DinD image inside this cluster
when running a job requesting GPU, got error could not select device driver "nvidia" with capabilities: [[gpu]]
Full error
http://localhost:2375/v1.40/containers/long-hash-string/start: Internal Server Error ("could not select device driver "nvidia" with capabilities: [[gpu]]")
exec to the DinD image inside of K8S pod, nvidia-smi is not available.
Some debugging and it seems it's due to the DinD is missing the Nvidia-docker-toolkit, I had the same error when I ran the same job directly on my local laptop docker, I fixed the same error by installing nvidia-docker2 sudo apt-get install -y nvidia-docker2.
I'm thinking maybe I can try to install nvidia-docker2 to the DinD 19.03 (docker:19.03-dind), but not sure how to do it? By multiple stage docker build?
Thank you very much!
update:
pod spec:
spec:
containers:
- name: dind-daemon
image: docker:19.03-dind
I got it working myself.
Referring to
https://github.com/NVIDIA/nvidia-docker/issues/375
https://github.com/Henderake/dind-nvidia-docker
First, I modified the ubuntu-dind image (https://github.com/billyteves/ubuntu-dind) to install nvidia-docker (i.e. added the instructions in the nvidia-docker site to the Dockerfile) and changed it to be based on nvidia/cuda:9.2-runtime-ubuntu16.04.
Then I created a pod with two containers, a frontend ubuntu container and the a privileged docker daemon container as a sidecar. The sidecar's image is the modified one I mentioned above.
But since this post is 3 year ago from now, I did spent quite some time to match up the dependencies versions, repo migration over 3 years, etc.
My modified version of Dockerfile to build it
ARG CUDA_IMAGE=nvidia/cuda:11.0.3-runtime-ubuntu20.04
FROM ${CUDA_IMAGE}
ARG DOCKER_CE_VERSION=5:18.09.1~3-0~ubuntu-xenial
RUN apt-get update -q && \
apt-get install -yq \
apt-transport-https \
ca-certificates \
curl \
gnupg-agent \
software-properties-common && \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && \
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable" && \
apt-get update -q && apt-get install -yq docker-ce docker-ce-cli containerd.io
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN set -eux; \
apt-get update -q && \
apt-get install -yq \
btrfs-progs \
e2fsprogs \
iptables \
xfsprogs \
xz-utils \
# pigz: https://github.com/moby/moby/pull/35697 (faster gzip implementation)
pigz \
# zfs \
wget
# set up subuid/subgid so that "--userns-remap=default" works out-of-the-box
RUN set -x \
&& addgroup --system dockremap \
&& adduser --system -ingroup dockremap dockremap \
&& echo 'dockremap:165536:65536' >> /etc/subuid \
&& echo 'dockremap:165536:65536' >> /etc/subgid
# https://github.com/docker/docker/tree/master/hack/dind
ENV DIND_COMMIT 37498f009d8bf25fbb6199e8ccd34bed84f2874b
RUN set -eux; \
wget -O /usr/local/bin/dind "https://raw.githubusercontent.com/docker/docker/${DIND_COMMIT}/hack/dind"; \
chmod +x /usr/local/bin/dind
##### Install nvidia docker #####
# Add the package repositories
RUN curl -fsSL https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add --no-tty -
RUN distribution=$(. /etc/os-release;echo $ID$VERSION_ID) && \
echo $distribution && \
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
tee /etc/apt/sources.list.d/nvidia-docker.list
RUN apt-get update -qq --fix-missing
RUN apt-get install -yq nvidia-docker2
RUN sed -i '2i \ \ \ \ "default-runtime": "nvidia",' /etc/docker/daemon.json
RUN mkdir -p /usr/local/bin/
COPY dockerd-entrypoint.sh /usr/local/bin/
RUN chmod 777 /usr/local/bin/dockerd-entrypoint.sh
RUN ln -s /usr/local/bin/dockerd-entrypoint.sh /
VOLUME /var/lib/docker
EXPOSE 2375
ENTRYPOINT ["dockerd-entrypoint.sh"]
#ENTRYPOINT ["/bin/sh", "/shared/dockerd-entrypoint.sh"]
CMD []
When I use exec to login into the Docker-in-Docker container, I can successfully run nvidia-smi (which previously return not found error then cannot run any GPU resource related docker run)
Welcome to pull my image at brandsight/dind:nvidia-docker
I wanted to install apache, php7, postgres12, node, java in ubuntu:18.04 base image dockerfile and wanted to show postgre data in php file. After building and running the container when I checked the process status, the port is missing. I started docker recently so I am new in this. Here is my Dockerfile
FROM ubuntu:18.04
ARG DEBIAN_FRONTEND=noninteractive
# # Install openjdk-8-jdk
RUN apt-get update && \
apt-get install -y openjdk-8-jdk
RUN apt-get -y install nodejs
RUN apt-get update && apt-get -qq -y install curl
RUN apt-get -y install apache2
RUN apt-get -y install php7.2
RUN apt-get -y install libapache2-mod-php7.2
RUN rm -f /var/www/html/index.html
COPY . /var/www/html
RUN apt-get update && apt-get install -y gnupg2 && apt-get install -y wget
Run wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main" > /etc/apt/sources.list.d/pgdg.list
RUN apt-get update && apt-get install -y postgresql-12 postgresql-client-12
USER postgres
RUN /etc/init.d/postgresql start &&\
psql --command "CREATE USER docker WITH SUPERUSER PASSWORD 'docker';" &&\
createdb -O docker docker
RUN echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/12/main/pg_hba.conf
RUN echo "listen_addresses='*'" >> /etc/postgresql/12/main/postgresql.conf
EXPOSE 80 5432
CMD ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"]
error
PS G:\Docker\test-docker-ubuntu-php\website> docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1fba1b8327fd test-custom-all "/usr/sbin/apache2ct…" 2 minutes ago Exited (1) 2 minutes ago test-custom
PS G:\Docker\test-docker-ubuntu-php\website> docker logs 1fba1b8327fd
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
Action '-D FOREGROUND' failed.
The Apache error log may have more information.
Can you help me with this?
Thanks
I installed Elasticsearch in my image based on ubuntu:16.04.
And start the service using
RUN service elasticsearch start
but, it was not started.
If I go into the container and run it, it starts.
I want to run the service and dump the index when I create the image, below is a part of my Dockerfile.
How do I start Elasticsearch in the Dockerfile?
#install OpenJDK-8
RUN apt-get update && apt-get install -y openjdk-8-jdk && apt-get install -y ant && apt-get clean
RUN apt-get update && apt-get install -y ca-certificates-java && apt-get clean
RUN update-ca-certificates -f
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME
#download ES
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
RUN apt-get install -y apt-transport-https
RUN echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-6.x.list
RUN apt-get update && apt-get install -y elasticsearch
RUN service elasticsearch start
The RUN command executes only during the build phase. It stops after the build is completed. You should use CMD (or ENTRYPOINT) instead:
CMD service elasticsearch start && /bin/bash
It's better wrapping the starting command in your own file and then only execute the file:
CMD /start_elastic.sh
I don't know why not take official oss image, but, this Docker file based on Debian work:
FROM java:8-jre
ENV ES_NAME=elasticsearch \
ELASTICSEARCH_VERSION=6.6.1
ENV ELASTICSEARCH_URL=https://artifacts.elastic.co/downloads/$ES_NAME/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz
RUN apt-get update && apt-get install -y --assume-yes openssl bash curl wget \
&& mkdir -p /opt \
&& echo '[i] Start create elasticsearch' \
&& wget -T 15 -O /tmp/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz $ELASTICSEARCH_URL \
&& tar -xzf /tmp/$ES_NAME-$ELASTICSEARCH_VERSION.tar.gz -C /opt/ \
&& ln -s /opt/$ES_NAME-$ELASTICSEARCH_VERSION /opt/$ES_NAME \
&& useradd elastic \
&& mkdir -p /var/lib/elasticsearch /opt/$ES_NAME/plugins /opt/$ES_NAME/config/scripts \
&& chown -R elastic /opt/$ES_NAME-$ELASTICSEARCH_VERSION/
ENV PATH=/opt/elasticsearch/bin:$PATH
USER elastic
CMD [ "/bin/sh", "-c", "/opt/elasticsearch/bin/elasticsearch --E cluster.name=test --E network.host=0 $ELASTIC_CMD_OPTIONS" ]
I believe most of the commands you'll be able to use on Ubuntu.
Don't forget to run sudo sysctl -w vm.max_map_count=262144 on your host
I have a docker file like this :
FROM ubuntu:12.04
MAINTAINER me <me#c.com>
RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install supervisor \
apache2 \
mysql-server \
php5 \
libapache2-mod-php5 \
php5-mysql \
php5-mcrypt
#ssh
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install openssh-server
RUN mkdir /var/run/sshd
RUN echo 'root:root' | chpasswd
RUN sed -i 's/PermitRootLogin without-password/PermitRootLogin yes/' /etc/ssh/sshd_config
RUN sed 's#session\s*required\s*pam_loginuid.so#session optional pam_loginuid.so#g' -i /etc/pam.d/sshd
ENV NOTVISIBLE "in users profile"
RUN echo "export VISIBLE=now" >> /etc/profile
EXPOSE 22 80
ADD ./supervisord.conf /etc/supervisor/conf.d/supervisord.conf
CMD ["/usr/bin/supervisord"]
My question is how do I add a couchdb server into this docker file?
I can get a built-in couchdb docker image from here : https://hub.docker.com/r/klaemo/couchdb/, but how do I create a image like this my self? I can't find any documentation regarding the process!
I spent 3 hours tried to googled but got no luck, so I will take the risk to ask even if this is a dump question!
Is there a specific version of couchdb that you want in your docker container?
If not, since you are using Ubuntu 12.04 as your base image, you can get the couchdb 1.0.1 binaries from the Ubuntu 12.04/precise [universe] repository easily by adding couchdb to your apt-get list like this:
FROM ubuntu:12.04
MAINTAINER me <me#c.com>
RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install supervisor \
apache2 \
mysql-server \
php5 \
libapache2-mod-php5 \
php5-mysql \
php5-mcrypt \
couchdb
#[--Rest of your dockerfile goes here unchanged--]
You can alternatively use the PPA maintained by the Apache CouchDB team to get latest stable releases for your base image based on the officially released tar balls. For this option you can use the following dockerfile:
# To install the ppa finder tool in your docker container
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install python-software-properties
RUN add-apt-repository ppa:couchdb/stable -y
RUN apt-get -y update
RUN DEBIAN_FRONTEND=noninteractive apt-get -y install supervisor \
apache2 \
mysql-server \
php5 \
libapache2-mod-php5 \
php5-mysql \
php5-mcrypt \
couchdb
#[--Rest of your dockerfile goes here unchanged--]
If you want the latest or a specific version of couchdb in your docker container then you might have to build couchdb from the source code. Note that this approach will require you to install many more packages (g++ erlang-dev erlang-manpages erlang-base-hipe erlang-eunit, libmozjs185-dev libicu-dev libcurl4-gnutls-dev libtool) onto your container to be able to build couchdb from source. However you may be able to purge/remove the packages that are required only to build couchdb. The complete list of dependencies can be found on the official couchdb build wiki on apache. If you really want THE latest version then you can refer to this dockerfile and add update your dockerfile accordingly. Here is a complete dockerfile [UNTESTED] for your ease of use:
FROM ubuntu:12.04
MAINTAINER me <me#c.com>
ENV COUCHDB_VERSION master
RUN groupadd -r couchdb && useradd -d /usr/src/couchdb -g couchdb couchdb
# download dependencies
RUN apt-get update -y -qq && apt-get install -y --no-install-recommends \
build-essential \
erlang-dev \
erlang-manpages \
erlang-base-hipe \
erlang-eunit \
erlang-nox \
erlang-xmerl \
erlang-inets \
libmozjs185-dev \
libicu-dev \
libcurl4-gnutls-dev \
libtool
RUN cd /usr/src && git clone https://git-wip-us.apache.org/repos/asf/couchdb.git \
&& cd couchdb && git checkout $COUCHDB_VERSION \
&& cd /usr/src/couchdb && ./configure && make
# You can optionally purge/remove the packages you installed to build the couchdb from source.
# permissions
RUN chmod +x /usr/src/couchdb/dev/run && chown -R couchdb:couchdb /usr/src/couchdb
USER couchdb
EXPOSE 5984 15984 25984 35984 15986 25986 35986
#[--Rest of your dockerfile can go here as required--]
I have create a apache server in docker, but the ip what i get in access log is just from my docker ip, not from remote client ip.
Is it possible to get remote ip?
I have try to use mod_remoteip and it is not works.
Dockerfile
FROM ubuntu:latest
ENV DEBIAN_FRONTEND noninteractive
# make sure the package repository is up to date and update ubuntu
RUN locale-gen de_DE.UTF-8
RUN echo "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
RUN apt-get update && apt-get install -y apt-transport-https ca-certificates
RUN apt-get update && apt-get install -y curl lsb-release supervisor openssh-server libapache2-mod-passenger git apache2 vim
# supervisor installation &&
# create directory for child images to store configuration in
RUN apt-get -y install supervisor && \
mkdir -p /var/run/sshd && \
mkdir -p /var/log/supervisor && \
mkdir -p /etc/supervisor/conf.d
# default command
CMD ["/usr/bin/supervisord"]