Docker Codes
# Import Ubuntu image to Docker
docker pull ubuntu:16.04
docker run -it ubuntu:16.04
# Instsall Python3 and pip3
apt-get update
apt-get install -y python3 python3-pip
# Install Selenium
pip3 install selenium
# Install BeautifulSoup4
pip3 install beautifulsoup4
# Install library for PhantomJS
apt-get install -y wget libfontconfig
# Downloading and installing binary
mkdir -p /home/root/src && cd &_
tar jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2
cd phantomjs-2.1.1-linux-x86_64/bin/
cp phantomjs /usr/local/bin/
# Installing font
apt-get install -y fonts-nanum*
Question
I am trying to import Ubuntu image to docker and install serveral packages inscluding python3, pip3, bs4, and PhantomJs. Then I want to save all this configurations in Docker as "ubuntu-phantomjs". As I am currently on Ubuntu image, anything that starts with 'docker' command do not work. How could I save my image?
Here is the dockerfile:
# Import Ubuntu image to Docker
FROM ubuntu:16.04
# Install Python3, pip3, library and fonts
RUN apt-get update && apt-get install -y \
python3 \
python3-pip \
wget libfontconfig \
fonts-nanum*
&& rm -rf /var/lib/apt/lists/*
RUN pip3 install selenium beautifulsoup4
# Downloading and installing binary
RUN mkdir -p /home/root/src && cd &_ tar jxvf phantomjs-2.1.1-linux-x86_64.tar.bz2 && cd phantomjs-2.1.1-linux-x86_64/bin/ && cp phantomjs /usr/local/bin/
Now after saving the code in file named dockerfile, open a terminal in the same directory as the one where file is stored, and run following command:
$ docker build -t ubuntu-phantomjs .
-t means that the target is ubuntu-phantomjs and . means that the context for docker is the current directory. The above dockerfile is not a standard one, and does not follow all good practices mentioned here. You can change this file according to your needs, read the documentations for more help.
Related
I am using the dev image so i can have the cuda compiler, now the issue is that when running the CI, as below I get that error, but if I build the standard container (commented line in dockerfile).
CONTAINER=$(docker run -d gpu-test)
docker exec $CONTAINER pytest
OCI runtime exec failed: exec failed: unable to start container process: exec: "pytest": executable file not found in $PATH: unknown
Dockerfile:
# Pulls the basic Image from NVIDIA repository
FROM rapidsai/rapidsai-dev:22.04-cuda11.5-devel-ubuntu20.04-py3.9
# FROM rapidsai/rapidsai:22.04-cuda11.5-runtime-ubuntu20.04-py3.9
# Updates OS libraries
RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
--fix-missing git python3-setuptools python3-pip build-essential libcurl4-gnutls-dev \
zlib1g-dev rsync vim nano cmake tabix
# Install libraries needed in the examples
RUN /opt/conda/envs/rapids/bin/pip install \
scanpy==1.9.1 wget pytabix dash-daq \
dash-html-components dash-bootstrap-components dash-core-components \
utils pytest
RUN /opt/conda/envs/rapids/bin/pip install \
torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116
WORKDIR /workspace
ENV HOME /workspace
I have the following Dockerfile:
FROM --platform=linux/x86_64 nvidia/cuda:11.7.0-devel-ubuntu20.04
COPY app ./app
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -y upgrade && apt-get install -y apt-utils
RUN apt-get install -y \
net-tools iputils-ping \
build-essential cmake git \
curl wget vim \
zip p7zip-full p7zip-rar \
imagemagick ffmpeg \
libomp5
# RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
COPY Miniconda3-latest-Linux-x86_64.sh .
RUN chmod guo+x Miniconda3-latest-Linux-x86_64.sh
RUN bash Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3
RUN export PATH=~/miniconda3/bin:$PATH
RUN conda --version
RUN conda update -n base conda
RUN conda create -y --name servier python=3.6
RUN conda activate servier
RUN conda install -c conda-forge rdkit
CMD ["bash"]
When I run: docker image build -t image_test_cuda2 . it breaks in the RUN conda --version.
The error is RUN conda --version: ... /bin/sh: 1: conda: not found. The problem is that RUN export PATH=~/miniconda3/bin:$PATH is not working. It is not creating conda link in the PATH.
If I build the image until RUN bash Miniconda3-latest-Linux-x86_64.sh -b -p ~/miniconda3 and manually I get access to the container using docker exec -it <id> /bin/bash and then from the #/ manually I run the same command #/export PATH=~/miniconda3/bin:$PATH it works good. If I manually run the next command inside the container RUN conda update -n base conda it works good.
The conclusion is that it seems that the command RUN export PATH=~/miniconda3/bin:$PATH is not working in Dockerfile - docker image build. How to solve this issue?
I am trying to use Selenium on one of my airflow tasks. I have airflow running on apache/airflow:2.1.4 docker image.
I get the following error when I use selenium in my airflow task (since I'm missing firefox)
FileNotFoundError: [Errno 2] No such file or directory: 'firefox': 'firefox'
How would I go about adding geckodriver and firefox to the airflow image?
I have my docker-compose build the following Dockerfile for airflow,
FROM apache/airflow:2.1.4
WORKDIR /python_dependencies
COPY ./requirements.txt .
RUN pip3 install -r requirements.txt
apache/airflow:2.1.4 is based on debian, so you just need to use apt-get install firefox-esr to get firefox command, while download pre-built binary from geckodriver github to install geckodriver.
Dockerfile:
FROM apache/airflow:2.1.4
USER root
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
ca-certificates curl firefox-esr \
&& rm -fr /var/lib/apt/lists/* \
&& curl -L https://github.com/mozilla/geckodriver/releases/download/v0.30.0/geckodriver-v0.30.0-linux64.tar.gz | tar xz -C /usr/local/bin \
&& apt-get purge -y ca-certificates curl
USER airflow
Verify:
$ docker build -t abc:1 .
$ docker run --rm -it --entrypoint=which abc:1 firefox
/usr/bin/firefox
$ docker run --rm -it --entrypoint=which abc:1 geckodriver
/usr/local/bin/geckodriver
I just made a very simple Docker file in my terminal, basically I did the following:
mkdir pgrouted
cd pgrouted
touch Dockerfile
Now I open the Docker file in the nano editor, and I add the following commands to the Docker file:
FROM ubuntu
MAINTAINER Gautam <gautamx07#yahoo.com>
LABEL Description="pgrouting excercise" Vendor="skanatek" Version="1.0"
ENV BBOX="-122.8,45.4,-122.5,45.6"
# Add pgRouting launchpad repository
RUN sudo apt-add-repository -y ppa:ubuntugis/ppa
RUN sudo apt-add-repository -y ppa:georepublic/pgrouting
RUN sudo apt-get update
# Install pgRouting package (for Ubuntu 14.04)
RUN sudo apt-get install postgresql-9.3-pgrouting
# Install osm2pgrouting package
RUN sudo apt-get install osm2pgrouting
# Install workshop material (optional, but maybe slightly outdated)
RUN sudo apt-get install pgrouting-workshop
# For workshops at conferences and events:
# Download and install from http://trac.osgeo.org/osgeo/wiki/Live_GIS_Workshop_Install
RUN wget --no-check-certificate https://launchpad.net/~georepublic/+archive/pgrouting/+files/pgrouting-workshop_2.0.6-ppa1_all.deb
RUN sudo dpkg -i pgrouting-workshop_2.0.6-ppa1_all.deb
# Review: Not sure weather this should be in the dockerfile
RUN cp -R /usr/share/pgrouting/workshop ~/Desktop/pgrouting-workshop
# Log in as user "user"
RUN psql -U postgres
# Create routing database
RUN CREATE DATABASE routing;
# Add PostGIS functions
RUN CREATE EXTENSION postgis;
# Add pgRouting core functions
CREATE EXTENSION pgrouting;
# Download using Overpass XAPI (larger extracts possible than with default OSM API)
wget --progress=dot:mega -O "sampledata.osm" "http://www.overpass-api.de/api/xapi?*[bbox=${BBOX}][#meta]"
The entire Dockerfile can be see HERE at a glance.
Now when I try to build the Dockerfile, like so:
docker build -t gautam/pgrouted:v1 .
The Dockerfile runs and then I get the below error:
Step 4 : RUN sudo apt-add-repository -y ppa:ubuntugis/ppa
---> Running in c93c3c5fd5e8
sudo: apt-add-repository: command not found
The command '/bin/sh -c sudo apt-add-repository -y ppa:ubuntugis/ppa' returned a non-zero code: 1
Why am I getting this error?
apt-add-repository is just not in the base Ubuntu image. You'll first need to install it. try apt-get install software-properties-common
By the way, you don't need to use sudo in the Dockerfile because the commands run as root by default unless you change to another user with the USER command.
Add these lines before running apt-add-repository command
RUN apt-get update && \
apt-get install -y software-properties-common && \
rm -rf /var/lib/apt/lists/*
thats worked for me:
RUN apt-get update --fix-missing && \
apt-get install -y software-properties-common && \
rm -rf /var/lib/apt/lists/* && \
add-apt-repository ppa:ondrej/php && \
apt install -y nginx php7.4-fpm php7.4-mysql php7.4-curl net-tools telnet php7.4-gd php-mail php7.4 php7.4-common php7.4-sqlite3 php7.4-curl php7.4-intl php7.4-mbstring php7.4-xmlrpc php7.4-mysql php7.4-gd php7.4-xml php7.4-cli php7.4-zip php7.4-soap unzip && \
rm -rf /var/lib/apt/lists/* && \
apt clean
currently I try to launch docker image on deamon with docker run -d ID (after to launch this commande: docker build -t toto .)
But when I launch this commande: docker exec -it ID bash, I've got this error:
Error response from daemon: no such id: toto
My Dockerfile look like that:
# Dockerfile
FROM debian:jessie
# Upgrade system
RUN apt-get update && apt-get dist-upgrade -y --no-install-recommends
# Install TOR
RUN apt-get install -y --no-install-recommends tor tor-geoipdb torsocks
# INSTALL POLIPO
RUN apt-get update && apt-get install -y polipo
# INSTALL PYTHON
RUN apt-get install -y python2.7 python-pip python-dev build-essential libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev libffi-dev libxslt-dev libxml2-dev && apt-get clean
# INSTALL GIT
RUN apt-get install -y git-core
# INSTALL NANO
RUN apt-get install -y nano
# INSTALL SUPERVISOR
RUN apt-get install -y supervisor
# INSTALL SCRAPY and dependencies
RUN pip install lxml && pip install pyopenssl && pip install Scrapy && pip install pyopenssl && pip install beautifulsoup4 && pip install lxml && pip install elasticsearch && pip install simplejson && pip install requests && pip install scrapy-crawlera && pip install avro && pip install stem
# INSTALL CURL
RUN apt-get install -y curl
# Default ORPort
EXPOSE 9001
# Default DirPort
EXPOSE 9030
# Default SOCKS5 proxy port
EXPOSE 9050
# Default ControlPort
EXPOSE 9051
# Default polipo Port
EXPOSE 8123
# Configure Tor and Polopo
RUN echo 'socksParentProxy = "localhost:9050"' >> /etc/polipo/config
RUN echo 'socksProxyType = socks5' >> /etc/polipo/config
RUN echo 'diskCacheRoot = ""' >> /etc/polipo/config
RUN echo 'ORPort 9001' >> /etc/tor/torrc
RUN echo 'ExitPolicy reject *:*' >> /etc/tor/torrc
ENV PYTHONPATH $PYTHONPATH:/scrapy
WORKDIR /scrapy
VOLUME ["/scrapy"]
Thanks in advance.
In order to facilitate the usage of docker exec, make sure you run your container with a name:
docker run -d --name aname.cont ...
I don't see an entrypoint or exec directove in the Dockerfile, so do mention what you want to run when using docker run -d
(I like to add '.cont' as a naming convention, to remember that it is a container name, not an image name)
Then a docker exec aname.cont bash should work.
Check that the container is still running with a docker ps -a
When creating the container you should use the image name:
docker run -d --name my_toto toto
You cannot impose an ID when creating it. It's Docker who assigns the ID.
Then connecting
docker exec -it my_toto bash
A more quick way to do that is running directly
docker run -d -it -name my_toto toto bash
The container will still existing after you exit.