I want to mount the local directory of a project to docker container before I used COPY command but when I make changes I have to rebuild those parts which involve some installation from bash scripts.
This is my docker-compose file
version "3.7"
services
tesseract:
container_name: tesseract
build:
context: ./app/services/tesseract/
dockerfile: Dockerfile
volumes:
- ./app/services/tesseract:/tesseract/
I don't have any errors when building and my WORKDIR tesseract is empty when i run container
This is my Dockerfile
FROM ubuntu:19.10
ENV DEBIAN_FRONTEND=noninteractive
ENV TESSERACT=/usr/share/tesseract
WORKDIR /tesseract
RUN apt-get update && apt-get install -y \
build-essential \
software-properties-common \
python3.7 \
python3-pip \
cmake \
autoconf \
automake \
libtool \
pkg-config \
libpng-dev \
tesseract-ocr \
libtesseract-dev \
libpango1.0-dev \
libicu-dev \
libcairo2-dev \
libjpeg8-dev \
zlib1g-dev \
libtiff5-dev \
wget \
git \
g++ \
vim
RUN git clone https://github.com/tesseract-ocr/tesseract $TESSERACT
COPY . /tesseract/
RUN chmod +x scripts/*
RUN scripts/compile_tesseract.sh
RUN scripts/langdata_lstm.sh scripts/start.sh
RUN pip3 install -r requirements.txt
ENV TESSDATA_PREFIX=/usr/share/tesseract/tessdata
Main objective of docker volume is
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers.
which means volumes are used to persist the data outside the lifecycle of a container. If you want to COPY a file or a directory into a container, please use COPY instruction.
If you’re copying in local files to your Docker image, always use COPY because it’s more explicit.
With Docker-compose, you can use a bind mount volume
https://docs.docker.com/compose/gettingstarted/#step-5-edit-the-compose-file-to-add-a-bind-mount#step-5-edit-the-compose-file-to-add-a-bind-mount
Related
Trying to run a decentralized vpn inside of a docker image. The issue im running to I think has to do with my running and installed kernels are different. I have to run modprobe wireguard in my docker file and its returning modprobe: FATAL: Module wireguard not found in directory /lib/modules/5.10.25-linuxkit I know the issue is related to the running kernels but im not sure what the fix would be. Heres my current Dockerfile.
FROM ubuntu:20.04
USER root
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \
git \
gcc \
make \
musl-dev \
unbound \
libtool \
g++ \
file \
automake \
autoconf \
libssl-dev \
libexpat-dev \
bison \
systemd \
iproute2 \
sudo \
wireguard-tools
RUN systemctl enable systemd-resolved
COPY . /sentinelnode /bin/
COPY . /.sentinelnode /root/
COPY . /hnsd /bin/
RUN chmod +x /bin/sentinelnode
RUN chmod +x /bin/hnsd
RUN sudo modprobe wireguard
RUN cd $HOME
CMD sentinelnode start
This question already has answers here:
Docker: Copying files from Docker container to host
(27 answers)
Closed 1 year ago.
I have a docker file
FROM ubuntu:20.04
################################
### INSTALL Ubuntu build tools and prerequisites
################################
# Install build base
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \
build-essential \
git \
subversion \
sharutils \
vim \
asciidoc \
binutils \
bison \
flex \
texinfo \
gawk \
help2man \
intltool \
libelf-dev \
zlib1g-dev \
libncurses5-dev \
ncurses-term \
libssl-dev \
python2.7-dev \
unzip \
wget \
rsync \
gettext \
xsltproc && \
apt-get clean && rm -rf /var/lib/apt/lists/*
ARG FORCE_UNSAFE_CONFIGURE=1
RUN git clone https://git.openwrt.org/openwrt/openwrt.git
WORKDIR /openwrt
RUN ./scripts/feeds update -a && ./scripts/feeds install -a
COPY .config /openwrt/.config
RUN mkdir files
WORKDIR /files
RUN mkdir etc
WORKDIR /etc
RUN mkdir uci-defaults
WORKDIR /uci-defaults
COPY xx_custom /openwrt/files/etc/uci-defaults/xx_custom
WORKDIR /openwrt
RUN make -j 4
RUN ls /openwrt/bin/targets/ramips/mt76x8
WORKDIR /root
CMD ["bash"]
I want to copy all the files inside the folder mt76x8 to the host. I want to that inside the dockerfile so that when I run the docker file I should get the generated files in my host.
How can I do that?
you can use the volume mount to access the docker-generated artifacts on the host machine.
you can also run the command
docker cp to copy the files to the host machine.
if don't want to use the docker command as mention only option is to use the volume.
you can also use docker create once the docker image is ready to create the writable layer and copy data.
You have two choices.
Use docker volumes to map the /openwrt/bin/targets/ramips/mt76x8 folder when you are running the container. i.e. docker run -v {VoluneName}:/openwrt/bin/targets/ramips/mt76x8. All of the files in the mt76x8 folder would be available in the volume folder. If you are using Linux then you will find the docker volumes in /var/lib/docker/volumes/
You can use docker cp command to copy data from container to the host machine. Here is an example
I'm new using dockers and trying to create a volume between Windows directory and Debian Container.
although the dockerfile does not have WORKDIR, I decided use default path inside container, for example, /home/ or /, whatever folder, path.
Is necessary create a WORKDIR to use VOLUME? I think no! what I need to do?
docker run --rm -v %echo%:/home/ --name rust_container rust
dockerfile:
FROM debian:buster-slim
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates \
gcc \
libc6-dev \
wget \
; \
\
url="https://static.rust-lang.org/rustup/dist/x86_64-unknown-linux-gnu/rustup-init"; \
wget "$url"; \
chmod +x rustup-init; \
./rustup-init -y --no-modify-path --default-toolchain nightly; \
rm rustup-init; \
chmod -R a+w $RUSTUP_HOME $CARGO_HOME; \
rustup --version; \
cargo --version; \
rustc --version; \
\
apt-get remove -y --auto-remove \
wget \
; \
rm -rf /var/lib/apt/lists/*;
You don’t need to set the workdir to use a volume, the workdir will default to the root folder. The directory you reference in the -v argument needs to exist in the container folder structure though because you are mounting a volume. So if /home/ doesn’t exist there is nothing for the volume to mount to. If you include a host point like /foo/bar:/home you will create a bind mount not a volume which mounts the host file system directory inside the container.
docker image inspect <name>
gives me 16GB
and about 20 layers
When I am logged as root, this
du -hs /
show me just 2GB
FYI, there are already very multi-lines RUN commands in Dockerfile.
can I squash all layers into one layer without touching Dockerfile, rebuilding etc?
or possibly by adding extra action to Dockerfile which clear/improve caching
Dockerfile is
FROM heroku/heroku:18
ENV PYENV_ROOT="/pyenv"
ENV PATH="/pyenv/shims:/pyenv/bin:$PATH"
ENV PYTHON_VERSION 3.5.6
ENV GPG_KEY <value>
ENV PYTHONUNBUFFERED 1
ENV TERM xterm
ENV EDITOR vim
RUN apt-get update && apt-get install -y \
build-essential \
gdal-bin \
binutils \
iputils-ping \
libjpeg8 \
libproj-dev \
libjpeg8-dev \
libtiff-dev \
zlib1g-dev \
libfreetype6-dev \
liblcms2-dev \
libxml2-dev \
libxslt1-dev \
libssl-dev \
libncurses5-dev \
virtualenv \
python-pip \
python3-pip \
python-dev \
libmysqlclient-dev \
mysql-client-5.7 \
libpq-dev \
libcurl4-gnutls-dev \
libgnutls28-dev \
libbz2-dev \
tig \
git \
vim \
nano \
tmux \
tmuxinator \
fish \
sudo \
libnet-ifconfig-wrapper-perl \
ruby \
libssl-dev \
nodejs \
strace \
tcpdump \
# npm & grunt
&& curl -L https://npmjs.com/install.sh | sh \
&& npm install -g grunt-cli grunt \
# ruby & foreman
&& gem install foreman \
# installing pyenv
&& curl https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash
COPY . /app
COPY ./requirements /requirements
COPY ./requirements.txt /requirements.txt
COPY ./docker/docker_compose/django/foreman.sh /foreman.sh
COPY ./docker/docker_compose/django/Procfile /Procfile
COPY ./docker/docker_compose/django/entrypoint.sh /entrypoint.sh
# ADD sudoer user django with password django
RUN groupadd -r django -g 1000 && \
useradd -ms /usr/bin/fish -p $(openssl passwd -1 django) --uid 1000 --gid 1000 -r -g django django && \
usermod -a -G sudo django && \
chown -R django:django /app
COPY --chown=django:django ./docker/docker_compose/django/fish /home/django/.config/fish
COPY --chown=django:django ./docker/docker_compose/django/tmuxinator /home/django/.tmuxinator
COPY ./docker/docker_compose/django/fish /root/.config/fish
WORKDIR /app
RUN sed -i 's/\r//' /entrypoint.sh \
&& sed -i 's/\r//' /foreman.sh \
&& chmod +x /entrypoint.sh \
&& chown django /entrypoint.sh \
&& chmod +x /foreman.sh \
&& chown django /foreman.sh \
&& chown -R django:django /home/django/ \
&& pyenv install ${PYTHON_VERSION%%} \
&& mkdir -p /app/log \
&& pyenv global ${PYTHON_VERSION%%} \
&& pyenv rehash \
&& ${PYENV_ROOT%%}/versions/${PYTHON_VERSION%%}/bin/pip install -U pip \
&& ${PYENV_ROOT%%}/versions/${PYTHON_VERSION%%}/bin/pip install -r /requirements.txt \
&& chown -R django:django /pyenv/ \
&& ${PYENV_ROOT%%}/versions/${PYTHON_VERSION%%}/bin/pip install -r /requirements/dev_requirements.txt
# this user receives ENVs from the top
USER django
ENTRYPOINT ["/entrypoint.sh"]
What I've tried so far:
The --squash option from experimental mode of docker build is rather not for me. That Dockerfile is one of more Dockerfiles inside docker-compose.
I've also checked this:
https://github.com/jwilder/docker-squash
but seems docker load cannot load a squashed image.
also, that squash gives me 8GB (still far away from expected ~2GB)
docker save <image_id> | docker-squash -t latest_tiny | docker load
update after answers:
when I've added this:
&& apt-get autoremove \ # ? to consider
&& apt-get clean \ # ? to consider
&& rm -rf /var/lib/apt/lists/*
to apt-get and --no-cache-dir to each pip, the result was 72GB (yes, even much more - docker images shows 36GB before pip command, and 72GB as final size).
my working directory is clear (regarding COPY). du -hs / (as a root) still has 2GB. And all images were removed before rebuilding.
Following the #Mihai approach, I was able to slim down the image from 16GB to 9GB.
There is a simple trick to get rid of the intermediate layers. It will bring down the size as well but with how much depends on how it was built.
Create a Dockerfile like this:
FROM your_image as initial
FROM your_image_base
COPY --from=initial / /
your_image_base should be something like 'alpine' - so the smallest image from which your image and its parents descend from.
Now build the image and check the history and size:
docker build -t your-image:2.0 .
docker image history your-image:2.0
docker image ls
This way you do create a new Dockerfile (if that is acceptable for your process) without touching the initial Dockerfile.
Let me know if this solves your issue.
UPDATE AFTER SEEING THE Dockerfile:
maybe I miss it but I don't see you cleaning up the apt-get cache after you perform the installations. Your big RUN command should end with "&& rm -rf /var/lib/apt/lists/*" on the same line so that it doesn't store the whole cache on the layer.
Definitely add && rm -rf /var/lib/apt/lists/* on the end of your main run command, like Mihai said. Another thing that may help (depending on how big your dependencies are) is installing with pip using the --no-cache-dir option . Also, make sure you understand build context and consider using either a .dockerignore or sending the context to another directory (totally depends on how you're directory is setup)
I've also had luck exploring an image using dive. Honestly this looks like a pretty big image so not sure how much you're going to be able to get it down
To squash a (Docker) container image, without re-building the image or manipulating the original Dockerfile,
You can extend from your image and squash it:
docker build --squash -t your_image_squashed - <<< "FROM your_image"
It's very easy, just use
docker commit YOUR_CONTAINER_ID NEW_IMAGE_ID
The docker will throw away the intermediate layers, you lost history but the size is small
I tried to installed a LAMP stack with a DockerFile in a directory on my desktop /home/username/Desktop/docker/lamp/:
FROM ubuntu:16.04
VOLUME ["/var/www"]
RUN apt-get update && \
apt-get dist-upgrade -y && \
apt-get install -y \
apache2 \
php7.0 \
php7.0-cli \
libapache2-mod-php7.0 \
php7.0-gd \
php7.0-json \
php7.0-ldap \
php7.0-mbstring \
php7.0-mysql \
php7.0-pgsql \
php7.0-sqlite3 \
php7.0-xml \
php7.0-xsl \
php7.0-zip \
php7.0-soap
COPY apache_default /etc/apache2/sites-available/000-default.conf
COPY run /usr/local/bin/run
RUN chmod +x /usr/local/bin/run
RUN a2enmod rewrite
EXPOSE 80
CMD ["/usr/local/bin/run"]
Then on my terminal, I run it:
$ docker build -t docker-lamp .
Step 4 : COPY apache_default /etc/apache2/sites-available/000-default.conf
lstat apache_default: no such file or directory
What I have done wrong? How can I fix this?
First of all, are you 100% sure the files apache_default and run are in the same directory as the Dockerfile?