I've multiple docker containers that host some flask apps which runs some machine learning services. Let's say container 1 is using pytorch, and container 2 is also using pytorch. When I build image, both pytorch take up some size on disk. For some reason, we split these 2 services into different containers, if I insist on this way, is it possible to only build pytorch once so that both container can import it? Thanks in advance, appreciate any help and suggestions!
You can build one docker image and install pytorch on it. Then use that image as base image for those two codes. In this way, pytorch only takes hard space once. And you save time no installing pytorch twice.
You can also build only one image, copy your codes in two different directories,
for example /app1 and /app2. Then in your docker compose, change work directory for each app.
Related
The NVIDIA NGC container catalog has a broad range of GPU-optimised containers for common activities such as deep learning. How does one find what is inside the Docker images?
For example, I require an image with Pytorch 1.4 and Python 3.6 or 3.7, but the Pytorch tags go from pytorch:17.10 to pytorch:21.06-py3 (where xx.xx is the container version). Is there somewhere a list of what is installed in each container, or even better the container Dockerfile that was used to build the images?
You can do a high level inspection of the image using:
docker history <IMAGE> also at one point I've used this tool:
https://github.com/wagoodman/dive which was quite nice in order to inspect different layers.
So basically you can inspect each layer to see the instructions used in order to build that specific image and search for commands that have been used to install different packages
The details of pytorch NGC containers are listed at PyTorch Release Notes at the bottom of the pytorch NGC page.
All other deep learning frameworks related documentation is also at NVIDIA Deep Learning Frameworks.
I have installed and configured a system in EC2s using Ansible. It is 1 EC2 master with a few EC2 workers. Sometimes when I use ansible to update or reinstall configuration, it fails because either some package has been removed from open-source repositories, or the package is updated so not compatible with some other packages. And I learned that using docker-container can resolve these kind of configuration problems.
However, according to what I learned, each docker image will create image of one application (I guess one application means one process). But mine is a system which has airflow master webserver, airflow worker webserver, flower webserver, rabbitmq, airflow celery, several configuration files, etc. how can I create docker images for that? Should I create one docker image for each process? How do I know which linux folder should I go to create each docker image? How do I know which applications/processes I need to create? And how to combine these images to make them work together as a system?
Or maybe in my case I should not use docker image, Instead I should just create an EC2 image?
Use docker-compose.
Compose is a tool for defining and running multi-container Docker applications
https://docs.docker.com/compose/
each docker image will create image of one application (I guess one application means one process)
That is basically correct. You should create one docker-container per application. In theory you can have multiple process per container, but that doesn't matter in this case.
how can I create docker images for that?
In your case you should make one docker-container for airflow master webserver, one for airflow worker webserver, one for flower webserver, etc. And the you use a docker-compose.yml to link them all together.
Should I create one docker image for each process?
generally yes. (It may depend on your exact setup though)
And how to combine these images to make them work together as a system?
docker-compose.
How do I know which linux folder should I go to create each docker image?
I don't understand that question
How do I know which applications/processes I need to create?
You could create a deployment-diagram and then start from there.
what are the advantages of using Layer in Docker particularly at the aspect of "running", not "building"?
I read Docker Docs, googled a lot, saw the article ( What are the advantages of having layers in a docker image )
Then, I figured out what the advantages of using Layers are when I build images.
But I don't know what the advantages are, at the aspect of running containers based on the images.
For example(when I build an Image),
There is the following line in my Dockerfile
FROM python:3.7-slim
If I have the python:3.7-slim image, I don't have to download another python image, so I can build it more efficiently and faster.
But it is the advantages about "building process". I want to know the advantages about "running".
What are the advantages of using Layers in Docker at the aspect of "running"?
There is no advantage at run time, even more it can be seen as a disadvantage.
Layers is a feature provided by union file systems. As you figured out, it is quite convenient to speed up docker image builds. But such a file system is slower than regular file system when it comes to I/O speed.
To overcome this drawback, there are docker volumes which are mount points within a container making use of a normal file system. Docker volumes also helps with persisting data because the data they hold is not on the container's union file system and thus, can survive container deletion (container union file system deletion).
I have been trying to train a 3DCNN network with a specific architecture. I wanted to create a dockerfile with all the steps necessary to have the network working. The issue is that If I run the neural network network in the host I have no problem, everything works fine. But doing almost the same on a docker container I always get the "segmentation fault (core dumped)" error.
Both installations are not exactly the same but the variations (maybe some extra package installed) shouldn't be a problem, right? Besides I don't have any error until it starts iterating, so it seems like is a memory problem. The GPU works on the docker container and is the same GPU as the host. the python code is the same.
The Docker container neural network network start training with the data but on the epoch 1 it gets the "segmentation fault (core dumped)".
So my question is the following: Is it possible to have critical differences between the host and a docker container even if they have exactly the same packages installed? Especially with relation to tensorflow and GPU. Because the error must be from outside the code, given that the code works in a similar environment.
Hope I explained myself enough to give the idea of my question, thank you.
A docker image will resolve, at runtime, will resolve its system calls by the host kernel.
See "How can Docker run distros with different kernels?".
In your case, your Error is
Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1, SSE4.2
See "How to compile Tensorflow with SSE4.2 and AVX instructions?"
(referenced by tensorflow/tensorflow issue 8037)
You could try and build an image from a Tensorflow built from source, using a docker multi-stage build.
I want to create separated containers with a single service in each (more or less). I am using the php7-apache image which seems to use a base image of debian:jessie, php7 and apache. Since apache and php in this case are pretty intertwined I don't mind using this container.
I want to start adding other services to their own containers (git for example) and was considering using a tiny base image like busybox or alpinebox for these containers to keep image size down.
That said, I have read that using the same base image as other containers only gives you the 'penalty' of the one time image download of the base OS (debian jessie) which is then cached - while using tiny OSes in other containers will download those OSes on top of the base OS.
What is the best practice in this case? Should I use the same base image (debian jessie) for all the containers in this case?
You may want to create a base image from scratch. Create a base image from scratch.
From docker documentation
You can use Docker’s reserved, minimal image, scratch, as a starting point for building containers. Using the scratch “image” signals to the build process that you want the next command in the Dockerfile to be the first filesystem layer in your image.
While scratch appears in Docker’s repository on the hub, you can’t pull it, run it, or tag any image with the name scratch. Instead, you can refer to it in your Dockerfile. For example, to create a minimal container using scratch:
This example creates the hello-world image used in the tutorials. If you want to test it out, you can clone the image repo