We are using Docker as part of our build pipeline. As we're using CentOS, the default Docker installation has set up devicemapper with a loop-lvm sparse file. Now that sparse file has reached it's size limit of 100G and we're not able to build any new images or containers. Due to the issues mentioned in Clean docker environment: devicemapper we cannot free any space by removing unused containers or images.
As I've learnt in the meantime, using loop-lvm is not a very good idea (http://www.projectatomic.io/blog/2015/06/notes-on-fedora-centos-and-docker-storage-drivers/) and we're planning the migration to direct-lvm. However, this will take some time to plan. Therefore I'm looking into possibilites to extend the currently used sparse file without loosing all data.
The current Docker docs have some instructions on how to extend the sparse file (https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#/for-a-loop-lvm-configuration). However, we are currently running Docker version 1.9 and the docs for this version do not contain these instructions. Any chance that they are still applicable for Docker 1.9?
Answering my own question: I was able to extend the loop-lvm file using the method described in the docs and did not encounter any issue so far.
Related
It seems that docker compose * mirrors many docker-compose * commands.
What is the difference between these two?
Edit: docker compose is not considered a tech preview anymore. The rest of the answer still stands as-is, including the not-yet-implemented command.
docker compose is currently a tech preview, but is meant to be a drop-in replacement for docker-compose. It is being built into the docker binary to allow for new features. It hasn't implemented one command yet, and has deprecated a few. These are quite rarely used though, in my experience.
The goal is that docker compose will eventually replace docker-compose, but no timeline for that yet and until that day you still need docker-compose for production.
Why they do that?
docker-compose is written in Python while most Docker developments are in Go. And they decided to recreate the project in Go with the same and more features, for better integration
Our build setup is backed into a large docker container (basically a 2 GB image coming with a complete X86 linux in itself).
We have two ways to actually build: the official approach is jenkins environment (running on X86 hardware). But we also have a little "side X86 server" running RH 7. Developers can log into that RH server and kick off specific builds (using said docker images) themselves.
Those RH servers will be shut down at some point, to be replaced with IBM Power8 machines (running RH7 Little Endian for power).
I am simply wondering: is there a chance that our existing build setup and docker images simply work on Power8? Or are the fundamental technical issues that make it unlikely and not even worth trying?
You can probably use your existing build methodology and scripts close to unchanged, but you'll need to rebuild the actual images.
You can't directly run x86 binaries on Power (at a very low level, the bytes of machine code are just different). Docker doesn't contain any sort of virtualization layer; it does a bunch of setup to isolate the container from the host, but then runs the binaries in an image directly.
If your Jenkins setup has enough parameters for image names and version tags, then you should be able to run the x86 and Power setups side-by-side; you need to encode the architecture somewhere in the built image name or tag; for instance, repo.example.com/app/build:20180904-power. (I don't know that one or the other is considered better if you control all of the machinery.) If you have a private repo, you could encode it earlier in the path, winding up with image names like repo.example.com/power/build:20180904.
You'd need to double-check that everywhere that has a Docker image reference has it correctly parameterized (which is a good practice anyways). That would include any direct docker run commands; any Docker Compose or Kubernetes YAML files or similar artifacts; and the FROM line of any Dockerfiles.
Existing build setup? Not sure!
Docker images? NO, don’t even try.
Docker images are actually multiple layers which stored on filesystem through corresponding storage driver and backing filesystem(shown in the output of docker info).
If storage driver/backing filesystem has been changed, which likely be true when OS changed, older docker images could not be valid any more. Meaning they must be rebuilt for sure.
I can't understand one thing. I read about images and AUFS file system and I think that I got it. However, when I look at iso file on ubuntu site it is meaningfully more than 100MB. Where is key ? In graphical enviroment? (eg. KDE)
Docker Images are minimal meaning, they contain only a few number of libraries (needed libraries). They don't include kernel, because containers use docker host's kernel.
You can download and inspect official ubuntu cloud image (source of library/ubuntu yekkety) from here.
Another thing to note: Base images usually don't include window managers and desktop environments.
I am very new to Docker and currently trying to get my head around if there is any best practice guide to update software that runs inside a docker container in a very large distributed environment. I already found couple of posts around updating a MySQL database in docker, etc. It gives a good hint for any software that stores data, but what if you want to update other parts or your own software package or services that are distributed and used by several other docker images through docker-compose?
Is there someone with real life experience doing that in such an environment who can help me or other newbies to understand the best practices in docker if there are any.
Thanks for your help!
You never update software in a running container. You pull down a new version from the hub. If we assume you're using the latest tag (which is a bad idea, always pin your versions) of your image and it's one of the official library images or the publicly available that uses automated builds you'll get the latest version of the container image when you pull the image.
This assume you've also separated the data out of your container either as a host volume or using the data container pattern.
The container should be considered immutable, if you change it's state it's no longer a true version of the image.
I'm new to Docker and trying to get my head around extending existing Images.
I understand you can extend an existing Docker image using the FROM command in a Dockerfile (e.g. How to extend an existing docker image?), but my question is -- in general, how can I install additional software / packages without knowing what the base operating system is of the base image or which package manager is available?
Or am I thinking about this the wrong way?
The best practice is to run the base image you want to start FROM (perhaps using docker exec) and see what package managers are available (if any). Then you can write your Dockerfile with the correct software installation procedure.
Think of it the same way you'd add software to any computer: you'd either log into it yourself and poke around, or write an installation program that can handle all of the expected variations.
In most cases, the source Dockerfile is provided and you can walk the chain backwards and gain a better understanding as you do.
For example, if we look at the official Redis image we see the information tab says
Supported tags and respective Dockerfile links
2.6.17, 2.6 (2.6/Dockerfile)
2.8.19, 2.8, 2, latest (2.8/Dockerfile)
So if you're interested in building off redis:latest you'd follow the second link and see that it in turn is built off of debian:wheezy.
Most user-created images will either include their Dockerfile on the hub page or from a link there.