I have two docker continers in privileged mode. And I installed LVM software in both. Its working fine, but the problem is when I create any volume group or logical volume in one disk suppose /dev/sdd1, I am not able to see into another container those volume groups. I am able to mount /dev/sdd1 in both.
It may be LVM-related problem.
Try to point at your PV (or VG) in second container manually: pvscan, vgscan /dev/sdd1
Also, LVs inside second container may be disabled
Commands to show them: lvs -a, lvdisplay -a
Last one useful to determine LV Status
So, enable it if not: lvchange -ay <VG name>/<LV name>
PS: please be specific. What not able to see means?
Related
We use containers to provision storage on our storage nodes but I can't for the life of my figure out a way to mount a device to the bare metal OS from a container. Both bare metal and containers are running oracle linux 7.5.
We cannot use ssh in any form for this. This is an isolated compute environment and the only access is thru the orchestration we use to manage containers.
I'm mainly a solaris guy so wondering if there is any linux magic I can work here.
I can mount any bare metal devices or filesystems into the container and I can run the container in privileged more.
Thx for any help
* clarification *
This is not about mounting a volume into a container.
This container is a temporary provisioning container, ie: it does stuff like mount iscisi volumes, create volume groups, create logical volumes and make filesystems.
This part is all working fine.
The last step this container needs to do is somehow tell the BARE METAL OPERATING SYSTEM TO MOUNT A DEVICE INTO IT'S FILESYSTEM. NOT IN THE CONTAINER.
Simplistic example: I need this container to somehow tell the OS to "mount /dev/sdg /data".
This mount does not need to be available to the container. The container is being destroyed once it allocate the storage and mounts it.
And we can't use SSH for this.
There are several problems you need to overcome.
By default, Docker does not have access to block devices on the
host.
A docker container is unable to modify its own mount namespace.
A docker container runs in a private mount namespace, so even after
solving (1) and (2), any mounts you make inside the container will
not be visible from the host.
Fortunately, there are solutions to all of the above!
We can solve (1) and (2) by passing the --privileged flag to
docker run. This removes all the restrictions that Docker normally
places on a container.
For solving (3), we need to use the --mount option instead of the
-v option, since we need to modify the style of mount propagation
used. Reading through the documentation on
bind-mounts,
we see that the --mount option supports the following options:
The type of the mount, which can be bind, volume, or tmpfs. This topic discusses bind mounts, so the type will always be bind.
The source of the mount. For bind mounts, this is the path to the file or directory on the Docker daemon host. May be specified as source or src.
The destination takes as its value the path where the file or directory will be mounted in the container. May be specified as destination, dst, or target.
The readonly option, if present, causes the bind mount to be mounted into the container as read-only.
The bind-propagation option, if present, changes the bind propagation. May be one of rprivate, private, rshared, shared, rslave, slave.
The consistency option, if present, may be one of consistent, delegated, or cached. This setting only applies to Docker for Mac, and is ignored on all other platforms.
The one we care about is the bind-propagation option. The values for
that are described later on in the same
document.
Reading through them, we probably want rshared.
Armed with this knowledge, I can run:
docker run -it \
--mount type=bind,source=/,dst=/host,bind-propagation=rshared \
--privileged alpine sh
Then inside the container I can run, for example:
mount /dev/sdd1 /host/mnt
And on the host I see the contents of /dev/sdd1 mounted on /mnt. The mount will persist after the container exits.
I have a Docker container and I am trying to make it so that all of the files in /var/www/ on the container will be saved on the host system at a location (/home/me), and vise-versa. Is it possible to have this shared space between the two?
Would you accomplish this with mount points, or is there a better method?
Thanks
You can use volumes for sharing between container and host.
docker run -v /home/me:/var/www <image>
If you have a fixed files/data, you can add to the image using dockerfile or committing after copying into container. If you want to share rw dir between host and container, you need to use the volumes. Your data will also be persisted even if you remove and recreate a new container.
There are three ways that you can do this
Use volumes. Official docs
Burn the files in your image. Basically include the creation of the files inside the Dockerfile. This means every container container from that image will have an initial state of sorts.
Use data-only containers. These are containers without a running process that contain the data that you need. This also uses volumes. But instead of mounting to the host, your containers mount on the data-only container (which in turn mounts on the host if you want to). This answer will be useful
Normally we would run container by using the following command:
Docker run -it ubuntu /bin/bash
Is there any option to specify where to run the container (like on which disk or partition)?
Do you mean where the container data/layers will be stored?
The layers are all inside /var/lib/docker/(aufs)
It's possible for you to mount a different larger/faster partition into this folder, but this is for the entire docker platform. if you are careful, you can mount the partition for a particular docker container.
It would be better if you would use "docker run -v folder:mount point" flag, since you can mount specific host folders as external volumes inside the container.
Both these can help you spread data over different partitions/disks.
I am not aware of a container specific option.
However, you can bind-mount (or create a symlink) a particular disk or partition to '/var/lib/docker'. This will make all the container storage to be on that partition.
If you want the container storage to be on multiple partitions, LVM is an option.
You can setup a volume group that spans multiple partitions. You can then ask the Docker daemon to create a thinly provisioned logical volume in one of these volume groups to be used as storage.
The following link provides more information : https://access.redhat.com/documentation/en/red-hat-enterprise-linux-atomic-host/7/getting-started-with-containers/chapter-7-managing-storage-with-docker-formatted-containers
Also, using a union mount like OverlayFS could be another solution : https://askubuntu.com/questions/109413/how-do-i-use-overlayfs
I want to test docker in my CentOS 7.1 box, I got this warning:
[root#docker1 ~]# docker run busybox /bin/echo Hello Docker
Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
Hello Docker
I want to know the reason and how to suppress this warning.
The CentOS instance is running in virtualbox created by vagrant.
The warning message occurs because your Docker storage configuration is using a "loopback device" -- a virtual block device such as /dev/loop0 that is actually backed by a file on your filesystem. This was never meant as anything more than a quick hack to get Docker up and running quickly as a proof of concept.
You don't want to suppress the warning; you want to fix your storage configuration such that the warning is no longer issued. The easiest way to do this is to assign some local disk space for use by Docker's devicemapper storage driver and use that.
If you're using LVM and have some free space available on your volume group, this is relatively easy. For example, to give docker 100G of space, first create a data and metadata volume:
# lvcreate -n docker-data -L 100G /dev/my-vg
# lvcreate -n docker-metadata -L1G /dev/my-vg
And then configure Docker to use this space by editing /etc/sysconfig/docker-storage to look like:
DOCKER_STORAGE_OPTIONS=-s devicemapper --storage-opt dm.datadev=/dev/my-vg/docker-data --storage-opt dm.metadatadev=/dev/my-vg/docker-metadata
If you're not using LVM or don't have free space available on your VG, you could expose some other block device (e.g., a spare disk or partition) to Docker in a similar fashion.
There are some interesting notes on this topic here.
Thanks. This was driving me crazy. I thought bash was outputting this message. I was about to submit a bug against bash. Unfortunately, none of the options presented are viable on a laptop or such where disk is fully utilized. Here is my answer for that scenario.
Here is what I used in the /etc/sysconfig/docker-storage on my laptop:
DOCKER_STORAGE_OPTIONS="--storage-opt dm.no_warn_on_loop_devices=true"
Note: I had to restart the docker service for this to have an effect. On Fedora the command for that is:
systemctl stop docker
systemctl start docker
There is also just a restart command (systemctl restart docker), but it is a good idea to check to make sure stop really worked before starting again.
If you don't mind disabling SELinux in your containers, another option is to use overlay. Here is a link that describes that fully:
http://www.projectatomic.io/blog/2015/06/notes-on-fedora-centos-and-docker-storage-drivers/
In summary for /etc/sysconfig/docker:
OPTIONS='--selinux-enabled=false --log-driver=journald'
and for /etc/sysconfig/docker-storage:
DOCKER_STORAGE_OPTIONS=-s overlay
When you change a storage type, restarting docker will destroy your complete image and container store. You may as well everything up in the /var/lib/docker folder when doing this:
systemctl stop docker
rm -rf /var/lib/docker
dnf reinstall docker
systemctl start docker
In RHEL 6.6 any user with docker access can access my private keys, and run applications as root with the most trivial of hacks via volumes. SELinux is the one thing that prevents that in Fedora and RHEL 7. That said, it is not clear how much of the additional RHEL 7 security comes from SELinux outside the container and how much inside the container...
Generally, loopback devices are fine for instances where the limit of 100GB maximum and a slightly reduced performance are not a problem. The only issue I can find is the docker store can be corrupt if you have a disk full error while running... That can probably be avoided with quotas, or other simple solutions.
However, for a production instance it is definitely worth the time and effort to set this up correctly.
100G may excessive for your production instance. Containers and images are fairly small. Many organizations are running docker containers within VM's as an additional measure of security and isolation. If so, you might have a fairly small number of containers running per VM. In which case even 10G might be sufficient.
One final note. Even if you are using direct lvm, you probable want a additional filesystem for /var/lib/docker. The reason is the command "docker load" will create an uncompressed version of the images being loaded in this folder before adding it to the data store. So if you are trying to keep it small and light then explore options other than direct lvm.
#Igor Ganapolsky Feb and #Mincă Daniel Andrei
Check this:
systemctl edit docker --full
If directive EnvironmentFile is not listed in [Service] block, then no luck (I also have this problem on Centos7), but you can extend standard systemd unit like this:
systemctl edit docker
EnvironmentFile=-/etc/sysconfig/docker
ExecStart=
ExecStart=/usr/bin/dockerd $OPTIONS
And create a file /etc/sysconfig/docker with content:
OPTIONS="-s overlay --storage-opt dm.no_warn_on_loop_devices=true"
I am adding a disk quota to my Ubuntu docker container. To add quota support, I need to edit the mount options and add usrquota as explained here: how-to-enable-user-and-group-quotas
Usually you would edit /etc/fstab and add the mount option.
My question, how would I add a mount option to a docker container?
You don't really mount container's disks anywhere. There is a feature request asking for setting quotas in Docker containers (https://github.com/docker/docker/issues/3804) so at the moment there is no easy way.
However, apparently there are a couple of workarounds.
Use Device Mapper as a limit
Docker containers have a maximum of 10GB of disk space, per container (that is the Device Mapper storage driver by default).
So your best option is to change the default value for new containers, but then, it is my understanding you would need to rebuild the containers.
So, if you want to enforce 5 gigabytes, you would write
docker -d --storage-opt dm.basesize=5G
Source
https://goldmann.pl/blog/2014/09/11/resource-management-in-docker/#_limiting_disk_space
User inside/quota outside
The trick is create a specific user account in each container, and assign a userid for that account (and obviously run the command with that account).
On the host, we would use setquota to limit this userid.
Source https://github.com/docker/docker/issues/471#issuecomment-22373948