I have docker-compose file with volumes section for given container:
video-streaming:
image: video-streaming
build:
context: ./video-streaming
dockerfile: Dockerfile-dev
container_name: video-streaming
volumes:
- /tmp/history/npm-cache:/root/.npm:z
I'm running docker on windows and image is linux based.
When I enter container and add file to /root/.npm and then close the container and run it again then the file is still there so this volume works. But the question is where can I find it's location on Windows host?
You should find the volumes in C:\ProgramData\docker\volumes. The filename will be a hash, which you can check with docker inspect.
If not, then note that you are simply mounting a host directory /tmp/history/npm-cache to your container. This directory is your volume.
When using docker for windows the question is if you are using the old Docker Toolbox or the newer ones that use WSL/WSL2
Docker Desktop configured Linux Containers and WSL/WSL2
The docker engine is actually not running on the windows, but inside the WSL instance, docker desktop makes docker commands available on the windows for ease of use.
So the volumes are probably inside that WSL instance (linux)
you can find out what WSL instances you have by typing wsl -l in powershell.
their file-system is available in \\\wsl$ path on windows.
In your case, the volume is not named, its in the exact location you specified for it.
/tmp/history/npm-cache but inside the WSL instance that docker engine is installed on.
Through WSL
in powershell write wsl ls /tmp/history, you should see npm-cache there.
wsl command allows piping linux commands that will be run on the actual linux wsl instance (default one) which is probably the one running the docker engine.
alternatively, you can connect to that linux by just typing wsl and going to that path cd /tmp/history
once inside the wsl instance you can write explorer.exe . to open explorer in that location (on windows)
notice that the path will always start with \\wsl$ so you can go to that path on windows and see all of you wsl instances and their file-systems, try to search for "npm-cache" in explorer, you might find it.
via Docker commands
docker volume ls will give you all of the available volumes. yours is not named, so its probably one of the 'UUID' ones. you can inspect each one to find its location (probably still inside the wsl instance)
docker volume inspact {the-uuid-of-the-volume}
ones you inspect it, you will see each volume has a Mountpoint field which points to the location of the volume (inside the wsl instance)
unnamed volumes are created with different permissions from your user, so you might need sudo to interact with them via the wsl terminal.
if its through windows file explorer on \\wsl$ you might not need extra permissions.
I am trying to do volume mapping from linux container to my local windows machine.
Here are the steps:
Installed latest version of Docker desktop for windows (2.4.0.0) and it's currently using WSL2 based engine.
Started a container using my own image on top of alpine image. Working directory is set to '/app' in linux container.
An output file (Report.html) is created under a folder (Reports) in my linux container once it is run. Able to view the file in container.
I would like to save the output file to a folder named 'Output' under my user directory in local windows machine.
Ran the following command in Power Shell in Admin mode:
docker run -it -v ~/Output:/app/Reports <imagename>
Issue:
Output file (Report.html) does not get copied to Output folder in local machine.
Note:
I don't see the option to select drive for file sharing in Docker settings.
Please guide me on how I can resolve this ?
Using absolute path in place of ~ worked, i.e. docker run -v C:/Users/12345/Output:/app/Reports
After recent Docker update (Docker Desktop for Mac) my stack broke.
I'm using my docker-compose config in two ways - with up command and with run command to execute some operations via containers (when it's not up yet).
I'm getting error:
ERROR: Configuration for volume my_code specifies "device" driver_opt /Users/me/Projects/project/backend/my_code, but a volume with the same name uses a different "device" driver_opt (/host_mnt/Users/me/Projects/project/backend/my_code). If you wish to use the new configuration, please remove the existing volume "my_code" first:
I have configured docker-compose with volumes shared between containers.
volumes:
my_code:
driver: local
driver_opts:
type: none
device: ${PWD}/project/backend/my_code
o: bind
Looks like for some reason up and run commands get's different path from $PWD in docker-compose. One get's prefixed with /host_mnt and the other doesn't. Is this a bug or maybe my config is invalid?
Docker for Mac 2.4.0.0 stable
Docker Compose 1.27.4
Catalina
I just had the same issue with the prefix /host_mnt on a Ubuntu system.
This is something related to Docker Desktop.
The solution for me was to uninstall docker and Docker desktop according to the documentation
https://docs.docker.com/desktop/install/ubuntu/
https://docs.docker.com/engine/install/ubuntu/
And afterwards also delete the docker config files manually.
rm -rf ~/.docker
Then I just installed the docker engine instead of the docker desktop.
The problem was caused by an update on Docker Desktop that adds that /host_mnt prefix for compatibility with Windows users.
I am running Windows Subsystem Linux (WSL) with Ubuntu as client OS under Windows 10. Now I installed Docker Desktop on the Windows host and enabled the WSL integration in the Docker settings. That works fine so far, I can access the Docker daemon running on the Windows host from my WSL Ubuntu client.
Now I am wondering where all the Docker volumes and other data is stored in this setup. Usually these are under /var/lib/docker, but it seems when using WSL this is not the case. When running df -h I can see the following Docker-related lines:
/dev/sdd 251G 3.1G 236G 2% /mnt/wsl/docker-desktop-data/isocache
/dev/sdc 251G 120M 239G 1% /mnt/wsl/docker-desktop/shared-sockets
/dev/loop0 244M 244M 0 100% /mnt/wsl/docker-desktop/cli-tools
So they are somewhere on the Windows host it seems.
... but where?
When I create a volume named shared_data in docker, I can find it under
\\\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\shared_data\\_data
You can find WSL2 volumes under a hidden network share. Open Windows Explorer, and type \\wsl$ into the location bar. Hit enter, and it should display your WSL volumes, including the ones for Docker for Windows.
If you are wondering where on the Windows host the docker volumes are located, for me they seem to be at:
C:\Users\username\AppData\Local\Docker\wsl\data\ext4.vhdx
and
C:\Users\username\AppData\Local\Docker\wsl\distro\ext4.vhdx
presumably, these are docker-desktop-data and docker-desktop respectively.
In theory, these WSL2 instances can be re-located to an alternate drive to free disk space as per this post; that is the standard method for exporting, unregistering, and re-importing an instance from a new location. This process is also described here (with regard to standard WSL instances).
(Caveat - I haven't yet done this with the docker WSL2 instances yet myself, only for Ubuntu using the method in the second link.)
Windows 10 + WSL2
I run docker-desktop on Windows 10 + WSL2. Just make sure you run the docker desktop, so the path would be accessible from a network.
I found my volume data under
\\wsl$\docker-desktop-data\data\docker\volumes
Note that you need to have docker desktop running before you will be able to discover those network direcotories:
Docker Desktop's WSL2 feature creates two new wsl2 containers docker-desktop and docker-desktop-data, which can be seen by the command wsl -l -v
NAME STATE VERSION
* Ubuntu-18.04 Running 2
docker-desktop Running 2
docker-desktop-data Running 2
This is where the docker daemon actually runs and where you can find the data you are looking for.
The volumes in the wsl2 kernel are mapped as follows:
docker run -ti -v host_dir:/app amazing-container will get mapped to /mnt/wsl/docker-desktop-data/data/docker/volumes/host_dir/_data/
The above is the right path, even though docker volume inspect amazing-container will tell you differently (/var/lib/docker/volumes/).
To conclude, the volumes are mapped to: /mnt/wsl/docker-desktop-data/data/docker/volumes/
Most answers on this topic are about the location from the Windows side, I needed to access the container log files (the issue is the same as for volumes) from my WSL distribution, the Windows path \\wsl$ was not an option.
The files could be found on Windows in \\wsl$\docker-desktop-data\version-pack-data\community\docker\containers.
From the WSL distribution, I could go to /mnt/wsl/docker-desktop-data/version-pack-data but it was empty.
Finally found a solution here:
From Windows, create a disk for docker-desktop-data:
net use w: \\wsl$\docker-desktop-data
From your WSL distribution, mount it to docker:
sudo mkdir /mnt/docker
sudo mount -t drvfs w: /mnt/docker
Now you can get everything you want, in my case log files:
ls -l /mnt/docker/version-pack-data/community/docker/containers/
total 0
drwxrwxrwx 4 root root 512 May 19 15:06 3f41ade0891c06725e828853524d73f185b415d035262f9c51d6b6e03654d505
In my case, i install docker-desktop on wsl2, windows 10 home. i find my image files in
\\wsl$\docker-desktop-data\version-pack-data\community\docker\overlay2
All image files are stored there, and have been seperated into several folders with long string names. When i look into every folder, i can find all the real image files in "diff" folders.
Although the terminal show the path "var/lib/docker", but the folder doesn't exsit and the actual files are not stored there. i think there is no error, the "var/lib/docker" is just linked or mapped to the real folder, kind like that.
In windows, we also use mklink to link two folders, it is similar, right?
You can find volumes and others data when using docker with WSL under docker-desktop-data
If you are running Docker on Windows host, using Docker Desktop, you can access the volumes at \\wsl$\docker-desktop-data\version-pack-data\community\docker\volumes\ (search this path from windows explorer and make sure docker engine is running).
When running Docker desktop app, the app creates its own Linux VM or using WSL to run the docker container and the path /var/lib/docker/volumes/ is from within that VM I think. The volumes are created as mountable .vhdx file at
C:\Users\username\AppData\Local\Docker\wsl\distro\
but accessing this directly is tricky.
Ref: Google how to access WSl files from Windows
Windows 10 + WSL2, Docker Desktop v4.13.1, free service tier, 2022-11-03:
I found my volumes at \\wsl$\docker-desktop-data\data\docker\volumes
Here is my docker-compose file (sorry for the images but the WSL terminal won't let me copy-paste indented text):
The intention being that external_stuff contains my mounts directory. When I look in the mounts directory I clearly see my drives:
However, when I run docker-compose up I only see a single folder ("c") as opposed to all of my drives, and when I navigate into that folder it appears empty:
I tried running sudo -E docker-compose up but that makes no difference.
What's going on here, and how do I fix it?
My system:
Docker desktop version 2.1.0.5
Windows build 1903 / OS Build 18362.476
I think I'm running wsl 1 but I really have no idea. If I run wsl -l from a powershell it spits out a bunch of command line options.
I'm running Ubuntu 18.04.2 LTS directly from the "Ubuntu" app in windows.
Dockerfile:
FROM python:3
Docker Desktop is running on a VM and you need to share your drives with it.
When running Docker Desktop through WSL, you will still need to share the drives you are using.
For this, you simply have to go into Docker Desktop Settings > Shared Drives and then allow sharing your drives.
Then you can work on Docker Desktop through WSL with linux commands, linux paths, etc.
Disclaimer : WSL and Docker Desktop are really unstable, with sharing volumes, permissions, inotify events, etc. You can find more information about theses problems on the answer of this question : Docker is not recompiling upon changing anything in angular project in windows