I met some issues with docker-desktop on Windows and performed a fresh install.
The only problem is that my images and containers are gone...
I'm only interested to recover one specific container using TensorFlow, containing jupyter notebook that I should have save.
Is there any way to restore it?
I'm sorry but you won't be able to restore it.
When you have Docker Desktop without WSL2 backend, the resources are stored under C:\ProgramData\docker and those are deleted at uninstall.
When you have Docker Desktop with WSL2 backend, the distribution where Docker is running is completely wiped out at uninstall.
I suppose you were using the official image under tensorflow/tensorflow (or the equivalent with GPU support), so next time don't forget to use a volume for those contents you'd like to persist or even better, have a disposable container that binds-mount your workspace.
Example:
Create a folder under C: where you want your Jupyter Workspace, let's say C:\Projects
Start the container by mounting that folder on the container and run Jupyter Notebook: docker run -it --rm -v C:\Projects:/usr/workspace -p 8888:8888 tensorflow/tensorflow:nightly-jupyter
When you access to your Notebook in the browser (under localhost:8888), open in Jupyter Notebook the directory /usr/workspace, so all the work you're doing will be also stored in the host in C:\Projects
When you finish, you can safely stop and delete your container, since the work is stored in the host and not only in the container.
Related
i previously ran jellyfin on my desktop comuputer and put a lot of work into the manual creation of collection (descriptions, folder pictures etc.). Now i want to implement jellyfin on my brandnew Synology DS220+ NAS, which is running Jellyfin on Docker. If my understanding of Docker is correct, it is running an instance of the jellyfin app. So while its running, i am not able to see the folders/files of jellyfin in my FileStation-Browser.
So my question is: How can i force Jellyfin/Docker to use the existing Jellyfin-Collection-Data from my desktopPC (which are bacically .xml files).
Thanks in advance!!
You need to map filepaths for config, cache and media. From the jellyfin docs:
docker run -d -v /srv/jellyfin/config:/config -v /srv/jellyfin/cache:/cache -v /media:/media --net=host jellyfin/jellyfin:latest
Arguments of -v flag means volume mapping. Left of colon(:) is your machine's path while right side is the path viewed by jellyfin in its container.
If you don't map internal volumes of a container with your machine, you cannot view those files and the data created will be destroyed on container shutdown.
So basically replace the config,cache and media paths of the above command with your folder paths
The docker run hello-world by #vanshaj is ment to be executed in the terminal window (ssh session) and not in a docker container.
Another approach would be to install the jellyfin package of synocommunity.com. This package does not use docker and is available since just a few days ago.
I am trying to execute ubuntu in docker. I use this command docker run -it ubuntu, and I want to install some packages and store some files. I know about volumes, but I have used it only in docker-compose. Is it possible to store all the container's data or how can I do that properly?
when you run a container, Docker creates a namespace and loads the image filesystem in that namespace. any changes you apply in a running container including installing some packages only remains during the lifetime of the container if you remove the container and rerun it they're gone.
if you want to your changes be permanent you have to commit the running container and actually create an image for that using this command:
As David pointed out in the comments
You should pretty much never run docker commit. It leads to images that can't be reproduced, and you'll be in trouble if there's a security fix you're required to take a year down the road.
sudo docker commit [CONTAINER_ID] [new_image_name]
if you have an app inside the container like MySQL and wants the data stored in that app be permanent you should map a volume from the host like this:
docker run -d -v /home/username/mysql-data:/var/lib/mysql --name mysql mysql
Need to know where docker volumes are located when using the docker machine on macOS.
The installation is using boot2docker, so the VM works behind.
Example:
docker volume create test-data
docker inspect shows a path, but where can I find the specific (physical) location?
It’s inside the virtual machine and isn’t directly accessible from the host.
Debug-level commands like docker volume inspect will give you a path, but they really are only for emergency debugging and not for routine use. If you have a way to get a shell in the VM you can see that path, but you really shouldn’t be directly accessing files there, and you shouldn’t be routinely docker inspecting anything.
macOS use a virtual machine it's different to linux where you can access to volumes from /var/lib/docker/volumes.
For macOS you should connect to a VM to find your volumes.
If you use persistent data volumes in Docker, and you want to access them with command-line.
If your docker host is Linux, that’s not a problem; you can find Docker volumes by /var/lib/docker/volumes path.
However, that’s not the case when you use Docker for Mac.
Try to cd /var/lib/docker/volumes from your MacOS terminal, you ‘ll get nothing.
You see, your Mac machine isn’t a real Docker host. Docker for Mac runs a virtual machine and hides it from you to make things simple.
So, to access persistent volumes created by Docker for Mac, you need to connect on that VM.
In order to accomplish this, we need to use a serial terminal on Mac. There’s a terminal application called “screen” that’s going to help us.
We need to “screen into” the Docker driver by executing a command:
screen
~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty
You should see a blank screen, just press Enter , and after a while, you should see a command line prompt
Now you’re inside Docker’s VM and you can cd into volumes dir by typing: cd /var/lib/docker/volumes
Profit, you got there!
If you need to transfer files from your MacOS host into Docker host you can refer to File Sharing
Hope this helps you!
If you have installed docker using snap then volumes are located at:
/var/snap/docker/common/var-lib-docker/volumes/
location of volumes when using docker official install
/var/lib/docker/volumes/
Normally, if you want to "know" where a volume lives, you would want to map a volume to the local filesystem. When you create a named volume you are just allocating "shared" storage. However, if your really need to know, run this command:
docker volume inspect test-data
I want to put a Docker image onto a USB HD and then be able to plug that into any [Linux] machine that has Docker and run the image. How would I go about doing that?
So far, I've discovered that you can "export" a Docker image into a flat file, but it appears you can't do anything with it until you "import" it again. That's no good. My ultimate goal is to run this stuff from a boot CD, which obviously won't have any writable storage to "import" the data into.
So remember that Docker is a service running on your [Linux] machine.
What you can do is are the following options:
Build and run the Dockerfile located on your USB Drive
docker build -t my_image --file /path/to/Dockerfile/on/usb/drive . && docker container run -d my_image
Create a docker-compose file and run the docker-compose from the Dockerfile on your USB Drive
docker-compose up -d --build -f /path/to/Dockerfile/on/usb/drive
In the end, the container will always run on the host machine, but you can take that USB drive to any machine and run the Dockerfile anywhere
OK, so it appears there's two main locations that the Docker daemon uses:
/var/lib/docker holds all the Docker images.
/var/run/docker holds... actually I'm not sure.
The solution I came up with is this:
Delete (!!) the /var/lib/docker folder.
Create a symlink named /var/lib/docker which points to where you actually want the data stored.
Then (and only then) start the Docker daemon.
This seems to result in Docker storing its data where you tell it to. In particular, if you symlink to a folder on an external USB device, Docker will store its state there. You can then repeat this procedure on another machine (maybe one without Internet access) and access the image(s).
Mind you, this stores the entire state of the Docker daemon, not just one image. But I haven't yet found a way around that.
You also wouldn't want to do this to a "real" computer; I want this for a boot CD, where next time you reboot, all the changes to the filesystem will just disappear again.
Another possibility: It's possible to run two Docker daemons on the same host, and to pass images between them. So you could start one daemon running on USB storage, load the necessary image(s) into it, and then on another machine start Docker running on the same USB device.
To run an alternative Docker daemon, you need the following incantations:
containerd \
--state-dir /mnt/Docker/containerd \
--listen unix:///mnt/Docker/containerd.sock
dockerd \
--pidfile /mnt/Docker/dockerd.pid \
--data-root /mnt/Docker/Data \
--exec-root /mnt/Docker/Exec \
--containerd /mnt/Docker/containerd.sock \
--host unix:///mnt/Docker/dockerd.sock
For this to work, the directory /mnt/Docker needs to already exist. The other files, sockets and directories appear to get created automatically.
Both containerd and dockerd accept a --debug option that makes them output a lot more info to the console. Both of these are daemons, so the commands above never return.
Once the new dockerd is running, you can talk to it as normal if you manually specify the socket:
docker --host unix:///mnt/Docker/dockerd.sock info
You might want to define that as a shell alias to save some typing.
You can copy an image from the "normal" Docker daemon to the new one you just created like so:
docker save ubuntu:latest | docker --host unix:///mnt/Docker/dockerd.sock load
I am running a Docker container in CoreOS (host) and mounted a host folder with a container's folder.
docker run -v /home/core/folder_name:/folder_name <container_name>
Now, each time I am changing (insert/delete) some file in that host folder (folder_name), I have to restart the container (container_name) to see the effects.
docker restart <container_name>
Is there any way from the host side or docker side to restart it automatically when there is a change (insert/delete) in the folder?
Restarting the docker container on a folder change is rather antithetical to the whole notion of the -v command in the first place. If you really really really need to restart the container in the manner you are suggesting then the only way to do it is from the docker host. There are a couple tools (I can name off the top of my head, there are definitely more) you could use to monitor the host folder and when a file is inserted or deleted you could trigger the docker restart <container_name> command. Those tools are incron and inotify-tools. Here is another question someone asked similar to yours and the answer recommended using one of the tools I suggested.
Now, there is no way that the files in the host folder are not being changed in the docker container as well. It must be that the program you are using in the docker container isn't updating it's view of the /folder_name folder after it starts up. Is it possible for you to force the program you are running in the docker container to refresh or update? The -v command works via bind mounting and has been a stable feature in docker for quite a while. With bind mounting, the home/core/folder_name folder IS (for all practical purposes) the same folder as /folder_name in the container.
run the command
docker run -t -i -v /home/core/folder_name:/folder_name <container_name> /bin/sh
This command gives you an interactive shell within the container. In this shell issue the command:
cd /folder_name; touch a_file
Now go to /home/core/folder_name on the docker host in a shell or some file browser. The file a_file will be there. You can delete that file on the host and go back to the shell running in the docker container and run ls /folder_name. The file a_file will not be there.
So, you either need to use inotify or incron to go about restarting your container anytime a file changes on the host, or figure out how to work with the program you are running in the docker container to have it update its view of the /folder_name folder.