Unable to start any container when Volumes are enabled Docker Toolbox - docker

I am running Docker Toolbox v. 1.13.1a on Windows 7 Pro Service pack 1 x64OS.
with Virtual Box Version 5.1.14 r112924
when I try to run any docker image e.g. official postgres image from Docker Hub with volumes disabled, it works fine!
But when I enable the volumes it fails.
I tried all official documentations
The VM has shared folder as required and has full access to it also
shared folder screenshot
In case of my example of postgresql it crashes with following log
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... LOG: could not link file "pg_xlog/xlogtemp.27" to "pg_xlog/000000010000000000000001": Operation not permitted
FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data"
I know its the problem with folder permissions. But kinda stuck!
A ton of thanks in advance

I've been busy with this problem all day and my conclusion that it's currently simply not possible to run postgresql inside a docker container while keeping your data persistent in a separate volume.
I even tried running the container without linking to a volume and copying the data that was originally in /var/lib/postgresql into a folder of my host OS (Windows 10 Home), then copy that into the folder that got then linked to the container itself.
Alas, I got the next error:
FATAL: data directory "/var/lib/postgresql/data/pgadmin" has wrong ownership
HINT: The server must be started by the user that owns the data directory.
In conclusion: There's something going wrong with the ownership and the correct user owning it and to be able to fix it, you'll need a unix commandline on Windows that is able to run docker (something currently not possible with Bash on Ubuntu on Windows that is running using Ubuntu 16.04 binaries).
Maybe, in the future, you'll be able to run the needed commands (found here, under Arbitrary --user Notes), but these are *nix commands and powershell (started by Kitematic) can't run those. Bash for Ubuntu for Windows could run those, but that shell has no connection to the docker daemon/service on windows...
TL;DR: Lost a day of work: It is currently impossible on Windows.

I have been trying to fix this issue also ..
At first I thought it was a symlink problem (because the first error fails on " could not link .. operation not permitted)
To be sure symlink is permitted you have to :
share a folder in virtualbox
run virtualbox as administrator (if you account is in administrator group) Right click virtualbox.exe and select run as Administrator
if your account is not administrator, add the symlink privilege with secpol.msc > "Local Policies-User Rights Assignments" add your user to "Create symbolic links"
enable symlink for your shared folder in virtualbox :
VBoxManage setextradata VM_NAME VBoxInternal2/SharedFoldersEnableSymlinksCreate/SHARED_FOLDER_NAME 1
Alternatively you can also use the c:\User\username folder which is shared and symlink enabled by default dockertools installation
Now I can create symlinks in the shared folder from the docker container .. but I still have the same error "could not link ... operation not permitted"
So the reason must be somewhere else ... in the file permissions as you said but I do not see why ?

Related

Docker Desktop on Hyper-V - bind mount do not propagate inotify on file copy

I have a Docker Desktop installed on my dev machine, with WSL 2 disabled. I have shared my entire C:/ drive:
Then I have a container that inside has a .net 6 (Core) application that uses the FileSystemWatcher to observe one directory, and when a file is pasted inside to read it.
I red in several articles in the internet that WSL2 do not support notification to propagate from the Windows file system to the underlying Linux distribution that docker is running on, hence there is no way that I can bind the directory that I have to "watch" with the app in the container. So I swithed to the old Hyper-V support of docker.
I run the container with the following command:
docker run `
--name mlc-importer `
-v C:/temp/DZBank:/opt/docker/mlc_importer/dfs/DZBank `
-v C:\temp\appsettings.json:/app/appsettings.json `
-v C:\temp\log4net.config:/app/log4net.config `
mlc-importer
The container starts and starts "watching" for new files. The strange thing is, that when I cut a file and paste it in the directory, the app in the container registers the new file and reads it, but when I copy the file and paste in in the directory, the app in teh container do not register it and read it.
Can someone help me because I can't find out what the problem might comes from.
Thanks in advance,
Julian
I managed to solve mu problem, and I'll post it here if somebody encounters the same problem.
The problem was in teh file itself. This I found out when I started a new container with only debian, and installed inotify-tools, and binded the same path. When I tried to copy the file and paste it in the binded dir the output was:
Three times MODIFY event.
When I tried to cut the file and paste in in the new dir the events were:
So with copy - three times MODIFY, with cut one CREATE and two MODIFY.
Then I inspected the copied file and saw this:
When I checked the checkbox and hit ok, everything is ok. And since in the container app (from the post), I hook to only "File created" callback, it not triggers when the file is only modified.
Hope this helps someone with a similar problem

Docker Failed to Initialize on Windows

Here, I have problem regarding pulling docker-dev in docker image for making my development environment but when I tried to pull docker-dev. I got the error like docker manifest not found.
Can anyone help me out with this error...plz
before this
I want to know about the docker failed to initialize error which i'm having right now...
the error is like,
I tried so many things like re-install the docker desktop or WSL updates, but didn't worked.
And error in the command be like...
So if someone can help me out with this....plz help me out
Got the same issue and fixed it by deleting %appdata%\Docker as mentioned by Github User "tocklime"
(Original Source : https://github.com/docker/for-win/issues/3088)
Short solution: delete %appdata%\Docker\settings.json and let Docker to create a new one.
Take a backup of the file for the next time it gets broken.
<tl;dr>
I face this issue almost every month and I hope this will get fixed definitely.
Following tmBlackCape answer, I checked the %appdata%\Docker directory and found settings.json damaged (editor tells it's a binary file and of course it shouldn't).
I deleted the file and Docker Service (still running) created a new one with default values. If the service isn't running, just launch it again.
You could need to change settings (via GUI, as recommended) to catch your needs.
I made a backup copy of my custom settings.json so next time I can replace the broken one without losing custom configuration.
Go to the directory C:\Users-------\AppData\Roaming\Docker and delete the file settings.json . Docker takes care of rewriting it at startup.
This manipulation solved the problem for me !
Docker failed to initialize
C:\Users[USER]\AppData\Local\Docker
C:\Users[USER]\AppData\Roaming\Docker
C:\Users[USER]\AppData\Roaming\Docker Desktop
Once deleted above directory, I didn’t have to do anything else, Docker Desktop started booting up as normal.
The error message I got was not exactly the same the OP got. For me, it said Docker failed to initialize. Docker Desktop is shutting down.
TL;DR
The powershell executable was missing from my local machies PATH. I had to add C:\Windows\System32\WindowsPowerShell\v1.0 and docker started again.
The longer story
I tried everything that was mentined in this thread and nothing worked for me. When I looked at the task manager of my local machine to see if any docker-related process was started, I noticed that Docker Desktop.exe itself was started. However, the com.docker.backend.exe not. Docker tries to start that exe in an infinite loop, but as soon as it started it crashed again after half a second.
I then took a look into com.docker.backend.exe.log, which is located in %localappdata%\Docker\log\host and noticed the following line:
[2022-07-07T10:46:57.936079700Z][com.docker.backend.exe][F] exec: "powershell": executable file not found in %PATH%
I then went ahead and just added the path to powershell to PATH (which is C:\Windows\System32\WindowsPowerShell\v1.0)`. As soon as I added that, everything started working again.
I have no idea how the path to the powershell executable got removed from PATH. I certainly did not do that myself.
This happened to me after Docker Desktop upgrade to version 3.6.0 (67351), too. (Which was surprising, because it worked before the upgrade.)
Due to the help in the top answer right now, I went to the above settings directory: %appdata%\Docker, looked at the logs and deleted/renamed the file settings.json -> Docker Desktop started immediatelly ; there had been a process retrying in the background.
In the time before that, the backend.exe.log had been all "unmarshal" something something:
settings.json: json: cannot unmarshal bool into Go struct field Content.proxyHttpMode of type string"
common/cmd/com.docker.backend/internal/settingsloader.GetSettings(0x0, 0x0, 0x0)
So, the above message 'tipped me off' as to where the actual error on start may be. Hmm...
For me the solutions here didn't help, but here's what helped.
Make sure the folder .docker/ in your home directory isn't marked as hidden in Windows. If it's hidden, Docker won't see it.
Make sure docker has Active Directory to .docker folder. For example, if the owner of .docker/ is SYSTEM and not your user, Docker won't be able to read it and crash.
For me deleting the folder %appdata%\Docker did not work.
Instead I had to run the following power shell command as admin.
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V -All
References.
https://stackoverflow.com/a/63845592/1977871
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/quick-start/enable-hyper-v#enable-hyper-v-using-powershell
My windows features with running docker desktop are as follows.
It resolved me by deleting the file
C:\Users{username}\AppData\Roaming\Docker\settings.json
I had the same issue. I'm on Docker Desktop 4.8.0. The following solution worked for me:
Uninstall Docker Desktop
Delete .docker folder from C:\Users\{Username}
Delete Docker folder from C:\Users\{Username}\AppData\Local
Delete Docker folder from C:\Users\{Username}\AppData\Roaming
Delete Docker folder from C:\Program Files
Run CCleaner on Registry and Custom Clean
Restart Computer
Install Docker
Restart Computer
Start Docker
That's all.
Go to C:\Users\asd\AppData\Local and delete Downloaded Installation directory.
C:\Users\asd\AppData\Roaming and delete Docker and Docker Desktop directory.
Then start docker.
Didn't found the AppData folder in users[myUser] or anywhere. re-install solve it for me
latest update 4.10.1 has issues, i downgraded to 4.6.1 and it worked however i think powershell is closely linked to this issue
It happened due to local data corruption. You can check in VM log inside %appdata%\Docker.
I was able to reset and restart by renaming setting.json and then restart the Docker desktop by deleting all the previous log folder and temp folder clean. If this does not help.
Try to kill all the processes running in through the task manager
and then run Power shell in administrator mode and then shoot the command if you are using an older version of Lsmanager Restart-Service LxssManager.
On my side, uninstalling HyperV did the trick :
Disable-WindowsOptionalFeature -Online -FeatureName Microsoft-Hyper-V-Hypervisor
I also (as suggested in the voted answer):
Uninstalled docker desktop,
Deleted any docker related content in %APPDATA% as suggested
Delete any docker related keyys in registry (may be not necessary)
Then reinstalled it
And it solved the problem (but without hyperV removal it didn't work).
There are a few great solutions; however, it didn't work for me. Most of the "fixes" suggested here are already implemented and it just didn't work. I chose a different approach.
I looked at the docker releases and saw what was actually affected by their latest version not working (4.10). These things are normal, especially if the software engine has been updated.
Going through Docker releases, the latest release that didn't touch the Docker engine is 4.8.0. I downloaded and works great.
Removing settings.json file or %AppData%\Docker folder did not work for me. After uninstalling/installing the docker, the issue was solved. (Note: I couldn't try other possible solutions that are required the admin privilege)
The below Steps worked for me,
Step 01:
Navigate to the below-mentioned paths and delete the below-mentioned directories,
C:\Users[USERNAME]\AppData\Local\Docker directory
C:\Users[USERNAME]\AppData\Roaming\Docker directory
C:\Users[USERNAME]\AppData\Roaming\Docker Desktop directory
Then restart the docker again
If you didn't find the AppData folder in users[USERNAME] or anywhere
Step 02: Type the below path in the "Run" to directly open the app data location for you
%appdata%\Docker
Then Delete the below-mentioned directories,
C:\Users[USERNAME]\AppData\Local\Docker directory
C:\Users[USERNAME]\AppData\Roaming\Docker directory
C:\Users[USERNAME]\AppData\Roaming\Docker Desktop directory
Then restart the docker again
Note: If these steps do not help you. Try to kill all the processes running in through the task manager. And again delete the directories and start the docker
if you still can't find the issue you may use this trick and i'm 100% sure this will work
first check your operating system is up to date means no remaining updates if you pass this step then come to 2 step open window command prompt in run as administrator
then type wsl --install then type wsl --list --online then wsl --install -d Debian
then open docker if error come
then install wsl 2 linux https://learn.microsoft.com/en-us/windows/wsl/install-manual#step-4---download-the-linux-kernel-update-package
then restart docker or your machine and then if error also occur then type then wsl --update i hope this will work Regard : Hammad Khadim

Change default volume mount point for docker rootless?

I saw this post with different solutions for standard docker installation:
How to change the default location for "docker create volume" command?
At first glance I struggle to repeat the steps to change the default mount point for the rootless installation.
Should it be the same? What would be the procedure?
I just got it working. I had some issues because I had the service running while trying to change configurations. Key takeaways:
The config file is indeed stored in ~/.config/docker/. One must make a daemon.json file here in order to change preferences. We would like to change the data-root option (and storage-driver, in case the drive does not have capabilities
To start and stop the headless service one runs systemctl --user [start | stop] docker.
a. Running the systemwide service starts a parallel and separate instance of docker, which is not rootless.
b. When stopping make sure to stop the docker.socketfirst.
Sources are (see Useage section for rootless)
and (config file information)
We ended up with the indirect solution. We have identified the directory where the volumes are mounted by default and created a symbolic link which points to the place where we actually want to store the data. In our case it was enough. Something like that:
sudo ln -s /data /home/ubuntu/.local/share/docker/volumes"

Can't attach CircleCI Workspace from Windows to Linux due to "Cannot change ownership to uid 3434"

I am using CircleCI's persistent workspace feature to run jobs with the same build folder between Linux and Windows executor types. I was able to go from Linux to Windows but when I went from Windows to Linux I got this error when CircleCI attempted to attach the workspace.
Applying workspace layers:
9ba3eddc-3658-43c2-858b-aea39250af3e
25c476af-8804-4125-b979-05a62a9ac204
Error applying workspace layer for job 25c476af-8804-4125-b979-05a62a9ac204: Error extracting tarball /tmp/workspace-layer-25c476af-8804-4125-b979-05a62a9ac204854634413 : tar: project/.circleci/config.yml: Cannot change ownership to uid 3434, gid 197121: Invalid argument
Looking at the error it's clear that the UIDs are not existing on the system. I attempted to run commands to create the same UID/GID it was erroring on but I still got an unable to change owner issue.
I was expecting CircleCI to move the files and ignore the user: group part when it was extracted as you can't guarantee the UID/GID exists.
I opened a support ticket but hoping for a faster solution to this issue.
I found a solution to this issue and it's forcing CircleCI to use TAR_OPTIONS environment variable to force the options to ignore the owner/group.
Here is what I added to my jobs steps that attach the workspace when the previous job run was Window.
build-app:
build:
docker:
- image: Dockerhub.com/myrepo/myimage:1.0.0
environment:
TAR_OPTIONS: --no-same-owner
using TAR_OPTIONS environment to inject the option --no-same-owner allowed CircleCI to extract the tarball without issue.

is /usr/bin/ authorized as a default location for volume sharing in docker prefrences?

I am sharing a docker-compose file with a team member to easily build our app. We're both on OSX and it works fine from my machine, but my colleague is getting the following error:
ERROR: for backend Cannot start service backend: b'Mounts denied: \r\nThe path /usr/bin/docker\r\nis not shared from OS X and is not known to Docker.\r\nYou can configure shared paths from Docker -> Preferences... -> File Sharing.\r\nSee https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.\r\n.'
I assume it is due to the following statement in the docker-compose.yaml.
volumes:
- "/usr/bin/docker:/usr/bin/docker"
I didn't have to alter my docker-> preferences-> file sharing to make this work. I only have the default dirs shared: /Users, /Volumes, /tmp, /private.
How come it isn't working on his machine? Does he have to add the /usr dir as a shared dir? If so, how come I don't have to?
UPDATE
The problem was that the docker executable was located in /usr/local/bin/ rather than /usr/bin. I have no idea why docker installed the executable differently despite both machines being OSX.
That's a misleading error, should have asked you to check if the path /usr/bin/docker exists on the host machine. Docker (at least on Mac) will attempt to create a directory if it doesn't exist. Apparently, your team mate isn't logged in with sufficient privileges to create /usr/bin/docker.

Resources