I am following a tutorial and using sqlc in my project. However, it's weird that I seem to mount an empty volume. After checking another post mounting the host directory, I found docker creates another empty folder, confirming that I did something wrong about it. Docker documentation doesn't help resolve this issue. Currently, my command with bash terminal:
docker run --rm -v $(pwd)://src -w //src kjconroy/sqlc init
docker run --rm -v $(pwd)://src -w //src kjconroy/sqlc generate
The first command runs successfully but creates another empty folder. The built container is running, and it's path is: \\wsl$\docker-desktop-data\data\docker\volumes on my Windows 10. However, the folder structure is different from the tutorial when I download the desktop docker, so I'll add extra information about how I construct the setting. The construction is using Make with docker:
postgres:
docker run --name postgreslatest -p 5432:5432 -e POSTGRES_USER=root -e POSTGRES_PASSWORD=secret -d postgres
createdb:
docker exec -it postgreslatest createdb --username=root --owner=root simple_bank
dropdb:
docker exec -it postgreslatest dropdb simple_bank
migrateup:
migrate -path db/migration -database "postgresql://root:secret#localhost:5432/simple_bank?sslmode=disable" -verbose up
migratedown:
migrate -path db/migration -database "postgresql://root:secret#localhost:5432/simple_bank?sslmode=disable" -verbose down
.PHONY: postgres createdb dropdb migrateup migratedown
Any help is appreciated.
I got it working. First of all, I still have no idea why bash command cannot correctly locate the sqlc.yaml file. However, under Windows 10 OS, I succeeded locate and generating files with the command provided by docs.
The command is: docker run --rm -v "%cd%:/src" -w /src kjconroy/sqlc generate using ONLY CMD and the command also works combined with the MakeFile.
I would like to pass a file from the host system to a container at runtime. I want to run a CLI tool within a container and use the file as an argument to the CLI tool. Is it possible to modify the following command: docker run -it --rm --name <container-name> to achieve what I want to do. The docker cp command doesn’t work for what I need since it doesn’t run from within the container and I need to pass the file name as an argument.
I usually use the -v "$PWD:$PWD" -w "$PWD" trick. Run container and volume mount the current host working directory into the container at the same path and set working directory to same path.
So for example if I want to transcode a wav file on the host to a mp3 file usig ffmpeg running in a container I would do:
docker run --rm -v "$PWD:$PWD" -w "$PWD" mwader/static-ffmpeg:4.2.2 -i file.wav file.mp3
You can also add -u $UID:$GROUPS if your unsure what default user the image runs as.
I am trying to set up a docker run configuration in Pycharm, i am pretty new to this functionality in pycharm, and i can't get it working.
In docker I would run the container with the following command
docker build -t test-container . && docker run --name container-pycharm -t -i --env-file .env -v $(pwd):/srv/app -p 8080:8080 --rm test-container ./serve-app
I set up this in pycharm, by adding the following line
--rm --env-file .env -i -t -p 8080:8080 -v $(pwd):/srv/app
to command line options section in the relevant docker Run/Debug Configuration Pycharm window. Unfortunately I get
Failed to deploy 'container-pycharm Dockerfile: Dockerfile': com.github.dockerjava.api.exception.BadRequestException: {"message":"create $(pwd): \"$(pwd)\" includes invalid characters for a local volume name, only \"[a-zA-Z0-9][a-zA-Z0-9_.-]\" are allowed. If you intended to pass a host directory, use absolute path"}
Clearly, I cant use $(pwd) in my command line options, any idea on how to solve this in pycharm?
Pycharm doesn't invoke docker directly via the command you see in the command preview, it goes through its custom parser, currently they haven't implemented the feature to read envs. Thus "If you intended to pass a host directory, use absolute path"
And -v is not officially supported as command line options in the current version. Ref
Use Bind mounts instead
What's the difference between these two docker run commands and why one is working and the other does not.
Working Command
docker run --publish=7474:7474 --volume=$HOME/neo4j_test/data:/data neo4j
Not Working
docker run --publish=7474:7474 --volume=C:/Users/USERNAME/neo4j_test/data:/data neoj
docker run --publish=7474:7474 --volume=C:\Users\USERNAME\neo4j_test\data:/data neo4j
Error for these commands
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Invalid bind mount spec "C:UsersUSERNAMEneo4j_testdata:/data": invalid mode: /data.
See 'C:\Program Files\Docker Toolbox\docker.exe run --help'.
In the non-working commands, I just replaced $HOME with the absolute path for my user profile folder C:/Users/USERNAME
UPDATE
I did inspect the value for $HOME by executing echo $HOME on Windows Powershell. And it is in fact C:\Users\USERNAME. I also did look at the link that #Titouan Freville has commented. So I used the command
docker run --publish=7474:7474 --volume=/c/Users/USERNAME/neo4j_test/data:/data neo4j
isntead of
docker run --publish=7474:7474 --volume=C:/Users/USERNAME/neo4j_test/data:/data neoj
and it is working now. Right now I'm just wondering where the transformation of $HOME from C:\Users\USERNAME to /c/Users/USERNAME happens
For anyone still having this problem with Docker-for-Windows, here are the 2 solutions that work:
Prefix your command
with MSYS_NO_PATHCONV=1
In full: MSYS_NO_PATHCONV=1 docker run -v /c/path:/path
Use double-slashes
// at the beginning
In full: docker run -v //c/path:/path
Source: https://github.com/moby/moby/issues/24029#issuecomment-250412919
To close the subject. Here is the solution ;) docker toolbox mount file on windows
Also, the interpolation of $HOME by docker on windows have to be compatible with it, so it should transform it by himself when you call it in docker command.
Now that I found a way to expose host files to the container (-v option) I would like to do kind of the opposite:
How can I edit files from a running container with a host editor?
sshfs could probably do the job but since a running container is already some kind of host directory I wonder if there is a portable (between aufs, btrfs and device mapper) way to do that?
The best way is:
$ docker cp CONTAINER:FILEPATH LOCALFILEPATH
$ vi LOCALFILEPATH
$ docker cp LOCALFILEPATH CONTAINER:FILEPATH
Limitations with $ docker exec: it can only attach to a running container.
Limitations with $ docker run: it will create a new container.
Whilst it is possible, and the other answers explain how, you should avoid editing files in the Union File System if you can.
Your definition of volumes isn't quite right - it's more about bypassing the Union File System than exposing files on the host. For example, if I do:
$ docker run --name="test" -v /volume-test debian echo "test"
The directory /volume-test inside the container will not be part of the Union File System and instead will exist somewhere on the host. I haven't specified where on the host, as I may not care - I'm not exposing host files, just creating a directory that is shareable between containers and the host. You can find out exactly where it is on the host with:
$ docker inspect -f "{{.Volumes}}" test
map[/volume_test:/var/lib/docker/vfs/dir/b7fff1922e25f0df949e650dfa885dbc304d9d213f703250cf5857446d104895]
If you really need to just make a quick edit to a file to test something, either use docker exec to get a shell in the container and edit directly, or use docker cp to copy the file out, edit on the host and copy back in.
We can use another way to edit files inside working containers (this won't work if container is stoped).
Logic is to:
-)copy file from container to host
-)edit file on host using its host editor
-)copy file back to container
We can do all this steps manualy, but i have written simple bash script to make this easy by one call.
/bin/dmcedit:
#!/bin/sh
set -e
CONTAINER=$1
FILEPATH=$2
BASE=$(basename $FILEPATH)
DIR=$(dirname $FILEPATH)
TMPDIR=/tmp/m_docker_$(date +%s)/
mkdir $TMPDIR
cd $TMPDIR
docker cp $CONTAINER:$FILEPATH ./$DIR
mcedit ./$FILEPATH
docker cp ./$FILEPATH $CONTAINER:$FILEPATH
rm -rf $TMPDIR
echo 'END'
exit 1;
Usage example:
dmcedit CONTAINERNAME /path/to/file/in/container
The script is very easy, but it's working fine for me.
Any suggestions are appreciated.
There are two ways to mount files into your container. It looks like you want a bind mount.
Bind Mounts
This mounts local files directly into the container's filesystem. The containerside path and the hostside path both point to the same file. Edits made from either side will show up on both sides.
mount the file:
❯ echo foo > ./foo
❯ docker run --mount type=bind,source=$(pwd)/foo,target=/foo -it debian:latest
# cat /foo
foo # local file shows up in container
in a separate shell, edit the file:
❯ echo 'bar' > ./foo # make a hostside change
back in the container:
# cat /foo
bar # the hostside change shows up
# echo baz > /foo # make a containerside change
# exit
❯ cat foo
baz # the containerside change shows up
Volume Mounts
mount the volume
❯ docker run --mount type=volume,source=foovolume,target=/foo -it debian:latest
root#containerB# echo 'this is in a volume' > /foo/data
the local filesystem is unchanged
docker sees a new volume:
❯ docker volume ls
DRIVER VOLUME NAME
local foovolume
create a new container with the same volume
❯ docker run --mount type=volume,source=foovolume,target=/foo -it debian:latest
root#containerC:/# cat /foo/data
this is in a volume # data is still available
syntax: -v vs --mount
These do the same thing. -v is more concise, --mount is more explicit.
bind mounts
-v /hostside/path:/containerside/path
--mount type=bind,source=/hostside/path,target=/containerside/path
volume mounts
-v /containerside/path
-v volumename:/containerside/path
--mount type=volume,source=volumename,target=/containerside/path
(If a volume name is not specified, a random one is chosen.)
The documentaion tries to convince you to use one thing in favor of another instead of just telling you how it works, which is confusing.
Here's the script I use:
#!/bin/bash
IFS=$'\n\t'
set -euox pipefail
CNAME="$1"
FILE_PATH="$2"
TMPFILE="$(mktemp)"
docker exec "$CNAME" cat "$FILE_PATH" > "$TMPFILE"
$EDITOR "$TMPFILE"
cat "$TMPFILE" | docker exec -i "$CNAME" sh -c 'cat > '"$FILE_PATH"
rm "$TMPFILE"
and the gist for when I fix it but forget to update this answer:
https://gist.github.com/dmohs/b50ea4302b62ebfc4f308a20d3de4213
If you think your volume is a "network drive", it will be easier.
To edit the file located in this drive, you just need to turn on another machine and connect to this network drive, then edit the file like normal.
How to do that purely with docker (without FTP/SSH ...)?
Run a container that has an editor (VI, Emacs). Search Docker hub for "alpine vim"
Example:
docker run -d --name shared_vim_editor \
-v <your_volume>:/home/developer/workspace \
jare/vim-bundle:latest
Run the interactive command:
docker exec -it -u root shared_vim_editor /bin/bash
Hope this helps.
I use sftp plugin from my IDE.
Install ssh server for your container and allow root access.
Run your docker container with -p localport:22
Install from your IDE a sftp plugin
Example using sublime sftp plugin:
https://www.youtube.com/watch?v=HMfjt_YMru0
The way I am doing is using Emacs with docker package installed. I would recommend Spacemacs version of Emacs. I would follow the following steps:
1) Install Emacs (Instruction) and install Spacemacs (Instruction)
2) Add docker in your .spacemacs file
3) Start Emacs
4) Find file (SPC+f+f) and type /docker:<container-id>:/<path of dir/file in the container>
5) Now your emacs will use the container environment to edit the files
docker run -it -name YOUR_NAME IMAGE_ID /bin/bash
$>vi path_to_file
The following worked for me
docker run -it IMAGE_NAME /bin/bash
eg. my image was called ipython/notebook
docker run -it ipython/notebook /bin/bash