docker: Error response from daemon: invalid volume specification - docker

I'm currently following this tutorial to run a model on Docker that was built using the Google Cloud AutoML Vision:
https://cloud.google.com/vision/automl/docs/containers-gcs-tutorial
I'm having trouble running the container, specifically running this command:
sudo docker run --rm --name ${CONTAINER_NAME} -p ${PORT}:8501 -v ${YOUR_MODEL_PATH}:/tmp/mounted_model/0001 -t ${CPU_DOCKER_GCR_PATH}
I have my environment variables set up right (did an echo $<env_var>). I do not have a /tmp/mounted_model/0001 directory on my local system. My model path is configured to be the model location on the cloud storage.

${YOUR_MODEL_PATH} must be a directory on the host on which you're running the container.
Your question suggests that you're using the Cloud Storage bucket path but you cannot do this.
Reviewing the tutorial, I think the instructions are confusing.
You are told to:
gsutil cp \
${YOUR_MODEL_PATH} \
${YOUR_LOCAL_MODEL_PATH}/saved_model.pb
So, your command should probably be:
sudo docker run \
--rm \
--interactive --tty \
--name=${CONTAINER_NAME} \
--publish=${PORT}:8501 \
--volume=${YOUR_LOCAL_MODEL_PATH}:/tmp/mounted_model/0001 \
${CPU_DOCKER_GCR_PATH}
NB I added --interactive --tty to make debugging easier; it's optional
NB ${YOUR_LOCAL_MODEL_PATH} not ${YOUR_MODEL_PATH}
NB The command should not be -t ${CPU_DOCKER_GCR_PATH} omit the -t
I've not run through this tutorial.

Related

"rootless" docker gets permission denied, but account running docker does not - why?

I am running docker "rootless" according to this guide: https://docs.docker.com/engine/security/rootless/
The user which actually runs docker is svc_test.
When I try and start a docker container which has diretory mounts which don't exists - the docker daemon (a.k.a. svc_test user) attempts to mkdir these directories, but fails with
docker: Error response from daemon: error while creating mount source path '/dir_path/dir_name': mkdir /dir_path/dir_name: permission denied.
When I (svc_test) them attempt to do mkdir /dir_path/dir_name I succeed without any issues.
What is going on here and why does this happen?
Clearly I am missing something, but I can't trace what is that exactly.
Update 1:
This is the specific docker cmd I use to run the container:
docker run -d --restart unless-stopped \
--name questdb \
-e QDB_METRICS_ENABLED=TRUE \
--network="host" \
-v /my_mounted_volume/questdb:/questdb \
-v /my_mounted_volume/questdb/public:/questdb/public \
-v /my_mounted_volume/questdb/conf:/questdb/conf \
-v /my_mounted_volume/questdb/db:/questdb/db \
-v /my_mounted_volume/questdb/log:/questdb/log \
questdb/questdb:6.5.2 /usr/bin/env QDB_PACKAGE=docker /app/bin/java \
-m io.questdb/io.questdb.ServerMain \
-d /questdb \
-f
For clarity: my final goal is to be able to run the docker container in question from the same user form which I run my docker daemon (the svc_test user). Hence how I stumbled on this problem.

Docker Bench for Security for image on Google Cloud

I have an image in Container Registry and deployed to App Engine flex.
How do I use Docker Bench for Security to check my containers security?
You can't use Docker Bench for the images that are uploaded in the Google Cloud Container Registry.
You can do it locally with the following command:
docker run -it --net host --pid host --userns host --cap-add audit_control \
-e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \
-v /etc:/etc:ro \
-v /usr/bin/docker-containerd:/usr/bin/docker-containerd:ro \
-v /usr/bin/docker-runc:/usr/bin/docker-runc:ro \
-v /usr/lib/systemd:/usr/lib/systemd:ro \
-v /var/lib:/var/lib:ro \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
--label docker_bench_security \
docker/docker-bench-security
For more information on Docker Bench usage you can check this
I think you should also be able to replicate this process with Cloud Build. You can check the documentation to see how to use it.
Cloud Build quickstart
Cloud Build config reference

Docker invalid characters for volume when using relative paths

Ive been given a docker container which is run via a bash script. The container should set up a php web app, it then goes on to call other scripts and containers. It seems to work fine for others, but for me its throwing an error.
This is the code
sudo docker run -d \
--name eluci \
-v ./config/eluci.settings:/mnt/eluci.settings \
-v ./config/elucid.log4j.settings.xml:/mnt/eluci.log4j.settings.xml \
--link eluci-database:eluci-database \
/opt/eluci/run_eluci.sh
This is the error
docker: Error response from daemon: create ./config/eluci.settings:
"./config/eluci.settings" includes invalid characters for a local
volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to
pass a host directory, use absolute path.
Im running docker on a centos VM using virtualbox on a windows 7 host.
From googling it seems to be something to do with the mount, however I dont want to change it in case the setting it breaks or is relied upon in another docker container. I still have a few more bash scripts to run, which should orchestrate the rest of the build process. As a complete newb to Docker, this has got me stumped.
The command docker run -v /path/to/dir does not accept relative paths, you should provide an absolute path. The command can be re-written as:
sudo docker run -d \
--name eluci \
-v "/$(pwd)/config/eluci.settings:/mnt/eluci.settings" \
-v "/$(pwd)/config/elucid.log4j.settings.xml:/mnt/eluci.log4j.settings.xml" \
--link eluci-database:eluci-database \
/opt/eluci/run_eluci.sh

docker --volume format for Windows

I'm trying to take a shell script we use at work to set up our development environments and re-purpose it to work on my Windows environment via Git Bash.
The way the containers are brought up in the shell script are as follows:
docker run \
--detach \
--name=server_container \
--publish 80:80 \
--volume=$PWD/var/www:/var/www \
--volume=$PWD/var/log/apache2:/var/log/apache2 \
--link=mysql_container:mysql_container \
--link=redis_container:redis_container \
web-server
When I run that as-is, it returns the following error message:
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: invalid bind mount spec "/C/Users/username/var/docker/environments/development/scripts/var/log/apache2;C:\\Program Files\\Git\\var\\log\\apache2": invalid volume specification: '/C/Users/username/var/docker/environments/development/scripts/var/log/apache2;C:\Program Files\Git\var\log\apache2': invalid mount config for type "bind": invalid mount path: '\Program Files\Git\var\log\apache2' mount path must be absolute. See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.
I did a bunch of googling and documentation reading, but I'm a little overwhelmed by Docker, and I think I got it wrong. I tried setting up the container as follows:
docker run \
--detach \
--name=server_container \
--publish 80:80 \
--volume=/c/users/username/var/www:/var/www \
--volume=/c/users/username/var/log/apache2:/var/log/apache2 \
--link=mysql_container:mysql_container \
--link=redis_container:redis_container \
web-server
It still errors out with a similar error message. If I remove the colon:/var/www it comes up, but it doesn't seem to map those directories properly, that is it doesn't know that C:\users\username\var\www = /var/www
I know I'm missing something painfully dumb here, but when I look at the documentation I just glaze over. Any help would be greatly appreciated.
For people using Docker on Windows 10, an extra / has to be included in the path:
docker run -it -v //c/Users/path/on/host:/app/path/in/docker/container command
(notice an extra / near c)
If you are using Git Bash and using pwd then use an extra / there as well:
docker run -p 3000:3000 -v /app/node_modules -v /$(pwd):/app 09b10e9fda85`
(notice / before $(pwd))
Well, I answered my own question moments after I posted it.
This is the correct format.
docker run \
--detach \
--name=server_container \
--publish 80:80 \
--volume=//c/users/username/var/www://var/www \
--volume=//c/users/username/var/log/apache2://var/log/apache2 \
--link=mysql_container:mysql_container \
--link=redis_container:redis_container \
web-server
Should have kept googling a few minutes longer.
If you want to make the path relative, you can use pwd and variables. For example:
CURRENT_DIR=$(pwd)
docker run -v /"$CURRENT_DIR"/../../test/:/test alpine ls test

Run bitcoind with bitcoind.conf in docker

I know docker, but less about bitcoind.
Now I want to use this docker image to start my own test environment:
The description tells me:
docker volume create --name=bitcoind-data
docker run -v bitcoind-data:/bitcoin --name=bitcoind-node -d \
-p 8333:8333 \
-p 127.0.0.1:8332:8332 \
kylemanna/bitcoind
Now I want to now how I have to add my bitcoind.conf?
This isn't provided anywere? Can I use it at container startup or docker exec?
The repository contains a documentation file dedicated to your issue: https://github.com/kylemanna/docker-bitcoind/blob/master/docs/config.md

Resources