I am using docker to run tensorflow and retrain inception module. I use the following code:
docker run -it \
--publish 6006:6006 \
--volume ${HOME}/tf_files:/tf_files \
--workdir /tf_files \
tensorflow/tensorflow:1.1.0 bash
Then I use
python retrain.py
bottleneck_dir=bottlenecks
how_many_training_steps=500
model_dir=inception
summaries_dir=training_summaries/basic
output_graph=retrained_graph.pb
output_labels=retrained_labels.txt
image_dir=flower_photos
When I run these codes, directory of flower_photos should be inside docker container. However, I want this directory to be in my home directory instead (/user/documents/flower_photos). What should I do?
You could use a volume in order to associate a host folder to a container folder:
docker run -it \
...
-v /user/documents/flower_photos:/path/to/inception/flower_photos
That way, the inception module would find an existing folder with your host content.
Related
I'm currently following this tutorial to run a model on Docker that was built using the Google Cloud AutoML Vision:
https://cloud.google.com/vision/automl/docs/containers-gcs-tutorial
I'm having trouble running the container, specifically running this command:
sudo docker run --rm --name ${CONTAINER_NAME} -p ${PORT}:8501 -v ${YOUR_MODEL_PATH}:/tmp/mounted_model/0001 -t ${CPU_DOCKER_GCR_PATH}
I have my environment variables set up right (did an echo $<env_var>). I do not have a /tmp/mounted_model/0001 directory on my local system. My model path is configured to be the model location on the cloud storage.
${YOUR_MODEL_PATH} must be a directory on the host on which you're running the container.
Your question suggests that you're using the Cloud Storage bucket path but you cannot do this.
Reviewing the tutorial, I think the instructions are confusing.
You are told to:
gsutil cp \
${YOUR_MODEL_PATH} \
${YOUR_LOCAL_MODEL_PATH}/saved_model.pb
So, your command should probably be:
sudo docker run \
--rm \
--interactive --tty \
--name=${CONTAINER_NAME} \
--publish=${PORT}:8501 \
--volume=${YOUR_LOCAL_MODEL_PATH}:/tmp/mounted_model/0001 \
${CPU_DOCKER_GCR_PATH}
NB I added --interactive --tty to make debugging easier; it's optional
NB ${YOUR_LOCAL_MODEL_PATH} not ${YOUR_MODEL_PATH}
NB The command should not be -t ${CPU_DOCKER_GCR_PATH} omit the -t
I've not run through this tutorial.
OS: Windows server 2016
I have an App wrote in Go and put in a docker container. The App has to access "D:\test.db". How can I do that?
Using docker volumes and by using the -v or --mount flag when you start your container.
A modified example from the Docker docs:
$ docker run -d \
--mount source=myvol2,target=/app \
nginx:latest
you just need to replace nginx:latext with your image name and adapt source and target as you need.
Another example (also from the docs) using -v and mounting in read-only mode:
$ docker run -d \
-v nginx-vol:/usr/share/nginx/html:ro \
nginx:latest
I build a image of my web project with all my dependencies in the image at /app. When running the container it's start blazing fast and I'm able to access the application instantly.
However I build all the thing directly in the Dockerfile so the host has nothing except the Dockerfile.
So I try to retrieve the project files like so docker run -v $(pwd):/app image_name but it seems the folder is overrided because when trying to serve the public folder it can't be found anymore. By just exclude the volume option it's start well.
Am I right when I'm thinking it override my container folder?
Why did this works for the GitLab Project? (https://docs.gitlab.com/omnibus/docker/README.html#prerequisites)
They got all the project in the container, and mount it on the host.
sudo docker run --detach \
--hostname gitlab.example.com \
--publish 443:443 --publish 80:80 --publish 22:22 \
--name gitlab \
--restart always \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
gitlab/gitlab-ce:latest
For named volumes, the image contents are copied from the image to the named volumes upon volume creation. This named volume will have the contents of /bin copied to the volume:
docker run -ti -v busyboxbin:/bin busybox sh
Bind mounted directories are mounted in place and override any image contents. So this example would fail (unless you already had a copy of the files in /tmp/empty)
docker run -ti -v /tmp/empty:/bin busybox sh
The gitlab container will populate the bind mounted volume contents after the image was started. The logs are easy to add what's there. The data directory may need to be initialised by the app. The image probably comes with pre canned config to populate and run with if no files exist.
I know docker, but less about bitcoind.
Now I want to use this docker image to start my own test environment:
The description tells me:
docker volume create --name=bitcoind-data
docker run -v bitcoind-data:/bitcoin --name=bitcoind-node -d \
-p 8333:8333 \
-p 127.0.0.1:8332:8332 \
kylemanna/bitcoind
Now I want to now how I have to add my bitcoind.conf?
This isn't provided anywere? Can I use it at container startup or docker exec?
The repository contains a documentation file dedicated to your issue: https://github.com/kylemanna/docker-bitcoind/blob/master/docs/config.md
I'm starting to learn how to use TensorFlow to do machine learning. And find out docker is pretty convenient to deploy TensorFlow to my machine. However, the example that I could found did not work on my target setting. Which is
Under ubuntu16.04 os, using nvidia-docker to host jupyter and tensorboard service together(could be two container or one container with two service). And files create from jupyter should be visible to host OS.
Ubuntu 16.04
Dokcer
nvidia-docker
Jupyter
Tensorboard
Jupyter container
nvidia-docker run \
--name jupyter \
-d \
-v $(pwd)/notebooks:/root/notebooks \
-v $(pwd)/logs:/root/logs \
-e "PASSWORD=*****" \
-p 8888:8888 \
tensorflow/tensorflow:latest-gpu
Tensorboard container
nvidia-docker run \
--name tensorboard \
-d \
-v $(pwd)/logs:/root/logs \
-p 6006:6006 \
tensorflow/tensorflow:latest-gpu \
tensorboard --logdir /root/logs
I tried to mount logs folder to both container, and let Tensorboard access the result of jupyter. But the mount seems did work. When I create new file in jupyter container with notebooks folder, host folder $(pwd)/notebooks just appear nothing.
I also followed the instructions in Nvidia Docker, Jupyter Notebook and Tensorflow GPU
nvidia-docker run -d -e PASSWORD='winrar' -p 8888:8888 -p 6006:6006 gcr.io/tensorflow/tensorflow:latest-gpu-py3
Only Jupyter worked, tensorboard could not reach from port 6006.
I was facing the same problem today.
Short answer: I'm going to assume you are using the same container for both Jupyter Notebook and tensorboard. So, as you wrote, you can deploy the container with:
nvidia-docker run -d --name tensor -e PASSWORD='winrar'\
-p 8888:8888 -p 6006:6006 gcr.io/tensorflow/tensorflow:latest-gpu-py3
Now you can access both 8888 and 6006 ports but first you need to initialize tensorboard:
docker exec -it tensor bash
tensorboard --logdir /root/logs
About the other option: running jupyter and tensorboard in different containers. If you have problems mounting same directories in different containers (in the past there was a bug about that), since Docker 1.9 you can create independent volumes unlinked to particular containers. This may be a solution.
Create two volumes to store logs and notebooks.
Deploy both images with these volumes.
docker volume create --name notebooks
docker volume create --name logs
nvidia-docker run \
--name jupyter \
-d \
-v notebooks:/root/notebooks \
-v logs:/root/logs \
-e "PASSWORD=*****" \
-p 8888:8888 \
tensorflow/tensorflow:latest-gpu
nvidia-docker run \
--name tensorboard \
-d \
-v logs:/root/logs \
-p 6006:6006 \
tensorflow/tensorflow:latest-gpu \
tensorboard --logdir /root/logs
As an alternative, you can also use the ML Workspace Docker image. The ML Workspace is a web IDE that combines Jupyter, TensorBoard, VS Code, and many other tools & libraries into one convenient Docker image. Deploying a single workspace instance is as simple as:
docker run -p 8080:8080 mltooling/ml-workspace:latest
All tools are accessible from the same port. You can find information on how to access TensorBoard here.