I started a docker container with the following command:
docker run -i -p 8989:8989 -p 8888:8888 -v /Users/sroche/FISHDATA/tiff_stacks:/home/neuroglancer_user/test_data -t openmind/neuroglancer:1.0 jupyter notebook --ip='0.0.0.0'
This creates a container running jupyter notebook and neuroglancer. These specifics are not too important, but may be helpful.
I have a notebook inside the container that generates a link, and this link is supposed to allow me to use my browser to view images at the address in the link. However, I am not allowed from connecting to the link, with the response:
(I am simply putting the link into a Chrome browser)
The link I am trying to access looks like:
http://127.0.0.1:42071/v/.../
How can I connect to a link that is generated within a Docker container?
Related
I am very new to docker and Jupyter notebook. I pulled the image from docker, it was able to direct me to the relevant Jupyter notebook. Problem is, whatever plots I am making in the notebook, I am not able to find the file in the system. A file with the name settings.cmnd should be made on my system. I am using Windows 10 home version. I am using the following command
docker run -it -v "//c/Users/AB/project":"//c/program files/Docker Toolbox" -p 8888:8888/tcp CONTAINER NAME
It is running fine as I am able to access the jupyter notebook but the file is still missing on my system.
Here the folder in which I want to save file is project
Kindly help.
I did not find an image called electronioncollider/pythiatutorial, so I'm assuming you meant electronioncollider/pythia-eic-tutorial.
Default working directory for that image is /code so the command on Windows should look like:
docker run --rm -v //c/Users/AB/project://code -p 8888:8888 electronioncollider/pythia-eic-tutorial:latest
Working dierctory can be changed with -w, so the following should work as well:
docker run --rm -w //whatever -v //c/Users/AB/project://whatever -p 8888:8888 electronioncollider/pythia-eic-tutorial:latest
Edit:
electronioncollider/pythia-eic-tutorial:latest image has only one version - one that is meant to run on linux/amd64. This means it's meant to run on 64-bit Linux installed on a computer with Intel or AMD processor.
You're not running it on Windows, but on a Linux VM that runs on your Windows host. Docker can access C:\Users\AB\project, because it's mounted inside the VM as c/Users/AB/project (although most likely it's actuall C:\Users that's mounted as /c/Users). Therein lies the problem - Windows and Linux permission models are incompatible, so the Windows directory is mounted with fixed permissions that allows all Linux users access. Docker then mounts that directory inside the container with the same permissions. Unfortunately Jupyter wants some of the files it creates to have a very specific set of permissions (for security reasons). Since the permissions are fixed to a specific value, Jupyter cannot change them and breaks.
There are two possible solutions
Get inside whatever VM the Docker is running inside, change directory to one not mounted from Windows, and run the container from there using the command from the tutorial/README:
docker run --rm -u `id -u $USER` -v $PWD:$PWD -w $PWD -p 8888:8888 electronioncollider/pythia-eic-tutorial:latest
and the files will appear in the directory that the command is run from.
Use the modified image I created:
docker run --rm -v //c/Users/AB/project://code -p 8888:8888 forinil/pythia-eic-tutorial:latest
You can find the image on Docker Hub here. The source code is available on GitHub here.
Edit:
Due to changes in my version of the image the proper command for it would be:
docker run -it --rm -v //c/Users/AB/project://code --entrypoint rivet forinil/pythia-eic-tutorial
I release a new version, so if you run docker pull forinil/pythia-eic-tutorial:latest, you'll be able to use both the command above, as well as:
docker run -it --rm -v //c/Users/AB/project://code forinil/pythia-eic-tutorial rivet
That being said I did not receive any permission errors while testing either the old or the new versions of the image.
I hope you understand that due to how Docker Toolbox works, you won't be able to use aliases the way the tutorial says you would on Linux.
For one thing, you'll only have access to files inside directory C:\Users\AB\project, for another file path inside the container will be different than outside the container, eg. file C:\Users\AB\project\notebooks\pythiaRivet.ipynb will be available inside the container as /code/notebooks/pythiaRivet.ipynb
Note on asking questions:
You've got banned from asking questions, because your questions are low quality. Please read the guidelines before asking any more.
I wanted to use following command to open jupyter:
docker run --runtime=nvidia --name tensorflow1 -it -p 8888:8888 -p 6006:6006 tensorflow/tensorflow:latest-gpu-py3-jupyter
I can't open it with browser. The systerm looks good because it says:
To access the notebook, open this file in a browser:
file:///root/.local/share/jupyter/runtime/nbserver-8-open.html
Or copy and paste one of these URLs:
http://(568ebbf84a86 or 127.0.0.1):8888/?token=17fc57d57c89f56c460748f464b488c59f8ddccf5793e7
But when I open it with external ip address, I can't connect and system says:
[W 06:15:52.336 NotebookApp] 404 GET http://110.249.212.46/testget?q=23333&port=8888 (110.249.212.46) 38.11ms referer=None
I have built external ip address and built firewall.
There is no problem if I use the following command:
docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu
and following test has pasted:
python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
how to solve this problem?
After one month debug, finally,finally,finally,finally,finally,finally,finally,finally,finally,finally,finally,finally
I fond the answer:
google cloud platform only open one port for docker deployment which is 8080
so if you want to open docker jupyter notebook, you should use the following method:
docker run --runtime=nvidia --name tensorflow1 -it -p 8080:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter
pay attention to 8080:8888
I've searched for a number of hours to find the answer to this because I want to respect the idea of non-redundant posts. I'm sure the fact that I'm relatively new to programming doesn't help but here's my issue (hopefully I present it in the correct way):
I am taking a Computational Finance class that requires the package 'qstk'. There is a docker image setup with everything needed for the course, under the docker file 'ruippeixotog/qstk'. I followed the following instructions to access this docker file via the Jupiter notebook and was successful but ever since that first use of the file, I can no longer access the notebook.
I have the current version of Docker downloaded and follow the same instructions to access the notebook via a web browser (I've tried Safari, Chrome, and Firefox) but always get the error code "
install docker whatever OS you have (it officially supports Windows,
Mac, Linux) by following the instructions on the Docker website start
it, still following the instructions on the website
List item
execute on command line: docker run -dt -p 8888:8888 --name qstk
ruippeixotog/qstk
open your browser at http://localhost:8888 and you have a python
interpreter with QSTK fully configured
The source repository is at: https://github.com/ruippeixotog/docker-qstk
My process:
I run docker with: docker run -dt -p 8888:8888 --name qstk ruippeixotog/qstk
I go to http://localhost:8888
And I get the response from the browser: "Failed to open page...server unexpectedly dropped the connection"
If I run:
I run docker with: docker run -dt -p 8888:8888 --name qstk ruippeixotog/qstk
jupyter notebook
It brings me to my tree of file folders (localhost:8888/tree), where the two files
"QSTK-0.2.5.tar.gz" and
"QSTK-0.2.5.tar.gz.cpgz"
Are located, along with a bunch of my other folders. When I try to edit the first of those two files, it opens a new tab and tells me "SSL is required." within the jupyter editor page for that file, showing nothing else
When I try to edit the second, it shows only "Error! /Users/.../QSTK-0.2.8
Can anyone help with this?
I have tried linking my docker containers but it seems to give error on access.
My structure is as following:
Database docker(Mysql) - Container name is um-mysql
Back-end docker(Tomcat) - Image name is cz-um-app
Front-end docker(Nginx) - Image name is cz-um-frontend
Linking of Back-end with Database docker is done as following and it works perfectly:
$ docker run -p 8080:8080 --name backendservices --link um-mysql:um-mysql cz-um-app
The linking of Front-end with Back-end is done as following:
$ docker run -p 80:80 --name frontend --link backendservices:backendservices cz-um-frontend
But, linking of Front-end with Back-end is not working.
I have a login page, on submit, it accesses a url http://backendservices:8080/MyApp
In console, it shows error as:
net::ERR_NAME_NOT_RESOLVED
Not sure why linking of back-end container with database works fine and not the same case of front-end with back-end. Do I need to configure some settings in Nginx for this?
The hosts entry is as following and I am able to ping backendservices too:
First you don't need to map 8080:8080 for backendservices: any EXPOSEd port in backendservices image is visible by any other container linked to it. No host port mapping needed.
Secondly, you can check in your front end if the backend has been register:
docker exec -it frontend bash
cat /etc/hosts
If it is not, check docker ps -a to see if backend is still runnong.
I've installed the tensorflow docker container on an ubuntu machine. The tensorflow docker setup instructions specify:
docker run -it b.gcr.io/tensorflow/tensorflow
This puts me into the docker container terminal, and I can run python and execute the Hello World example. I can also manually run .\run_jupyter.sh to start the jupyter notebook. However, I can't reach the notebook from host.
How do I start the jupyter notebook such that I can use the notebook from the host machine? Ideally I would like to use docker to launch the container and start jupyter in a single command.
For a Linux host Robert Graves answer will work, but for Mac OS X or Windows there is more to be done because docker runs in a virtual machine.
So to begin launch the docker shell (or any shell if you are using Linux) and run the following command to launch a new TensorFlow container:
docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow ./run_jupyter.sh
Then for Mac OS X and Windows you need to do the following only once:
Open VirtualBox
Click on the docker vm (mine was automatically named "default")
Open the settings by clicking settings
In the network settings open the port forwarding dialog
Click the + symbol to add another port and connect a port from your mac to the VM by filling in the dialog as shown below. In this example I chose port 8810 because I run other notebooks using port 8888.
then open a browser and connect to http://localhost:8810 (or whichever port you set in the host port section
Make your fancy pants machine learning app!
My simple yet efficient workflow:
TL;DR version:
Open Docker Quickstart Terminal. If it is already open, run $ cd
Run this once: $ docker run -it -p 8888:8888 -p 6006:6006 -v /$(pwd)/tensorflow:/notebooks --name tf b.gcr.io/tensorflow/tensorflow
To start every time: $ docker start -i tf
If you are not on windows, you should probably change /$(pwd) to $(pwd)
You will get an empty folder named tensorflow in your home directory for use as a persistent storage of project files such as Ipython Notebooks and datasets.
Explanation:
cd for making sure you are in your home directory.
params:
-it stands for interactive, so you can interact with the container in the terminal environment.
-v host_folder:container_folder enables sharing a folder between the host and the container. The host folder should be inside your home directory. /$(pwd) translates to //c/Users/YOUR_USER_DIR in Windows 10. This folder is seen as notebooks directory in the container which is used by Ipython/Jupyter Notebook.
--name tf assigns the name tf to the container.
-p 8888:8888 -p 6006:6006 mapping ports of container to host, first pair for Jupyter notebook, the second one for Tensorboard
-i stands for interactive
Running TensorFlow on the cloud
After further reading of docker documentation I have a solution that works for me:
docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow ./run_jupyter.sh
The -p 8888:8888 and -p 6006:6006 expose the container ports to the host on the same port number. If you just use -p 8888, a random port on the host will be assigned.
The ./run_jupyter.sh tells docker what to execute within the container.
With this command, I can use a browser on the host machine to connect to http://localhost:8888/ and access the jupyter notebook.
UPDATE:
After wrestling with docker on windows I switched back to a Ubuntu machine with docker. My notebook was being erased between docker sessions which makes sense after reading more docker documentation. Here is an updated command which also mounts a host directory within the container and starts jupyter pointing to that mounted directory. Now my notebook is saved on the host and will be available next time start up tensorflow.
docker run -p 8888:8888 -p 6006:6006 -v /home/rob/notebook:/notebook b.gcr.io/tensorflow/tensorflow sh -c "jupyter notebook /notebook"
Jupyter now has a ready to run Docker image for TensorFlow:
docker run -d -v $(pwd):/home/jovyan/work -p 8888:8888 jupyter/tensorflow-notebook
These steps worked for me if you are a total docker noob using a windows machine.
Versions: Windows 8.1, docker 1.10.3, tensorflow r0.7
Run Docker Quickstart Terminal
After it is loaded, note the ip address. If you can't find it use this docker-machine ip and make a note. Lets call it 'ip address'. Will look something like this: 192.168.99.104 (I made up this ip address)
Paste this command on the docker terminal:
docker run -p 8888:8888 -p 6006:6006 b.gcr.io/tensorflow/tensorflow.
If you are running this for the first time, it will download and install the image on this light weight vm. Then it should say 'The Jupyter notebook is running at ....' -> This is a good sign!
Open your browser at: <your ip address (see above)>:8888. Eg. 192.168.99.104:8888/
Hopefully you can see your ipython files.
To get this to run under hyper-v. Perform the following steps:
1) Create a docker virtual machine using https://blogs.msdn.microsoft.com/scicoria/2014/10/09/getting-docker-running-on-hyper-v-8-1-2012-r2/ this will get you a working docker container. You can connect to it via the console or via ssh. I'd put at least 8gb of memory since I'm sure this will use a lot of memory.
2) run "ifconfig" to determine the IP address of the Docker VM
3) On the docker shell prompt type:
docker run -p 8888:8888 -p 6006:6006 -it b.gcr.io/tensorflow/tensorflow
4) Connect to the Jupyter Workbench using http:/[ifconfig address]:8888/
To tidy up the things a little bit, I want to give some additional explanations because I also suffered a lot setting up docker with tensorflow. For this I refer to this video which is unfortunately not selfexplanatory in all cases.
I assume you already installed docker. The really interesting general part of the video starts at minute 0:44 where he finally started docker. Until there he only downloads the tensorflow repo into the folder, that he then mounts into the container. You can of course put anything else into the container and access it later in the docker VM.
First he runs the long docker command docker run –dit -v /c/Users/Jay/:/media/disk –p 8000 –p 8888 –p 6006 b.gcr.io/tensorflow/tensorflow. The “run” command starts containers. In this case it starts the container “b.gcr.io/tensorflow/tensorflow”, whose address is provided within the tensorflow docker installation tutorial. The container will be downloaded by docker if not already locally available.
Then he gives two additional kinds of arguments: He mounts a folder of the hostsystem at the given path to the container. DO NOT forget to give the partition in the beginning (eg. "/c/").
Additionally he declares ports being available later from the host machine with the params -p.
From all this command you get back the [CONTAINER_ID] of this container execution!
You can always see the currently running containers by running “docker ps” in the docker console. Your container created above should appear in this list with the same id.
Next Step: With your container running, you now want to execute something in it. In our case jupyter notebook or tensorflow or whatever: To do this you make docker execute the bash on the newly created container: docker exec –ti [CONTAINER_ID] bash. This command now starts a bash shell on your container. You see this because the “$” now changed to root#[CONTAINER_ID]:. From here is no way back. If you want to get back to the docker terminal, you have to start another fresh docker console like he is doing in minute 1:10. Now with a bash shell running in the container you can do whatever you want and execute Jupiter or tensorflow or whatever. The folder of the host system, you gave in the run command, should be available now under “/media/disk”.
Last step accessing the VM output. It still did not want to work out for me and I could not access my notebook. You still have to find the correct IP and Port to access the launched notebook, tensorboard session or whatever. First find out the main IP by using docker-machine –ls. In this list you get the URL. (If it is your only container it is called default.) You can leave away the port given here. Then from docker ps you get the list of forwarded ports. When there is written 0.0.0.32776->6006/tcp in the list, you can access it from the hostmachine by using the port given in the first place (Awkyard). So in my case the executed tensorboard in the container said “launched on port 6006”. Then from my hostmachine I needed to enter http://192.168.99.100:32776/ to access it.
-> And that’s it! It ran for me like this!
It gives you the terminal prompt:
FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i
docker run -it tensorflow/tensorflow:r0.9-devel
or
FOR /f "tokens=*" %i IN ('docker-machine env --shell cmd vdocker') DO %i
docker run -it b.gcr.io/tensorflow/tensorflow:latest-devel
You should have 'vdocker' or change vdocker to 'default'.
For some reason I ran into one additional problem that I needed to overcome beyond the examples provided, using the --ip flag:
nvidia-docker run --rm \
-p 8888:8888 -p 6006:6006 \
-v `pwd`:/root \
-it tensorflow/tensorflow:latest-devel-gpu-py3 sh -c "jupyter notebook --ip 0.0.0.0 ."
And then I can access via http://localhost:8888 from my machine. In some ways this makes sense; within the container you bind to 0.0.0.0 which represents all available addresses. But whether I need to do this seems to vary (e.g I've started notebooks using jupyter/scipy-notebook without having to do this).
In any case, the above command works for me, might be of use to others.
As an alternative to the official TensorFlow image, you can also use the ML Workspace Docker image. The ML Workspace is an open-source web IDE that combines Jupyter, VS Code, TensorFlow, and many other tools & libraries into one convenient Docker image. Deploying a single workspace instance is as simple as:
docker run -p 8080:8080 mltooling/ml-workspace:latest
All tools are accessible from the same port and integrated into the Jupyter UI. You can find the documentation here.