Integrating Spyder (python IDE) with Docker container on the same mahcine - docker

I'm trying to start a Spyder kernel from a docker container and connect the Spyder IDE on the host to this kernel on the container (I tried follow this post, and the instructions on Spyder's site - Connect Spyder to a console in a docker container on a remote host).
However, I encountered the following problems:
Getting access to the kernel-pid.json file which is in the container. I tried to map the ~/.local/shared/jupyter/runtime folder on the host to the one on the container but I'm not sure it's the right way to do it (maybe I need to connect it via SSH, like in remote connection).
If I do map this folder on the host to the container, I need to give permissions (using sudo chmod) to the kernal-pid.json file, or else it says it doesn't have the appropriate permissions when I tried to connect to the kernel on the docker container.
If I give the appropriate permissions, it looks like it goes well, but then the process is stuck forever in "connecting to kernel" message (all of this is inside the Spyder IDE).
I have a docker version of 19.03, which have anaconda3 and python 3.7 installed on it, and on the host I also have anaconda3 (to run the Spyder IDE).

Related

Attaching IDE to my backend docker container stops that container's website from being accessible from host

Summary
I'm on mac. I have several docker containers, I can run all of them using docker-compose up and everything works as expected: for instance, I can access my backend container by searching http://localhost:8882/ on my browser, since port 8882 is mapped to the same port on host by using:
ports:
- "8882:8882"
Problems start when I try to attach an IDE to the backend container so as to be able to develop "from inside" that container.
I've tried using vscode's plugin "Remote - Containers" following this tutorial and also pycharm professional, which comes with the possibility to run docker configurations out of the box. On both cases I had the same result: I run the IDE configuration to attach to the container and its local website suddenly stops working, showing "this site can't be reached".
When using pycharm, I noticed that Docker Desktop shows that the backend container changed its port to 54762. But I also tried that port with no luck.
I also used this command:
docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
to get the container ip (172.18.0.4) and tried that with both ports, again, same result.
Any ideas?
Pycharm configuration
Interpreter:
This works in the sense that I can watch the libraries code installed inside the container:
Run/Debug configuration. This configuration succeeds in the sense that I can start it and it seems to be attached correctly to the backend container... though the problem previously described appears.
So, there were many things taking part in this, since this is an already huge and full of technical debt project.
But the main one is that the docker-compose I was using was running the server using uwsgi on production mode, which tampered many things... amongst which were pycharm's ability to successfully attach to the running container, debug, etc.
I was eventually able to create a new docker-compose.dev.yml file that overrided the main docker-compose file, only to change the backend server command for flask on development mode. That fixed everything.
Be mindful that for some reason flask run command inside a docker container does not allow you to see the website properly until you pass a -host=0.0.0.0 option to it. More in https://stackoverflow.com/a/30329547/5750078

VSCode cannot access Docker image inside wsl2 image

I am running a cuda enabled docker container inside wsl2 and when i try to use VSCode to run scripts inside it i cannot connect to it. The message in the docker plugin window is:
Error: connect EACCESS /var/run/docker.sock
I added the default user in the wsl to the docker group. I double-checked and i know for sure that i can run docker as a user. How do I elevate the VSCode privillages to make it able to access the docker inside the wsl image?
Btw. I cannot use docker desktop because it does not allow to use the gpu inside the container.
UPDATE
So with the set of plugins locally:Docker, Remote - WSL, Remote - Containers, Remote SSH, Remote Development, Remote SSH - editing configuration files
in the WSL: Docker Explorer, Docker, Docker Extension Pack
I am able to log from VSCode directly to the console and perform other basic operations (starting containers etc). Attaching VSCode is still impossible as for now. At least the menu option under RMB throws an error

VSCode in-container-debugging over SSH machine

I am trying to setup and advanced configuration with VScode insider and I ma facing an issue.
My setup is:
VSCode Running in my local machine Windows 10 with a Django source code hosted on my machine. I have no docker client on this machine, and I don't wont to install one...
A virtual machine with Ubuntu is running a docker daemon, docker client and docker-compose. My workspace is shared over vboxfs and mounted on my Ubuntu
A python docker container is running in the Ubuntu machine and running the mounted code.
I tried to use the Remote Extension to debug the python code inside the container. However, when a run my vscode inside on the remote SSH Taget (so the ubuntu machine), I am able to manage docker objects (images, containers, etc...) using the Docker extension of vscode, but I can't see the option: Remote-Containers: Open Folder in Container. It's not found in the F1 command... I can see the other related command like: Remote-Containers: Settings.
Do you have any idea ? Or my setup is not supported by the extension ? It seems like it supports SSH development or Container development but not mixing both together, right ?
Is there any other VSCode config to debug in my targeted setup ?
Regards

VS Code: connect a docker container in a remote server

I want to work in a container in a remote server.
But it doesn't work.
Environment:
Local: Windows 10
Local Terminal for ssh: WSL in Windows 10
Server: Ubuntu 18.04
I checked these two articles.
https://code.visualstudio.com/docs/remote/containers-advanced
https://code.visualstudio.com/docs/containers/ssh
I followed these steps.
I installed [Remote Development] extension in VS Code.
Remote-SSH: Connect to host. It works fine.
I Installed [Docker] extension on the remoter server.
Now I can see my containers and images in a docker tab.
I clicked one container and clicked [Attach Visual Studio Code] and it says There are no running containers to attach to.
I resolved this problem by switching to the remote server's Docker context on my local machine:
docker context create some-context-label --docker "host=ssh://user#remote_server_ip"
docker context use some-context-label
docker ps
# A list of remote containers on my local machine! It works!
After that:
Connect via Remote-SSH to the container server
Right click relevant container -> the "Attach Visual Studio Code"
That works for me.
(Note: One would think that I should be able to just use my local VSCode (skip step 1) to connect to said remote container after switching my local context, but VSCode complains Failed to connect. Is docker running? in the Docker control pane.)
I solve this issue using SSH tunneling following the steps found in https://florian-kriegel.de/blog/?p=234
Summarizing:
Set (or add) "docker.host": "tcp://localhost:23750" in settings.json
in VSCode.
Open a SSH tunnel like this in your local machine
changing the user and hostname by the remote machine (where the docker daemon is running) credentials:
ssh -NL localhost:23750:/var/run/docker.sock user#hostname.
Now, in the docker tab, you will be able to see and attach to containers in the remote machine.
Note that the Remote SSH Extension is not used in this case.
This might sound very strange, but for me, I had to open a folder on the remote SSH server prior to using the Remote Containers extension in VS Code. If I didn't do that, then it would constantly try to find the docker service running locally, even though the terminal tab was connected to the remote SSH server.
This seems very weird, because if you're conncted via SSH in VS Code, then the extension should assume you're trying to attach to the container on the remote server. Shouldn't have to open a remote folder first.
By "opening a folder" on the remote server, the Remote Containers extension was then able to attach VS code to the container running on the remote SSH server. I didn't have to do any of the steps in any of those articles. Just simply use Remote SSH to connect VS Code remotely via SSH, open a folder, and then use Remote Containers.
Solution using the "Remote SSH" and the "Remote Explorer" extension in Visual Studio Code.
Following the steps above (https://stackoverflow.com/a/61728799/11687201) I figured out how to make use of the SSH Remote and Remote Explorer Extension. The first step is the same as above:
Open the settings.json file in VSCode, press F1 and select ">Preferences: Open Settings (JSON)" and add/edit the following line:"docker.host": "tcp://localhost:23750"
Open the ssh config file, click on the "Remote Explorer" Extension, then click on the "SSH Targets" "Configure" button and open the ssh config file.
Add the following line to your ssh connection:
LocalForward localhost:23750 /var/run/docker.sock
Remark: Previously I used the solution described earlier in this thread (https://stackoverflow.com/a/61728799/11687201). I had to reboot both machines the local machine and remote machine before the solution described below worked out.
Afterwards I have to use multiple VSCode Windows:
Local Machine: Start VSCode and use the "Remote Explorer" to connect to the remote machine using a new VSCode window
VSCode window connected to remote (SSH)
→ startup the Docker container of your choice
(I was not able to "Attach Visual Studio Code" from this VSCode window)
VSCode window connected to local machine
→ Click on the "Docker" extension, the docker containers running on the remote get listed. Attach VSCode to a running container using one of the folling options:
Right-click on the desired container and chose "Attach Visual Studio Code"
Press F1 and chose">Remote-Containers: Attach to Running Container..." and select the container of your choice afterwards
A third VSCode window will open being attached to the Docker container.
Pros and cons of this solution
(+) Using the "Remote Explorer" extension I can directly connect and open a previously used project folder on my remote machine with one click
(-) 3 VSCode windows (local machine, remote ssh and remote container) are needed instead of 2 VSCode windows
Do you see the error message as of following?
Failed to connect. Is Docker running?
Error: connect EACCES /var/run/docker.sock
Error Message on VSCode
It's because VSCode uses /var/run/docker.sock of remote host to communicate with the Docker service.
There're two methods.
Method 1. (Secure, Need reboot or logging out) After executing following code of dockerode npm getting error "connect EACCES /var/run/docker.sock" on ubuntu 14.04
Method 2. (Instant effect. Use it if you're not dealing with production server)
Run the following command on SSH console.
sudo chmod o+rw /var/run/docker.sock
For some reason, this problem is fixed for me when I open a folder in the remote window before trying to attach to a container.
I found Daniel's answer really helpful but didn't work for me. I put my two cents.
TL;DR
Create a new docker context for the remote machine where remote container is running.
docker context create some-context-label --docker "host=ssh://user#remote_server_ip"
docker context use some-context-label
Just open VSC, go to Docker (you should have installed the extension) tab and you'll see listed all running containers from the remote context you recently created.
Right click on your desired container and attach visual studio code
You can also use the remote-explorer tab, just select containers from the dropdown at the top left.
Why not to ssh remote host
When attaching visual studio code to a container, you can check logs by clicking the notification Setting up Remote-Containers (show log) at the bottom left. There, you can check that:
...
[26154 ms] Start: Run: ssh some-remote-host /bin/sh
[26160 ms] Start: Run in host: id -un
Here, my guess is that it's trying to ssh to the remote host from itself ,since we already connected via remote-ssh.
If you can reach the remote node running Docker engine via SSH why you need yet another SSH server inside the container? From the host running your container, it is possible and safe to use tty, i.e. attach.
I don't think that this is not a good idea to use SSHD running inside the container although it is possible. To be useful SSHD has to listen to non-conflict port in every container. Otherwise, 2 containers occasionally exposing the same port on the same node will conflict like any other service running on same the node.
Of course, ports can be randomized using -P option but it is not so convenient. It is also less convenient to manage keys and users at the container level than at host level where all machinery is provided by the Host software.
Loading every container with SSHD increases the container size. In Kubernetes, every container is reachable without any SSHD running inside containers via pass Pod->Container because Pod, has IP and containers are attachable by id, i.e. "Docker-host->container"
Step 1 - Docker daemon on the remote machine
make sure your remote Docker daemon can accept connections from your host
for testing purposes, I use the following command on the remote
machine to force Docker daemon to listen on port 4243 on all IPs,
beware this is not secure
There is no support for reading a file from /etc/sysconfig or elsewhere to modify the command line. Fortunately, systemd gives us the tools we need to change this behavior.
The simplest solution is probably to create the file /etc/systemd/system/docker.service.d/docker-external.conf (the exact filename doesn't matter; it just needs to end with .conf) with the following contents:
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:4243 -H unix:///var/run/docker.sock
And then:
systemctl daemon-reload
systemctl restart docker
Step 3 - Opening Docker Ports Using FirewallD
firewall-cmd --permanent --zone=public --change-interface=docker0
firewall-cmd --permanent --zone=public --add-port=4243/tcp
firewall-cmd --reload
Step 4 - Set (or add) "docker.host": "tcp://localhost:4243" in settings.json in VSCode.

Can not run docker command on windows 7

can not run docker on windows7 command after installation, and there is nothing in virtualbox.
I have tried uninstall and install other versions, run install application administrately and try another dirver but above does not work.
...
when installation finished, i clicked the docker quickstart terminal, and it show me this error message:
dial tcp 127.0.0.1:53822: connectex: No connection could be made because the tar
get machine actively refused it.
Looks like something went wrong in step ´Checking if machine default exists´...
Press any key to continue...
Just a reminder, there are sort of 3 ways to use docker on Windows, if one doesn't work you may want to try another one:
use docker Desktop: it requires windows 10 Pro (In fact Hyper-V maybe it's an option for you)
use docker Toolbox: the one you may have installed on windows 7, it uses Virtualbox and a docker machine VM.
use Virtualbox directly: install a linux distribution with shared folder only for your docker usage
Now since you are on option 2, you should have a virtualbox installed, so you can check if the underlying docker machine is launched by launching virtualbox, it should list the installed vm.
If the vm is not started: you will have to search the problem on virtualbox side, error logs or popup: trying to launch the vm manually directly from virtualbox application will help you
If the vm is started: there may be a connection problem between your host (windows) and your vm (boot2docker), you could try to connect to the vm via bash using "boot2docker ssh" as it is indicated in the quick start. If it works, the connection problem could come from a firewall.
Check if the quick start happened correctly when you have luanched it.
I don't want to make proselitism on other solution, but it may be simpler to use a linux directly in virtualbox (option 3), at least at the beginning to get more familar with docker.
The other simpler option would be to use windows 10
I'm sorry it's not a complete solution, but indications, and it was too long for a comment.
docker server is not running properly. check is you can run docker ps or docker version command successfully.

Resources