I have a unique Docker issue. I am developing an application which needs to connect to multiple Docker containers. The gist is, that this application will use the Docker SDK to spin up containers and connect to them as needed.
However, due to the nature of the application, we should assume that each one of these containers is compromised and unsafe. Therefore, I need to separate them from the host network (so they cannot access my devices and the WAN). I still have the constraint of needing to connect to them from my application.
It is a well-known problem that the macOS networking stack doesn't support connecting to a docker network. Normally, I'd get around this by exposing a port I need. However, this is not possible with my application, as I am using internal networks with Docker.
I'd like to accomplish something like the following. Imagine Container 2 and Container 3 are on their own private internal network. The host (which isn't a container) is controlling the Docker SDK and can query their internal IPs. Thus, it can easily connect to these machines without this network being exposed to the network of the host. Fortunately, this sort of setup works on Linux. However, I'd like to come up with a cross platform solution that works on macOS.
I had a similar situation. What I ended up doing was:
The app manages a dynamic container-to-port mapping (just a hash table).
When my app (on the host) wants to launch a container, it finds an unused port in a pre-defined range (e.g. 28000-29000).
Once it has a port, it maps the container's port to some port in a pre-determined range (e.g. -p 28003:80).
When my app needs to refer to a container, it uses localhost:<port> (e.g. localhost:28001).
It turns out to not be a lot of code, but if you go that route, make sure you encapsulate the way you refer to containers (i.e. don't hard-code the hostname and port, use a class that generates the string).
All that said, you should really do some testing with a VM deployment option before you rule it out as too slow.
Docker is a wonderful tool for running/deploying your application in a well-defined, controlled environment, and is well supported by e.g. the GitLab CI or by MS Azure.
We would like to use it also in the development phase, so that all developers have the same environment available. Of course, we want to keep the image as light as possible and we do not want e.g. any IDE or other development tool inside of it.
So the actual development takes place outside of docker.
Running our (python) application inside of docker is no problem, but debugging it is not trivial: I do not know of a way to attach a debugger to an application running inside docker. In theory this should be possible, but how does one do it?
Additional info: we use visual studio code, that does have some docker, plugin, but nothing of this sort is mentioned.
Turns out that this is possible, following the same steps needed for remote debugging.
The IP address of the docker image can be retrieved through:
docker inspect <container_id> | grep -i ip
just be sure to add at the beginning of your application:
import ptvsd
# Allow other computers to attach to ptvsd at this IP address and port, using the secret
ptvsd.enable_attach(secret=None, address = ('0.0.0.0', 3000))
ptvsd.wait_for_attach()
'0.0.0.0' means on all interfaces.
For vscode, the last steps consists in adapting the python: Attach configuration, specifying the address and the remote and local roots for your script.
However, for some mysterious reason my breakpoints are ignored.
Windows Firewall is blocking my attempt to allows Docker for Windows to share C: on windows 10 machine.
Works fine when Windows Firewall off. When its on I get
A firewall is blocking file Sharing between Windows and the
containers. See documentation for more info.
The documentation says
You do not need to open port 445 on any other network. By default,
allow connections to 10.0.75.1 port 445 (the Windows host) from
10.0.75.2 (the virtual machine).
I am "googled out" on trying to find how to do that - can someone advise?
You don't actually need to share the C drive but only reinstall (or potentially even uncheck - click OK - then check) the File and Print sharing service on the Hyper-V virtual network card.
See this article.
Also if there are restrictions on your network profiles (i.e. public), consider changing the default "unidentified network" for the "vEthernet (DockerNAT)" card to private via this PowerShell command before doing the above:
Set-NetConnectionProfile -interfacealias "vEthernet (DockerNAT)" -NetworkCategory Private
Ok, so after running in the same issue,
I have found a Solution.
This is what I did:
Step 1:
Open ESET. Then click on Setup
Step 2:
Click on Network protection
Step 3:
Click on Troubleshooting wizard
Step 4:
Find the Communication 10.0.75.2 (Default docker IP setting)
Just check what the IP Range is defined inside your docker settings. Then look for for the IP which resides in that range.
Step 5:
Click on the Unblock button, then you should receive this screen.
This solved the issue for myself.
You can then go to the Rules and check the rule that was added.
PS: This is my first post, sorry for any incorrect procedures.
Only this solution helps me:
Go to Hyper-V Manager -> Virtual Switch Manager -> DockerNAT -> Connection Type: change from internal to private, apply, change back to internal, apply
Restart MobyLinuxVM
Restart Docker
Set Docker network profile to 'Private'. Run command in PowerShell as admin
Set-NetConnectionProfile -interfacealias "vEthernet (DockerNAT)" -NetworkCategory Private
Reset File and Printer Sharing for Microsoft Networks on DockerNAT connection
Go to Docker -> Settings -> Shared Drives and share C:
My solution was to disconnect from the VPN; that was causing the issue
I was not using any third party firewalls when running into this error. I was convinced it was a Windows Firewall issue, though disabling Windows Firewall did not work for me. I finally found this blog post after much research: Docker on windows 10 error: A firewall is blocking file Sharing ...
It ended up NOT having to do with the built in Windows Firewall.
The Fix
Uncheck File and Printer Sharing for Microsoft Networks from the vEthernet (DockerNAT) network adapter (you can find the connection in the Windows Network and Sharing Center).
Recheck it and make sure it is enabled.
My C drive stopped being shared with Docker after a recent Windows 10 update. I was getting the same problem saying it was blocked by the Windows firewall when attempting to reshare it.
Looking through the above solutions, I found something that worked for me that is simpler than anything else I saw on this page. In Control Panel\All Control Panel Items\Network and Sharing Center, on the vEthernet (DockerNAT) connection, I unchecked the property File and Printer Sharing for Microsoft Networks and saved the setting. Then I checked the property again to reenable it and saved it again.
At this point, I was able to reshare the C drive in Docker settings. I have no idea why this worked but it was not a firewall problem, which already have an entry for DockerSmbMount.
I had this problem with Kaspersky; turning off Kaspersky worked, so I knew it was the firewall. In my case, Kaspersky was already blocking port 445 for some reason. I had to go to Packet Rules for the Firewall, Local Services (TCP) and remove 445 from the list of blocks ports.
For Windows 10 machines on domain networks, when creating the Hyper-V virtual Ethernet adapter, it gets categorized as a public network. You have to change it to a private network to allow the more relaxed Windows Firewall rules, and therefore allow file sharing.
Run the following command in PowerShell:
Set-NetConnectionProfile -interfacealias "vEthernet (DockerNAT)" -NetworkCategory Private
Change the name in quotes if your machine's virtual Hyper-V network connection is called something else.
My problem was Cisco Anyconnect VPN interfered with internal docker networking
to fix this go to:
Cisco Anyconnect Settings > Preferences >
check Allow local (LAN) access when using VPN
As stated in one other answer Docker doesn't play nice with a VPN.
If you're using Nordvpn you have to disable "Invisibility on LAN" and probably "Internet Kill Switch".
If you've done so it should work even with the VPN active.
That depends on what firewall do you have installed. In my case I do have disabled the built-in Windows Firewall and I am using ESET Smart Security so my rules looks like:
Create a rule for IN connection since you should allow Docker to connect to your host and set it to Allow
Setup the port properly as explained in docs meaning 445:
Setup the remote IP address:
Maybe this is not the answer since it's not related to Windows Firewall but could give you a clue in what to do.
In my case, I disabled "Block TCP 445" on Windows Defender Firewall with Advanced Security and it worked. Then enabled it again after setting shared drives on Docker.
None of the above worked for me.
What finally did the trick was opening the properties of the "vEthernet (DockerNAT)" network and ticking the box "Hyper-V Extensible Virtual Switch" at the bottom of the list under the "Networking" tab.
Not sure if this is the actual fix or whether it just somehow reset the network adapter for me... but it worked!
Seem like many having this issue when running Cisco AnyConnect. I got the same problem and here is how I solved:
The cause: The subnet being used by Docker is in the list of Secured Routes managed by Cisco AnyConnect (I believe this list is managed by your VPN's admin).
The solution: Change the subnet used by Docker to not overlap with the list being managed by AnyConnect.
For example, in my case, I changed from 10.0.75.0 (which was overlapped with 10.0.0.0/8) to 172.31.0.0/28.
I had the same problem and tried all the fixes - and it turned out that more than one was necessary:
Add a firewall rule (Norton Security for me)
Make the network private
Share the drive
I've written a full explanation at http://kajabity.com/2017/08/unblock-docker-for-windows-firewall-issues-with-host-volumes/
My G drive stopped being shared with Docker after a recent Windows 10 update. I was getting the same problem saying it was blocked by the Windows firewall when attempting to reshare it.
Then I had tried to solve this issues by couple of suggestion but i cant resolve that issue after that I have tried to Reset credentials below of Shared Drives and my issue was solved.
So If you want then you can try to do this-
Everything everyone posted DID NOT WORK for me.
THIS DID.
I installed the EDGE release. I then went to WINDOWS DEFENDER and disabled the firewall for DOCKER NAT. (not my actual ethernet adapter, just the docker nat)
Once I disabled windows defender/firewall THAT way it worked fine.
ugh. really hopes that helps some of you!
Even after ensuring that the inbound firewall rule is set up properly and even after uninstalling and reinstalling the File and Printing Sharing Service it didn't work for me.
Solution: on top of that I also had to do a third thing. I had to deactivate the checkbox Prevent incoming connections when on a public network in the specific firewall settings for public networks. After doing that it started working for me as well. See screenshots attached at the end of this message.
Don't know how long this option has been there already. I'm currently working on Win 10 Pro 1709 16299.402.
1. Open specific firewall settings for public networks
2. Uncheck this checkbox
I had same issue with F-secure, DeepGuard was blocking the Docker service. My solution was:
Open F-secure client and click "Tasks"
Choose "Allow a program to start"
Choose from list "com.docker.service" and press "Remove"
After that restart Docker client and try to apply for file share.
Also very good troubleshoot guide here: Error: A firewall is blocking file sharing between Windows and the containers
I had the same problem with firewall not allowing me to share my C drive. I tried all listed above solutions and nothing worked for me. Then I uninstalled docker completely from my machine Control Panel\Programs\Programs and Features ->
select Docker for Windows -> Uninstall
Then go to docker website and click Get Docker for Windows (Stable)
After that I was able to share drive C
For AVG Internet Security, enabling Internet Connection Sharing Mode under the Firewall settings did the trick for me.
Using Kaspersky Internet Security you can resolve this easily if you change the vEthernet (DockerNAT) network adapter type to Trusted network.
Settings > Protection > Firewall > Networks > vEthernet (DockerNAT) > Select "Trusted Network"
In short, use the Edge version.
The Edge version of Docker for Windows since 2.1.5.0 (2019-11-04) has a new implementation of file sharing without requiring firewall modifications
https://docs.docker.com/docker-for-windows/edge-release-notes/
"New file sharing implementation: Docker Desktop introduces a new file sharing implementation which uses gRPC, FUSE, and Hypervisor sockets instead of Samba, CIFS, and Hyper-V networking. The new implementation offers improved I/O performance. Additionally, when using the new file system:
Users don’t have to expose the Samba port, and therefore do not experience issues related to IT firewall or the drive-sharing policy.
There is no need to provide user credentials to Docker Desktop. File access rights are automatically enforced when accessing mounted folders through containers.
"
You can download the edge version: https://download.docker.com/win/edge/Docker%20Desktop%20Installer.exe
Note as of now, the edge version of 2.1.7.0 will be used for next stable release: "Note: Docker Desktop Edge 2.1.7.0 is the release candidate for the upcoming major Stable release.". So the risk of using Edge is relatively low. Or you can wait for next stable release, to avoid all these firewall issues.
25 Jan 2020
It seems, it is an issue in the old versions, I installed the last version :
Docker version 19.03.5, build 633a0ea
and it worked for me without any configuration.
This (link below) seems to be the best fix i have found so far. It is persistant across reboots. It is best explained here: https://gist.github.com/biggyspender/8b5b2ed9ff63de31045d41304e3915b3
The vEthernet network interface seems to be dynamically created each time the system is started, and it is created in the 'Public' group as opposed to in the 'Private' group (where it works). The edits in one of the Docker startup scripts (called out in the link above) automate the powershell command / fix noted by David Tesar above....
I had more luck adding the function to the script as opposed to editing the script and changing 'Internal' to 'Private'.
If non of the above works, just make sure you're not connected to a VPN. That's exactly what happened to me, i was connected to a VPN using Cisco AnyConnect client, also make sure you set an static DNS in the docker settings.
I tried everything listed here and on https://github.com/docker/for-win/issues/360
Nothing worked for me.
I run my dev environment on a docker container, and while on Corporate VPN (Cisco AnyConnect), I need the to mount my local drive on the container to access my project files.
Here's a docker hack that worked for me. Add --publish 8000:8000 to your existing docker run command.
So
docker run -v C:/Users/kumar.joshi/KumarData:/mnt --name dev <image:latest>
will become
docker run -v C:/Users/kumar.joshi/KumarData:/mnt --name dev --publish 8000:8000 <image:latest>
Make sure the port is not used else you will get this error:
Bind for 0.0.0.0:8000 failed: port is already allocated
for those who can not solve this issue by any means, you can try this: manually map drive into the docker host:
https://github.com/docker/for-win/issues/466#issuecomment-416682825
The research is here: https://github.com/docker/for-win/issues/466#issuecomment-398305463
I found it quite easy. Just go to you network connections. You can go Control Panel/Network and Sharing. You will find various connections. Search for Docker connection. Select which ever is default. After selecting network, go to Properties. In the properties section enable the option Hyper-V Extensible Virtual Switch. This will help virtual container to use network card.
What did it for me (after several hours of trial-n-error) was changing the Subnet Mask from 255.255.255.240 to 255.255.255.0 (which should not change anything).
As part of the trial-n-error, I had done everything else listed on article, but without any success .. but this last step did it .. and reverting back to 255.255.255.240 does not break the good cycle.
I admit, it makes no sense .. but it might be related to an internal state only being triggered by the network change.
Anyway, if i have helped just one, then it was worth the effort.
Docker Desktop edge, 2.0.4.1 (34207)
Running on a Win7(host) and Virtual Box with Windows Server 2016 TP5(guest vm). On WS2016 i run docker, using the windowsservercore base image for the containers.
I'm trying out kind of a fleet management system where I try to fire up multiple simulated vehicles inside separate docker images.
These vehicle simulators uses a simulated gps which send data to a virtual com port, from which the simulated vehicles read.
In windows XP/7/Server 2016 I have successfully made use of com0com(2.2.2.0 signed) for creating the virtual com ports.
My problem is getting the virtual com ports installed inside a docker container, alternatively accessing them as devices on the host
My first attempt has been running com0com inside docker, installation of the drivers work fine using silent mode, but when creating virtual com port pairs using the installed application I run into problems. The command stalls and if I view the Application eventlog I can see a few rows of this kind for each time I try to run the command.
<date> <time> Information Windows Error Rep. .. 1001 Fault bucket , type 0
When instead trying to create the devices in the host and instead passing the devices I seem to fail to access them
Using com0com --list within the container I can see the devices but not with the same name as I in the host
In the associated Dockerfile I rename them using the com0com setupc.exe to the same name as identified on the host
The mode command issued on the host lists the ports properly, using the mode command from insde the container does on the other hand fail to list them, only showing one item: CON
I then tried passing them using the --device=//./COM128://./COM128 argument to docker run
NOTE!: Is this the correct "path" to the devices?
The noticable result of this is zero, the com0com setupc command gives the same result, same with mode
The strange thing is that when I run powershell and list the serialports i get the expected comports as available on the host, with the correct names, but trying to open one of them gives an exception
PS C:\sims\com0com> [System.IO.Ports.SerialPort]::getportnames()
COM128
COM129
PS C:\sims\com0com> $comports=[System.IO.Ports.SerialPort]::getportnames()
PS C:\sims\com0com> $port= new-Object System.IO.Ports.SerialPort $comports[0],9600,None,8,one
PS C:\sims\com0com> $port.open()
Exception calling "Open" with "0" argument(s): "The port 'COM128' does not exist."
Finally tried just about the same as above but with another virtual com port software(eltima), setting them up in the host, the final result when trying to open the port using ps in the container is the same as for the com0com configured ports
Update:
Tried a new approach, running the container in hyper-v instead, using the -isolation flag to docker run, given the thesis that you would be able to install the com0com drivers in a hyper-v container, not having to try to share them from the host.
Running in hyper-v forced us to change to running the nanoserver base image instead of the windowsservercore base image.
Currently stuck on not being able to run the com0com install tool in nanoserver since the installer(NSIS-based) is a 32bit application, nanoserver only able to run 64bit applications.
Quick glance indicates that a new version(3.0) of NSIS could be used along with the NSIS scripts included in the com0com repo.
All ideas on how to move forward are welcome!
P.S: Same question posted on msdn containers forum
Changed to using named pipes to get it to work since no one else seems to be using docker for windows with serial ports.