How to connect IntelliJ's Docker plugin to Docker Desktop Windows - docker

I'm running the latest Windows 10 Pro build (1903), the latest Docker Engine build (v19.03.8), and the latest IntelliJ (2019.3.4). I have set Expose daemon on tcp://localhost:2375 without TLS and Apply/Restart-ed the Engine. Trying to switch to a Windows container seemingly hangs the daemon, throwing an error, after which I need to destroy all settings and config files before I can start the daemon again.
Yet, when I'm trying to set tcp://localhost:2375 in my Docker plugin, the connection simply fails (probably with a timeout, but there's no log of it). Yet, simply using docker info and other commands from the CLI works as intended, so I'm fairly certain the Engine is running.
For reference, the output of docker info:
$ docker system info
Client:
Debug Mode: false
Plugins:
app: Docker Application (Docker Inc., v0.8.0)
buildx: Build with BuildKit (Docker Inc., v0.3.1-tp-docker)
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 19.03.8
Storage Driver: overlay2
Backing Filesystem: <unknown>
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.19.76-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.943GiB
Name: docker-desktop
ID: NGP3:BQCE:JSUO:6BSV:IUU6:2UEZ:4QTQ:N6IO:TA3T:A7I7:4GXS:IYD6
Docker Root Dir: /var/lib/docker
Debug Mode: true
File Descriptors: 34
Goroutines: 50
System Time: 2020-03-27T15:56:23.690394533Z
EventsListeners: 3
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
What else can I try (short of using Docker Toolkit again) to get the integration up and running? How can I even test where the connection might be dropping?

So after three days of research, I have an answer. Apparently, Windows 10 reserves a port range of 2344-2444, which prevents the Docker daemon from actually exposing the TCP socket, despite the settings. I have a feeling it also relates to the daemon being unable to start after a reboot. You can verify if this is the root cause of the issue by executing the following in an elevated prompt/powershell: netsh interface ipv4 show excludedportrange protocol=tcp - if the output show a range that includes 2375, you are affected,
Remediation (this will include two reboots!):
Disable HyperV and reboot, to free up the port allocations: dism.exe /Online /Disable-Feature:Microsoft-Hyper-V
Manually allocate port 2375: netsh int ipv4 add excludedportrange protocol=tcp startport=2375 numberofports=1
Re-enable HyperV and reboot to take advantage of the fix: dism.exe /Online /Enable-Feature:Microsoft-Hyper-V /All
Optional: after this process, you can re-enable TLS on said socket, and the Docker plugin will be able to connect just fine.

2 years later (June 2022), this integration is easier to setup, assuming you have the latest Docker Desktop for Windows.
It uses as a back-end either WSL2 or HyperV.
The article "Getting Started with Visual Studio Code and IntelliJ IDEA Docker Plugins" from Tyler Charboneau details the IntelliJ IDEA part.
It will require a manual installation of the Docker plugin if you’re using the Community Edition.
Once you’ve installed the Docker plugin, you’ll need to connect it to Docker Desktop.
Follow these steps:
Navigate to IntelliJ IDEA > Preferences.
Expand the Build, Execution, Deployment group.
Click Docker, and then click the small "+" icon to the right.
Choose the correct Docker daemon for your platform (for example, Docker for Mac).
The installation may take a few minutes. Once it’s complete, you’ll see the "Connection successful" message toward the middle-bottom of the Preferences pane:

Related

Error response "proxyconnect tcp: EOF" from daemon when running docker run hello-world in docker quickstart terminal

I'm running docker toolbox on windows version 10.0.15063 behind corporate proxy, and I'm getting the following error when i run the command docker run hello-world
Unable to find image 'hello-world:latest' locally
C:\Program Files\Docker Toolbox\docker.exe: Error response from daemon: Get https://registry-1.docker.io/v2/: proxyconnect tcp: EOF.
See 'C:\Program Files\Docker Toolbox\docker.exe run --help'
I'm getting the same error on attempting to login to docker using docker login.
I've already tried exporting HTTPS_PROXY and HTTP_PROXY in /Docker Toolbox/start.sh and adding the same to my environment variables but none of these approaches seem to be working.
Here's my docker info
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 18.09.6
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.116-boot2docker
Operating System: Boot2Docker 18.09.6 (TCL 8.2.1)
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 989.4MiB
Name: default
ID: PNCR:EWZF:D4O3:7KDP:UW7W:4NEO:YGTT:CHWV:VF5U:YTE6:GD66:WKHS
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: http://xxxx:xxxx
HTTPS Proxy: https://xxxxx:xxxx
No Proxy: 192.168.99.100
Registry: https://index.docker.io/v1/
Labels:
provider=virtualbox
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Here's what i get on ssh-ing into the default machine and running sudo vi /var/lib/boot2docker/profile
CACERT=/var/lib/boot2docker/ca.pem
DOCKER_HOST='-H tcp://0.0.0.0:2376'
DOCKER_STORAGE=aufs
DOCKER_TLS=auto
SERVERKEY=/var/lib/boot2docker/server-key.pem
SERVERCERT=/var/lib/boot2docker/server.pem
export "HTTP_PROXY=http://proxy2:8080"
export "HTTPS_PROXY=https://proxy2:8080"
export "NO_PROXY=192.168.99.100"
I'm I missing something? I've been working on this for the last hour without any luck.
I actually figured it out.
I was initially using corporate network while working on this, and the networking team has blacklisted every ip and then white-listed the trusted ones. That was what was causing the whole problem while trying to connect to an external registry.
These are the steps I followed to fix the issue:
I switched to my personal network then disabled the proxy ie. Under Internet Properties > Switch to automatic Configuration > Automatically detect settings.
Removed the HTTP_PROXY and HTTPS_PROXY exports from start.sh in C:\Program Files\Docker Toolbox
Opened Windows Firewall > Advanced Settings > Inbound Rules > Add New Rule; Then
Created a Custom rule with a range ip range of 1-100, then named it Docker.
Deleted the .Virtualbox folder from C:\Users\myusername*
Restarted Docker Quickstart Terminal. It created a new default docker-machine and updated the configuration settings, and now pulling images from external registries is working fine.

'docker: invalid reference format' when docker repository is not explicitly specified

I'm running a Docker in Docker container in kubernetes through the Gitlab CI (there is a main image and dind is as a service). In the pod I can build docker images and push them, however any further manipulation or a simple docker run triggers an error of invalid reference format.
There are two additional insecure private repositories specified in the dind service specification. If I try to run docker images without a repo prefix specified, it fails. (note that the images are there, it's just their tag which doesn't have the repo specified)
bash-4.2# docker run hello-world
docker: invalid reference format.
See 'docker run --help'.
bash-4.2# docker run docker.io/hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/get-started/
bash-4.2#
I would expect the basic docker pull/run commands to pass and to use the images cached in the daemon even when their current tag doesn't contain a repository specified.
Added docker info
Containers: 3
Running: 1
Paused: 0
Stopped: 2
Images: 21
Server Version: 18.09.5
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
seccomp
Profile: default
Kernel Version: 4.14.35-1844.3.2.el7uek.x86_64
Operating System: Alpine Linux v3.9 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 20
Total Memory: 125.5GiB
Name: runner-vppr4q7s-project-14503-concurrent-08qxbj
ID: HQ3U:HATA:TZUT:OLVS:MHRZ:F4DY:ENBD:Z5WH:FQBC:SGI3:BUUY:QIME
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: xxxxxxxx
HTTPS Proxy: xxxxxxx
No Proxy: localhost,127.0.0.1,.xxxxxxxxxx
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
xxxxxxxxxxxx:5000
yyyyyyyyyyyy:5000
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
Access to the remote API is equivalent to root access on the host. Refer
to the 'Docker daemon attack surface' section in the documentation for
more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
Registries:
Additionally:
bash-4.2# docker tag docker.io/hello-world:latest hello-world:v1
Error parsing reference: "hello-world:v1" is not a valid repository/tag: invalid reference format
bash-4.2# docker tag docker.io/hello-world:latest some:2000/hello-world:v1
bash-4.2# docker tag docker.io/hello-world:latest docker.io/hello-world:v1
(ok)
And docker images doesn't show the docker.io/hello-world:latest and docker.io/hello-world:v1 but shows some:2000/hello-world:v1

Unable to start Docker Container in Windows 10

I am using Windows 10 pro and successfully installed Docker Client 18.09.0 and fetched hello-world docker image.
But when I try to run the image in container using docker run, it is giving the below error
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: CreateComputeSystem 7b206637bedeb11c5f4bb8a5c12f941da3980a5c0e6e18d823f3323b6640a9de: The virtual machine could not be started because a required feature is not installed.
(extra info: {"SystemType":"Container","Name":"7b206637bedeb11c5f4bb8a5c12f941da3980a5c0e6e18d823f3323b6640a9de","Owner":"docker","IgnoreFlushesDuringBoot":true,"LayerFolderPath":"C:\\ProgramData\\Docker\\windowsfilter\\7b206637bedeb11c5f4bb8a5c12f941da3980a5c0e6e18d823f3323b6640a9de","Layers":[{"ID":"ba045b84-94ef-5e96-a203-a8ef5cf53b41","Path":"C:\\ProgramData\\Docker\\windowsfilter\\2cbe39538cedc860f14e954ceed1044a5760df8830e8dc21bcbd4d21e88bf8f3"},{"ID":"959d85fc-a8bf-595a-84f9-a083080f2e27","Path":"C:\\ProgramData\\Docker\\windowsfilter\\3fc0987aeffab6be6b2bb0626867739cbad8dd80f42951e4e803b1e61b64543f"},{"ID":"40a5cfc0-ad6b-5b5e-85ff-dcd5826f380a","Path":"C:\\ProgramData\\Docker\\windowsfilter\\3e839c40c3c413a579f0f60a6ad8ec03daa496dcb61cfc621c35788beb6ae0d4"},{"ID":"be5e886a-ec0d-50e8-a735-c2c9a8b717de","Path":"C:\\ProgramData\\Docker\\windowsfilter\\12eddd7dc5f665f34ffebe1ff1600de14da8d7998950b9a3a180407b2781993a"}],"HostName":"7b206637bede","HvPartition":true,"EndpointList":["3C0F3EDA-3D0F-4C93-8908-C4DCB4FF6C8E"],"HvRuntime":{"ImagePath":"C:\\ProgramData\\Docker\\windowsfilter\\3e839c40c3c413a579f0f60a6ad8ec03daa496dcb61cfc621c35788beb6ae0d4\\UtilityVM"},"AllowUnqualifiedDNSQuery":true}).
I am not sure what sure about the issue or what feature is not installed. When I searched the internet all the errors are about hyperv which is there in my environment. My docker info gives the below details by the way:
PS C:\Windows\system32> docker info
Containers: 14
Running: 0
Paused: 0
Stopped: 14
Images: 2
Server Version: 18.09.0
Storage Driver: windowsfilter
Windows:
Logging Driver: json-file
Plugins:
Volume: local
Network: ics l2bridge l2tunnel nat null overlay transparent
Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: hyperv
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows 10 Pro Version 1809 (OS Build 17763.134)
OSType: windows
Architecture: x86_64
CPUs: 4
Total Memory: 23.54GiB
Name: DESKTOP-6MOD0L8
ID: 4QC3:QQKX:2BS2:P2JG:RUZA:3MK2:RAQ7:ZW7V:Q6YZ:5S56:Z3GQ:WXDC
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: -1
Goroutines: 26
System Time: 2018-12-10T12:14:05.5454663+05:30
EventsListeners: 1
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
Just in case someone is using a Windows 10 Virtual Machine like me you can use the following command in power shell.
Set-VMProcessor -VMName my_virtual_machine -ExposeVirtualizationExtensions $true
Remember to run as Administrator before executing the command else it will give you an error saying you don't have permissions.
This helped me with the same problem (I have virtualization enabled, but you can double-check on the Task Manager -> Performance tab):
Uninstall hyper-v (found in Windows Features)
Restart PC
Install hyper-v
Restart PC
Now you hopefully can build your container.
Source: https://github.com/docker/for-win/issues/2956#issuecomment-514572084
If you're running Docker with Hyper-V you may need to go into your BIOS and enable virtualization
https://blogs.technet.microsoft.com/canitpro/2015/09/08/step-by-step-enabling-hyper-v-for-use-on-windows-10/
I ran into this error with a recent Docker for Windows install and that was the culprit.
I was trying to run RavenDB docker container in Windows 10. My Windows 10 is a virtual machine running in Parallels Desktop on Mac OS.
To be able to run it I had to enable the Nested Virtualization feature on Parallels:
Nested Hyper-V support in Parallels Desktop virtual machines
Having that disabled was causing the RavenDB container to not start with that generic message:
"the virtual machine could not be started because a required feature is
not installed."
By the way, to get the log while starting docker on the command line you can do this:
docker run -it ravendb/ravendb:windows-nanoserver-latest > C:/Temp/docker-log.txt 2>&1d
This may help you debug what's causing the issue...
Try the following:
Uninstall docker, reboot
Uncheck Hyper-V and Windows Container in Windows Features, reboot
Completely remove docker, reboot
Re-install docker, maybe try edge

Docker Hub login behind a proxy

I recently installed Docker 18.04.0-ce on my Ubuntu 17.10 VM. While working behind a proxy I got stuck trying to log in into the Docker Hub registry. So when I run:
docker login -u <username> -p <cool password>
And I get this message:
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http:
request canceled while waiting for connection (Client.Timeout exceeded while
awaiting headers)
As far as my understanding goes from the research I did is that when you are behind a proxy and you want to use Docker you need to configure it accordingly even though having set your environment variable like so:
http_proxy=<my_proxy>
https_proxy=<my_proxy>
HTTP_PROXY=<my_proxy>
HTTPS_PROXY=<my_proxy>
Second, it looks like that Docker has multiple levels of proxy configurations meaning different configuration for build, containers, and the daemon.
I went and configured all of those to match my proxy, but I still get that message. YES, I DID RESTART the service and the daemon.
UPDATE:
$ docker info
Containers: 8
Running: 0
Paused: 0
Stopped: 8
Images: 12
Server Version: 18.04.0-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.13.0-39-generic
Operating System: Ubuntu 17.10
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 7.79GiB
Name: docker-vb
ID: 447I:6DFI:JZ7V:F6SZ:BUCB:IFB2:4HGT:MXK2:Y5H5:EECC:FIQN:SZOH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: http://<proxy_ip>:<proxy_port>
HTTPS Proxy: http://<proxy_ip>:<proxy_port>
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
The docker login typically needs to be run with sudo priviledges, so ensure that the HTTPS_PROXY environment is actually preserved. Simply done by
export HTTPS_PROXY=<your proxy>
Or simply pass it on the command line:
sudo HTTPS_PROXY=<your proxy> docker login -u <user> <registry>
The docker daemon doesn't need to be restarted.
The above works fine (we use extensivelye proxies at work).
There may be some issues with your network reachability, proxy, and possibly the registry. The docker log can provide some clues:
Ubuntu: sudo journalctl -fu docker.service
RHEL: /var/log/messages | grep docker

Docker build leads to "no space left on device" on Windows 10

I´m facing a weird problem when I want to build my image on Windows. I haven´t used Docker for anything else, so the installation can be considered as fresh. There are no volumes at all and no images yet.
When I´m trying to build my application from my Dockerfile, it finishes with this error
docker build ./
Sending build context to Docker daemon 1.4 GB
Error response from daemon: Error processing tar file(exit status 1): write /.bowerrc: no space left on device
I´ve read that you can increase the basesize of docker, but I haven´t found a solution for that for Windows (Why is this even limited by default?)
docker info prints some stuff, but it doesn´t show anything about the basesize under "Storage Driver" at all
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.13.1
Storage Driver: overlay2
Backing Filesystem: tmpfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.8-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.934 GiB
Name: moby
ID: GUXQ:KPKS:PHBV:BMEF:QHHM:B2YG:MWPB:2W5H:Z3GX:27YS:QBT6:O4RV
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 13
Goroutines: 21
System Time: 2017-02-19T20:15:57.8764828Z
EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
According to some posts in the internet, this was once or is the way to go on Linux, but it doesn´t work on Windows
docker daemon --storage-opt dm.basesize=20G
What is wrong with my docker installation and how can I increase the basesize?
I had the same problem on Windows. One of this two settings should solve this problem:
Increase Memory that Docker is using. If is it 2GB, add more
Increase "Disk image max size" - initially is 60, move it to 80 GB
This should be enough, but depends on complexity of what you are building. In some cases increase of Swap would be needed. The initial Swap of Docker is 1024 MB
Thanks
I reset to factory default , then select : delete all image.
At advanced tab : increase 'Disk image max size' (I modify to 80 GB)
Then build docker again
I´ve reinstalled Docker and now it seems to work
This wouldn't have been the underlying issue for the OP back in 2017, but if you're experiencing this problem with Windows 10 1903, then you may be running into Docker for Windows issue #4100 ("Windows 1903 fails when storage-opt used").
On January 6th, 2020, a commit was pushed to the docker-ce GitHub repository that, from the comments (below), appears to implement a workaround to the underlying bug in Windows.
microsoft/hcsshim#718 wclayer: Work around Windows bug when expanding sandbox size
fixes microsoft/hcsshim#708 Windows Host Compute Service bug breaks docker (and other) sandboxes bigger than 20G on Windows 1903
fixes microsoft/hcsshim#624The hcsshim on Windows 10 1903 always fails to build Docker image
fixes/addresses docker/for-win#3884 An error occurred while attempting to build Docker image (especially this comment and the next comments after: docker/for-win#3884 (comment))
fixes/addresses docker/for-win#4100 Windows 1903 fails when storage-opt used
fixes moby/moby#36831 hcsshim::PrepareLayer failed in Win32: The parameter is incorrect (moby/moby#36831 (comment))
fixes Stannieman/audacity-with-asio-builder#5 Docker won't build container
fixes MicrosoftDocs/visualstudio-docs#3523 Error when running build with storage-opts set
fixes moby/moby#39524 Docker build windows 19.03 --storage-opt size>20G
Unfortunately, no new release of Docker has yet been made that includes this commit, so those of us on 1903 and experiencing this bug are kind of stuck for the time being.
Assuming you're doing this with a dev environment, just run:
docker system prune -a -f
Caution: Do not run this on a production environment

Resources