Access to internet from containers connected to bridge - docker

I’m now cracking my nuts trying to get containers connected to custom bridge network reach to internet.
Small background, I have internet facing server running the Docker daemon. As all my applications are web applications I need to have single point for maintaining SSL certificates and access rights to them.
As a solution I deployed Nginx reverse proxy (nginx-proxy-manager) that is handling all incoming traffic, serving the SSL certificates and deployed all web containers into the same custom network bridge.
It turned out, that in this way applications can communicate each other (so the reverse proxy can pass the traffic to them which is what I wanted), but they are unable to reach out the internet.
Is there any possible way how to with this setup allow containers speak to internet?
Many thanks for eventual hints.
lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
docker version
Client: Docker Engine - Community
Version: 20.10.22
API version: 1.41
Go version: go1.18.9
Git commit: 3a2c30b
Built: Thu Dec 15 22:28:20 2022
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.22
API version: 1.41 (minimum version 1.12)
Go version: go1.18.9
Git commit: 42c8b31
Built: Thu Dec 15 22:26:12 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.15
GitCommit: 5b842e528e99d4d4c1686467debf2bd4b88ecd86
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker network ls
NETWORK ID NAME DRIVER SCOPE
d95bb7471827 bridge bridge local
2f660f266735 host host local
36c3efe85b83 none null local
d9eaf06a2693 reverseproxy-nw bridge local
# docker network inspect d9eaf06a2693
[
{
"Name": "reverseproxy-nw",
"Id": "d9eaf06a26936b155fac6568980d5f8fd685531bdc797911592e112a3b70a8a3",
"Created": "2022-08-15T20:26:29.046077109+02:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Attachable": true,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
Switching to the host network works, so it looks that this problem is related to the bridge network.

Related

Kubernetes not using docker registry

I have kubernetes cluster with 7 worker nodes running behind proxy. Deploying application on cluster and scaling application is consuming too much of internet bandwidth. Therefore I decided to deploy Docker Registry acting as pull through cache server. But deployments are not pulling images from the registry. What is the issue here?
Docker daemon.json
...
"registry-mirrors": [
"https://myregistry",
"https://myregistry:443"
]
Docker version
Client: Docker Engine - Community
Version: 20.10.5
API version: 1.40
Go version: go1.13.15
Git commit: 55c4c88
Built: Tue Mar 2 20:33:55 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 19.03.14
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 5eb3275d40
Built: Tue Dec 1 19:19:17 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.18.0
GitCommit: fec3683
Kubernetes version
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.15", GitCommit:"73dd5c840662bb066a146d0871216333181f4b64", GitTreeState:"clean", BuildDate:"2021-01-13T13:14:05Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"
Docker registry configuration
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /data
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
proxy:
remoteurl: https://index.docker.io/v1/
To access the Docker Registry, your Kubernetes cluster needs to be able to make an outbound connection to the registry. If your proxy server is blocking this connection, then the deployments will not be able to retrieve the images from the registry. You may need to configure your proxy server to allow outbound connections to the registry.
The API server is not able to access any of the services running inside of the Docker container. This could be due to a misconfiguration or a firewall blocking the connection. In order for the API server to access the services running inside the container, it needs to have access to the Docker daemon, which is responsible for managing the containers. In order to allow the API server to access the Docker services deployed registry on another host with docker-compose, you will need to configure the appropriate ports to be accessible from the API server. You will need to open port 2375 on the Docker host, which is the default port used by the Docker Remote API. Additionally, you will need to ensure that port 5000 is open on the Docker host, as this is the default port used by the Docker Registry. Once you have opened up the necessary ports, you should be able to access the Docker services deployed registry on the Docker host from the API server.
Set the imagePullPolicy to Never, otherwise Kubernetes will try to download the image.
Refer to this document for more information.

Unable to login to docker via Ubuntu 18.04.5 LTS

I am having an issue on login to docker from my Ubuntu 18.04.5 LTS
This is the command I run to login docker
sudo docker login -u myname -p mypass
This is the error I've got
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I did searches on google and seems it is not quite a common issue to everyone.
This is my docker version
Client: Docker Engine - Community
Version: 19.03.12
API version: 1.40
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:45:36 2020
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.12
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 48a66213fe
Built: Mon Jun 22 15:44:07 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
Basically, there are what I have done so far:
run sudo docker run hello-world to verify that my docker is runner properly
Hello from Docker!
This message shows that your installation appears to be working correctly.
...
run sudo vi ~/.docker/config.json to add "HttpHeaders
{
"auths": {},
"HttpHeaders": {
"User-Agent": "Docker-Client/19.03.12 (linux)"
}
}
run sudo vi /etc/docker/daemon.json to add dns then reload daemon and restar docker
{
"dns": ["8.8.8.8","8.8.4.4" ]
}
run curl https://registry-1.docker.io/v2/ and sure enough I've got this
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
As people were talking about proxy server, I also did a test run curl https://google.com aand get respons, does it mean that my server is not reside behind a proxy?
Unfortunately, after try several combinations of above solutions, I'm still not able to login to docker.
Does anyone has any advice?

Docker host network container service access under Windows

If I run docker container using the host network (--network host), for any services running in the container their exposed port can be directly accessed from host right?
I always thought so until I'm running docker container using the host network under Windows --
The ip a s eth0 shows that my container IP address is 192.168.65.3
The route | awk '/^default/ { print $2 }' gives 192.168.65.1
However, my host machine has an IP of 10.66.xx.xx
I.e., the container IP address and host IP are completely different. Unlike what the https://www.metricfire.com/blog/understanding-dockers-net-host-option/ says.
Anyway, if I'm running any services in the container, how to expose their port so that they can be directly accessed from host? (I thought with host network (--network host), you no longer need to map port from container to host)
thx
docker version
Client: Docker Engine - Community
Version: 19.03.8
API version: 1.40
Go version: go1.12.17
Git commit: afacb8b
Built: Wed Mar 11 01:23:10 2020
OS/Arch: windows/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.8
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: afacb8b
Built: Wed Mar 11 01:29:16 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
Host networking is not supported on Windows:
The host networking driver only works on Linux hosts, and is not supported on Docker Desktop for Mac, Docker Desktop for Windows, or Docker EE for Windows Server.
https://docs.docker.com/network/network-tutorial-host/
I would suggest trying the -p option to docker run, since that is supported on Windows.
Alternately, one forum user suggests using VirtualBox in bridged mode to install Linux, which can then use host networking. YMMV.

Docker bound mount - can not see changes on browser

I'm using docker-toolbox on windows home.
I was able to run a jekyll-serve web server image to see the default page on browser, but when I try to edit file on VS Code, I can not see the changes after refreshing the browser.
Any idea why I can not see the changes after refresh?
Step to reproduce:
First I've git cloned this repository into my c:/Users/shaharshokrani/udemy-docker-mastery/bind-mount-sample1 (I'm able to see the files with ls on 'cmder' console).
Then I was able to run this image with:
docker run -v //c/users/shaharshokrani/udemy-docker-mastery/bindmount-sample-1:/site bretfisher/jekyll new .
docker container run -p 80:4000 --name myjekyll -v //c/users/shaharshokrani/udemy-docker-mastery/bindmount-sample-1:/site bretfisher/jekyll-serve
And I'm able to see the default welcome page on http://192.168.99.100/.
I've tried to edit and save using VS Code this 2017-03-05-welcome-to-jekyll.markdown but I can not see the changes after refreshing the browser.
I also checked the VM for shared network - it shows c:/users/.
Even the Mounts on inspect looks good:
"Mounts": [
{
"Type": "bind",
"Source": "/c/users/shaharshokrani/udemy-docker-mastery/bindmount-sample-1",
"Destination": "/site",
"Mode": "",
"RW": true,
"Propagation": "rprivate"
}
],
And the image dockerfile's CMD has the --force_polling flag.
Both the images (bretfisher/jekyll-serve, bretfisher/jekyll) are latest.
The docker container logs -f myjekyll looks good:
Bundle complete! 4 Gemfile dependencies, 28 gems now installed.
Bundled gems are installed into `/usr/local/bundle`
Configuration file: /site/_config.yml
Source: /site
Destination: /site/_site
Incremental build: disabled. Enable with --incremental
Generating...
Jekyll Feed: Generating feed for posts
done in 1.031 seconds.
Auto-regeneration: enabled for '/site'
Server address: http://0.0.0.0:4000/
Server running... press ctrl-c to stop.
Docker version:
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24302
Built: Fri Mar 23 08:31:36 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
Server: Docker Engine - Community
Engine:
Version: 19.03.3
API version: 1.40 (minimum version 1.12)
Go version: go1.12.10
Git commit: a872fc2f86
Built: Tue Oct 8 01:01:20 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
GitCommit: fec3683
Issue identified
Bind mounting actually does not work for docker toolbox:
file change events in mounted folders of host are not propagated to
container by Docker for Windows
Solution
This script is intended to be the answer to this issue: docker-windows-volume-watcher.
Side note
This is a common issue with data manipulated outside of your container.
For jekyll, in particular, even the solution described in the issue below does not work for windows-based systems.
https://github.com/jekyll/jekyll-watch/issues/17
In short you need to execute jekyll with the --force_polling flag (Does not work with Windows Hosts). You can find it in the jekyll docs here
https://jekyllrb.com/docs/configuration/options/
On Linux based systems it works out of the box since the image used in the question bretfisher/jekyll-serve already utilizes the --force_polling flag.
just ran
docker run --rm -it -e JEKYLL_NEW=true -p 8080:4000 -v (pwd):/site bretfisher/jekyll new .
to create a new jekyll site and
docker run --rm -it -e JEKYLL_NEW=true -p 8080:4000 -v (pwd):/site bretfisher/jekyll-serve
to run it mounted to a directory on my machine (linux) and was able to edit a file with changes propagating to jekyll.

artifactory do not pulling some layers of windows image

I want to use artifactory with offline clients.
I had installed ws2016 and docker.
I don't have the proxy on this machine.
and when I am trying to pull image - docker start download (for example iis:nanoserver) all layers except two. Client trying to download them directly from the internet. I find the reason - because there is in manifest strong URL to Microsoft or others resources in manifest)
What will be the solution for this?
PS C:\Windows\system32> docker -l debug pull docker.artifactory.mydomain.com/microsoft/aci-helloworld:windows
windows: Pulling from microsoft/aci-helloworld
bce2fbc256ea: Retrying in 1 second
4a8c367fd46d: Retrying in 1 second
a8a90ba3a09e: Download complete
f694b71407bb: Download complete
1297730844f7: Download complete
757c0e11bc6f: Download complete
23daf900b85b: Download complete
4c4e4246add8: Download complete
739a2e484f2e: Download complete
726e0a195fd7: Download complete
8f617470c3a5: Download complete
1504687851f5: Download complete
dial tcp 23.64.230.126:443: connectex: No connection could be made because the target machine actively refused it.
docker version
Client:
Version: 17.03.1-ee-3
API version: 1.27
Go version: go1.7.5
Git commit: 3fcee33
Built: Thu Mar 30 19:31:22 2017
OS/Arch: windows/amd64
Server:
Version: 17.03.1-ee-3
API version: 1.27 (minimum version 1.24)
Go version: go1.7.5
Git commit: 3fcee33
Built: Thu Mar 30 19:31:22 2017
OS/Arch: windows/amd64
Experimental: false
manifest.json
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
"size": 252691002,
"digest": "sha256:bce2fbc256ea437a87dadac2f69aabd25bed4f56255549090056c1131fad0277",
"urls": [
"https://go.microsoft.com/fwlink/?linkid=837858"
]
},
{
"mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
"size": 141758132,
"digest": "sha256:b0b5e40cb939a7befa4e01212d6f65f30022bbd04b5f15985b45ce9cfd3fcabc",
"urls": [
"https://go.microsoft.com/fwlink/?linkid=860052"
]
Found official solution with docker 17.6
Edit the daemon.json file add the follow line:
{
"allow-nondistributable-artifacts": ["myregistrydomain.com:5000"]
}
When you push images to the registries in the list, their non-distributable layers will be pushed to the registry.
Right click your installed docker desktop
click Docker engine
Add comma "," near "experimental": true**,** #comma used to seperate the line
Enter in to the new line add
"allow-nondistributable-artifacts": ["localhost.com:5000"]
Mostly local host is nothing but your public or private ip
click apply&restart

Resources