How to communicate between Docker container and Linux host over SCTP - docker

I tried to listen to SCTP on the docker container and connect to there from the Linux host machine. But it seems the connection timed out.
Is there any way to communicate between host and container over SCTP?
FYI: It looks container-to-container SCTP communication works fine.
Detailed information is the following:
Dockerfile for test-container
FROM ubuntu:focal
RUN apt update -y && apt install -y ncat
docker run
$ sudo docker run --rm --name sctp-server -p 9999:9999/sctp test-container:latest ncat --sctp -lv 9999
SCTP request (timeout)
$ ncat --sctp 127.0.0.1 9999
Ncat: TIMEOUT.
docker version
Client: Docker Engine - Community
Version: 19.03.13
API version: 1.40
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:02:52 2020
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.13
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 4484c46d9d
Built: Wed Sep 16 17:01:20 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.7
GitCommit: 8fba4e9a7d01810a393d5d25a3621dc101981175
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
FYI: Container-to-Container SCTP communication (works fine)
$ sudo docker network create -d bridge sctp
$ sudo docker run --rm --name sctp-server --net=sctp sctp-test:latest ncat --sctp -lv 9999
$ sudo docker run --rm --name sctp-client --net=sctp sctp-test:latest ncat --sctp 172.18.0.2 9999

Finally, I found the cause of this problem.
The reason for the timeout is it used the same SCTP port between host and container.
When I launched a container with
different ports like sudo docker run --rm --name sctp-server -p 19999:9999/sctp test-container:latest ncat --sctp -lv 9999 and run ncat --sctp 127.0.0.1 19999 on the host machine, it worked fine.
I'm not confident but I suspect the behavior of iptables.

Related

Unable to start Flink JobManager container in Docker

I am unable to start a Flink JobManager Docker container on M1 MacBook running Monterey. Below is the docker command pulled from the Flink Docs and the resulting java.io.IOException
docker run \
--rm \
--name=jobmanager \
--network flink-network \
--publish 8081:8081 \
--env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
flink:1.16.0-scala_2.12 jobmanager
INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting StandaloneSessionClusterEntrypoint down with application status FAILED. Diagnostics java.io.IOException: Could not create the working directory /tmp/jm_ba47b82cf8d85068faa1c41d30126b5d.
at org.apache.flink.runtime.entrypoint.WorkingDirectory.createDirectory(WorkingDirectory.java:58)
at org.apache.flink.runtime.entrypoint.WorkingDirectory.<init>(WorkingDirectory.java:39)
at org.apache.flink.runtime.entrypoint.WorkingDirectory.create(WorkingDirectory.java:88)
at org.apache.flink.runtime.entrypoint.ClusterEntrypointUtils.lambda$createJobManagerWorkingDirectory$2(ClusterEntrypointUtils.java:241)
at org.apache.flink.runtime.entrypoint.DeterminismEnvelope.map(DeterminismEnvelope.java:49)
at org.apache.flink.runtime.entrypoint.ClusterEntrypointUtils.createJobManagerWorkingDirectory(ClusterEntrypointUtils.java:239)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:356)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:282)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:232)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:229)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:729)
at org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:59)
My docker version is:
Client:
Cloud integration: v1.0.29
Version: 20.10.22
API version: 1.41
Go version: go1.18.9
Git commit: 3a2c30b
Built: Thu Dec 15 22:28:41 2022
OS/Arch: darwin/arm64
Context: default
Experimental: true
Why does the container not allow creating the required directory?
Looks like this was related to disk space issue. Running docker system prune cleared some space and the container is running nicely.

Why can't I lookup other container by DNS in container

According to official Docker's doc, Docker will create DNS server when it started which makes it able to query other container directly by container ID or name.
containers that use a custom network use Docker’s embedded DNS server, which forwards external DNS lookups to the DNS servers configured on the host.
But when I trying to use nslookup directly in container it failed to lookup but wget still success! What makes it different?
Reproduce steps:
docker network create my-net
docker run -d --name web --network my-net httpd
docker run -it --rm --network my-net busybox
after inside busybox:
$ wget -q -O - web
<html>...some content...</html>
It works great! but use nslookup will failed:
$ nslookup web
Server: 127.0.0.11
Address: 127.0.0.11:53
Non-authoritative answer:
*** Can't find web: No answer
This is my docker's version:
$ docker version
Client: Docker Engine - Community
Version: 20.10.21
API version: 1.41
Go version: go1.19.2
Git commit: baeda1f82a
Built: Tue Oct 25 17:53:02 2022
OS/Arch: darwin/amd64
Context: colima
Experimental: true
Server:
Engine:
Version: 20.10.18
API version: 1.41 (minimum version 1.12)
Go version: go1.18.6
Git commit: e42327a6d3c55ceda3bd5475be7aae6036d02db3
Built: Sun Sep 11 07:10:00 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: 5fd4c4d144137e991c4acebb2146ab1483a97925
docker-init:
Version: 0.19.0
GitCommit:
While reproducing your issue I noticed that nslookup failed for any query (e.g., nslookup google.com also failed. Afterwards, I tried spinning up an ubuntu container on the same network and there both wget and nslookup worked fine. I do not know the exact reason why this is so, but my guess is that wget and nslookup rely on some system functionalities which are different for busybox and for ubuntu.

Docker swarm service environment variable is not visible within container

I am creating a docker service with an environment variable:
docker service create --env TEST=123 myservice
And I verify the environment variable was set with
$ docker service inspect myservice
...
ContainerSpec:
Env: TEST=123
...
But then the environment variable does not show up within the docker container. In particular, the code running inside the docker container prints os.Environ() and in the logs I see only the standard environment variable:
$ docker service logs myservice
[HOSTNAME=48bcddab9204 SHLVL=1 HOME=/root PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PWD=/app]
So the environment variable "TEST" is not set. How can I set up a service that defines this environment variable?
The output of docker version is
Client:
Version: 20.10.12
API version: 1.41
Go version: go1.17.5
Git commit: e91ed5707e
Built: Mon Dec 13 22:31:40 2021
OS/Arch: linux/amd64
Context: synology
Experimental: true
Server:
Engine:
Version: 20.10.3
API version: 1.41 (minimum version 1.12)
Go version: go1.15.13
Git commit: a3bc36f
Built: Thu Aug 19 07:11:25 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.4.3
GitCommit: ea3508454ff2268c32720eb4d2fc9816d6f75f88
runc:
Version: v1.0.0-rc93
GitCommit: 31cc25f16f5eba4d0f53e35374532873744f4b31
docker-init:
Version: 0.19.0
GitCommit: ed96d00
Variables should be defined in the environment when you use --env. If that's not happening, we'd need a complete example to reproduce. Here's an example showing that it works:
$ docker service create --name env-test --env TESTVAR=123 busybox tail -f /dev/null
nj9l6z57d9pviztyp9pglmv4r
overall progress: 1 out of 1 tasks
1/1: running [==================================================>]
verify: Service converged
$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
697c1bf221b1 busybox:latest "tail -f /dev/null" 20 seconds ago Up 19 seconds env-test.1.vlrepc8mqqvx7gysh2qsymja8
$ docker exec -it env-test.1.vlrepc8mqqvx7gysh2qsymja8 env
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=697c1bf221b1
TERM=xterm
TESTVAR=123
HOME=/root

Metabase on docker not getting exposed

I'm having some trouble trying to get metabase running on docker.
I've installed Docker for Windows 7 following this tutotial.
Here is the docker version:
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24302
Built: Fri Mar 23 08:31:36 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.05.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.10.1
Git commit: f150324
Built: Wed May 9 22:20:42 2018
OS/Arch: linux/amd64
Experimental: false
Now when I try to get the metabase running on docker, using
docker run -d -p 3000:3000 --name metabase metabase/metabase
and using
docker logs -f metabase
I get the message after all the setup:
Metabase Initialization COMPLETE
The problem is when I try to access http://0.0.0.0:3000 I get a ERR_ADDRESS_INVALID. I looked for all the ports that are "LISTENING" using
netstat -aon | find /i "listening"
I don't see the 0.0.0.0:3000 listed there.
Docker ps returns:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6a283cf6b8d7 metabase/metabase "/app/run_metabase.sh" 4 hours ago Up 4 hours 0.0.0.0:3000->3000/tcp metabase
I'm not sure if this is a problem with metabase image or with docker itself.
Does anyone have an idea on what this could be?
This 0.0.0.0 is not a valid IP address, but a non-routable meta-address
Your application is reachable on your machines IP address / hostname on port 3000.
If you are unsure, what your hostname is, you can run in powershell;
hostname
And then access it using http://your_hostname:3000

Error with Docker daemon for docker installation on Fiware cloud

I am new with the Fiware and docker technologies so I need some help.
I am following the instructions from this link http://simple-docker-hosting-on-fiware-cloud.readthedocs.io/en/v1.0/manuals/install in order to create a docker-host machine on Fiware cloud but when I run the following command:
docker-machine create -d openstack --openstack-flavor-id="2" --openstack-image-name="base_ubuntu_14.04" --openstack-net-name="node-int-net-01" --openstack-floatingip-pool="public-ext-net-01" --openstack-sec-groups="docker-sg" --openstack-ssh-user "ubuntu" docker-host
I receive the following error:
Error creating machine: Error running provisioning: Unable to verify the Docker daemon is listening: Maximum number of retries (10) exceeded
Although, I can see the instance of the docker-host machine on Fiware cloud, but when I run the following command:
eval "$(docker-machine env docker-host)"
the following error comes up:
Error checking TLS connection: Error checking and/or regenerating the certs: There was an error validating certificates for host "147.27.60.136:2376": dial tcp 147.27.60.136:2376: connectex: No connection could be made because the target machine actively refused it.
You can attempt to regenerate them using 'docker-machine regenerate-certs [name]'.
Be advised that this will trigger a Docker daemon restart which might stop running containers.**
I also tried to regenerate the certificates:
docker-machine regenerate-certs docker-host
but I received the following error:
Error getting SSH command to check if the daemon is up: ssh command error:
command : sudo docker version
err : exit status 1
output : Client:
Version: 18.04.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 3d479c0
Built: Tue Apr 10 18:21:14 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?**
Image with the result for the Command: docker-machine ls
What am I doing wrong?
I use docker community edition for windows 10.
The docker version is:
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:06:28 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm
Server:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:14:32 2018
OS/Arch: linux/amd64
Experimental: false
First make sure you've opened your docker port (tcp/2376) in your default security group
Let me suggest you using base_ubuntu_16.04 instead of base_ubuntu_14.04
Anyway, it won't run properly at first. There is a problem with the latests versions of docker and docker-machine. As a workaround, after running your docker-machine command, you can do this to fix the problem:
ssh docker-host 'sudo apt-get -y install linux-image-extra-$(uname -r) linux-image-extra-virtual ; sudo modprobe aufs ; sudo service docker start'
However, you might find furhter problems due to MTU configuration in your docker host. To solve them, you can lower your MTU with these commands:
docker-machine ssh docker-host "sudo sed -i 's/--label provider=openstack/--label provider=openstack\n--mtu=1400/g' /etc/default/docker"
docker-machine ssh docker-host "sudo service docker restart"
docker-machine ssh docker-host "sudo ip link set mtu 1400 dev docker0"

Resources