Using UDP broadcast for pods/peers discovery in Kubernetes - docker

Question

I need to use UDP broadcast for peer discovery.
Environment:
docker-desktop with a single node Kubernetes cluster
My code looks as follows:
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class MainApp {
public static void main(String[] args) throws ExecutionException, InterruptedException {
int inPort = Integer.parseInt(System.getenv("IN_PORT"));
int outPort = Integer.parseInt(System.getenv("OUT_PORT"));
String name = System.getenv("NAME");
Client client = new Client(name, outPort);
Server server = new Server(name, inPort);
ExecutorService service = Executors.newFixedThreadPool(2);
service.submit(client);
service.submit(server).get();
}
static class Client implements Runnable {
final String name;
final int port;
Client(String name, int port) {
this.name = name;
this.port = port;
}
#Override
public void run() {
System.out.println(name + " client started, port = " + port);
try (DatagramSocket socket = new DatagramSocket()) {
socket.setBroadcast(true);
while (!Thread.currentThread().isInterrupted()) {
byte[] buffer = (name + ": hi").getBytes();
DatagramPacket packet = new DatagramPacket(buffer, buffer.length,
InetAddress.getByName("255.255.255.255"), port);
socket.send(packet);
Thread.sleep(1000);
System.out.println("packet sent");
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
static class Server implements Runnable {
final String name;
final int port;
Server(String name, int port) {
this.name = name;
this.port = port;
}
#Override
public void run() {
System.out.println(name + " server started, port = " + port);
try (DatagramSocket socket = new DatagramSocket(port)) {
byte[] buf = new byte[256];
while (!Thread.currentThread().isInterrupted()) {
DatagramPacket packet = new DatagramPacket(buf, buf.length);
socket.receive(packet);
String received = new String(packet.getData(), 0, packet.getLength());
System.out.println(String.format(name + " received '%s' from %s:%d", received,
packet.getAddress().toString(),
packet.getPort()));
}
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
}
Kubernetes pod settings:
For peer-1:
spec:
containers:
- name: p2p
image: p2p:1.0-SNAPSHOT
env:
- name: NAME
value: "peer-1"
- name: IN_PORT
value: "9996"
- name: OUT_PORT
value: "9997"
For peer-2 :
spec:
containers:
- name: p2p-2
image: p2p:1.0-SNAPSHOT
env:
- name: NAME
value: "peer-2"
- name: IN_PORT
value: "9997"
- name: OUT_PORT
value: "9996"
I used a different in/out ports for simplicity's sake. In reality, it should be the same port, e.g.: 9999
I see that each pod has a unique IP address
kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
p2p-deployment-2-59bb89f9d6-ghclv 1/1 Running 0 2m26s 10.1.0.38 docker-desktop <none> <none>
p2p-deployment-567bb5bd77-5cnsl 1/1 Running 0 2m29s 10.1.0.37 docker-desktop <none> <none>
Logs from peer-1:
peer-1 received 'peer-2: hi' from /10.1.0.1:57565
Logs from peer-2:
peer-2 received 'peer-1: hi' from /10.1.0.1:44777
Question: why peer-1 receives UDP packets from 10.1.0.1 instead of 10.1.0.37 ?
If I log into peer-2 container: kubectl exec -it p2p-deployment-2-59bb89f9d6-ghclv -- /bin/bash
Then
socat - UDP-DATAGRAM:255.255.255.255:9996,broadcast
test
test
...
in peer-1 logs I see peer-1 received 'test' from /10.1.0.1:43144.
Again why network address is 10.1.0.1 instead of 10.1.0.37.
Could you please tell me what I'm doing wrong?
Note: when using the same port to send/receive UDP packets, some peer can receive a packet from its own IP address. In other words, a peer can only discover its own IP address but always gets 10.1.0.1 for packets received from other peers/pods

Answer 1

For some reason, UDP broadcast doesn't work as expected in Kubernetes infrastructure, however multicast works fine.
Thanks Ron Maupin for suggesting multicast.
Here you can find java code + kube config