EKS connection refused when trying to talk to Jaeger agent daemonset - docker

Question

I recently deployed the jaeger agent as a daemonset on my k8s cluster alongside a collector. When trying to send spans to the agent using:
- name: JAEGER_AGENT_HOST
valueFrom:
fieldRef:
fieldPath: status.hostIP
When looking at the application logs I see:
failed to flush Jaeger spans to server: write udp <Pod-Ip>:42531-><Node-Ip>:6831: write: connection refused
All nodes can access each other as the security group does not block ports between them, when using a sidecar agent the spans are sent without issue.
Replicate:
Deploy agent using:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: jaeger-agent
labels:
app: jaeger
app.kubernetes.io/name: jaeger
app.kubernetes.io/component: agent
namespace: observability
spec:
selector:
matchLabels:
app: jaeger
app.kubernetes.io/name: jaeger
app.kubernetes.io/component: agent
template:
metadata:
labels:
app: jaeger
app.kubernetes.io/name: jaeger
app.kubernetes.io/component: agent
spec:
containers:
- name: jaeger-agent
image: jaegertracing/jaeger-agent:1.18.0
args: ["--reporter.grpc.host-port=<collector-name>:14250"]
ports:
- containerPort: 5775
protocol: UDP
- containerPort: 6831
protocol: UDP
- containerPort: 6832
protocol: UDP
- containerPort: 5778
protocol: TCP
Then deploy hotrod application:
apiVersion: apps/v1
kind: Deployment
metadata:
name: hotrod
labels:
app: hotrod
version: v1
spec:
replicas: 1
selector:
matchLabels:
app: hotrod
template:
metadata:
labels:
app: hotrod
spec:
containers:
- name: hotrod
image: jaegertracing/example-hotrod:latest
imagePullPolicy: Always
env:
- name: JAEGER_AGENT_HOST
valueFrom:
fieldRef:
fieldPath: status.hostIP
ports:
- containerPort: 8080

Answer 1

Looks like your DaemonSet misses the hostNetwork property, to be able to listen on the node IP.
You can check that article for further info: https://medium.com/#masroor.hasan/tracing-infrastructure-with-jaeger-on-kubernetes-6800132a677