Configuring end to end SSL using Nginx Ingress controller - docker

Question

I am running a local deployment and trying to redirect HTTPS traffic to my backend pods.
I don't want SSL termination at the Ingress level, which is why I didn't use any tls secrets.
I am creating a self signed cert within the container, and Tomcat starts up by picking that and exposing on 8443.
Here is my Ingress Spec
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-name
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
#nginx.ingress.kubernetes.io/service-upstream: "false"
kubernetes.io/ingress.class: {{ .Values.global.ingressClass }}
nginx.ingress.kubernetes.io/affinity: "cookie"
spec:
rules:
- http:
paths:
- path: /myserver
backend:
serviceName: myserver
servicePort: 8443
I used the above annotation in different combinations but I still can't reach my pod.
My service routes
# service information for myserver
service:
type: ClusterIP
port: 8443
targetPort: 8443
protocol: TCP
I did see a few answers regarding this suggesting annotations, but that didn't seem to work for me. Thanks in advance!
edit: The only thing that remotely worked was when I overwrote the ingress values as
nginx-ingress:
controller:
publishService:
enabled: true
service:
type: NodePort
nodePorts:
https: "40000"
This does enable https, but it picks up kubernetes' fake certs, rather than my cert from the container
Edit 2:
For some reason, the ssl-passthrough is not working. I enforced it as
nginx-ingress:
controller:
extraArgs:
enable-ssl-passthrough: ""
when I describe the deployment, I can see it in the args but when I check with kubectl ingress-nginx backends as described in https://kubernetes.github.io/ingress-nginx/kubectl-plugin/#backends, it says "sslPassThrough:false"

Answer 1

SSL Passthrough requires a specific flag to be passed to the nginx controller while starting since it is disabled by default.
SSL Passthrough is disabled by default and requires starting the controller with the --enable-ssl-passthrough flag.
Since ssl-passthrough works on layer 4 of the OSI model and not on the layer 7 (HTTP) using it will invalidate all the other annotations that you set on the ingress object.
So at your deployment level you have specify this flag under args:
containers:
- name: controller
image: us.gcr.io/k8s-artifacts-prod/ingress-nginx/controller:v0.34.1#sha256:0e072dddd1f7f8fc8909a2ca6f65e76c5f0d2fcfb8be47935ae3457e8bbceb20
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
args:
- /nginx-ingress-controller
- --enable-ssl-passthrough

Answer 2

There are several thingns you need to setup if you want to use ssl-passthrough.
First is to set proper host name:
spec:
rules:
- host: example.com <- HERE
http:
...
It's mentioned in documentation:
SSL Passthrough leverages SNI [Server Name Indication] and reads the virtual domain from the
TLS negotiation, which requires compatible clients. After a connection
has been accepted by the TLS listener, it is handled by the controller
itself and piped back and forth between the backend and the client.
If there is no hostname matching the requested host name, the request
is handed over to NGINX on the configured passthrough proxy port
(default: 442), which proxies the request to the default backend.
Second thing is setting --enable-ssl-passthrough flag as already mentioned in separate answer by #thomas.
Just edit the nginx ingress deployment and add this line to args list:
- --enable-ssl-passthrough
Third thing that has to be done is use to use the following annotations in your ingress object definition:
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
IMPORTANT: In docs you can read:
Attention
Because SSL Passthrough works on layer 4 of the OSI model (TCP) and
not on the layer 7 (HTTP), using SSL Passthrough invalidates all the
other annotations set on an Ingress object.
This means that all other annotations are useless from now on. This applies to annotations like force-ssl-redirect, affinity and also to paths you defined (e.g. path: /myserver). Since traffic is end-to-end encrypted, all ingress sees is some gibberish and all it can do is pass this data to the application based on dns name (SNI).

Answer 3

Looks like this is an open issue
https://github.com/kubernetes/ingress-nginx/issues/5686
So, I had to revert back to using my certs as default certificates and mounting it as tls sercets.