Rails 6 / Kubernetes : Readiness HTTP probe failed with statuscode: 403 - ruby-on-rails

Question

I'm working on the setup of a new Rails project, hosted with Google Kubernetes Engine. Everything was going fine until I switched my deployed server to production mode, with RAILS_ENV=production.
My Kubernetes pods don't reach the ready state anymore. The readiness probe is forbidden to hit the server apparently, since it return a 403 code.
When I run kubectl describe pod <name> on a stuck pod, I get this :
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 5m25s default-scheduler Successfully assigned front to gke-interne-pool
Normal Pulling 5m24s kubelet Pulling image "registry/image:latest"
Normal Pulled 5m24s kubelet Successfully pulled image "registry/image:latest"
Normal Created 5m24s kubelet Created container front
Normal Started 5m24s kubelet Started container front
Warning Unhealthy 11s (x19 over 4m41s) kubelet Readiness probe failed: HTTP probe failed with statuscode: 403
The return of kubectl logs <name> for this pod shows indeed no request from the probe.
But when I launch a console with kubectl exec -it deploy/front -- bash, I can make a curl -s http://localhost:3000, which works perfectly, is displayed in the logs and returns 200.
My setup works in development mode but not in production, and so the Rails 6 app config is the main suspect. Something that I don't understand in the production mode of Rails 6 forbid my readiness probes to contact my pod.
Just in case, the readiness part of deployment.yaml :
spec:
containers:
- name: front
image: registry/image:latest
ports:
- containerPort: 3000
readinessProbe:
httpGet:
path: "/"
port: 3000
initialDelaySeconds: 30
periodSeconds: 15

Answer 1

Another potential cause of readiness 403 errors with Rails 6 is the allowed hosts list defined in config.
Typically, you'll have a line of code in config/environments/production.rb that looks something like:
config.hosts << "www.mydomain.com"
This leads to the rejection of any requests that come from hosts other than "www.mydomain.com". The readiness checks come from a private IP address within your cluster so they're going to be rejected given the above config.
One way to get around this is by adding an additional hosts entry that allows traffic from any private IP addresses:
config.hosts << "www.mydomain.com"
config.hosts << /\A10\.\d+\.\d+\.\d+\z/

Answer 2

I can't notice a specific error whereby your implementation failed after switching the mode RAILS_ENV = production. But after checking the error 403 I was able to find a hack, which seems that it worked for some users in their use case, you can try that by leaving your code in yaml like,
readinessProbe:
httpGet:
path: "/"
port: 3000
scheme: "HTTP"
initialDelaySeconds: 30
periodSeconds: 15
Even though I was not able to find an error on your deploy, the error could be directed to your credentials, so validate the route that you put in Rails and the permissions it has and identify if these change depending on whether it is in production or development environment.
as a last option would be to clarify your suspicions with the Rails app, because I don't see what affects when changing the environment variable to Production.