Haproxy: Contents load after "timeout server" hit - timeout

Question

I'm totally new in the world of haproxy. I'm using it as loadbalancer with the following setup:
loadablanacer (haproxy 2.2.9, deb11)
2x webserver with php-fpm and the Roundcube app (a webmail client, deb11)
db server (MariaDB, deb11)
I managed to set up all what I need for my test but when I open the website (Roundcube) using the loadbalancer some content remains in pending until the timeout server setting hit. After that, the content continue to load correctly (it could happens for more than one file, tipically .js).
The log, after the timeout, return the following lines:
Dec 30 15:54:27 lb haproxy\[513\]: [my.public.ip.addr:61687](https://my.public.ip.addr:61687) \[30/Dec/2022:15:54:21.210\] webmail-fe webmail-be/webmail1 0/0/1/3/6005 200 7804 - - sDVN 6/6/4/4/0 0/0 "GET /plugins/jqueryui/themes/elastic/jquery-ui.min.css?s=1658607433 HTTP/1.1"
That sDVN, if I got it right, means that the webserver didnt' reply with any data and then it got killed (https://cbonte.github.io/haproxy-dconv/1.8/configuration.html#8.5):
s : the server-side timeout expired while waiting for the server to send or receive data.
D : the session was killed by haproxy because the server was detected as down and was configured to kill all connections when going down.
On the webserver, the respective HTTP request:
webmail1.local - - [30/Dec/2022:15:54:21 +0100] "GET /plugins/jqueryui/themes/elastic/jquery-ui.min.css?s=1658607433 HTTP/1.1" 200 7848 "http://loadbalancer.local/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"
If I visit the website directly on the webserver everything works fine without waiting time.
All the server are connected via LAN and there are no latency or packet loss.
I'll appreciate any hint or help. If need more info I'll gather'em asap.
Here my configuration:
global
maxconn 5000
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 3s
timeout client 6s
timeout server 6s
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
frontend webmail-fe
bind *:80
mode http
default_backend webmail-be
backend webmail-be
balance leastconn
option forwardfor
cookie SERVERUSED insert indirect nocache
# http-request cache-use webmail-cache
# http-response cache-store webmail-cache
# http-request set-header X-Forwarded-Port %[dst_port]
# http-request add-header X-Forwarded-Proto https if { ssl_fc }
# option httpchk HEAD / HTTP/1.1\r\nHost:localhost
option httpchk HEAD /
default-server check maxconn 50
server webmail1 webmail1.local:8080 cookie webmail1
# server webmail2 192.168.0.53:8080 cookie webmail2
# listen stats
# bind :32700
# stats enable
# stats uri /
# stats hide-version
# stats auth someuser:password
#cache webmail-cache
# total-max-size 128
# max-object-size 1000
# max-age 14
No error log from webserver or any strage log on haproxy other thab the line already posted.
No hanging connection using ss or any clear problem seen using tcpdump
I expect that, with so few connection (it's a test env) and small webapp all the resources are loaded fast as directly visited skipping the loadbalancer.

Answer 1

Upgraded to haproxy 2.4 and now everything works fine!