I'm trying to put a HAProxy loadbalancer in front of my RabbitMQ cluster(which is setup with nodes in separate docker containers). I cannot find many examples for haproxy config for the above setup,
global
debug
defaults
log global
mode tcp
timeout connect 5000
timeout client 50000
timeout server 50000
frontend main
bind *:8089
default_backend app
backend app
balance roundrobin
mode http
server rabbit-1 172.18.0.2:8084
server rabbit-2 172.18.0.3:8085
server rabbit-3 172.18.0.4:8086
In this example, what should I give in the place of the ip addresses for docker containers?
Related
We are using haproxy to switch between a local MQTT broker and a cloud broker based on availability (with preference to the local server). haproxy.cfg looks something like this:
global
log 127.0.0.1 local1
maxconn 1000
daemon
debug
#quiet
tune.bufsize 1024576
stats socket /var/run/haproxy.sock mode 600 level admin
defaults
log global
mode tcp
option tcplog
retries 3
option redispatch
timeout connect 5000
timeout client 50000
timeout server 50000
# Listen to all MQTT requests (port 1883)
listen mqtt
bind *:1883
mode tcp
balance first # Connect to first available
timeout client 3h
timeout server 3h
option clitcpka
option srvtcpka
# MQTT server 1 - local wifi
server wifi_broker localserver.local:1883 init-addr libc,last,none check inter 3s rise 5 fall 2 maxconn 1000 on-marked-up shutdown-backup-sessions on-marked-down shutdown-sessions
# MQTT server 2 - cloud
server aws_iot xxxxx.amazonaws.com:8883 backup check backup ssl verify none ca-file ./root-CA.crt crt ./cert.pem inter 5s rise 3 fall 2
listen stats
bind :9000
mode http
stats enable # Enable stats page
stats hide-version # Hide HAProxy version
stats realm Haproxy\ Statistics # Title text for popup window
stats uri /haproxy_stats # Stats URI
Everything works fine if the local broker is available when haproxy starts up. However, if the wifi connection to the local machine is down when haproxy starts up, init-addr none still allows it to start using the backup server (aws_iot). The local server is marked as "Down for Maintenance" and no more health checks are performed. Even after the network is up and running, haproxy is unaware of it and does not switch back from the cloud server.
Is there any way to make it consider unresolved domain name the same as a normal "down" condition?
One alternative I see right now is to have a script polling the domain name in the background and sending an "enable server" command to the haproxy control socket once it is up. This seems overly roundabout for something that should be really simple!
Update:
Running the command echo "enable server mqtt/wifi_server" | socat /var/run/haproxy.sock stdio doesn't switch the backends after the local connection is up and running. haproxy just never switches back to the local server with anything short of restarting it.
Update 2:
Changed init-addr none to init-addr libc,last,none
You are using "init-addr none" so the server will start without any valid IP address when it is in a down state. In addition, your current config makes HAProxy able to resolve hostnames only at startup as mentioned here.
So to make HAProxy resolves localserver.local after startup to get right the IP and send health checks you need to configure a resolvers section in HAProxy.
I am trying to set up a Docker Nginx Proxy server to forward incoming requests to their corresponding Docker Container on 192.168.1.120 or to the Router's Web-Admin at 192.168.1.1
So right now I am in a bit of a pickle, but I need to set this up regardless. I have this setup right now
Router 192.168.1.1 (Web Admin + Port Forwarding)
Server1 LAMP - (Router Forwards -> port 80 for LAMP Server)
Server2 Docker - (Router Forwards -> 20 SSH, 8080, 9000 Docker Admin)
So I have to configure the port forwarding through my Router's web interface, which is accessible on port 8080. But the issue is that right now I moved to Florida, and I had stupidly added a port-forwarding rule on 8080 to forward to Shipyard Docker Manager, which I eventually planned to install an Nginx-Proxy Forwarding Docker container. I never got the forwarding Docker container working, and I eventually switched to Portainer on port 9000 which I had to configure because it was the only other port I had already set forwarded before I lost access to my Router's web interface, and thus lost the ability to forward ports.
The downside is that I cannot access my Router's web interface. The upside is that - I still have to implement an Nginx-Proxy port forwarding Container anyways, to set up dynamic port 80 forwarding to different Docker containers based on the URL.
So I want to mvoe my LAMP server on as a new Docker Container, and then I will also have a few other Rails Docker containers - but I need to configure a Docker Container to forward the app to differnt servers based on the port. I assume I need to have 2 dockers running - one for port 80 forwarding, and then one for port 8080 forwarding - this is not a problem.
I have not been able to correctly configure my Nginx config to forwarding an incoming request from my domain-name that I have point to my server (my.domain.com below), needs to get forwarded to my router 192.168.1.1. Any help / suggestions on how to configure my Nginx-Proxy Docker Container to forward this correctly, or what I should setup here to forward incoming requests to a web-server dynamically based on the URL. I can install any Docker containers I need for this.
My current Config /etc/nginx/nginx.conf, running on a Nginx-Proxy Docker Container on port 8080 (Google to find the Docker Image for nginx-proxy)
# My Nginx Config to forward my.domain.com
http {
resolver 127.0.0.1;
access_log /var/logs/nginx/access.log;
server {
listen 8080;
server_name my.domain.com;
return 301 http://192.168.1.1:8080/$request_uri;
}
}
I get these errors:
[error] 55#55: *2274 datacenter.URL.com could not be resolved (110: Operation timed out), client: 166.172.189.185, server: datacenter.URL.com, request: "GET / HTTP/1.1", host: "datacenter.URL.com:8080"
[error] 55#55: recv() failed (111: Connection refused) while resolving, resolver: 192.168.1.1:8080
EDIT: I just noticed that I can only have one Docker Container running at-a-time for each port. So I need to figure out how to forward requests to different servers's + ports based on the Domain Name. So each URL forwarding rule entry needs to be able to go to different servers all running on all different ports.
I'm trying to setup HAProxy inside a Docker host.
Using HAProxy 1.7 and Docker 1.12
My haproxy.cfg looks like:
# Simple configuration for an HTTP proxy listening on port 81 on all
# interfaces and forwarding requests to a single backend "servers" with a
# single server "server1" listening on 127.0.0.1:8000
global
daemon
maxconn 256
resolvers docker
# nameserver dnsmasq 127.0.0.1:53
nameserver dns 127.0.0.1:53
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
default-server init-addr none
frontend http-in
bind *:80
default_backend www_somedomain1_com
# Define hosts
acl host_www_somedomain1_com hdr(host) -i somedomain1.com
acl host_www_somedomain1_com hdr(host) -i www.somedomain1.com
acl host_www_somedomain2_com hdr(host) -i www.somedomain2.com
## figure out which one to use
use_backend www_somedomain1_com if host_www_somedomain1_com
use_backend www_somedomain2_com if host_www_somedomain2_com
backend www_somedomain1_com
# Utilizing the Docker DNS to resolve below host
# server server1 www-somedomain1-com maxconn 32 check port 80
server server1 www-somedomain1-com resolvers docker check maxconn 32
backend www_somedomain2_com
# Utilizing the Docker DNS to resolve below host
# server server1 www-somedomain2-com maxconn 32 check resolvers docker resolve-prefer ipv4
server server1 www-somedomain2-com maxconn 32 check port 80
I want to use Docker's embedded DNS system - which, in my understanding, is only enabled when using a user defined network.
So I create a network (using the default bridge driver)
docker network create mynetwork
When I run my two named docker containers, (my-haproxy and www-somedomain1-com) I add them to that network with the --net flag.
Docker run commands:
docker run --name myhaproxy --net mynetwork -p 80:80 -d haproxy
docker run --name www-somedomain1-com --net mynetwork -d nginx
I know the Docker dns is functional because I can resolve from one container to the other when I hop on them in a bash shell. I can't get the right combo/config in HAProxy to enable the dynamic DNS resolution.
HAProxy stats page always shows the downstream backends as brown/resolution issue....
Some things that have helped:
- the "default-server init-addr none" helps pass the haproxy config check on startup.
Any guidance is greatly appreciated!
I think your issue is that you are using 127.0.0.1:53 for your resolver dns, when it needs to be 127.0.0.11:53 for the docker bridge network.
Here is my haproxy setup for dev docker stuff:
global
quiet
defaults
log global
mode http
option forwardfor
timeout connect 60s
timeout client 60s
timeout server 60s
default-server init-addr none
resolvers docker_resolver
nameserver dns 127.0.0.11:53
frontend https-proxy
bind 0.0.0.0:80
bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/dev_server.pem
redirect scheme https if !{ ssl_fc }
acl is_api_server hdr(host) -i mywebsite
use_backend api_server if is_api_server
backend api_server
server haproxyapi api-server-dev:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4
I'm using HAProxy 1.6.4 and want to enable the stats. (/haproxy?stats)
Here is my cfg:
global
log 127.0.0.1 local2
daemon
maxconn 256
defaults
log global
timeout connect 5000
timeout client 10000
timeout server 10000
frontend http-in
bind *:8080
default_backend testb
backend testb
balance roundrobin
server s1 123.456.789.0:443 maxconn 32
server s2 123.456.789.1:443 maxconn 32
listen statistics
bind *:8080
mode http
stats enable
If I run statistics on other port than 8080 it works, but how can I run it on the same port as my frontend (8080), which is running in the default mode tcp?
You can do it by redirecting to your self and using access list like this:
global
log 127.0.0.1 local2
daemon
maxconn 256
defaults
log global
timeout connect 5000
timeout client 10000
timeout server 10000
listen stats :1936
mode http
stats enable
stats hide-version
stats realm Haproxy\ Statistics
stats uri /
stats auth myUser:myPassword
frontend http-in
bind *:8080
acl is_www hdr_end(host) -i www.mysite.com
acl is_stat hdr_end(host) -i stat.mysite.com
use_backend srv_www if is_www
use_backend srv_stat if is_stat
backend srv_www
balance roundrobin
server s1 123.456.789.0:443 maxconn 32
server s2 123.456.789.1:443 maxconn 32
backend srv_stat
server Local 127.0.0.1:1936
When going to your server with www, it takes you to the web server.
But using stat, it redirects you from your input port 8080 to 1936 whee stat is running
This is just an educated guess. You can't serve the stats page in tcp mode because it's proxying at layer4. In this mode, haproxy only knows IPs and ports from incoming packets and routes it accordingly based on defined rules.
Unlike http mode (layer7), it has more info to work on like HTTP headers where path is available and use it to know when to serve the /haproxy?stats page.
If you are happy to use a your path, it really easy.
this should work
global
log 127.0.0.1 local2
daemon
maxconn 256
defaults
log global
timeout connect 5000
timeout client 10000
timeout server 10000
frontend http-in
bind *:8080
stats enable
stats uri /stats
default_backend testb
backend testb
balance roundrobin
server s1 123.456.789.0:443 maxconn 32
server s2 123.456.789.1:443 maxconn 32
you can then access the haproxy stats from
http://< hostname >:8080/stats
(tested on haproxy 2.5.5)
I am trying to implement sticky sessions with HAProxy.
I have a HAProxy instance that routes to two different Node.js servers, each running socket.io. I am connecting to these socket servers (via the HAProxy server) using an iOS app (https://github.com/pkyeck/socket.IO-objc).
Unlike when using a web browser, the sticky sessions do not work, it is like the client is not handling the cookie properly and so the HAProxy server just routes the request to wherever it likes. Below you can see my HAProxy config (I have removed the IP addresses):
listen webfarm xxx.xxx.xxx.xxx:80
mode http
stats enable
stats uri /haproxy?stats
stats realm Haproxy\ Statistics
stats auth haproxy:stats
balance roundrobin
#replace XXXX with customer site name
cookie SERVERID insert indirect nocache
option httpclose
option forwardfor
#replace with web node private ip
server web01 yyy.yyy.yyy.yyy:8000 cookie server1 weight 1 maxconn 1024 check
#replace with web node private ip
server web02 zzz.zzz.zzz.zzz:8000 cookie server2 weight 1 maxconn 1024 check
This is causing a problem with the socket.io handshake, because the initial handshake routes to server1 then subsequent heartbeats from the client go to server2. This causes server2 to reject the client because the socket session ID is invalid as far as server 2 is concerned, when really all requests from the client should go to the same server.
Update the haproxy config file /etc/haproxy/haproxy.cfg by the following:
global
daemon
maxconn 256
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
frontend http-in
bind *:80
default_backend servers
option forwardfor
backend servers
cookie SRVNAME insert
balance leastconn
option forwardfor
server node1 127.0.0.1:3001 cookie node1 check
server node2 127.0.0.1:3002 cookie node2 check
server node3 127.0.0.1:3003 cookie node3 check
server node4 127.0.0.1:3004 cookie node4 check
server node5 127.0.0.1:3005 cookie node5 check