I have a fairly standard HAProxy / Web Cluster setup, which is running perfectly fine - as long as I don't enable the Shorewall firewall on the web cluster servers.
As soon as I do, error messages appear in the HAProxy server's syslog, and the websites served via HAProxy returns 503.
Take the HAProxy server and one of the web servers as an example.
The HAProxy config looks like this:
global daemon maxconn 8192 log 127.0.0.1 local5 info defaults mode http timeout connect 5000ms timeout client 50000ms timeout server 50000ms log global option httplog option dontlognull option http-server-close option tcplog clf option httplog clf stats enable stats refresh 10s stats uri /haprsts listen http-in bind *:80 mode http option forwardfor default_backend www_servers backend www_servers balance leastconn cookie SERVERID insert indirect nocache server server1 XXX.XXX.XXX.XXX:9009 maxconn 128 check cookie server1 server server2 YYY.YYY.YYY.YYY:9009 maxconn 128 check cookie server2 listen https-in mode http bind *:443 ssl crt /etc/haproxy/certs/cert.pem option forwardfor reqadd X-Forwarded-Proto:\ https default_backend www_servers_ssl backend www_servers_ssl balance leastconn cookie SERVERID insert indirect nocache server server1 XXX.XXX.XXX.XXX:9009 maxconn 128 check inter 3000 fall 2 rise 2 cookie server1 server server2 YYY.YYY.YYY.YYY:9009 maxconn 128 check inter 3000 fall 2 rise 2 cookie server2 XXX.XXX.XXX.XXX and YYY.YYY.YYY.YYY are public IP addresses.
Apache and the vhosts configs on the web server, are configured to listen to port 9009, as per the HAProxy config. Everything is running fine up to this point.
Now, I switch on Shorewall Firewall on the web server, with a firewall rules file that looks like this:
#SECTION ALL
#SECTION ESTABLISHED
#SECTION RELATED
SECTION NEW
ACCEPT net:ZZZ.ZZZ.ZZZ.ZZZ $FW:XXX.XXX.XXX.XXX TCP 9009
ACCEPT net:WWW.WWW.WWW.WWW $FW:XXX.XXX.XXX.XXX TCP 9009
# Drop Ping from the "bad" net zone.. and prevent your log from being flooded..
Ping(DROP) net $FW
# Permit all ICMP traffic FROM the firewall TO the net zone
ACCEPT $FW net icmp
ZZZ.ZZZ.ZZZ.ZZZ is the primary HAProxy server, and WWW.WWW.WWW.WWW is the secondary HAProxy server (and not relevant right now to this problem).
And all web traffic dies when the firewall is enabled on the web server.
The HAProxy server starts sending files like these to the syslog: Message from syslogd@localhost at Sep 30 12:59:38 ... haproxy[14631]: backend www_servers has no server available!
And the web server returns 503 errors.
There are other rules in the rules file, which for instance allows SSH on a certain port from a specific IP address, and those rules all work, so I assume (although I could be wrong) it has little to do with the Shorewall firewall rules per say.
Is it so, that HAProxy needs additional ports open on the web server (this is what I suspect) and if so what would those undocumented ports be, or does anyone know what could be wrong here?
Any help would be much appreciated.
Cheers, /j.
