What makes you think you are limited to 64K clients? You should be able to serve more than that. It's not the port count that is the limiting factor, but the memory and CPU power that limits the amount of connections you can have open at any given time. Check: http://www.kegel.com/c10k.html which is dated, just think of it as a c100k or c1M problem instead. :-)
By the way the haproxy site has an excellent article on the subject of load balancing and haproxy's architecture: http://haproxy.1wt.eu/download/1.2/doc/architecture.txt
By the way, regardingRegarding connection limit, this is a theoretical limit that normally you wouldn't reach as you'd run out of resources before that.
"The TCP standard sets up unique connection identifiers as the tuple of local IP address, local TCP port number, remote IP address, and remote TCP port number. In your example, the local numbers are both fixed, which leaves approximately 2^32 remote IP (version 4) addresses, and 2^16 TCP port numbers, or an approximate total potential simultaneous TCP connections of 281,474,976,710,656 (2^48, or 2.81 * 10^14, or 281 trillion)."