2

We recently migrated from Perlbal to HAProxy due to Perlbal's memory usage. It's been a fairly flawless migration though lately we've run into an issue that I can't seem to figure out (it doesn't help I'm not familiar with HAproxy's umpteen configuration options).

We're a media website serving a fair amount of traffic - we use HAProxy to spread the load across our media servers. Upon starting HAProxy everything is snappy and works accordingly - though after about 10-20 minutes things start to slow down and media is served very slowly. I'm fairly certain it's HAProxy since a simple restart of the service rectifies the issue.

I'm using a fairly vanilla HAProxy configuration with no bells or whistles:

 defaults log global mode http option httplog option dontlognull retries 3 option redispatch maxconn 2000 contimeout 5000 clitimeout 50000 srvtimeout 50000 listen media 0.0.0.0:80 balance roundrobin server media_1 x.x.x.x:8080 weight 1 maxconn 1024 check server media_2 x.x.x.x:8080 weight 1 maxconn 1024 check

Could anyone shed some light onto what the issue may be?

Improve this question

4 Answers 4

Reset to default
2

Your config is very simple indeed. I see that you don't have "option httpclose" and you have a very low "maxconn", maybe you're simply running out of connections under high load ? You should definitely enable this option. You should also enable the stats page, by adding "stats uri /haproxy-stats" in the "listen" section. Then you connect your browser to that URL on your site and you'll see all the stats with concurrent connections, connections per second, errors, ... That will help you a lot.

Oh, what version is this ? 1.3.16 has a nasty bug which makes it eat a lot of CPU on some traffic patterns. You should be using either 1.3.18 or 1.3.15.X.

Willy

Improve this answer
1
  • I've toyed with the httpclose option with no luck, it's currently in my production configuration, though I don't believe this to be causing the issue. I do indeed have stats enabled, I just emitted it since it wasn't pertitent to the issue. It wasn't running out of max connections / no errors if I recall - I'll have to do some testing to verify this though. I am running a older version - it didn't even occur to me to check the bug reports (silly I know).. I'll take a peak and see if this may be the issue. I'll post back letting you know your suggestion(s) help, thanks! Commented Jun 10, 2009 at 16:00
1

Since it happens over time, it's probably resource depletion of some sort.

Are you monitoring the various free memory statistics and processor usage?

Improve this answer
1
  • Indeed, which is why I'm fairly certain it's something silly in the HAProxy configuration. Commented Jun 9, 2009 at 15:15
1

I'd say that the first thing to do is enable the HAProxy stats page, and perhaps logging, too.

Improve this answer
1

It may also be possible that your backend web servers are refusing too many connections from a single IP. With HAProxy (without tproxy), requests are seen coming from the load balancing node. This could create such problems.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.