Skip to content

ERR no server connection avaliable #120

@zkl94

Description

@zkl94

你好,我现在有redis cluster部署在k8s上,statefulset部署,版本5.0.9-alpine,用predixy作为proxy。predixy一开始运行良好,但是过一段时间后日志内就会大量报错,甚至直接拒绝连接 ERR no server connection avaliable,需要重启pod才能把error消除。但是过一段时间后error又会重现。 查询了一下之前的issue,发现其他人也有类似的问题,但似乎这个这个问题还没解决。请问这个问题如何解决?

以下是部分日志:

2020-09-28 18:19:43.269529 E Handler.cpp:437 h 1 s cnx-bionix-facial-monitor-redis-4.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 19 will be close with status 4 EventError 2020-09-28 18:19:43.269557 E Handler.cpp:437 h 1 s 10.187.144.105:6379 42 will be close with status 4 EventError 2020-09-28 18:19:43.269558 E Handler.cpp:437 h 1 s 10.187.144.46:6379 45 will be close with status 4 EventError 2020-09-28 18:19:43.269559 E Handler.cpp:437 h 1 s 10.187.144.26:6379 78 will be close with status 4 EventError 2020-09-28 18:19:43.269617 N Handler.cpp:264 server cnx-bionix-facial-monitor-redis-4.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 mark failure 2020-09-28 18:19:43.269644 N Handler.cpp:275 h 1 close s cnx-bionix-facial-monitor-redis-4.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 19 and c None -1 with status 4 EventError 2020-09-28 18:19:43.269659 N Handler.cpp:264 server 10.187.144.105:6379 mark failure 2020-09-28 18:19:43.269670 N Handler.cpp:275 h 1 close s 10.187.144.105:6379 42 and c None -1 with status 4 EventError 2020-09-28 18:19:43.269676 N Handler.cpp:264 server 10.187.144.46:6379 mark failure 2020-09-28 18:19:43.269677 N Handler.cpp:275 h 1 close s 10.187.144.46:6379 45 and c None -1 with status 4 EventError 2020-09-28 18:19:43.269682 N Handler.cpp:264 server 10.187.144.26:6379 mark failure 2020-09-28 18:19:43.269683 N Handler.cpp:275 h 1 close s 10.187.144.26:6379 78 and c None -1 with status 4 EventError 2020-09-28 18:19:43.269805 E Handler.cpp:437 h 1 s cnx-bionix-facial-monitor-redis-1.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 16 will be close with status 4 EventError 2020-09-28 18:19:43.269821 N Handler.cpp:264 server cnx-bionix-facial-monitor-redis-1.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 mark failure 2020-09-28 18:19:43.269823 N Handler.cpp:275 h 1 close s cnx-bionix-facial-monitor-redis-1.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 16 and c None -1 with status 4 EventError 2020-09-28 18:19:43.269925 E Handler.cpp:437 h 0 s 10.187.145.109:6379 36 will be close with status 4 EventError 2020-09-28 18:19:43.269940 N Handler.cpp:264 server 10.187.145.109:6379 mark failure 2020-09-28 18:19:43.269941 N Handler.cpp:275 h 0 close s 10.187.145.109:6379 36 and c None -1 with status 4 EventError 2020-09-28 18:19:43.270104 E Handler.cpp:437 h 1 s 10.187.145.170:6379 40 will be close with status 4 EventError 2020-09-28 18:19:43.270126 N Handler.cpp:264 server 10.187.145.170:6379 mark failure 2020-09-28 18:19:43.270135 N Handler.cpp:275 h 1 close s 10.187.145.170:6379 40 and c None -1 with status 4 EventError 2020-09-28 18:19:43.270254 N Logger.cpp:152 MissLog count 3 

predixy.conf:

predixy.conf: | Name redisproxy Bind 0.0.0.0:6379 WorkerThreads 2 MaxMemory 20G ClientTimeout 300 BufSize 65536 LogDebugSample 0 Authority { Auth xxxxxxxxx { Mode admin } } ClusterServerPool { Password xxxxxxxxx MasterReadPriority 60 StaticSlaveReadPriority 50 DynamicSlaveReadPriority 50 RefreshInterval 1 ServerTimeout 30 ServerFailureLimit 10 ServerRetryTimeout 1 KeepAlive 120 Servers { + cnx-bionix-facial-monitor-redis-0.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-1.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-2.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-3.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-4.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-5.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-6.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-7.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-8.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 + cnx-bionix-facial-monitor-redis-9.cnx-bionix-facial-monitor-redis-headless.cnx-bionix-facial-monitor.svc.cluster.local:6379 } } 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions