Issue connecting to RedisCluster that has more than one network interface #2480
Description
Version: What redis-py and what redis version is the issue happening on?
redis-py 4.3.4, 4.3.5
redis 7.0.4
Platform: What platform / version? (For example Python 3.5.1 on Windows 7 / Ubuntu 15.10 / Azure)
Python 3.6.8, 3.8.2 Linux
Python 3.10.7 MacOS
Description: Description of your issue, stack traces from errors and code that reproduces the issue
I have a redis cluster running on a server that has two network interfaces.
Locally on the server, the following code works for connecting to the cluster:
>>> import redis >>> rc = redis.RedisCluster(host="127.0.0.1", port=30001) >>> rc = redis.RedisCluster(host="my.public.ip.address", port=30001) >>> rc = redis.RedisCluster(host="my.private.ip.address", port=30001) On a separate host that can only reach the cluster via its public IP address, the following succeeds:
>>> import redis >>> rc = redis.RedisCluster(host="my.public.ip.address", port=30001) On a third host that can only reach the cluster via its private IP address, the following FAILS:
>>> import redis >>> rc = redis.RedisCluster(host="my.private.ip.address", port=30001) And produces the following errors:
<class 'redis.exceptions.ConnectionError'> Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. <class 'redis.exceptions.ConnectionError'> Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. <class 'redis.exceptions.ConnectionError'> Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. <class 'redis.exceptions.ConnectionError'> Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. <class 'redis.exceptions.ConnectionError'> Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 611, in connect sock = self.retry.call_with_retry( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/retry.py", line 46, in call_with_retry return do() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 612, in <lambda> lambda: self._connect(), lambda error: self.disconnect(error) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 677, in _connect raise err File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 665, in _connect sock.connect(socket_address) ConnectionRefusedError: [Errno 111] Connection refused During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1067, in _execute_command connection = get_connection(redis_node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 50, in get_connection return redis_node.connection or redis_node.connection_pool.get_connection( File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 1387, in get_connection connection.connect() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/connection.py", line 617, in connect raise ConnectionError(self._error_message(e)) redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:30001. Connection refused. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1450, in initialize copy_kwargs = copy.deepcopy(kwargs) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 237, in _deepcopy_method return type(x)(x.__func__, deepcopy(x.__self__, memo)) File "/usr/lib/python3.8/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 172, in deepcopy y = _reconstruct(x, memo, *rv) File "/usr/lib/python3.8/copy.py", line 270, in _reconstruct state = deepcopy(state, memo) File "/usr/lib/python3.8/copy.py", line 146, in deepcopy y = copier(x, memo) File "/usr/lib/python3.8/copy.py", line 230, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python3.8/copy.py", line 161, in deepcopy rv = reductor(4) TypeError: cannot pickle '_thread.lock' object During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 572, in __init__ self.commands_parser = CommandsParser(self) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/commands/parser.py", line 16, in __init__ self.initialize(redis_connection) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/commands/parser.py", line 19, in initialize commands = r.execute_command("COMMAND") File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1029, in execute_command raise e File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1019, in execute_command res[node.name] = self._execute_command(node, *args, **kwargs) File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1103, in _execute_command self.nodes_manager.initialize() File "/usr/local/share/pynq-venv/lib/python3.8/site-packages/redis/cluster.py", line 1486, in initialize raise RedisClusterException( redis.exceptions.RedisClusterException: ERROR sending "cluster slots" command to redis server 127.0.0.1:30001. error: cannot pickle '_thread.lock' object >>> From this same third host that produces the above error, I can successfully connect with redis-cli via the following command:
redis-cli -c -h my.private.ip.address -p 30001