Description
Bug report
Bug description:
When system proxy bypass list is set, the urllib.request library on macOS and Windows resolves the hostname to an IP address and the IP address to a hostname (on Windows) before checking it against the system proxy bypass list (see here and here).
This causes DNS leak and HTTP requests to hang while waiting for DNS timeout in some air-gaped environments. This behavior also differs from other system applications (tested on macOS Sonoma with Safari and Windows Server 2022 with the Edge browser).
Test process on macOS and Windows:
Creating an A record from <my-test-domain>.net to <my-test-ip>.
macOS with Safari:
In the system network setting:
- "Web proxy (HTTP)" is set to 172.16.0.1:8000
- "Secure web proxy (HTTPS)" is set to 172.16.0.1:8000
- "Bypass proxy settings" is set to
<my-test-ip>
In Safari:
- visiting
http://<my-test-ip>: does not use the proxy - visiting
http://<my-test-domain>.net: uses the proxy
Windows Server 2022 with Edge browser:
in system network setting:
- "HTTP proxy" is set to 172.16.0.1:8000
- "Do not use proxy server" is set to
<my-test-ip>
In Edge browser:
- visiting
http://<my-test-ip>: does not use the proxy - visiting
http://<my-test-domain>.net: uses the proxy
urllib.request on Windows also resolves the IP address back to FQDN before check, here's a test for that:
Windows Server 2022 with Edge browser:
Update the Host file so the IP address can be resolved back to FQDN (socket.getfqdn("<my-test-ip>") == "<my-test-domain>.net").
In system network setting:
- "HTTP proxy" is set to 172.16.0.1:8000
- "Do not use proxy server" is set to .net
In Edge browser:
- visiting
http://<my-test-ip>: uses the proxy - visiting
http://<my-test-domain>.net: does not use the proxy
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS, Windows
Linked PRs
- gh-115197: Stop resolving host in urllib.request proxy bypass #115210
- [3.12] gh-115197: Stop resolving host in urllib.request proxy bypass (GH-115210) #116066
- [3.11] gh-115197: Stop resolving host in urllib.request proxy bypass (GH-115210) #116067
- [3.9] gh-115197: Stop resolving host in urllib.request proxy bypass (GH-115210) #116068
- [3.8] gh-115197: Stop resolving host in urllib.request proxy bypass (GH-115210) #116069
- [3.10] gh-115197: Stop resolving host in urllib.request proxy bypass (GH-115210) #116070