Network timeouts: how to debug?

Question

I'm experiencing network timeouts when trying to fetch libraries from https://rubygems.org and I'd like to figure out why.

It only seems to affect requests made from my home network. If I take my computer to a different network, I can fetch from rubygems without issue.

It also seems to be somewhat specific to rubygems.org -- if I make the same requests to a different gem server (https://gems.ruby-china.org), I can fetch without timing out.

I watched the requests/responses while fetching gems (running bundle --verbose), and saw several successful requests, followed by some rate-limit errors (429 Too Many Requests), then some more successes before it hangs, and eventually times out.

I ran ss --tcp --processes at the same time and saw steadily-repeating SYN-SENT statuses associated with the bundle process.

State Recv-Q Send-Q Peer Address:Port SYN-SENT 0 1 [2a04:4e42::514]:https users:(("bundle",pid=17701,fd=8)) SYN-SENT 0 1 [2a04:4e42::514]:https users:(("bundle",pid=17701,fd=8)) ... 

Since I'm still getting familiar with networking, I could be looking at this from the wrong angle. Any pointers would be greatly appreciated.

Answer 1

Observe the rate limits rubygems.org imposes. Count the requests with packet captures and proxies, See if you have broken software, or lots of users behind NAT, that exceeds the limits.

Collect performance data by requesting something basic from the API (perhaps https://rubygems.org/api/v1/downloads.json) on a regular basis from many different hosts across the Internet.

Services exist that help with this. ThousandEyes has a free tier, you can spin up a web test to their API on your network and a dozen cloud endpoints. The path visualization graphs can be enlightening.