pip install using SSH reverse tunnel through a jumphost

Question

I've searched for an answer to this, but haven't had any luck.

I'm trying to install some PyPi packages on a server(tachi) which does not have internet access, and it is only reachable through a ssh jumphost(pella) which uses 2FA(google-authenticator).

Normally, if there was no jumphost in the way, I'd do something like this to let pip fetch packages through my local machine(koto):

ssh [email protected] -R tachi.example.org:8888

and then on tachi:

pip3 --proxy socks5://localhost:8888 install --upgrade pip

This works fine.

However, when I add a jumphost(pella) to the mix, it does not work. On the jumphost I have enabled these in sshd_config:

AllowTcpForwarding yes AllowAgentForwarding yes PermitTunnel yes 

(these are also enabled on tachi)

So I've tried to run these commands:

tony@koto:~$ ssh pella.example.org -R pella.example.org:8888 tony@pella:~$ ssh tachi.example.org -R 8888:tachi.example.org:8888 tony@tachi:~$ pip3 --proxy socks5://localhost:8888 install --upgrade pip 

But pip doesn't connect and eventually times out with:

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x71b236020dd0>: Failed to establish a new connection: [Errno 99] Cannot assign requested address')': /simple/pip/ 

I've also tried:

tony@koto:~$ ssh [email protected] -R tachi.example.org:8888 -J [email protected] 

But pip still doesn't connect.

If I look at the output from ssh -vvvv, it clearly states that the reverse tunnels are created, and on both the jumphost(pella) and the server(tachi) I can see using ss -anp that the tunnel is there on port 8888.

I'm probably not using the tunnelling correctly, but I haven't been able to find a solution of figure out how to do it.

Any SSH-gurus around who's got any ideas?

TIA,

/tony edit: forgot to mention that tachi also have forwarding etc. enabled

Answer 1

A few of things should get you closer to a solution:

(1) In your ssh commands, use -D localhost:8888 instead of -R .... -R & -L are for generic TCP port forwarding, where as -D does the extra SOCKS magic.

(2) -J <jumphost_user>@<jump_host> is more likely to work smoothly, and a bit more likely give you meaningful error message.

(3) For -R <p1>:host:<p2> and -L <p1>:host:<p2> you should almost always use localhost. (unless you really do want to open the server port to machines other than src,jump,dest.

(4) Use ssh -G ... to check the result of reading both /etc/ssh/config and ~/.ssh/config at each "hop":

• tony@koto:~$ ssh -G ... pella.example.org
• tony@pella:~$ ssh -G ... tachi.example.org

...to see if your forwarding/tunneling args are getting read/processed/accepted as you intended

(5) Order of args: Maybe ssh lets you be sloppy, but you should almost always have -flag args before the non-flag args (ssh -R <p1>:localhost:<p2> targethost)

Good luck!