Some SSH tunnels reporting connection refused between Linux VM and OSX Lion

Question

I'm trying to connect to a set of services running on a Centos 6.3 VM guest (VMware Fusion 4) from a Mac OS X Lion host. I've setup .ssh/config script in /var/root:

Host testbed Hostname testbed User <username> Port 15922 # web server LocalForward localhost:80 testbed:80 # zend server admin console LocalForward localhost:10081 testbed:10081 # zend debugger LocalForward localhost:10000 testbed:10000 # mongodb LocalForward localhost:28017 testbed:28017 # couchdb LocalForward localhost:10080 testbed:5984 # scrapyd LocalForward localhost:6800 testbed:6800 # eclipse connection for zend debugger RemoteForward testbed:10137 localhost:10137 

I then run sudo ssh testbed to connect the tunnels. Sudo is required to tunnel port 80, hence root.

Some of the tunnels work fine. I can view localhost:80, localhost:6800 and localhost:10081 in Firefox. However some of the tunnels don't work. They timeout in the browser and produce this error in the shell:

channel 19: open failed: connect failed: Connection refused 

I thought initially this was an artefact of the port number, so for 5984, I tunnelled it back to localhost:10080. The behaviour was the same (timeout + error). I have verified all the services are running correctly using a text-only browser (lynx) in the shell.

I did wonder if this was a firewall issue. The Guest VM has a firewall, but it opens happily for outgoing connections. The host OS firewall is 'Off' (System Preferences -> Security -> Firewall).

netstat doesn't show any collisions on the ports

tcp6 0 0 ::1.6800 *.* LISTEN tcp4 0 0 127.0.0.1.6800 *.* LISTEN tcp6 0 0 ::1.10080 *.* LISTEN tcp4 0 0 127.0.0.1.10080 *.* LISTEN tcp6 0 0 ::1.28017 *.* LISTEN tcp4 0 0 127.0.0.1.28017 *.* LISTEN tcp6 0 0 ::1.10000 *.* LISTEN tcp4 0 0 127.0.0.1.10000 *.* LISTEN tcp6 0 0 ::1.10081 *.* LISTEN tcp4 0 0 127.0.0.1.10081 *.* LISTEN tcp6 0 0 ::1.80 *.* LISTEN tcp4 0 0 127.0.0.1.80 *.* LISTEN 

I've tested VM Fusion in two networking configs (NAT and Bridged) and both exhibit the same behaviour.

Is it possible that:

1. there's an IPTABLES firewall on the host OS (no iptables command in Mac OS X)?
2. there's some kind of protection in VMware Fusion that locks out certain ports from guest to host?
3. there's protection built into the linux services, so that some localhost only services won't be tunnelled?

Any kind of guidance appreciated. For completeness, here's a log of the SSH transaction using the verbose option:

OpenSSH_5.6p1, OpenSSL 0.9.8r 8 Feb 2011 debug1: Reading configuration data /var/root/.ssh/config debug1: Applying options for testbed debug1: Reading configuration data /etc/ssh_config debug1: Applying options for * debug1: Connecting to testbed [XX.XX.XX.XXX] port 15922. debug1: Connection established. debug1: permanently_set_uid: 0/0 debug1: identity file /var/root/.ssh/id_rsa type -1 debug1: identity file /var/root/.ssh/id_rsa-cert type -1 debug1: identity file /var/root/.ssh/id_dsa type -1 debug1: identity file /var/root/.ssh/id_dsa-cert type -1 debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3 debug1: match: OpenSSH_5.3 pat OpenSSH* debug1: Enabling compatibility mode for protocol 2.0 debug1: Local version string SSH-2.0-OpenSSH_5.6 debug1: SSH2_MSG_KEXINIT sent debug1: SSH2_MSG_KEXINIT received debug1: kex: server->client aes128-ctr hmac-md5 none debug1: kex: client->server aes128-ctr hmac-md5 none debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP debug1: SSH2_MSG_KEX_DH_GEX_INIT sent debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY debug1: Host '[testbed]:15922' is known and matches the RSA host key. debug1: Found key in /var/root/.ssh/known_hosts:7 debug1: ssh_rsa_verify: signature correct debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug1: SSH2_MSG_NEWKEYS received debug1: Roaming not allowed by server debug1: SSH2_MSG_SERVICE_REQUEST sent debug1: SSH2_MSG_SERVICE_ACCEPT received debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug1: Next authentication method: publickey debug1: Offering DSA public key: /path/to/my/private-key.ppk debug1: Server accepts key: pkalg ssh-dss blen 817 debug1: Authentication succeeded (publickey). Authenticated to testbed ([10.12.1.154]:15922). debug1: Local connections to localhost:80 forwarded to remote address testbed:80 debug1: Local forwarding listening on 127.0.0.1 port 80. debug1: channel 0: new [port listener] debug1: Local forwarding listening on ::1 port 80. debug1: channel 1: new [port listener] debug1: Local connections to localhost:10081 forwarded to remote address testbed:10081 debug1: Local forwarding listening on 127.0.0.1 port 10081. debug1: channel 2: new [port listener] debug1: Local forwarding listening on ::1 port 10081. debug1: channel 3: new [port listener] debug1: Local connections to localhost:10000 forwarded to remote address testbed:10000 debug1: Local forwarding listening on 127.0.0.1 port 10000. debug1: channel 4: new [port listener] debug1: Local forwarding listening on ::1 port 10000. debug1: channel 5: new [port listener] debug1: Local connections to localhost:28017 forwarded to remote address testbed:28017 debug1: Local forwarding listening on 127.0.0.1 port 28017. debug1: channel 6: new [port listener] debug1: Local forwarding listening on ::1 port 28017. debug1: channel 7: new [port listener] debug1: Local connections to localhost:10080 forwarded to remote address testbed:5984 debug1: Local forwarding listening on 127.0.0.1 port 10080. debug1: channel 8: new [port listener] debug1: Local forwarding listening on ::1 port 10080. debug1: channel 9: new [port listener] debug1: Local connections to localhost:6800 forwarded to remote address testbed:6800 debug1: Local forwarding listening on 127.0.0.1 port 6800. debug1: channel 10: new [port listener] debug1: Local forwarding listening on ::1 port 6800. debug1: channel 11: new [port listener] debug1: Remote connections from testbed:10137 forwarded to local address localhost:10137 debug1: channel 12: new [client-session] debug1: Requesting [email protected] debug1: Entering interactive session. debug1: remote forward success for: listen 10137, connect localhost:10137 debug1: All remote forwarding requests processed debug1: Sending environment. debug1: Sending env LANG = en_GB.UTF-8 Last login: Mon Sep 3 12:27:10 2012 from something.somenet [me@testbed ~]$ debug1: Connection to port 10080 forwarding to testbed port 5984 requested. debug1: channel 13: new [direct-tcpip] channel 13: open failed: connect failed: Connection refused debug1: channel 13: free: direct-tcpip: listening port 10080 for testbed port 5984, connect from 127.0.0.1 port 54639, nchannels 14 debug1: Connection to port 10080 forwarding to testbed port 5984 requested. debug1: channel 13: new [direct-tcpip] channel 13: open failed: connect failed: Connection refused debug1: channel 13: free: direct-tcpip: listening port 10080 for testbed port 5984, connect from 127.0.0.1 port 54640, nchannels 14 

Answer 1

Are some of the services bound to localhost on the guest? If you do an ssh forward from (host) localhost:8080 to (guest) 10.x.y.z:80 and the service is listening on (guest) 127.0.0.1:80 , it will give you connection refused.

Try making a tunnel from 127.0.0.1:port to 127.0.0.1:port , where the first is host and the second is guest.