What causes the ssh error "REMOTE HOST IDENTIFICATION HAS CHANGED" on connections with a port specifier (i.e not the default of 22)

Question

I have migrated an old Amazon Linux server running OpenSSH_7.4p1, OpenSSL 1.0.2k-fips to a new Ubuntu 24 LTS box running OpenSSH_9.6p1 Ubuntu-3ubuntu13.11, OpenSSL 3.0.13. As part of the migration I have copied the contents of /etc/ssh/ssh_host* from the old box to the new box. I have also copied the old user home directories (inc root's) to ensure their ssh public and private keys remain, in addition to all of the ~/.ssh/known_hosts files.

The public DNS hostname and the public IP of the new server are the same as the old server, as I have just reassigned the elastic IP from the old server to the new within AWS.

Users connect to the server using a keypair via a hostname, e.g

ssh [email protected] -i ~/.ssh/id_rsa

Because I copied the host keys from the old server to the new one, this works fine and the users are none the wiser. So far so good. BUT: Some clients can connect only if they do so on the default port of 22. If they try to use, for example, -p443 then they are met with the error

Host key for [server.example.com]:443 has changed and you have requested strict checking.

We run ssh listening on 22 and 443 as connecting over 443 is, as I'm sure you all know, a handy way to get through rudimentary non-TLS decrypting firewalls that block outbound 22. Some of our clients are located on such networks.

Connecting to the old server using the same host name but with either port works fine without the fingerprint error. So I believe the cause may be something in /etc/sshd_config or perhaps a change in the newer version of sshd?

If I examine a client that does connect on both 22 and 443 versus one that will only connect without error/warning on 22, they are both running fairly fresh installs of Ubuntu 24 and report OpenSSH_9.6p1 Ubuntu-3ubuntu13.11, OpenSSL 3.0.13.

ssh-keyscan -p443 and ssh-keyscan run against the old server and new both return the same fingerprints.

I'm at a loss and cannot find anyone experiencing this same error. Of course, the noise generated by questions about how to solve the general "REMOTE HOST IDENTIFICATION HAS CHANGED!" error does not help to sift out more interesting questions.

Thank you

Answer 1

ssh to remote.example.com and ssh to remote.example.com on port 443 is not the same thing. As far as the ssh client is concerned, these are different hosts.

If you inspect the contents of ~/.ssh/known_hosts (or ~/.ssh/known_hosts2) you may notice entries for both in this format:

remote.example.com ssh-... [remote.example.com]:443 ssh-... 

Therefore the solution would be to save both keys in your known_hosts file.

P.S. I may be wrong to suggest the above, but if you try using ssh -vv it may help you see what the difference and problems are.