ssh: authorized keys vs known hosts

In ssh, what is the difference between the known_hosts file and the authorized_keys file? This may seem a basic part of understanding ssh, but it recently tripped me up while automating the configuration for an ssh connection between servers which will then be used for further automation.

I created a new ssh key-pair on the client machine with the familiar command ssh-keygen -t ed25519 and copied the keytype and public key to the authorized_keys file in the home directory of my login user on the host.

[james@test ~]$ cat ~/.ssh/authorized_keys
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIK+8gTqN9R80CZ4AAb2NNyRVXWjsYsSijLyTaeMdauJ4

Now, if I just wanted to manually connect to the host, this would be enough. The first time we run ssh to connect to the new host we are met with this familiar message:

[james@beepboop ~]$ ssh test.mydomain.com
The authenticity of host 'test.mydomain.com (b490:8736:4e8c:4246::1)' can't be established.
ED25519 key fingerprint is SHA256:K6cZ6TBMZr09k4NuQqo7SfodWo5Wl6UHRs1Ja0p9kLU.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

This is ssh giving us a chance to inspect the fingerprint of the public key that the host presented to us and verify that we really are talking to the correct host. If so, we type yes and ssh adds the public key data for the host to the known_hosts file:

Warning: Permanently added 'test.mydomain.com' (ED25519) to the list of known hosts.

known_hosts now looks like this:

[james@beepboop ~]$ cat ~/.ssh/known_hosts
test.mydomain.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAejS63Hz3qQzTSXysvtgtmcMvKW4fZJDd4aMcicWr4L

But we want to automate! To automate things, we don't want to have to verify the key and type yes. We want to take the public key from the host and put it in our known_hosts file before we use ssh to connect. If we do that ssh to the new host will just work.

This is where I got tripped up... Where does the host's public key reside? I had never thought about it before. We can already ssh to this host via a different client, so the host must already have a key - we don't need to create any new keys.

As usual, the answer could be found in the man pages. It turns out that sshd by default has host key-pairs in /etc/ssh and they have stems ssh_host_ecdsa_key, ssh_host_ed25519_key, and ssh_host_rsa_key. There are multiple keys, one for each host key algorithm. If we inspect the .pub public keys of these pairs we will see that they match what ssh added to our known_hosts file. We just need to take the contents of the public key from one of the pairs to create the automation and put it in known_hosts ourself.

authorized keys

• authorized_keys lives on the host at ~/.ssh/authorized_keys. It is unique per user and the ssh user determines the $HOME directory in ~ expansion. For example, if the client does ssh alice@somewhere.com, sshd will look at /home/alice/.ssh/authorized_keys (assuming alice's home directory is configured to /home/alice).

• authorized_keys contains client key-pair info and is used by sshd on the host to decide if a login is allowed or not.

• client key-pairs are the ones we typically see the most: by default they live in ~/.ssh and have a stem like id_rsa or id_ed25519 (but you can name them whatever you like).

• Not only does authorized_keys let us authorize keys, it also lets us do other cool and useful things like restricting what command the connecting client can perform. This is an essential part of hardening ssh access and employing the security principle of least privilage. This is relevant both for ssh connections in automation and for manual ssh connections that one might administer for users.

known hosts

• known_hosts lives on the client at ~/.ssh/known_hosts. It is unique per user. There is also a globally shared one at /etc/ssh/ssh_known_hosts.

• known_hosts contains host key-pair info and is used by the ssh client to decide if the host is legit or not.

• host key-pairs normally sit in /etc/ssh/ and have stems such as ssh_host_rsa_key, ssh_host_ecdsa_key, ssh_host_ed25519_key. It's common to see one of each type there and probably uncommon to see them with different names.

• Not only does known_hosts let us list the hosts we know, it can also let us "mark" keys as @revoked or from a @cert-authority.

The names themselves were a clue and now serve as a great reminder of their function. Shout-out to the ssh devs for good naming!

Check out the manual page man sshd for more info. It describes the authorized_keys file format and the known_hosts file format it detail.