I've been using OpenSSH already for a while and although I've seen mentions of "public key authentication" and "RSA encryption" several times in it's config files, I never decided to figure out what it did exactly, and stuck to password authentication. But now the guys at work explained how it works and after reading more about it, I'm totally hooked on it!
It's a feature in ssh protocol version 2 (thus it's around for already a while, e.g. we can all use it without updating something) which essentially comes down to this: you generate an asymmetric key pair and distribute the public key to all remote hosts. When logging in to that host, the host will encrypt a random integer, which only you can decrypt (using the private key) and hence prove your identity. Too secure your private key you'll store it encrypted with a password. Ssh-agent (which is bundled with openssh) is the tool that interacts with ssh to perform this task: when logging in to a host, ssh-agent will open the private key for you automatically if it can decrypt it with the password it receives from you But the problem is you'll have to load (enter your password and decrypt the key) each time again.
This is where keychain comes in, or you can use SSH Agent (don't confuse this with the ssh agent that comes with openssh) if you're a Mac user and like gui's. These tools basically ask you your passwords for all private keys you wish to use in a session (with session I mean "the whole time of using your computer"), decrypt the encrypted key on your hard disk and cache the decrypted key in ram, so it can be used in all terminals you open.
For more information:
OpenSSH key management, Part 1: Understanding RSA/DSA authentication
OpenSSH key management, Part 2: Introducing ssh-agent and keychain
OpenSSH key management, Part 3: Agent forwarding and keychain improvements (freaks only ;-))