Why Multiple SSH Keys Don’t Work with Git, and How to Make Them

Good security hygiene includes keeping your credentials separate. You should have separate identities for work and personal tasks. Unfortunately, that’s not so easy with SSH and Git. If you generate multiple SSH keys and try to clone a GitHub repository, you may find that you don’t have access — even though you have the correct key available.

Why this happens is an interesting lesson in layered software design. Understanding the problem led me to a clean fix that suits my workflow.

How SSH Works

SSH isn’t just a way to authenticate and secure remote access to a Git repository. Because it’s so flexible, SSH is a valuable tool for Unix users and has been for a very long time.

In the past, SSH’s most common use was logging in to a remote system for administration or diagnosis (or even some kinds of development). But you can also run pretty much anything on the other end of an SSH connection, including Git. When you do, you get the benefit of SSH’s strong authentication and encryption.

SSH can authenticate with a password, but the reason you’re probably using it for Git is to avoid doing that every time you push or pull. You do this by generating an SSH public/private key pair with ssh-keygen, then supplying your Git provider (e.g., GitHub) with the public half so your computer can use the private half.

How Git Works with SSH

Once you’ve done this, your Git provider loads your SSH public keys into its own list of keys. When you connect, SSH comes first. Your SSH client tells your Git provider what keys it has access to, and the provider’s SSH server accepts one of them. This works great if you’ve only got a single account with your Git provider (even if you have multiple keys for that account) because they all resolve to the same account.

But when you have two (for example, a work account and a personal account), there’s a chance your Git provider’s SSH server will pick one that doesn’t have access to the repository you want to access. The SSH layer doesn’t know which account you want to use. It doesn’t communicate with Git until after authentication.

(I can hear the system design gears turning in your head from here. Why not have SSH and Git work together to pick a key that’s valid for the repository you’re trying to access? That’s definitely a thing that could solve some cases. But it also means highly-secure SSH code is intermingling with highly-complex Git code, which can compromise the former. It’s an expensive feature to add. It’s also an impossible decision if you have different rights to the same repository across accounts.)

For SSH to authenticate the right account, you need to help it along a little.

The SSH Agent

There’s another really useful component to SSH that you might not be using: the SSH agent. The agent’s job is to keep your SSH private keys and use them — but not give them up — when requested.

SSH can also read your keys directly from your .ssh directory, but only if they’re unencrypted. If you’ve encrypted them at rest (using the -p flag to ssh-keygen), SSH will ask for a passphrase to unlock them. From a user perspective, this isn’t much better than being asked for a password every time.

That said, it’s good security hygiene to keep your keys encrypted at rest. If you don’t, any non-sandboxed software you run can steal them.

Thankfully, keeping your keys encrypted is very easy. macOS implements the SSH agent as part of the keychain and can store the passphrases in the keychain. The keychain will only use your passphrases to unlock your keys. Similar facilities exist in other operating systems.

SSH private keys are added to the agent with ssh-add. If they’re unencrypted, they’ll just get added, and SSH will connect to the agent and offer to authenticate to a server with them. If they’re encrypted, you’ll get asked a passphrase, but only once per agent session.

Storing your passphrases in your keychain instead means you’ll only ever have to do it once. This is done on macOS with the special -K flag, which tells ssh-add to look for previously-used passphrases and add new passphrases to the keychain.

Switching Keyrings

Okay, this is all great. We’ve taken our SSH key security to the next level. But how does this help us with Git?

I’ve seen a few solutions to this problem, including creating fake hostnames to distinguish between the keys. These solutions all require some setup and don’t strike me as particularly clean.

I carry car keys and home keys with me on two linked keyrings. When I’m using the car, I grab the car end. When I’m entering my home, I grab the home end. Similarly, I use work credentials when I’m being paid and personal credentials when I’m doing my own thing. What I need is to be able to pick the keyring I want to use.

To set this up, I gave my personal keys standard file names, such as id_rsa. My work keys got a prefix of atomic, such as atomic_rsa. I wrote the following script:


# clear out old keyring
ssh-add -D

# add requested keys, or id keys by default
ssh-add -K $HOME/.ssh/${1-id}_{dsa,ecdsa,ed25519,rsa}

Now, when I’m working, I issue ssh-keyring atomic. Any existing keys are cleared from my agent, and my work keys are loaded. Just typing ssh-keyring will use my personal keys instead.

This solution keeps me from having to modify system files or create complex ssh configurations. It’s also a really good fit for making a clean break between personal time and work time.