Present Perfect


Picture Gallery
Present Perfect

ssh friction

Filed under: Hacking,sysadmin — Thomas @ 12:57


I haven't been too good this year at removing friction from my workflow. Today I wanted to change that. And the random friction thrown my way today has to do with ssh.

You see, somewhere along the line I read that it is a good idea to create separate keys for separate identities. So I have an identity for all work-related stuff (which I consider 'ring 1': it's unlikely to change but everyone can get fired or change jobs), one for personal stuff on machines I actually control ('ring 0': they'd have to pry it out of my dead hands), another for my 'public online default' identity ('ring 2': I can always pull a whytheluckystiff and pull myself of the net and reinvent myself), and then per-project identities ('ring 3': I may lose interest in being a fedora or gstreamer contributor without massive changes in my personality).

I started splitting ring 3 per project when it made sense - for example, Fedora recently enforced a key change even if your account wasn't compromised and even if you already have a strong passphrase on your key (like I had), and of course a massive flamefest ensued. I shrugged and decided to split off a new key and set that on all my machines.

But the problem is, this whole tower of ssh doesn't really work well in practice. I chose a long passphrase for the new fedora keys, but obviously I do not want to type that every time I clone a package or commit changes. So I use ssh-agent. In theory, ssh-agent adds your keys and asks you for the passphrase once, and is then able to offer those identities to the other side.

The problem is a lot of ssh servers out there only give you a few tries. So your ssh agent will offer identity by identity until it gets refused. If my fedora identity was added as the fourth identity I lose - I can't clone a package.

Specifying IdentityFile in the ssh config is useless. It is poorly documented, but IdentityFile files actually come after your ssh-agent identities. So your agent blasts all the wrong keys at the host first, and you get denied.

So you can specify IdentityOnly to make sure that only the identity file you want is being used. Sadly in that case it will not use the ssh-agent at all, so it will ask you for the password to your key file - the whole reason you want agents to be used in the first place.

Now obviously ssh has all the pieces it needs to Do The Right Thing. If my config says to use this identity and this identity only, ssh should be able to request ssh-agent to present that identity, and that identity only, and make the login happen without any password.

Surely I must be missing something obvious. Surely one of you uberhackers out there has set up the same thing as me. Why don't you comment about it here and help the rest of us?


  1. I don’t use an idendity per project, what’s the argument to justify that?
    What I do, is an identity per host I am on. i.e. a separate keypair for my laptop, workstation and phone. (all encrypted of course to elevate security, and using ssh-agent)
    To me this makes more sense. Suppose somehow one of my boxes gets compromised (read: the ssh keys on one of my boxes), that does not affect my other keys. So you can just remove the trust of that keypair everywhere (at work, on arch linux servers, on github, etc), but I’ll still be able to do my stuff if I use another machine. Compromising one of my machines does not mean comprising my other systems (ssh keys)
    If you have multiple keys on the same system for different projects (and they are all encrypted), and one of them can get compromised (by means of a keylogger, ssh-agent with a trojan in it, a camera filming you entering your passphrase, ..) than it’s very likely the others can get compromised anyway.

    Comment by Dieter@be — 2011-12-22 @ 13:59

  2. Dieter, in this particular case for example Fedora forced everyone to regenerate keys. If a project enforces that for whatever reason, I prefer to separate my identity for them into a separate one, because I don’t feel like recalling my key everywhere just because some other doofus put his ssh private key on a fedora server that got hacked.

    Comment by Thomas — 2011-12-22 @ 16:20

  3. Hey, sounds like a good excuse to do some Twisted Conch hacking. :) I bet it’s a lot easier to implement the key-use behavior you want for it than to figure out the mysterious OpenSSH configuration incantation. :)

    Comment by Jean-Paul Calderone — 2011-12-22 @ 14:37

  4. JP, haha, great answer :) My personal approach to security though is – this stuff is so hard, anyone else does a better job than I do. I’m scared of making security mistakes. But maybe I should give this one in particular a go this christmas…

    Comment by Thomas — 2011-12-22 @ 16:18

  5. ssh-agent sets two environment variables (SSH_AGENT_PID and SSH_AUTH_SOCK). There is no reason you can’t run several SSH agents in the background by starting ssh-agent and ssh-add-ing one agent/keyset per “ring”, storing the pair of SSH environment variables elsewhere (other environment variable names or a script file). Note that you’ll have to ssh-add the keys by path since ssh-add defaults to the standard key locations (~/.ssh/id_dsa and ~/.ssh/id_rsa).

    Then when you want to use a particular ring, copy in the pair of saved environment variables to the real names (SSH_AGENT_PID and SSH_AUTH_SOCK) and then call ssh or other ssh using command.

    Comment by Rod Morehead — 2011-12-22 @ 15:16

  6. I believe you’re mistaken: OpenSSH with the IdentitiesOnly option will use keys from the SSH agent. At least, that’s how it works for me. Here are the relevant bits of my SSH config – BTW all keys have passphrases and are loaded into the SSH agent:

    Host bitbucket.org
    IdentityFile ~/.ssh/id_rsa_bitbucket
    User hg

    Host github.com
    IdentityFile ~/.ssh/id_rsa_github
    User git

    Host *.fedoraproject.org *.fedorahosted.org
    IdentityFile ~/.ssh/id_rsa_fedora
    User syskill

    Host *
    IdentitiesOnly yes
    IdentityFile ~/.ssh/id_dsa
    IdentityFile ~/.ssh/id_rsa

    Comment by syskill — 2011-12-22 @ 17:08

  7. ssh-agent loaded keys will be tried before any file specified with -i or IdentityFile. So in your case, if you connect to bitbucket, it will work because of the agent, not because of your IdentityFile statement. Try it – remove that line and it will still work.
    If you were to put IdentitiesOnly in your bitbucket section, you would be asked for a password – the agent keys are skipped. Try it and see.

    Why is this a problem ? Because if you have multiple keys, you’re likely to be refused by a server for trying too many keys through your agent.

    Please give it a try and let me know if my hunches are right.

    Comment by Thomas — 2011-12-22 @ 17:21

  8. When you use ssh-agent to authenticate to an SSH server, the server will first list the identities that are stored in the agent (SSH2_AGENTC_REQUEST_IDENTITIES), and then ask to sign with an identity that is stored in the agent and listed in authorized-keys (SSH2_AGENTC_SIGN_REQUEST). As such, the server cannot refuse you for trying too many keys, since it is the server that chooses the key.

    Comment by Raf — 2011-12-22 @ 20:40

  9. Are you sure about that? According to `ssh -v`, I am only using the configured file through the IdentityFile statement (and it’s not alphabetically first, or anything, because it works for multiple hosts).

    The difference between my config and comment #6 is that the IdentitiesOnly statement is global in mine (i.e., not under a Host line).

    Comment by Elliott — 2011-12-22 @ 23:12

  10. If you have IdentitiesOnly, it will only ever look at the identity you specify with IdentityFile or -i, or the default files. It will not look at your agent, and you will be asked for your passphrase. Not sure what you mean by ‘that’, but it seems to be in line with what I posted no?

    Comment by Thomas — 2011-12-22 @ 23:30

  11. Please make a new post if you find a solution. I have run into the same problem, and running multiple instances of the agent is rather inconvenient. At worst fixing this would, I think, require changes to the software that uses the SSH agent – i.e. patching OpenSSH to Do The RIght Thing somehow.

    Comment by Tuure Laurinolli — 2011-12-23 @ 01:02

  12. I have a similar config to what Elliot describes in #8, and things work for me. On the first use, the agent prompts for my passphrase and from then uses the key from the agent (I have 6 keys — although 2 of them are now never used).

    Comment by Danielle — 2011-12-23 @ 02:27

  13. Yes, I agree with Danielle on this one. It “Just Works”, one might say. Unfortunately, even `ssh -vvv` does not seem to indicate that it’s asking the key agent for anything. But I definitely do not get asked on the second and subsequent times.

    I didn’t see Raf’s comment (which is definitely incorrect) earlier, so my “that” was really referring to your post.

    Are you perhaps not using gnome-keyring as your key agent?

    Comment by Elliott — 2011-12-29 @ 04:51

RSS feed for comments on this post. TrackBack URL

Leave a comment