Hey! Listen! This looks like it’s been fixed in macOS 11.4! You can read the rest of the post for a fun deep-dive though.
Using a Yubikey for user identity is great. Your Yubikey generates a private key that never
leaves the device, outputs a CSR that your IT org can sign, and boom you’ve got a pretty
solid story around user identity. For a while macOS has shipped an SSH agent
that (I think?) accesses the Yubikey’s certificates through the Keychain and/or
CryptoTokenKit API. It is helpfully named
I had successfully tested that my Yubikey worked with my VPN, and that in-browser mutual TLS auth worked as expected. It seemed like a slam dunk to give SSH the old college try, until…
$ ssh-add -s /usr/lib/ssh-keychain.dylib Enter passphrase for PKCS#11: Could not add card "/usr/lib/ssh-keychain.dylib": agent refused operation
Oh, uh… ok… why’s that?
$ ssh-agent -d ... debug1: process_message: socket 1 (fd=4) type 20 failed PKCS#11 add of "/usr/lib/ssh-keychain.dylib": realpath: No such file or directory
Fascinating. So, the library is not in /usr/lib (or anywhere in /usr). Yet, you could run
man ssh-keychain and read more about using this PKCS11 Provider. I found this amusing:
haha, macos 11 kills /usr/lib/ssh-keychain.dylib but keeps the developer documentation. at least i can read about what used to work :)— Pumpkin Patch Cable (@patcable) November 14, 2020
I posted about this in a few places, and fortunately mendel chimed in to say “some libraries live only in the cache in macOS 11, but not in the filesystem, might it be that?” and linked me to two blog posts:
It turns out a thing that changed in macOS Big Sur is that all the system libraries live in a
cache located in
/System/Library/dyld/dyld_shared_cache_x86_64. When an application calls
libc goes and pulls the library from the cache instead of the file system. This is interesting
from a security perspective, but: is my SSH provider hiding in that cache too?
I ended up doing the steps in the second post to build
dyld_shared_cache_util, which was a lot of fun
as a first time Xcode user1. But, fortunately the effort paid off because I did find
in the cache. The existence of the man page (and the bug) now made way more sense.
In OpenSSH 7.5 and later, OpenSSH checks that whatever PKCS11 libraries you pass it are
on an allowlist to mitigate CVE-2016-10009.
If the library or it’s path is on the allowlist,
ssh-agent passes the library to
which actually calls libc’s
dlopen(). But, since the library doesn’t exist in
in the first place, my guess is that it never makes it that far.
Hopefully Apple has a fix for this in an upcoming release, since I preferred relying on the OS for token access. In the meantime, dropping OpenSC’s OnePIN PKCS11 library into /usr/local/lib works fine.
It was not fun. ↩︎