Tommy Skaug
a4adbc2b1c
All checks were successful
Export / Explore-GitHub-Actions (push) Successful in 34s
230 lines
8.1 KiB
Markdown
230 lines
8.1 KiB
Markdown
## Key Takeaways
|
|
|
|
* SSH has a key-signing concept that in combination with a
|
|
smartcard provides a lean, off-disk process
|
|
* A SSH-CA provides the possibility of managing access
|
|
without a central point of failure
|
|
* The use of SSH Jumphost is an easier way to tunnel
|
|
sessions end-to-end encrypted, while still maintaining
|
|
visibility and control through a central point
|
|
|
|
## Introduction
|
|
|
|
This post is an all-in-one capture of my recent discoveries with
|
|
SSH. It is an introduction for a technical audience.
|
|
|
|
It turns out that SSH is ready for a zero trust and
|
|
microsegmentation approach, which is important for
|
|
management of servers. Everything described in this post is
|
|
available as open source software, but some parts require a
|
|
smartcard or two, such as a Yubikey (or a Nitrokey if you
|
|
prefer open source. I describe both).
|
|
|
|
I also go into detail on how to configure the CA key without
|
|
letting the key touch the computer, which is an important
|
|
principle.
|
|
|
|
The end-result should be a more an architecture providing a better
|
|
overview of the infrastructure and a second logon-factor
|
|
independent of phones and OATH.
|
|
|
|
## SSH-CA
|
|
|
|
My exploration started when I read a 2016-article by
|
|
Facebook engineering [1]. Surprised, but concerned with the
|
|
configuration overhead and reliability I set out to test the
|
|
SSH-CA concept. Two days later all my servers were on a new
|
|
architecture.
|
|
|
|
SSH-CA works predictably like follows:
|
|
|
|
[ User generates key on Yubikey ]
|
|
|
|
|
|
|
|
v
|
|
[ ssh-keygen generates CA key ] --------> [ signs pubkey of Yubikey ]
|
|
| - for a set of security zones
|
|
| - for users
|
|
| |
|
|
| |
|
|
| v
|
|
v pubkey cert is distributed to user
|
|
[ CA cert and zones pushed to servers ] - id_rsa-cert.pub
|
|
- auth_principals/root (root-everywhere)
|
|
- auth_principals/web (zone-web)
|
|
|
|
The commands required in a nutshell:
|
|
|
|
# on client
|
|
$ ssh-keygen -t rsa
|
|
|
|
# on server
|
|
$ ssh-keygen -C CA -f ca
|
|
$ ssh-keygen -s ca -I <id-for-logs> -n zone-web -V +1w -z 1 id_ecdsa.pub
|
|
|
|
# on client
|
|
cp id_ecdsa-cert.pub ~/.ssh/
|
|
|
|
Please refer to the next section for a best practice storage
|
|
of your private key.
|
|
|
|
On the SSH server, add the following to the SSHD config:
|
|
|
|
TrustedUserCAKeys /etc/ssh/ca.pub
|
|
AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u
|
|
|
|
What was conceptually new for me was principals and
|
|
authorization files per server. This is how it works:
|
|
|
|
1. Add a security zone, like zone-web, during certificate
|
|
signing - "ssh-keygen * -n zone-web *". Local username does
|
|
not matter
|
|
2. Add a file per user on the SSH server, where zone-web
|
|
is added where applicable -
|
|
e.g. "/etc/ssh/auth_principals/some-user" contains "zone-web"
|
|
3. Login with the same user as given in the zone file - "ssh some-user@server"
|
|
|
|
This is the same as applying a role instead of a name to the
|
|
authorization system, while something that IDs the user is
|
|
added to certificate and logged when used.
|
|
|
|
This leaves us with a way better authorization and
|
|
authentication scheme than authorized_keys that everyone
|
|
uses. Read on to get the details for generating the CA key
|
|
securely.
|
|
|
|
|
|
## Keeping Private Keys Off-disk
|
|
|
|
An important principle I have about private keys is to
|
|
rather cross-sign and encrypt two keys than to store one on
|
|
disk. This was challenged for the SSH-CA design. Luckily I found
|
|
an article describing the details of PKCS11 with ssh-keygen
|
|
[2]:
|
|
|
|
> If you're using pkcs11 tokens to hold your ssh key, you
|
|
> may need to run ssh-keygen -D $PKCS11_MODULE_PATH
|
|
> ~/.ssh/id_rsa.pub so that you have a public key to
|
|
> sign. If your CA private key is being held in a pkcs11
|
|
> token, you can use the -D parameter, in this case the -s
|
|
> parameter has to point to the public key of the CA.
|
|
|
|
Yubikeys on macOS 11 (Big Sur) requires the yubico-piv-tool
|
|
to provide PKCS#11 drivers. It can be installed using
|
|
Homebrew:
|
|
|
|
$ brew install yubico-piv-tool
|
|
$ PKCS11_MODULE_PATH=/usr/local/lib/libykcs11.dylib
|
|
|
|
Similarly the procedure for Nitrokey are:
|
|
|
|
$ brew cask install opensc
|
|
$ PKCS11_MODULE_PATH=/usr/local/lib/opensc-pkcs11.so
|
|
|
|
Generating a key on-card for Yubikey:
|
|
|
|
$ yubico-piv-tool -s 9a -a generate -o public.pem
|
|
|
|
For the Nitrokey:
|
|
|
|
$ pkcs11-tool -l --login-type so --keypairgen --key-type RSA:2048
|
|
|
|
Using the exported CA pubkey and the private key on-card a
|
|
certificate may now be signed and distributed to the user.
|
|
|
|
$ ssh-keygen -D $PKCS11_MODULE_PATH -e > ca.pub
|
|
|
|
$ ssh-keygen -D $PKCS11_MODULE_PATH -s ca.pub -I example -n zone-web \
|
|
-V +1w -z 1 id_rsa.pub
|
|
Enter PIN for 'OpenPGP card (User PIN)':
|
|
Signed user key .ssh/id_rsa-cert.pub: id "example" serial 1 for zone-web
|
|
valid from 2020-10-13T15:09:00 to 2020-10-20T15:10:40
|
|
|
|
The same concept goes for a user smart-card, except that is
|
|
a plug and play as long as you have the gpg-agent
|
|
running. When the id_rsa-cert.pub (the signed certificate of
|
|
e.g. a Yubikey) is located in ~/.ssh, SSH will find the
|
|
corresponding private key automatically. The workflow will
|
|
be something along these lines:
|
|
|
|
[ User smartcard ] -----------> [ CA smartcard ]
|
|
^ id_rsa.pub |
|
|
| | signs
|
|
|------------------------------|
|
|
sends back id_rsa-cert.pub
|
|
|
|
|
|
## A Simple Bastion Host Setup
|
|
|
|
The other thing I wanted to mention was the -J option of
|
|
ssh, ProxyJump.
|
|
|
|
ProxyJump allows a user to confidentially, without risk of a
|
|
man-in-the-middle (MitM), to tunnel the session through a
|
|
central bastion host end-to-end encrypted.
|
|
|
|
Having end-to-end encryption for an SSH proxy may seem
|
|
counter-intuitive since it cannot inspect the
|
|
content. I believe it is the better option due to:
|
|
|
|
* It is a usability compromise, but also a security
|
|
compromise in case the bastion host is compromised.
|
|
* Network access and application authentication (and even
|
|
authorization) goes through a hardened point.
|
|
* In addition the end-point should also log what happens on
|
|
the server to a central syslog server.
|
|
* A bastion host should always be positioned in front of the
|
|
server segments, not on the infrastructure perimeter.
|
|
|
|
A simple setup looks like the following:
|
|
|
|
[ client ] ---> [ bastion host ] ---> [ server ]
|
|
|
|
|
|
Practically speaking a standalone command will look like
|
|
follows:
|
|
|
|
ssh -J jump.example.com dest.example.com
|
|
|
|
|
|
An equivalent .ssh/config will look like:
|
|
|
|
Host j.example.com
|
|
HostName j.example.com
|
|
User sshjump
|
|
Port 22
|
|
|
|
Host dest.example.com
|
|
HostName dest.example.com
|
|
ProxyJump j.example.com
|
|
User some-user
|
|
Port 22
|
|
|
|
With the above configuration the user can compress the
|
|
ProxyJump SSH-command to "ssh dest.example.com".
|
|
|
|
## Further Work
|
|
|
|
The basic design shown above requires one factor which is
|
|
probably not acceptable in larger companies: someone needs
|
|
to manually sign and rotate certificates. There are some
|
|
options mentioned in open sources, where it is normally to
|
|
avoid having certificates on clients and having an
|
|
authorization gateway with SSO. This does however introduce
|
|
a weakness in the chain.
|
|
|
|
I am also interested in using SSH certificates on iOS, but
|
|
that has turned out to be unsupported in all apps I have
|
|
tested so far. It is however on the roadmap of Termius,
|
|
hopefully in the near-future. Follow updates on this subject
|
|
on my Honk thread about it [4].
|
|
|
|
For a smaller infrastructure like mine, I have found the
|
|
manual approach to be sufficient so far.
|
|
|
|
|
|
[1] Scalable and secure access with SSH: https://engineering.fb.com/security/scalable-and-secure-access-with-ssh/
|
|
[2] Using a CA with SSH: https://www.lorier.net/docs/ssh-ca.html
|
|
[3] Using PIV for SSH through PKCS #11:
|
|
https://developers.yubico.com/PIV/Guides/SSH_with_PIV_and_PKCS11.html
|
|
[4] https://cybsec.network/u/tommy/h/q1g4YC31q45CT4SPK4
|