I am looking for some recommendations on how to secure the data of my physical servers (against physical theft), that I am about to set up. I am new to selfhosting but have a few years of experience running Linux on a desktop.
My usecase is a simple debian(?) server at home with Paperless ngx and Tailscale for when I am away from home.
The question is how to encrypt the data while still being able to keep the server updated.
Coming from Desktop my first thought was to simply enable FDE on install. But that would mean supplying the password everytime the server needs to reboot for an update. Could someone provide some insights on how often updates to debian require a reboot?
My second thought was to use an encrypted data partition. That way the server could reboot and I could use wireguard to ssh in and open the partition even when I am away from home for a longer time.
I am open to other ideas!
You must log in or register to comment.
You can use FDE and setup a minimal ssh server like dropbear to run at startup. This way, you can supply the password via a keyboard connected to the machine OR via ssh. This gives you a similar workflow to the data partition you mentioned, but encrypts the entire system.
I am looking for some recommendations on how to secure the data of my physical servers (against physical theft)…
Lock the case shut. Run a security wire (like for laptops) to the case and mount the other end to something big / immovable.
Truly paranoid: 24/7 video surveillance.
Super truly paranoid: same as above but in a locked room
I use LUKS on my systems. I use mandos and wireguard in intramfs to connect to a mandos server to unlock LUKS during boot.
What’s mandos? I don’t find anything useful when searching for it.
https://www.recompile.se/mandos
I use this to get wireguard in intramfs. I just skip the dropbear related stuff. https://github.com/r-pufky/wireguard-initramfs
Other options are LUKS with Tang and Clevis, or LUKS with SSH and Dropbear.
Sorry, I have no details.
Edit: Tang/Clevis are local software and a network server that provide keys. If stolen, won’t boot.
SSH and Dropbear make it so you can login to provide keys.
If you want to encrypt only the data partition you can use an approach like https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/#encrypted-zfs to ulock it at boot.
TL;DR: store half of the decryption key on the computer and another half online and write a script that at boot fetches the second half and decrypt the drive. There is a timewindow where a thief could decrypt your data before you remove the key if they connect your computer to the network, but depending on your thread model can be acceptable. you can also decrypt the root portion with a similar approach but you need to store the script in the initramfs and it is not trivial.
Another option I’ve seen suggested is storing the decryption key on a USB pendrive and connect it with a long extension cord to the server. The assumption is that a thief would unplug all the cables before stealing your server.
Luks FDE, and install dropbear-initramfs, configure ssh authorized_keys and rebuild initramfs. Then you can access initramfs via ssh to type luks password.
A more detailed guide for dropbear: https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/
If I remember correctly, the only outdated bit of information is that the IP configuration doesn’t happen anymore in the initramfs configuration but you must pass a parameter at the kernel by editing /etc/default/grub and passing
GRUB_CMDLINE_LINUX_DEFAULT="ip=192.168.x.x:::::"where 192.168.x.x is the IP address that you want at boot
I’m using https://github.com/dracut-crypt-ssh/dracut-crypt-ssh on some of my servers. The initrd opens an ssh port where you can login and enter the passphrase. Setting it up is non-trivial, but it works well. Haven’t tried it on Debian but there should be something similar.
I’d go with the Full Disk Encryption. You can be sure everything is encrypted that way. Any additional complexity adds ways to mess up and compromise security. Entering the password is a bit cumbersome. But that’s part of the deal. I just carry my computer keyboard to my NAS and enter the password each time I need to reboot. Which doesn’t happen that often. There also used to be some tutorial somewhere on how to put a Dropbear SSH server into the initrd so you can enter the password over network.
Unless the crook happens to be extremely nerdy or its law enforcement, already being a Linux formatted partition feels it should be enough for a rando breaking in and stealing a computer.
That being said, something like a PiKVM connected to your server (and Tailscale) could let you enable both UEFI/boot password and propt for LUKS decryption upon boot.
I wouldn’t rely on the thief not knowing how to read linux partitions. That very well may be the case, but the person they sell your hardware to will know better, considering they are in the market of purchasing used server hardware.
I self host and my threat model is the thief selling my server to someone who knows what to do with it, but not knowing how to extract encryption keys from the memory of a running server before unpluging it. That being said I haven’t figured out encryption yet so watching this thread.
yea they’ll plug in the drive and windows will popup “this drive needs to be formatted” and the rest is history
Since I already use ZFS for my data storage, I just created a private dataset for sensitive data. I also have my services split based on if it’s sensitive or not, so the non sensitive stuff comes up automatically and the sensitive stuff waits for me to log in and unlock the dataset.
Not using ZFS but a similar approach: All my data (paperless, and other docker container data) is encrypted with LUKS on a separate disk. The OS is running unencrypted on the SD card (using a Raspberry Pi). This way I can swap out the system and relink the docker container data if needed. Yes, I do need to unlock after a reboot, but since the system is fully up, that’s done easily via ssh.
Still looking into ways to unlock it automatically on certain criteria…
On Dell server hardware with the right cards/licensing, you can remove the need for physical access to the server to input an FDE password by leaning on iDRAC. This provides access to the console remotely during the boot process (and thereafter).
Alternatives exist that supposedly do the same thing, but I’ve never had to try them. Airconsole, pikvm, blikvm etc.
You can keep this interface unexposed by using wireguard to dial in when you’re away, as per your original thinking. Just make sure the endpoint isn’t on the server you’re rebooting…
Full disk encryption with a LVM inside a LUKS arrition is pretty easy to setup, arch wiki is very helpful for that ! You can even encrypt most of the bootloader, so the drive is 100% useless if stolen.
Good question.
Debian doesn’t often require a reboot, but the longer you go, and if you need kernel modules (nvidia is the worst at this) you might need to reboot to keep everything in sync.
My suggestion: raspberry pi, like 1st edition, keep the key very secure, give it a usb serial console. When the server reboots, enter the password that way. It’s your emergency console.
This has been on my mind, I have yet to do it but the implementation seems trivial.
You can use typical luks full disk encryption with a password. Luks actually has five password slots. Passwords do not have to be actual text, they can be a file or even part of a file.
So my idea is, buy some really cheap, low profile USB flash drives and store some seemingly innocuous data like cat pictures or public domain books, IDK and it doesn’t matter what the actual data is. Use full disk encryption and set a regular password, then add a second password that is a file or part of a file that lives on the flash drives, and have it set up to look for that file on boot as an option for unlocking.
Now the disc is fully encrypted but will boot/reboot without interruption as long as the flash drive is installed. You can remove the flash drive when you’re feeling paranoid, or even better only install it when you are going to be away for a while. If you leave with the machine having the flash drive but are feeling worried, you can remote into the machine and edit / delete the file or just clear the key slot from Luks.
That’s what’s been on my mind, anyway. I think the typical suggestion/solution is to just use drop bear and remotely unlock using that, or don’t use full disk encryption and selectively encrypt your data instead (partitions or userspace encryption).
I’m not going to proofread this so I hope it makes sense
I would do FDE yeah. My current laptop setup is with systemd-boot and a special initramfs that allows me to unlock it with a yubikey, with fallback to password. Fair warning, this exact configuration is not particularly easy to setup.
There are also modules which enable early network connectivity along with a SSH server, meaning you login and unlock it remotely. I have not tried this.
Debian does not frequently require rebooting under normal circumstances. Kernel updates are not that frequent, and you can usually put it off for a bit if you don’t want to deal with it.
