Monday December 10 2018
Booting MS Windows, FreeBSD, and Linux off of the same drive.
First off, you’re probably thinking “WHY!?” The answer? Because I can. Also, because Virtualization sucks. Plus my laptop only has room for one 2.5” SATA 3 (6gb/s) drive, or a slow SATA 2 (3gb/s) mSATA drive–which I do not plan on using
Why does virtualization suck?
Well let’s say I want to run Windows for a particular piece of software, and it’s resource intensive. CPU wise you’re going to end up with a pretty decent performance hit if you’re running it in a VM. Say nothing of the almost non-existant support for accelerated graphics inside of a virtual machine if I dare try to do something graphics intensive such as play a video game.
Next up is networking, if you’re accustomed to most major desktop virtualization software you might know that it does some NAT trickery to make it “just work”–that is until it doesn’t, or need say IPv6. It’s a disaster and doesn’t solve the problem of having incompatible or poorly written non-portable software in the first place. Something I cover in more depth in future posts, so stay tuned.
Moving on,
What is covered here is setting up a laptop with a single disk to boot Windows, FreeBSD, and Linux via UEFI using the GPT partitioning scheme.
Installing Microsoft Windows
Fresh MS Windows install. I’m sure you’ll be able to handle this one. Just be sure to install it in GPT/UEFI mode. An easy way to do this is to download the ISO from Microsoft, and then write it to a flash drive ( be sure to select UEFI only mode ) via the Rufus tool.
Be sure to set the partition size accordingly and leave plenty of room for the other operating systems. Windows requires about 60GB minimum these days. I just gave it 200gb on my X230’s 1TB ssd.
Once it’s installed you may wish to set the system time to use UTC–especially if you’re installing a lot of Unix-like operating systems that expect that functionality.
Simply write this out to a registry file:
utc.reg
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\TimeZoneInformation]
"RealTimeIsUniversal"=dword:00000001
You should then be able to double click on it, ignore the warning and install it.
Installing FreeBSD
This one is a bit tricky. The quick version is that you’ll boot to the install medium and perform a by hand installation.
The partitioning is the only real gotcha, so-to-speak:
# gpart show ada0
=> 34 1953525101 ada0 GPT (932G)
34 2014 - free - (1.0M)
2048 1021952 1 ms-recovery (499M)
1024000 204800 2 efi (100M)
1228800 32768 3 ms-reserved (16M)
1261568 410118144 4 ms-basic-data (196G)
411379712 2097152 5 freebsd-ufs (1.0G)
413476864 1048576000 6 freebsd-zfs (500G)
Above you can see how my partitions are setup, I was able to create them via:
# gpart add -t freebsd-ufs -s 1G ada0
# gpart add -t freebsd-zfs -s 500G ada0
Then (optionally) encrypt and create your filesystems:
# geli init -b -e AES-XTS -l 256 -s 4096 /dev/ada0p6
# geli attach /dev/ada0p6
# zpool create -O compress=lz4 -O canmount=off -R /mnt zroot ada0p6.eli
# zfs create -o canmount=off zroot/ROOT
# zfs create -o mountpoint=/ zroot/ROOT/default
# zfs create -o mountpoint=/home zroot/home
# for _n in usr var ; do
zfs create -o canmount=off -o mountpoint=/$_n zroot/$_n
done
# zfs create zroot/usr/ports
# zfs create zroot/var/log
# newfs -L boot /dev/ada0p5
Then go ahead and mount the filesystems:
# mkdir /mnt/boot_real && mount /dev/ufs/boot /mnt/boot_real
# tar -C /mnt -xJf /usr/freebsd-dist/base.txz
# tar -C /mnt -xJf /usr/freebsd-dist/kernel.txz
# mv /mnt/boot /mnt/boot_real/ && ln -s boot_real/boot /mnt/boot
# mkdir /mnt/boot/efi && mount -t msdosfs /dev/ada0p2 /mnt/boot/efi
Move Microsoft’s main bootloader out of the way, we’re going to use this to get going with FreeBSD. ( You’ll still be able to select Windows from the boot menu as it writes an EFI bootloader variable )
# mv /mnt/boot/efi/EFI/Boot/bootx64.efi /mnt/boot/efi/EFI/Boot/msft_bootx64.efi
# cp /mnt/boot/boot1.efi /mnt/boot/efi/EFI/Boot/bootx64.efi
Adjust boot/loader.conf for our ZFS root and encrypted root partition:
# ed /mnt/boot/loader.conf
a
aesni_load="YES"
geom_eli_load="YES"
zfs_load="YES"
vfs.root.mountfrom="zfs:zroot/ROOT/default"
geom_eli_passphrase_prompt="YES"
.
w
q
#
Adjust /etc/rc.conf in much the same way: ( #TODO: add other flags here, and
sysctl variables )
# ed /etc/rc.conf
a
hostname="$MY_HOSTNAME"
zfs_enable="YES"
.
w
q
#
Set our system’s timezone
# ln -s /usr/share/zoneinfo/America/Detroit /etc/localtime
Make sure our filesystems are in the etc/fstab ( Note that ZFS is not
managed through the fstab )
# ed etc/fstab
a
/dev/ufs/boot /boot_real ufs rw 0 0
# Read only, if we want to reference it.
# Otherwise we can do `mount -u -o rw /boot/efi` to write to it
/dev/ada0p2 /boot/efi msdosfs ro 0 0
.
w
q
Finally,
# sync && reboot
And that should take care of most of the install of FreeBSD. There are still a lot of other little pieces that need to be put in place if you want a usable desktop experience. Stay tuned for that post as well.
Installing Linux
These days I find myself using Gentoo most of the time. It’s stable, easy
to specify the build options I want, it’s not subject to the bloat of
systemd and binary package are trivial.
For my x230 I purposely wanted to do something different I’ve used Void
Linux in the past for a few things. It’s a somewhat unique distribution that
uses runit as the default init scheme, libressl as the TLS implementation,
and optionally musl as the system’s libc. I’m currently using the musl
libc variant although the setup should be similar if you go with glibc
First boot to some sort of Linux distribution, this actually matters very little as we’ll be installing via a chroot.
Partition the disk
# gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.4
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 1953525168 sectors, 931.5 GiB
Model: CT1000MX500SSD1
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): C0C481E6-25C4-4837-8610-3F2D5DCAE3B6
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1953525134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2014 sectors (1007.0 KiB)
Number Start (sector) End (sector) Size Code Name
1 2048 1023999 499.0 MiB 2700 Basic data partition
2 1024000 1228799 100.0 MiB EF00 EFI system partition
3 1228800 1261567 16.0 MiB 0C01 Microsoft reserved ...
4 1261568 411379711 195.6 GiB 0700 Basic data partition
5 411379712 413476863 1024.0 MiB A503
6 413476864 1462052863 500.0 GiB A503
7 1462052864 1464150015 1024.0 MiB 8300
8 1464150016 1883580415 200.0 GiB 8300
Basically, create a 1GB boot partition and a 200GB partition for the OS much like you did with FreeBSD a moment ago.
Setup the filesystems, encryption, and LVM.
There will be free space in the volume group with the configuration below, this makes snapshotting easy.
# mkfs.ext4 /dev/sda7
# cryptsetup luksFormat /dev/sda8
# cryptsetup luksOpen /dev/sda8 root
# pvcreate /dev/mapper/root
# vgcreate vg /dev/mapper/root
# lvcreate vg --name root --size 10G
# lvcreate vg --name home --size 100G
# mkfs.xfs /dev/vg/root
# mkfs.xfs /dev/vg/home
Then go ahead and mount the filesystems:
# mount /dev/vg/root /mnt
# mkdir /mnt/{boot,home}
# mount /dev/vg/home /mnt/home
# mount /dev/sda7 /mnt/boot
# mkdir /mnt/boot/efi
# mount /dev/sda2 /mnt/boot/efi
Download and extract the minimal root filesystem
Be sure to check the website for a mirror
near you and change the URL accordingly. You might need to browse the
live/current/ path to figure out the current ROOTFS tarball
# cd /mnt
# wget http://mirror.clarkson.edu/voidlinux/live/current/void-x86_64-musl-ROOTFS-20181111.tar.xz
# tar xJf void-*ROOTFS*
Chroot into the new system and setup the basics
# cd /mnt
# mount -t proc none proc
# mount -o bind /sys sys
# mount -o bind /dev dev
# chroot /mnt /bin/bash
bash-4.4# export PS1="(CHROOT) # "
Setup nameservers
(CHROOT) # echo nameserver 8.8.8.8 > /etc/resolv.conf
Setup the package repository to point at your local mirror
(CHROOT) # echo repository=http://mirror.clarkson.edu/voidlinux/current/musl \
> /etc/xbps.d/00-repository-main.conf
Install the base system, lvm2, and a few other useful packages
(CHROOT) # xbps-install -Syu \
base-system lvm2 curl wget ed grub gptfdisk lzop cryptsetup \
lzip openvpn nmap pv rrdtool rsync sipcalc whois xz wireless_tools \
net-tools wpa_supplicant grub-x86_64-efi
Configure bootloader, initrd and /etc/fstab
Kind of a hack to setup the fstab, but it works:
(CHROOT) # cat /proc/mounts | grep xfs >> /etc/fstab
The Void Linux kernel by default does not build in the necessary module to
handle booting off of a logical volume with associated snapshots. To make
matters worse, it does not load it automatically even if you put it in the
initial ramdisk. Thankfully dracut has an option to force load kernel modules
Simply:
# echo 'force_drivers="dm_snapshot"' >> /etc/dracut.conf
Then go ahead and rebuild the initrd:
(CHROOT) # dracut -f --kver $(ls -t /lib/modules | sed 1q)
Now to check the generated initrd and make sure that it has LVM and Cryptsetup
(CHROOT) # lsinitrd /boot/initrd* | grep lvm >/dev/null || echo "No LVM support"
(CHROOT) # lsinitrd /boot/initrd* | grep cryptsetup >/dev/null || echo "No crypt support"
(CHROOT) #
Instlalling grub
It’s pretty straightforward:
(CHROOT) # grub-install --target=x86_64-efi --efi-directory=/boot/efi \
--bootloader-id void
Configuring grub
We’re going to need to tell the grub configuration generator how to get to the
filesystem. Adjust /etc/default/grub like so:
GRUB_CMDLINE_LINUX_DEFAULT="rd.luks.uuid=$LUKS_UUID rd.luks.allow-discards rd.luks.crypttab=0"
where $LUKS_UUID is actually your UUID, you can get this via:
(CHROOT) # blkid -o export /dev/sda8 | grep '^UUID'
You can leave or remove the extra arguments that are already there. ( Not
listed above) You can change the loglevel option if you want to have a little
bit more verbose boot.
You will need to remove the rd.luks.allow-discards if you’re not booting to a
solid state drive that supports trim. There are also some security concerns
with enabling it centering around the fact that it will espose what data
is and is not encrypted.
Finally, create the grub configuration file
(CHROOT) # grub-mkconfig -o /boot/grub/grub.cfg
Installing rEFInd
Now that we have more operating systems than we eaisly can select from the boot menu it’s about time we fix that problem by using a UEFI boot manager.
Why not just set some EFI variables and be done with it? Because UEFI is buggy. And on many systems it’s not that simple–or even respected.
I downloaded rEFInd from here I used the “A binary zip file”
I didn’t use the install scripts, instead I just copied the binary over to the EFI directory like so:
$ cd ~/Downloads/
$ bsdtar -xf refind-bin-0.11.4
$ cd refind-bin-0.11.4
$ sudo cp refind/refind_x64.efi /boot/efi/EFI/Boot/bootx64.efi
I then proceeded to create a configuration file like so:
timeout 20
menuentry Void {
loader /EFI/void/grubx64.efi
icon /EFI/refind/icons/os_void.png
}
menuentry FreeBSD {
loader /EFI/freebsd/bootx64.efi
icon /EFI/refind/icons/os_freebsd.png
}
menuentry "Windows 10" {
loader \EFI\Microsoft\Boot\bootmgfw.efi
icon \EFI\refind\icons\os_win.png
}
If you would like the icons and themes as mentioned above, then you can copy over the `refind/icons/’ folder:
$ sudo cp -R refind/icons /boot/efi/EFI/Boot/
If you’re on a system that supports EFI variables you can always add a special entry for it as well:
$ sudo -i
# cd /boot/efi/EFI
# mkdir refind
# cp -R Boot/* refind/
# efibootmgr --create --disk /dev/sda --part 2 --loader \
/EFI/refind/bootx64.efi --label "rEFInd Boot Manager"
This will obviously not carry over to other systems you install the
drive to so I highly recommend that you use the default EFI/Boot
directory.
Final thoughts
This is obviously not a common configuration. Very few people have any desire, let alone a need to run multiple operating systems on the same machine, let alone three of them.
I run Linux and FreeBSD production servers for some clients.
Although this configuration is not strictly necessary–I could always
login to another host via ssh. It does offer a little bit more experience
with the operating system in question when you run it with a desktop
environment–especially on a laptop.
If I didn’t know how to setup this kind of thing I would be much worse at my job. When shit hits the fan, everything is broken–it’s 04:00 and you’re trying to restore from a backup after you repeatedly told the client to upgrade the hardware before it failed–you don’t have time to place blame or complain. You just have to figure out the damn bootloader and get the system running. Besides, clients are a lot more receptive to the expensive service call and a plan to prevent the problem in the future if you resolve it quickly and without much fuss.
So go on, mess with your systems, multi boot. You’ll learn something and it might come in handy if you work on these kinds of things.
If anyone is interested in how my desktop is setup, feel free to send me a
message and I’ll do a writeup on i3, the applications I use, etc. I don’t
think it’s anything special however and as such do not plan on writing
about it at this time.