AoE is one of the protocols supported by Linux to access storage via network. It uses plain Ethernet for communication and includes a discovery mechanism to find all available targets. I use it to provide disk space to VMs running on different machines.
Next step is to boot via AoE. It is no real problem to use AoE in running systems. However with some help it is even possible to actually boot disk-less machines via AoE. The PXE implementation iPXE provides the AoE support to actually boot from. I will describe the necessary parts.
1. Setup vblade
The AoE target used it vblade.
vblade needs access to raw sockets. As I prefer to not have run anything as root if it is not necessary, I use filesystem capabilities to allow it access to the network.
setcap cap_net_raw+ep /usr/sbin/vblade
vblade gets the mac address of the initiator, the shelf and slot number, the network device and the block device.
/usr/sbin/vblade -m $mac 0 0 eth0 $dev
2. Setup a tftp server and iPXE
apt-get install atftpd ipxe
ln -s /usr/lib/ipxe/undionly.kpxe /var/lib/tftpboot
3. Setup ISC dhcpd
The dhcp server needs to be configured. It needs to hand out two distinct parameter sets. The first is used to chain-load iPXE into the normal PXE stack. The second is for iPXE and sets the root path to the AoE device. They are selected on the iPXE marker in the request.
if exists user-class and option user-class = "iPXE" {
filename "";
option root-path "aoe:e0.0";
} else {
filename "undionly.kpxe";
}
4. Support AoE via initramfs-tools
The initramfs needs to initialize AoE support. It needs to enable the network device used for communication with the AoE server and wait until it is up. After that it needs to load the aoe module and run aoe-discover. We should have all devices now.
The root device can now be used like any other normal device. After the AoE device is initialized, it can be found via UUID and all the other ways. So no further modifications are necessary over the usage of local disks. The initramfs finds the device as usual and boots from it.
The initramfs support is still a prototyp, but seems to work. For initramfs-tools it needs a hook to include all necessary stuff in the initramfs and a script to actually do the work. Both are shown here.
/etc/initramfs-tools/hooks/aoe:
#!/bin/sh
case $1 in
prereqs)
echo "udev"
exit 0
;;
esac
. /usr/share/initramfs-tools/hook-functions
copy_exec /sbin/aoe-discover
manual_add_modules aoe
/etc/initramfs-tools/scripts/local-top/aoe:
#!/bin/sh
case $1 in
prereqs)
echo "udev"
exit 0
;;
esac
ifconfig eth0 up
sleep 10
modprobe aoe
aoe-discover
udevadm settle --timeout=30
Problems
Not all parts of this works 100%. Some parts works not for all hardware.
via Bastian Blank