KVM the hard way
The goal of this article is to document the process of setting up a basic KVM instance the hard way. That is, without using GUI tools, magic scripts, VNC’d installers, etc.
Pretty much all of the KVM guides I’ve seen assume you’re sitting at the physical box you’re deploying the VMs on and have access to the local display (i.e. they use GUI tools). Or they assume you just want a magic script to do it all for you. Or they assume you want to sit there and click through the debian-installer over a VNC session.
None of those options sound particularly fun or interesting. Below is a step-by-step guide to getting a KVM instance running the hard way. Most of the information can actually be found by pulling apart Hardy’s ubuntu-vm-builder script. I’m using Debian Lenny as Etch doesn’t have some of the necessary tools, for example kpartx for mapping partitions on a loopback device. Also we assume a 64-bit host and guest, though it should be fairly obvious how to use other numbers of bits. I decided to document the process mainly as a reference, but also as an education in the underlying process.
Let’s begin. First, we need a disk image to work with. Create a 5GB raw image using qemu-img:
$ qemu-img create -f raw lenny-base.raw 5G
Now, create a loopback device for it:
# losetup /dev/loop0 lenny-base.raw
Partition the loopback device using fdisk. I use a single primary partition in this article.
# fdisk /dev/loop0
Now we need to create device-mapper entries for each of the partitions on the loopback device:
# kpartx -a /dev/loop0
If you made a single partition on /dev/loop0, there will now be a device-mapper block device at /dev/mapper/loop0p1, which you can go ahead and make a filesystem on and bootstrap to whatever flavour of Debian/Ubuntu you’d like. You’ll also want to remember the UUID of the root filesystem for later. For example,
# mke2fs -j /dev/mapper/loop0p1
# vol_id --uuid /dev/mapper/loop0p1 > target.uuid
# mkdir /mnt/target
# mount /dev/mapper/loop0p1 /mnt/target
# debootstrap lenny /mnt/target http://ftp.nz.debian.org/debian
At this point, the bootstrapped filesystem will need some manual setting up. In particular you’ll need to
- Set up /etc/hostname
- Set up /etc/hosts
- Set up /etc/fstab (you’ll want to use the UUID you saved before)
- Enable serial getty in /etc/inittab (important, we’ll use this for initial login)
Copy the following into the target’s /etc/kernel-img.conf:
do_symlinks = yes
relative_links = yes
do_bootfloppy = no
do_initrd = yes
link_in_boot = no
postinst_hook = update-grub
postrm_hook = update-grub
do_bootloader = no
Now we need to install a kernel and set up the boot-loader.
# chroot /mnt/target
target # apt-get install linux-image-amd64 grub
target # mkdir -p /boot/grub
target # cp /usr/lib/grub/x86_64-pc/* /boot/grub
Exit from the target’s chroot. Now we’re back in the host, we need to install grub into the MBR of the disk image. You’ll need that UUID for the root filesystem from before. This step requires the host’s /dev to be bind-mounted into the target’s filesystem
# mount --bind /dev /mnt/target/dev
# echo "(hd0) lenny-base.raw" >> device.map
# grub --device-map=device.map
grub>root (hd0,0)
grub>setup (hd0)
grub>quit
# echo "(hd0) UUID=UUID of target root fs goes here" >> /mnt/target/boot/grub/device.map
# chroot /mnt/target
target # update-grub
update-grub will have written a basic menu.lst, but because it’s using the host’s /dev it will be pointing to the wrong place. Edit the target’s /boot/grub/menu.lst to use the UUID of the filesystem and not use the loop0 device. So, open up the target’s /boot/grub/menu.lst, search for the line:
# kopt_2_6 root=/dev/mapper/loop0p1 ro
and replace /dev/mapper/loop0p1 with
UUID=the uuid of the root filesystem
so it will look something like (make sure the # is still at the start of the line):
# kopt_2_6 root=UUID=81af6388-cca5-4bf2-99dc-a47c81c00445 ro
Also, replace the line groot=(loop0p1) with groot=(hd0,0). Again, it’s “commented out”. Now we need to update grub again.
# chroot /mnt/target
target # update-grub
target # exit
Almost done… At this point check that you’ve enabled the serial getty in the target’s /etc/inittab file as we’re going to boot the VM and use the serial console to do the initial login. Of course, you could install SSH in the chroot, set up networking and forget about the serial console, but it’s interesting none the less. The other alternative is to use VNC to connect to the console, but that’s the easy way out, though you will get boot messages, so if something goes wrong you’ll get to see why.
Unmount everything:
# umount /mnt/target/dev
# umount /mnt/target
# kpartx -d /dev/loop0
# losetup -d /dev/loop0
Now boot your shiny new VM:
# kvm -nographic -serial pty -drive file=lenny-base.raw,if=virtio,index=0,boot=on -daemonize
If all goes well you’ll see something like:
char device redirected to /dev/pts/10
Use minicom to connect to that pseudo-terminal and login to your new VM. Done! Sure it would have been easier if you’d just used a script or a graphical tool, but we got there in the end.
At this point the VM isn’t overly useful without networking, but there’s plenty of documentation in the qemu man pages about the options to enable a virtual NIC. There’s also options for changing the amount of RAM the guest is allocated, the number of CPUs, virtual disks, etc. Go RTFM.
Another option from here is to create a libvirt XML description of your VM and use virsh to manage it. This makes networking and management a bit easier, but isn’t necessary.
Enjoy!
Add comment August 28th, 2008