QEMU guest with networking and virtfs

edit Introduction

This article describes how to run a QEMU guest with full networking and VirtFS support. VirtFS (Plan 9 folder sharing over Virtio - I/O virtualization framework) allows you to mount folders from the host system on the guest system, this is significantly faster than folder sharing over the network e.g. NFS.

edit Enable kernel modules

The NixOS install process will typically enable the relevant kvm modules for your system (kvm-amd or kvm-intel) in hardware-configuration.nix (). You can also add "tun" and "virtio" if you wish to use the networking setup described below.

boot.kernelModules = [ "kvm-amd" "tun" "virtio" ];

You can check for the existence of /dev/kvm after rebooting. You may also need to enable hardware virtualization in your BIOS or similar.

edit Prepare a guest image

qemu-img create -f qcow2 /mnt/OSImages/mint.qcow2 5G
qemu-kvm -m 1024 -drive file=/mnt/OSImages/mint.qcow2  -cdrom /mnt/OSImages/mint-livecd.iso -boot d

The default amount of memory available to the guest is 128MB so you will probably want to specify something more than that with the "m" flag. By default the guest will have access to the network so you can install the OS at this stage if you like.

Alternatively you can download a QEMU image with an OS already installed e.g. Debian: http://people.debian.org/~aurel32/qemu/

qemu-kvm -m 1024 -drive file=/mnt/OSImages/debian_squeeze_amd64_standard.qcow2

edit Prepare host network

There are a number of options to enable access to the guest OS from the host, using VDE (Virtual Distributed Ethernet) is probably the easiest and most versatile.

As root add a "kvm" group so that you only have to do the networking setup as root:

# groupadd kvm
# usermod -G kvm username

To change the groups of the user without logging out and back in again (as username):

$ su username -

Start the vde daemon (as root):

# vde_switch -tap tap0 -mod 660 -group kvm -daemon
# ip addr add 10.0.2.1/24 dev tap0
# ip link set dev tap0 up

You should now see the tap0 interface with `ip addr`/`ifconfig` and it should have been assigned the IP address specified. It is important that you don't use the same subnet as your host network interface (eth0/wlan0) is using.

Forward traffic:

# sysctl -w net.ipv4.ip_forward=1
# iptables -t nat -A POSTROUTING -s 10.0.2.0/24 -j MASQUERADE

You can configure NixOS to do these steps on boot by placing the commands in the networking.localCommands section:

networking = {
  localCommands =
    ''
      ${pkgs.vde2}/bin/vde_switch -tap tap0 -mod 660 -group kvm -daemon
      ip addr add 10.0.2.1/24 dev tap0
      ip link set dev tap0 up
      ${pkgs.procps}/sbin/sysctl -w net.ipv4.ip_forward=1
      ${pkgs.iptables}/sbin/iptables -t nat -A POSTROUTING -s 10.0.2.0/24 -j MASQUERADE
    '';
};

edit Guest network config

You can now start the VM as a normal user by adding the following options `-net nic,model=virtio -net vde`:

qemu-kvm -m 1024 -net nic,model=virtio -net vde -drive file=/mnt/OSImages/debian_squeeze_amd64_standard.qcow2

Within the VM you can setup the network:

root@guest# ip addr add 10.0.2.100/24 dev eth0
root@guest# ip link set dev eth0 up
root@guest# ip route add default via 10.0.2.1
root@guest# echo 'nameserver 8.8.8.8' >> /etc/resolv.conf

You can now access this VM on 10.0.2.100

edit VirtFS

VirtFS enables folders to be shared between the host and guest very efficiently. Some extra qemu options are required:

qemu-kvm -m 1024 -net nic,model=virtio -net vde -drive file=/mnt/OSImages/debian_squeeze_amd64_standard.qcow2,if=virtio -virtfs local,path=/home/username/shared/,security_model=passthrough,mount_tag=host_share

This will share /home/username/shared on the host with the guest, passing through permissions. See the QEMU wiki for more details.

To mount this on the guest:

mount -t 9p -o trans=virtio host_share /mnt/host_share -oversion=9p2000.L

NOTE:

There appears to be a problem with using VirtFS where fsync fails

e.g. when running `git clone` 
fatal: fsync error on '/some/path/.git/objects/pack/tmp_idx_h8upsf': Invalid argument

edit Tips

edit Shortcuts

Reset the size of the QEMU window: ctrl-alt-u

edit Snapshots

You can save the current state of the VM (including the CPU state) and then load it later.

Enter the QEMU monitor with ctrl-alt-2
savevm snap1
Return to the QEMU VM with ctrl-alt-1
Do something noticable: `ls /`
Return to the QEMU monitor and `savevm snap2`
Load snap1 `loadvm snap1`

You can also load the snapshot directly from the command line:

qemu-kvm -m 1024 -net nic,model=virtio -net vde -drive file=/mnt/OSImages/debian_squeeze_amd64_standard.qcow2,if=virtio -virtfs local,path=/home/username/shared/,security_model=passthrough,mount_tag=host_share -loadvm snap1

edit References

Building a Virtual Army (Dave Reisner's post provided almost all of the details here. He also has additional explanations and useful scripts.)
VirtFS

Contents