NixOS VM tests

The NixOS source tree contains tests that use automatically instantiated virtual machines to test various features of the system. These tests define virtual networks of virtual NixOS machines and test scripts that use them. This approach is described in an ISSRE-2010 paper. While it uses NixOS in the VMs, it is not limited to testing NixOS: if you have a (distributed) application or system-level package that you wish to test automatically, you can use the NixOS VM testing framework. Note that NixOS is not required on the host system - any Linux system (such as Ubuntu) with KVM support should work.

Running a test

If you don't have NixOS as your host OS, you should first install the Nix package manager and obtain the Nixpkgs and NixOS source trees:

$ svn co https://svn.nixos.org/repos/nix/nixpkgs/trunk nixpkgs
$ svn co https://svn.nixos.org/repos/nix/nixos/trunk nixos

To speed up builds by using pre-built binaries, you should do:

$ nix-pull http://nixos.org/releases/nixpkgs/channels/nixpkgs-unstable/MANIFEST

Now you can run a test as follows:

$ nix-build nixos/tests/ -A nfs.test

This performs an automated test of the NFS client and server functionality in the Linux kernel, including file locking semantics (e.g., whether locks are maintained across server crashes). It will first build or download all the dependencies of the test (e.g., all packages needed to run a NixOS VM). The test is defined in nixos/tests/nfs.nix. If the test succeeds, nix-build will place a symlink ./result in the current directory pointing at the location in the Nix store of the test results (e.g., screenshots, test reports, and so on). In particular, a pretty-printed log of the test is written to log.html, which you can view by doing something like

$ firefox ./result/log.html

Interactive testing

Screenshot of running the NFS test environment interactively.

It is also possible to run the test environment interactively, allowing you to experiment with the VMs. For example:

$ nix-build nixos/tests/ -A nfs.driver
$ ./result/bin/nixos-run-vms

The script nixos-run-vms starts the three virtual machines defined in the NFS test using QEMU/KVM. The root filesystem of the VMs is created on the fly and kept across VM restarts in ./hostname.qcow2.

Finally, you can run the test itself interactively. This is particularly useful when developing or debugging a test.

$ nix-build nixos/tests/ -A nfs.driver
$ ./result/bin/nixos-test-driver
starting VDE switch for network 1
>

You can now type Perl statements to execute commands to start or manipulate the VMs:

> startAll;
(the VMs start booting)
> $server->waitForJob("nfs-kernel-nfsd");
> $client1->succeed("flock -x /data/lock -c 'sleep 100000' &");
> $client2->fail("flock -n -s /data/lock true");
> $client1->shutdown;
(this releases client1's lock)
> $client2->succeed("flock -n -s /data/lock true");

The function testScript executes the entire test script and drops you back into the test driver command line upon its completion. This allows you to inspect the state of the VMs after the test (e.g. to debug the test script).

Writing tests

TODO