NixOS

Learn

Integration testing using virtual machines (VMs)

One of the most powerful features in the Nix ecosystem is the ability to provide a set of declarative NixOS configurations and use a simple Python interface to interact with them using QEMU as the backend.

Those tests are widely used to ensure that NixOS works as intended, so in general they are called NixOS tests. They can be written and launched outside of NixOS, on any Linux machine (with MacOS support coming soon).

Integration tests are reproducible due to the design properties of Nix, making them a valuable part of a Continuous Integration (CI) pipeline.

Testing a typical web application backed by PostgreSQL

This tutorial follows PostgREST tutorial, a generic RESTful API for PostgreSQL.

If you skim over the official tutorial, you’ll notice there’s quite a bit of setup in order to test if all the steps work.

We are going to set up:

  • A VM named server running postgreSQL and postgREST.

  • A VM named client running HTTP client queries using curl.

  • A testScript orchestrating testing logic between client and server.

Writing the test

Create postgrest.nix:

  1let
  2  # Pin nixpkgs, see pinning tutorial for more details
  3  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/0f8f64b54ed07966b83db2f20c888d5e035012ef.tar.gz";
  4  pkgs = import nixpkgs {};
  5
  6  # Single source of truth for all tutorial constants
  7  database      = "postgres";
  8  schema        = "api";
  9  table         = "todos";
 10  username      = "authenticator";
 11  password      = "mysecretpassword";
 12  webRole       = "web_anon";
 13  postgrestPort = 3000;
 14
 15  # NixOS module shared between server and client
 16  sharedModule = {
 17    # Since it's common for CI not to have $DISPLAY available, we have to explicitly tell the tests "please don't expect any screen available"
 18    virtualisation.graphics = false;
 19  };
 20
 21in pkgs.nixosTest ({
 22  # NixOS tests are run inside a virtual machine, and here we specify system of the machine.
 23  system = "x86_64-linux";
 24
 25  nodes = {
 26    server = { config, pkgs, ... }: {
 27      imports = [ sharedModule ];
 28
 29      networking.firewall.allowedTCPPorts = [ postgrestPort ];
 30
 31      services.postgresql = {
 32        enable = true;
 33
 34        initialScript = pkgs.writeText "initialScript.sql" ''
 35          create schema ${schema};
 36
 37          create table ${schema}.${table} (
 38              id serial primary key,
 39              done boolean not null default false,
 40              task text not null,
 41              due timestamptz
 42          );
 43
 44          insert into ${schema}.${table} (task) values ('finish tutorial 0'), ('pat self on back');
 45
 46          create role ${webRole} nologin;
 47          grant usage on schema ${schema} to ${webRole};
 48          grant select on ${schema}.${table} to ${webRole};
 49
 50          create role ${username} inherit login password '${password}';
 51          grant ${webRole} to ${username};
 52        '';
 53      };
 54
 55      users = {
 56        mutableUsers = false;
 57        users = {
 58          # For ease of debugging the VM as the `root` user
 59          root.password = "";
 60
 61          # Create a system user that matches the database user so that we
 62          # can use peer authentication.  The tutorial defines a password,
 63          # but it's not necessary.
 64          "${username}".isSystemUser = true;
 65        };
 66      };
 67
 68      systemd.services.postgrest = {
 69        wantedBy = [ "multi-user.target" ];
 70        after = [ "postgresql.service" ];
 71        script =
 72          let
 73            configuration = pkgs.writeText "tutorial.conf" ''
 74                db-uri = "postgres://${username}:${password}@localhost:${toString config.services.postgresql.port}/${database}"
 75                db-schema = "${schema}"
 76                db-anon-role = "${username}"
 77            '';
 78          in "${pkgs.haskellPackages.postgrest}/bin/postgrest ${configuration}";
 79        serviceConfig.User = username;
 80      };
 81    };
 82
 83    client = {
 84      imports = [ sharedModule ];
 85    };
 86  };
 87
 88  # Disable linting for simpler debugging of the testScript
 89  skipLint = true;
 90
 91  testScript = ''
 92    import json
 93    import sys
 94
 95    start_all()
 96
 97    server.wait_for_open_port(${toString postgrestPort})
 98
 99    expected = [
100        {"id": 1, "done": False, "task": "finish tutorial 0", "due": None},
101        {"id": 2, "done": False, "task": "pat self on back", "due": None},
102    ]
103
104    actual = json.loads(
105        client.succeed(
106            "${pkgs.curl}/bin/curl http://server:${toString postgrestPort}/${table}"
107        )
108    )
109
110    assert expected == actual, "table query returns expected content"
111  '';
112})

A few notes:

  • Between the machines defined inside the nodes attribute, hostnames are resolved based on their attribute names. In this case we have client and server.

  • The testing framework exposes a wide set of operations used inside the testScript. A full set of testing operations is part of VM testing operations API Reference.

Running tests

To set up all machines and execute the test script:

$ nix-build postgrest.nix

You’ll notice an error message if something goes wrong.

In case the tests succeed, you should see at the end:

...
test script finished in 10.96s
cleaning up
killing client (pid 10)
killing server (pid 22)
(0.00 seconds)
/nix/store/bx7z3imvxxpwkkza10vb23czhw7873w2-vm-test-run-unnamed

Developing and debugging tests

When developing tests or when something breaks, it’s useful to interactively fiddle with the script or access a terminal for a machine.

To interactively start a Python session with a testing framework:

$ $(nix-build -A driverInteractive postgrest.nix)/bin/nixos-test-driver
...
starting VDE switch for network 1
>>>

You can run any of the testing operations. The testScript attribute from our postgrest.nix definition can be executed with test_script() function.

To start all machines and enter a telnet terminal to a specific machine:

>>> start_all()
...
>>> server.shell_interact()
server: Terminal is ready (there is no prompt):

uname -a
Linux server 5.10.37 #1-NixOS SMP Fri May 14 07:50:46 UTC 2021 x86_64 GNU/Linux

Next steps

View original article on nix.dev