Wombat’s Book of Nix

If you’ve opened this PDF, you already have your own motivation for learning Nix. Here’s how it helps me. As a researcher, I tend to work on a series of short-term projects, mostly demos and prototypes. For each one, I typically develop some software using a compiler, often with some open source libraries. Often I use other tools to analyse data or generate documentation, for example.

Problems would arise when handing off the project to colleagues; they would report errors when trying to build or run the project. Belatedly I would realise that my code relies on a library that they need to install. Or perhaps they had installed the library, but the version they’re using is incompatible.

Using containers helped with the problem. However, I didn’t want to develop in a container. I did all my development in my nice, familiar, environment with my custom aliases and shell prompt. and then I containerised the software. This added step was annoying for me, and if my colleague wanted to do some additional development, they would probably extract all of the source code from the container first anyway. Containers are great, but this isn’t the ideal use case for them.

Nix allows me to work in my custom environment, but forces me to specify any dependencies. It automatically tracks the version of each dependency so that it can replicate the environment wherever and whenever it’s needed.

Flakes are labeled as an experimental feature, so it might seem safer to avoid them. However, they have been in use for years, and there is widespread adoption, so the aren’t going away any time soon. Flakes are easier to understand, and offer more features than the traditional Nix approach. After weighing the pros and cons, I feel it’s better to learn and use flakes; and this seems to be the general consensus.

To follow along with the examples in this book, you will need access to a computer or (virtual machine) with Nix (or NixOS) installed and flakes enabled.

I recommend the installer from zero-to-nix.com. This installer automatically enables flakes.

More documentation (and another installer) available at nixos.org.

To enable flakes on an existing Nix or NixOS installation, see the instructions in the NixOS wiki.

The description part is self-explanatory; it’s just a string. You probably won’t need nixConfig unless you’re doing something fancy. I’m going to focus on what goes into the inputs and outputs sections, and highlight some of the things I found confusing when I began using flakes.

This section specifies the dependencies of a flake. It’s an attribute set; it maps keys to values.

To ensure that a build is reproducible, the build step runs in a pure environment with no network access. Therefore, any external dependencies must be specified in the “inputs” section so they can be fetched in advance (before we enter the pure environment).

Each entry in this section maps an input name to a flake reference. This commonly takes the following form.

As a first example of a flake reference, all (almost all?) flakes depend on “nixpkgs”, which is a large Git repository of programs and libraries that are pre-packaged for Nix. We can write that as

where version is replaced with the version number that you used to build the package, e.g. 22.11. Information about the latest nixpkgs releases is available at https://status.nixos.org/. You can also write the entry without the version number

or more simply,

You might be concerned that omitting the version number would make the build non-reproducible. If someone else builds the flake, could they end up with a different version of nixpkgs? No! remember that the lockfile (flake.lock) uniquely specifies all flake inputs.

Git and Mercurial repositories are the most common type of flake reference, as in the examples below.

You can find more examples of flake references in the Nix Reference Manual.

Each of the inputs is fetched, evaluated and passed to the outputs function as a set of attributes with the same name as the corresponding input.

This section is a function that essentially returns the recipe for building the flake.

We said above that inputs are passed to the outputs, so we need to list them as parameters. This example references the import-cargo dependency defined in the previous example.

So what actually goes in the highlighted section? That depends on the programming languages your software is written in, the build system you use, and more. There are Nix functions and tools that can simplify much of this, and new, easier-to-use ones are released regularly. We’ll look at some of these in the next section.

Let’s make a simple modification to the script. This will give you an opportunity to check your understanding of flakes.

The first step is to enter a development shell.

The flake.nix file specifies all of the tools that are needed during development of the package. The nix develop command puts us in a shell with those tools. As it turns out, we didn’t need any extra tools (beyond the standard environment) for development yet, but that’s usually not the case. Also, we will soon need another tool.

A development environment only allows you to develop the package. Don’t expect the package outputs (e.g. executables) to be available until you build them. However, our script doesn’t need to be compiled, so can’t we just run it?

That worked before; why isn’t it working now? Earlier we used nix shell to enter a runtime environment where hello-flake was available and on the $PATH. This time we entered a development environment using the nix develop command. Since the flake hasn’t been built yet, the executable won’t be on the $PATH. We can, however, run it by specifying the path to the script.

We can also build the flake using the nix build command, which places the build outputs in a directory called result.

Rather than typing the full path to the executable, it’s more convenient to use nix run.

Here’s a summary of the more common Nix commands.

Now we’re ready to make the flake a little more interesting. Instead of using the echo command in the script, we can use the Linux cowsay command. Here’s the hello-flake file, with the modified line highlighted.

Let’s test the modified script.

What went wrong? Remember that we are in a development shell. Since flake.nix didn’t define the devShells variable, the development shell only includes the Nix standard environment. In particular, the cowsay command is not available.

To fix the problem, we can modify flake.nix. We don’t need to add cowsay to the inputs section because it’s included in nixpkgs, which is already an input. However, we also want it to be available in a develoment shell. The highlighted modifications below will accomplish that.

Now we restart the development shell and see that the cowsay command is available and the script works. Because we’ve updated source files but haven’t git commited the new version, we get a warning message about it being “dirty”. It’s just a warning, though; the script runs correctly.

Alternatively, we could use nix run.

Note, however, that nix run rebuilt the package in the Nix store and ran that. It did not alter the copy in the result directory, as we’ll see next.

If we want to update the version in result, we need nix build again.

Let’s git commit the changes and verify that the warning goes away. We don’t need to git push the changes until we’re ready to share them.

If you’re getting confused about when to use the different commands, it’s because there’s more than one way to use Nix. I tend to think of it as two different development workflows.

My usual, high-level workflow is quite simple.

In the high-level workflow, I don’t use a development shell because I don’t need to directly invoke development tools such as compilers and linkers. Nix invokes them for me according to the output definition in flake.nix.

Occasionally I want to work at a lower level, and invoke compiler, linkers, etc. directly. Perhaps want to work on one component without rebuilding the entire package. Or perhaps I’m confused by some error message, so I want to temporarily bypass Nix and work directly with the compiler. In this case I temporarily switch to a low-level workflow.

I generally only use nix build if I just want to build the package but not execute anything (perhaps it’s just a library).

It is a bit annoying to modify flake.nix and ether rebuild or reload the development environment every time you need another tool. However, this Nix way of doing things ensures that all of your dependencies, down to the exact versions, are captured in flake.lock, and that anyone else will be able to reproduce the development environment.

Next, we’ll create a simple Python program.

Before we package the program, let’s verify that it runs. We’re going to need Python. By now you’ve probably figured out that we can write a flake.nix and define a development shell that includes Python. We’ll do that shortly, but first I want to show you a handy shortcut. We can lauch a temporary shell with any Nix packages we want. This is convenient when you just want to try out some new software and you’re not sure if you’ll use it again. It’s also convenient when you’re not ready to write flake.nix (perhaps you’re not sure what tools and packages you need), and you want to experiment a bit first.

The command to enter a temporary shell is

nix-shell -p packages

If there are multiple packages, they should be separated by spaces.

Let’s enter a shell with Python so we can test the program.

Next, create a Python script to build the package. We’ll use Python’s setuptools, but you can use other build tools. For more information on setuptools, see the Python Packaging User Guide, especially the section on setup args.

We won’t write flake.nix just yet. First we’ll try building the package manually.

The missing module error happens because we don’t have build available in the temporary shell. We can fix that by adding “build” to the temporary shell. When you need support for both a language and some of its packages, it’s best to use one of the Nix functions that are specific to the programming language and build system. For Python, we can use the withPackages function.

Note that we’re now inside a temporary shell inside the previous temporary shell! To get back to the original shell, we have to exit twice. Alternatively, we could have done exit followed by the nix-shell command.

After a lot of output messages, the build succeeds.

Now we should write flake.nix. We already know how to write most of the flake from the examples we did earlier. The two parts that will be different are the development shell and the package builder.

Let’s start with the development shell. It seems logical to write something like the following.

Note that we need the parentheses to prevent python.withPackages and the argument from being processed as two separate tokens. Suppose we wanted to work with virtualenv and pip instead of build. We could write something like the following.

For the package builder, we can use the buildPythonApplication function.

If you put all the pieces together, your flake.nix should look something like this.

Let’s try out the new flake.

Why can’t it find flake.nix? Nix flakes only “see” files that are part of the repository. We need to add all of the important files to the repo before building or running the flake.

We’d like to share this package with others, but first we should do some cleanup. When the package was built (automatically by the nix run command), it created a flake.lock file. We need to add this to the repo, and commit all important files.

You can test that your package is properly configured by going to another directory and running it from there.

If you move the project to a public repo, anyone can run it. Recall from the beginning of the tutorial that you were able to run hello-flake directly from my repo with the following command.

Modify the URL accordingly and invite someone else to run your new Python flake.

This shell provides access to two packages from nixpkgs: hello and cowsay.

Here’s a demonstration using the shell.

The command-line equivalent would be nix-shell -p hello cowsay

Here’s a demonstration using the shell.

Here’s a demonstration using the shell.

This shell provides access to three Haskell packages that are on my hard drive.

This shell provides access to four Haskell packages that are on my hard drive. The fourth package depends on the first three to build.

This shell has the environment variable FOO set to “bar”

Here’s a demonstration using the shell.