Wombat’s Book of Nix

Before learning to write Nix flakes, let’s learn how to use them. I’ve created a simple example of a flake in this git repository. To run this flake, you don’t need to install anything; simply run the command below. The first time you use a package, Nix has to fetch and build it, which may take a few minutes. Subsequent invocations should be instantaneous.

That’s a lot to type every time we want to use this package. Instead, we can enter a shell with the package available to us, using the nix shell command.

In this shell, the command is on our $PATH, so we can execute the command by name.

Nix didn’t install the package; it merely built and placed it in a directory called the ``Nix store''. Thus we can have multiple versions of a package without worrying about conflicts. We can find out the location of the executable, if we’re curious.

Once we exit that shell, the hello-flake command is no longer available.

Actually, we can still access the command using the store path we found earlier. That’s not particularly convenient, but it does demonstrate that the package remains in the store for future use.

Let’s clone the repository and see how the flake is defined.

This is a simple repo with just a few files. Like most git repos, it includes LICENSE, which contains the software license, and README.md which provides information about the repo.

The hello-flake file is the command we were executing earlier. This particular executable is just a shell script, so we can view it. It’s an extremly simple script with just two lines.

Now that we have a copy of the repo, we can execute this script directly.

Not terribly exciting, I know. But starting with such a simple package makes it easier to focus on the flake system without getting bogged down in the details. We’ll make this script a little more interesting later.

Let’s look at another file. The file that defines how to package a flake is always called flake.nix.

If this is your first time seeing a flake definition, it probably looks intimidating. Flakes are written in a functional language called Nix^[1]. Yes, ``Nix'' is the name of both the package manager and the language it uses. We’ll look at this in more detail shortly. For now, I’d like to focus on the inputs section.

There are just two entries, one for nixpkgs and one for flake-utils. The first one, nixpkgs refers to the collection of standard software packages that can be installed with the Nix package manager. The second, flake-utils, is a collection of utilities that simplify writing flakes. The important thing to note is that the hello-flake package depends on nixpkgs and flake-utils.

Finally, let’s look at flake.lock, or rather, just part of it.

If flake.nix seemed intimidating, then this file looks like an invocation for Cthulhu. The good news is that this file is automatically generated; you never need to write it. It contains information about all of the dependencies for the flake, including where they came from, the exact version/revision, and hash. This lockfile uniquely specifies all flake dependencies, (e.g., version number, branch, revision, hash), so that anyone, anywhere, any time, can re-create the exact same environment that the original developer used.

No more complaints of ``but it works on my machine!''. That is the benefit of using flakes.

The basic structure of a flake is shown below.

The description part is self-explanatory; it’s just a string. You probably won’t need nixConfig unless you’re doing something fancy. I’m going to focus on what goes into the inputs and outputs sections, and highlight some of the things I found confusing.

The previous section presented a very high-level view of flakes, focusing on the basic structure. In this section, we will add a bit more detail.

Flakes are written in the Nix programming language, which is a functional language. As with most programming languages, there are many ways to achieve the same result. Below is an example you can follow when writing your own flakes. I’ll explain the example in some detail.

We discussed how to specify flake inputs ❶ in the previous section, so this part of the flake should be familiar. Remember also that any dependencies in the input section should also be listed at the beginning of the outputs section ❷.

Now it’s time to look at the content of the output section. If we want the package to be available for multiple systems (e.g., x86_64-linux'', aarch64-linux'', x86_64-darwin'', and aarch64-darwin''), we need to define the output for each of those systems. Often the definitions are identical, apart from the name of the system. The eachDefaultSystem function ❸ provided by flake-utils allows us to write a single definition using a variable for the system name. The function then iterates over all default systems to generate the outputs for each one.

The devShells variable specifies the environment that should be available when doing development on the package. If you don’t need a special development environment, you can omit this section. At ❹ you would list any tools (e.g., compilers and language-specific build tools) you want to have available in a development shell. If the compiler needs access to language-specific packages, there are Nix functions to assist with that. These functions are very language-specific, and not always well-documented. We will see examples for some languages later in the tutorial. In general, I recommend that you do a web search for ``nix language'', and try to find resources that were written or updated recently.

The packages variable defines the packages that this flake provides. The package definition ❺ depends on the programming languages your software is written in, the build system you use, and more. There are Nix functions and tools that can simplify much of this, and new, easier-to-use ones are released regularly. Again, I recommend that you do a web search for ``nix language'', and try to find resources that were written or updated recently.

The apps variable identifies any applications provided by the flake. In particular, it identifies the default executable ❻ that nix run will run if you don’t specify an app.

The list below contains are a few functions that are commonly used in this section.

Now that we have a better understanding of the structure of flake.nix, let’s have a look at the one we saw earlier, in the hello-flake repo. If you compare this flake definition to the colour-coded template presented in the previous section, most of it should look familiar.

This flake.nix doesn’t have a devShells section, because development on the current version doesn’t require anything beyond the ``bare bones'' linux commands. Later we will add a feature that requires additional development tools.

Now let’s look at the section I labeled ```SOME UNFAMILIAR STUFF’' and see what it does.

This flake uses mkDerivation (1) which is a very useful general-purpose package builder provided by the Nix standard environment. It’s especially useful for the typical ./configure; make; make install scenario, but for this flake we don’t even need that.

The name variable is the name of the flake, as it would appear in a package listing if we were to add it to Nixpkgs or another package collection. The src variable (2) supplies the location of the source files, relative to flake.nix. When a flake is accessed for the first time, the repository contents are fetched in the form of a tarball. The unpackPhase variable indicates that we do want the tarball to be unpacked.

The buildPhase variable is a sequence of Linux commands to build the package. Typically, building a package requires compiling the source code. However, that’s not required for a simple shell script. So buildPhase consists of a single command, :, which is a no-op or ``do nothing'' command.

The installPhase variable is a sequence of Linux commands that will do the actual installation. In this case, we create a directory (3) for the installation, copy the hello-flake script (4) there, and make the script executable (5). The environment variable $src refers to the source directory, which we specified earlier (2).

Earlier we said that the build step runs in a pure environment to ensure that builds are reproducible. This means no Internet access; indeed no access to any files outside the build directory. During the build and install phases, the only commands available are those provided by the Nix standard environment and the external dependencies identified in the inputs section of the flake.

I’ve mentioned the Nix standard environment before, but I didn’t explain what it is. The standard environment, or stdenv, refers to the functionality that is available during the build and install phases of a Nix package (or flake). It includes the commands listed below^[2].

Only a few environment variables are available. The most interesting ones are listed below.

Let’s make a simple modification to the script. This will give you an opportunity to check your understanding of flakes.

The first step is to enter a development shell.

The flake.nix file specifies all of the tools that are needed during development of the package. The nix develop command puts us in a shell with those tools. As it turns out, we didn’t need any extra tools (beyond the standard environment) for development yet, but that’s usually not the case. Also, we will soon need another tool.

A development environment only allows you to develop the package. Don’t expect the package outputs (e.g. executables) to be available until you build them. However, our script doesn’t need to be compiled, so can’t we just run it?

That worked before; why isn’t it working now? Earlier we used nix shell to enter a runtime environment where hello-flake was available and on the $PATH. This time we entered a development environment using the nix develop command. Since the flake hasn’t been built yet, the executable won’t be on the $PATH. We can, however, run it by specifying the path to the script.

We can also build the flake using the nix build command, which places the build outputs in a directory called result.

Rather than typing the full path to the executable, it’s more convenient to use nix run.

Here’s a summary of the more common Nix commands.

Now we’re ready to make the flake a little more interesting. Instead of using the echo command in the script, we can use the Linux cowsay command. The sed command below will make the necessary changes.

Let’s test the modified script.

What went wrong? Remember that we are in a development shell. Since flake.nix didn’t define the devShells variable, the development shell only includes the Nix standard environment. In particular, the cowsay command is not available.

To fix the problem, we can modify flake.nix. We don’t need to add cowsay to the inputs section because it’s included in nixpkgs, which is already an input. However, we do need to indicate that we want it available in a develoment shell. Add the following lines before the packages = rec { line.

Here is a `diff'' showing the changes in `flake.nix.

We restart the development shell and see that the cowsay command is now available and the script works. Because we’ve updated source files but haven’t `git commit`ed the new version, we get a warning message about it being ``dirty''. It’s just a warning, though; the script runs correctly.

Alternatively, we could use nix run.

Note, however, that nix run rebuilt the package in the Nix store and ran that. It did not alter the copy in the result directory, as we’ll see next.

If we want to update the version in result, we need nix build again.

Let’s git commit the changes and verify that the warning goes away. We don’t need to git push the changes until we’re ready to share them.

At last we are ready to create a flake from scratch! Start with an empty directory and create a git repository.

Next, we’ll create a simple Python program.

Before we pachage the program, let’s verify that it runs. We’re going to need Python. By now you’ve probably figured out that we can write a flake.nix and define a development shell that includes Python. We’ll do that shortly, but first I want to show you a handy shortcut. We can lauch a temporary shell with any Nix packages we want. This is convenient when you just want to try out some new software and you’re not sure if you’ll use it again. It’s also convenient when you’re not ready to write flake.nix (perhaps you’re not sure what tools and packages you need), and you want to experiment a bit first.

The command to enter a temporary shell is

nix-shell -p packages

If there are multiple packages, they should be separated by spaces. Note that the command used here is nix-shell with a hyphen, not nix shell with a space; those are two different commands. In fact there are hyphenated and non-hyphenated versions of many Nix commands, and yes, it’s confusing. The non-hyphenated commands were introduced when support for flakes was added to Nix. I predict that eventually all hyphenated commands will be replaced with non-hyphenated versions. Until then, a useful rule of thumb is that non-hyphenated commands are for for working directly with flakes; hyphenated commands are for everything else.

Let’s enter a shell with Python so we can test the program.

Next, create a Python script to build the package. We’ll use Python’s setuptools, but you can use other build tools. For more information on setuptools, see the Python Packaging User Guide, especially the section on setup args.

We won’t write flake.nix just yet. First we’ll try building the package manually.

The missing module error happens because we don’t have build available in the temporary shell. We can fix that by adding `build'' to the temporary shell. When you need support for both a language and some of its packages, it’s best to use one of the Nix functions that are specific to the programming language and build system. For Python, we can use the `withPackages function.

Note that we’re now inside a temporary shell inside the previous temporary shell! To get back to the original shell, we have to exit twice. Alternatively, we could have done exit followed by the nix-shell command.

After a lot of output messages, the build succeeds.

Now we should write flake.nix. We already know how to write most of the flake from the examples we did earlier. The two parts that will be different are the development shell and the package builder.

Let’s start with the development shell. It seems logical to write something like the following.

Note that we need the parentheses to prevent python.withPackages and the argument from being processed as two separate tokens. Suppose we wanted to work with virtualenv and pip instead of build. We could write something like the following.

For the package builder, we can use the buildPythonApplication function.

If you put all the pieces together, your flake.nix should look something like this.

Let’s try out the new flake.

Why can’t it find flake.nix? Nix flakes only ``see'' files that are part of the repository. We need to add all of the important files to the repo before building or running the flake.

We’d like to share this package with others, but first we should do some cleanup. When the package was built (automatically by the nix run command), it created a flake.lock file. We need to add this to the repo, and commit all important files.

You can test that your package is properly configured by going to another directory and running it from there.

If you move the project to a public repo, anyone can run it. Recall from the beginning of the tutorial that you were able to run hello-flake directly from my repo with the following command.

Modify the URL accordingly and invite someone else to run your new Python flake.

You can use nix-shell to run scripts in arbitrary languages, providing the necessary dependencies. This is particularly convenient for standalone scripts because you don’t need to create a repo and write a separate flake.nix.

The script should start with two `shebang'' (#!) commands. The first should invoke `nix-shell. The second should declares the scrpt interpreter and any dependencies. Here are some examples.