diff --git a/index.html b/index.html index 11b9fa6..43df2e6 100644 --- a/index.html +++ b/index.html @@ -898,7 +898,7 @@ see what it does.

        packages = rec {
-          hello = pkgs.stdenv.mkDerivation rec {  ❶
+          hello = pkgs.stdenv.mkDerivation rec { ❶
             name = "hello-flake";
 
             src = ./.; ❷
@@ -917,7 +917,7 @@ see what it does.
-

This flake uses mkDerivation (❶) which is a very useful +

This flake uses mkDerivation ❶ which is a very useful general-purpose package builder provided by the Nix standard environment. It’s especially useful for the typical ./configure; make; make install scenario, but for this flake we don’t @@ -926,7 +926,7 @@ even need that.

The name variable is the name of the flake, as it would appear in a package listing if we were to add it to Nixpkgs or another package -collection. The src variable (❷) supplies the location of the source +collection. The src variable ❷ supplies the location of the source files, relative to flake.nix. When a flake is accessed for the first time, the repository contents are fetched in the form of a tarball. The unpackPhase variable indicates that we do want the tarball to be @@ -941,10 +941,10 @@ which is a no-op or “do nothing” command.

The installPhase variable is a sequence of Linux commands that will do -the actual installation. In this case, we create a directory (❸) for the -installation, copy the hello-flake script there (❹), and make the -script executable (❺). The environment variable $src refers to the -source directory, which we specified earlier (❷).

+the actual installation. In this case, we create a directory ❸ for the +installation, copy the hello-flake script there ❹, and make the +script executable ❺. The environment variable $src refers to the +source directory, which we specified earlier ❷.

Earlier we said that the build step runs in a pure environment to ensure diff --git a/source/flake-structure/main.adoc b/source/flake-structure/main.adoc new file mode 100644 index 0000000..cdaf3f8 --- /dev/null +++ b/source/flake-structure/main.adoc @@ -0,0 +1,125 @@ +== Flake structure + +The basic structure of a flake is shown below. + +.... +{ + description = ... # package description + inputs = ... # dependencies + outputs = ... # what the flake producesgo + nixConfig = ... # advanced configuration options +} +.... + +The `description` part is self-explanatory; it’s just a string. You +probably won’t need `nixConfig` unless you’re doing something fancy. I’m +going to focus on what goes into the `inputs` and `outputs` sections, +and highlight some of the things I found confusing when I began using flakes. + +=== Inputs + +This section specifies the dependencies of a flake. It’s an _attribute +set_; it maps keys to values. + +To ensure that a build is reproducible, the build step runs in a _pure_ +environment with no network access. Therefore, any external dependencies +must be specified in the "`inputs`" section so they can be fetched in +advance (before we enter the pure environment). + +Each entry in this section maps an input name to a _flake reference_. +This commonly takes the following form. + +.... +NAME.url = URL-LIKE-EXPRESSION +.... + +As a first example of a flake reference, all (almost all?) flakes depend on "`nixpkgs`", +which is a large Git repository of programs and libraries that are +pre-packaged for Nix. We can write that as + +.... +nixpkgs.url = "github:NixOS/nixpkgs/nixos-VERSION"; +.... + +where `NN.MM` is replaced with the version number that you used to build +the package, e.g. `22.11`. Information about the latest nixpkgs releases +is available at https://status.nixos.org/. You can also write the entry +without the version number + +.... +nixpkgs.url = "github:NixOS/nixpkgs/nixos-VERSION"; +.... + +or more simply, + +.... +nixpkgs.url = "nixpkgs"; +.... + +You might be concerned that omitting the version number would make the +build non-reproducible. If someone else builds the flake, could they end +up with a different version of nixpkgs? No! remember that the lockfile +(`flake.lock`) _uniquely_ specifies all flake inputs. + +Git and Mercurial repositories are the most common type of flake +reference, as in the examples below. + +A Git repository:: + `git+https://github.com/NixOS/patchelf` +A specific branch of a Git repository:: + `git+https://github.com/NixOS/patchelf?ref=master` +A specific revision of a Git repository:: + + + `git+https://github.com/NixOS/patchelf?ref=master&rev=f34751b88bd07d7f44f5cd3200fb4122bf916c7e` +A tarball:: + `https://github.com/NixOS/patchelf/archive/master.tar.gz` + +You can find more examples of flake references in the +https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-flake.html#examples[Nix +Reference Manual]. + +[NOTE] +==== +Although you probably won’t need to use it, there is another syntax for +flake references that you might encounter. This example + +.... +inputs.import-cargo = { + type = "github"; + owner = "edolstra"; + repo = "import-cargo"; +}; +.... + +is equivalent to + +.... +inputs.import-cargo.url = "github:edolstra/import-cargo"; +.... +==== + +Each of the `inputs` is fetched, evaluated and passed to the `outputs` +function as a set of attributes with the same name as the corresponding +input. + +=== Outputs + +This section is a function that essentially returns the recipe for +building the flake. + +We said above that `inputs` are passed to the `outputs`, so we need to +list them as parameters. This example references the `import-cargo` +dependency defined in the previous example. + +[subs=quotes] +.... +outputs = { self, nixpkgs, import-cargo }: { + #... outputs ...# +}; +.... + +So what actually goes in the highlighted section? +That depends on the programming languages your software is written in, +the build system you use, and more. There are Nix functions and tools +that can simplify much of this, and new, easier-to-use ones are released +regularly. We’ll look at some of these in the next section. diff --git a/source/generic-flake/main.adoc b/source/generic-flake/main.adoc new file mode 100644 index 0000000..f751b00 --- /dev/null +++ b/source/generic-flake/main.adoc @@ -0,0 +1,100 @@ +== A generic flake + +The previous section presented a very high-level view of flakes, +focusing on the basic structure. In this section, we will add a bit more +detail. + +Flakes are written in the Nix programming language, which is a +functional language. As with most programming languages, there are many +ways to achieve the same result. Below is an example you can follow when +writing your own flakes. I’ll explain the example in some detail. + +[subs=quotes] +---- +{ + description = "#_brief package description_#"; + + inputs = { + [.highlight01]#nixpkgs#.url = "github:NixOS/nixpkgs"; + [.highlight02]#flake-utils#.url = "github:numtide/flake-utils"; + [.highlight03]#..._other dependencies_...# ❶ + }; + + outputs = { self, [.highlight01]#nixpkgs#, [.highlight02]#flake-utils#, [.highlight03]#..._other dependencies_...# ❷ }: + flake-utils.lib.eachDefaultSystem (system: ❸ + let + pkgs = import nixpkgs { inherit system; }; + python = pkgs.python3; + in + { + devShells = rec { + default = pkgs.mkShell { + packages = [ #_packages needed for development shell_#; ❹ ])) + ]; + }; + + packages = rec { + [.highlight04]#myPackageName# = #_package definition_#; ❺ + default = [.highlight04]#myPackageName#; + }; + + apps = rec { + [.highlight04]#myPackageName# = flake-utils.lib.mkApp { drv = self.packages.${system}.[.highlight04]#myPackageName#; }; + default = [.highlight04]#myPackageName#; + }; + } + ); +} +---- + +We discussed how to specify flake inputs `❶` in the previous section, so +this part of the flake should be familiar. Remember also that any +dependencies in the input section should also be listed at the beginning +of the outputs section `❷`. + +Now it’s time to look at the content of the output section. If we want +the package to be available for multiple systems (e.g., +"`x86_64-linux`", "`aarch64-linux`", "`x86_64-darwin`", and +"`aarch64-darwin`"), we need to define the output for each of those +systems. Often the definitions are identical, apart from the name of the +system. The eachDefaultSystem function `❸` provided by flake-utils allows +us to write a single definition using a variable for the system name. +The function then iterates over all default systems to generate the +outputs for each one. + +The `devShells` variable specifies the environment that should be +available when doing development on the package. If you don’t need a +special development environment, you can omit this section. At `❹` you +would list any tools (e.g., compilers and language-specific build tools) +you want to have available in a development shell. If the compiler needs +access to language-specific packages, there are Nix functions to assist +with that. These functions are very language-specific, and not always +well-documented. We will see examples for some languages later in the +tutorial. In general, I recommend that you do a web search for +"`nix language`", and try to find resources that were written or updated +recently. + +The `packages` variable defines the packages that this flake provides. +The package definition `❺` depends on the programming languages your +software is written in, the build system you use, and more. There are +Nix functions and tools that can simplify much of this, and new, +easier-to-use ones are released regularly. Again, I recommend that you +do a web search for "`nix language`", and try to find resources that +were written or updated recently. + +The `apps` variable identifies any applications provided by the flake. +In particular, it identifies the default executable ❻ that `nix run` +will run if you don’t specify an app. + +Below is a list of some functions that are commonly used in +this section. + +General-purpose:: + The standard environment provides `mkDerivation`, which is especially + useful for the typical `./configure; make; make install` scenario. + It’s customisable. +Python:: + `buildPythonApplication`, `buildPythonPackage`. +Haskell:: + `mkDerivation` (Haskell version, which is a wrapper around the + standard environment version), `developPackage`, `callCabal2Nix`. diff --git a/source/hello-flake-repo/main.adoc0 b/source/hello-flake-repo/main.adoc0 new file mode 100644 index 0000000..adb9631 --- /dev/null +++ b/source/hello-flake-repo/main.adoc0 @@ -0,0 +1,90 @@ +== The hello-flake repo + +Let’s clone the repository and see how the flake is defined. + +.... +$ git clone https://codeberg.org/mhwombat/hello-flake +$ cd hello-flake +$ ls +.... + +This is a simple repo with just a few files. Like most git repos, it +includes `LICENSE`, which contains the software license, and `README.md` +which provides information about the repo. + +The `hello-flake` file is the executable we ran earlier. +This particular executable is just a shell script, so we can view it. +It’s an extremly simple script with just two lines. + +[source,bash,linenums] +.hello-flake +.... +$# cat hello-flake +.... + +Now that we have a copy of the repo, we can execute this script +directly. + +.... +$ ./hello-flake +.... + +Not terribly exciting, I know. But starting with such a simple package +makes it easier to focus on the flake system without getting bogged down +in the details. We’ll make this script a little more interesting later. + +Let’s look at another file. The file that defines how to package a flake +is always called `flake.nix`. + +[source,nix,linenums] +.flake.nix +.... +$# cat flake.nix +.... + +If this is your first time seeing a flake definition, it probably looks +intimidating. Flakes are written in a functional language called +Nixfootnote:[For an introduction to the Nix language, see +https://nixos.org/guides/nix-language.html[Nix language basics].]. Yes, +"`Nix`" is the name of both the package manager and the language it +uses. We’ll look at this in more detail shortly. For now, I’d like to +focus on the inputs section. + +.... +inputs = { + nixpkgs.url = "github:NixOS/nixpkgs"; + flake-utils.url = "github:numtide/flake-utils"; + }; +.... + +There are just two entries, one for `nixpkgs` and one for `flake-utils`. +The first one, `nixpkgs` refers to the collection of standard software +packages that can be installed with the Nix package manager. The second, +`flake-utils`, is a collection of utilities that simplify writing +flakes. The important thing to note is that the `hello-flake` package +_depends_ on `nixpkgs` and `flake-utils`. + +Finally, let’s look at `flake.lock`, or rather, just part of it. + +[source,linenums] +.flake.lock +.... +$# head -n 40 flake.lock +.... + +If `flake.nix` seemed intimidating, then this file looks like an +invocation for Cthulhu. The good news is that this file is automatically +generated; you never need to write it. It contains information about all +of the dependencies for the flake, including where they came from, the +exact version/revision, and hash. This lockfile _uniquely_ specifies all +flake dependencies, (e.g., version number, branch, revision, hash), so +that _anyone, anywhere, any time, can re-create the exact same +environment that the original developer used._ + +No more complaints of "`but it works on my machine!`". That is the +benefit of using flakes. + +//// +Good adoc0 scripts clean up after themselves. +$ cd .. ; rm -rf hello-flake # clean up +//// diff --git a/source/hello-flake-return/main.adoc b/source/hello-flake-return/main.adoc new file mode 100644 index 0000000..0780deb --- /dev/null +++ b/source/hello-flake-return/main.adoc @@ -0,0 +1,135 @@ +== Another look at hello-flake + +Now that we have a better understanding of the structure of `flake.nix`, +let’s have a look at the one we saw earlier, in the `hello-flake` repo. +If you compare this flake definition to the colour-coded template +presented in the previous section, most of it should look familiar. + +[subs=quotes] +.... +{ + description = "a very simple and friendly flake"; + + inputs = { + nixpkgs.url = "github:NixOS/nixpkgs"; + flake-utils.url = "github:numtide/flake-utils"; + }; + + outputs = { self, nixpkgs, flake-utils }: + flake-utils.lib.eachDefaultSystem (system: + let + pkgs = import nixpkgs { inherit system; }; + in + { + packages = rec { + hello = + . . . + _SOME UNFAMILIAR STUFF_ + . . . + }; + default = hello; + }; + + apps = rec { + hello = flake-utils.lib.mkApp { drv = self.packages.${system}.hello; }; + default = hello; + }; + } + ); +} +.... + +This `flake.nix` doesn’t have a `devShells` section, because development +on the current version doesn’t require anything beyond +the "`bare bones`" linux commands. Later we will add a feature that requires +additional development tools. + +Now let’s look at the section I labeled `SOME UNFAMILIAR STUFF` and +see what it does. + +.... + packages = rec { + hello = pkgs.stdenv.mkDerivation rec { ❶ + name = "hello-flake"; + + src = ./.; ❷ + + unpackPhase = "true"; + + buildPhase = ":"; + + installPhase = + '' + mkdir -p $out/bin ❸ + cp $src/hello-flake $out/bin/hello-flake ❹ + chmod +x $out/bin/hello-flake ❺ + ''; + }; +.... + +This flake uses `mkDerivation` `❶` which is a very useful +general-purpose package builder provided by the Nix standard +environment. It’s especially useful for the typical +`./configure; make; make install` scenario, but for this flake we don’t +even need that. + +The `name` variable is the name of the flake, as it would appear in a +package listing if we were to add it to Nixpkgs or another package +collection. The `src` variable `❷` supplies the location of the source +files, relative to `flake.nix`. When a flake is accessed for the first +time, the repository contents are fetched in the form of a tarball. The +`unpackPhase` variable indicates that we do want the tarball to be +unpacked. + +The `buildPhase` variable is a sequence of Linux commands to build the +package. Typically, building a package requires compiling the source +code. However, that’s not required for a simple shell script. So +`buildPhase` consists of a single command, `:`, +which is a no-op or "`do nothing`" command. + +The `installPhase` variable is a sequence of Linux commands that will do +the actual installation. In this case, we create a directory `❸` for the +installation, copy the `hello-flake` script there `❹`, and make the +script executable `❺`. The environment variable `$src` refers to the +source directory, which we specified earlier `❷`. + +Earlier we said that the build step runs in a pure environment to ensure +that builds are reproducible. This means no Internet access; indeed no +access to any files outside the build directory. During the build and +install phases, the only commands available are those provided by the +Nix standard environment and the external dependencies identified in the +`inputs` section of the flake. + +I’ve mentioned the Nix standard environment before, but I didn’t explain +what it is. The standard environment, or `stdenv`, refers to the +functionality that is available during the build and install phases of a +Nix package (or flake). It includes the commands listed +belowfootnote:[For more information on the standard environment, see the +https://nixos.org/manual/nixpkgs/stable/#sec-tools-of-stdenv[Nixpkgs +manual]]. + +* The GNU C Compiler, configured with C and C++ support. +* GNU coreutils (contains a few dozen standard Unix commands). +* GNU findutils (contains find). +* GNU diffutils (contains diff, cmp). +* GNU sed. +* GNU grep. +* GNU awk. +* GNU tar. +* gzip, bzip2 and xz. +* GNU Make. +* Bash. +* The patch command. +* On Linux, stdenv also includes the patchelf utility. + +Only a few environment variables are available. The most interesting +ones are listed below. + +* `$name` is the package name. +* `$src` refers to the source directory. +* `$out` is the path to the location in the Nix store where the package +will be added. +* `$system` is the system that the package is being built for. +* `$PWD` and `$TMP` both point to a temporary build directories +* `$HOME` and `$PATH` point to nonexistent directories, so the build +cannot rely on them. diff --git a/source/hello-flake/main.adoc0 b/source/hello-flake/main.adoc0 new file mode 100644 index 0000000..9d3fd57 --- /dev/null +++ b/source/hello-flake/main.adoc0 @@ -0,0 +1,53 @@ +== Hello, flake! + +Before learning to write Nix flakes, let’s learn how to use them. I’ve +created a simple example of a flake in this git +https://codeberg.org/mhwombat/hello-flake[repository]. To run this +flake, you don’t need to install anything; simply run the command below. +The first time you use a flake, Nix has to fetch and build it, which +may take time. Subsequent invocations should be instantaneous. + +.... +$ nix run "git+https://codeberg.org/mhwombat/hello-flake" +.... + +That’s a lot to type every time we want to use this package. Instead, we +can enter a shell with the package available to us, using the +`nix shell` command. + +.... +$ nix shell "git+https://codeberg.org/mhwombat/hello-flake" +.... + +In this shell, the command is on our `$PATH`, so we can execute the +command by name. + +.... +$ hello-flake +.... + +Nix didn’t _install_ the package; it merely built and placed it in a +directory called the "`Nix store`". Thus we can have multiple versions +of a package without worrying about conflicts. We can find out the +location of the executable, if we’re curious. + +.... +$ which hello-flake +.... + +Once we exit that shell, the `hello-flake` command is no longer +directly available. + +.... +$# echo '$ exit' +$# echo '$ hello-flake' +sh: line 3: hello-flake: command not found +.... + +However, we can still run the command using the store path we found +earlier. That’s not particularly convenient, but it does demonstrate +that the package remains in the store for future use. + +.... +/nix/store/0xbn2hi6h1m5h4kc02vwffs2cydrbc0r-hello-flake/bin/hello-flake +....