diff --git a/source/hello-flake.adoc0 b/source/hello-flake.adoc0 new file mode 100644 index 0000000..3a65c35 --- /dev/null +++ b/source/hello-flake.adoc0 @@ -0,0 +1,53 @@ +== Hello, flake! + +Before learning to write Nix flakes, let’s learn how to use them. I’ve +created a simple example of a flake in this git +https://codeberg.org/mhwombat/hello-flake[repository]. To run this +flake, you don’t need to install anything; simply run the command below. +The first time you use a package, Nix has to fetch and build it, which +may take a few minutes. Subsequent invocations should be instantaneous. + +.... +$ nix run "git+https://codeberg.org/mhwombat/hello-flake" +.... + +That’s a lot to type every time we want to use this package. Instead, we +can enter a shell with the package available to us, using the +`nix shell` command. + +.... +$ nix shell "git+https://codeberg.org/mhwombat/hello-flake" +.... + +In this shell, the command is on our `$PATH`, so we can execute the +command by name. + +.... +$ hello-flake +.... + +Nix didn’t _install_ the package; it merely built and placed it in a +directory called the ``Nix store''. Thus we can have multiple versions +of a package without worrying about conflicts. We can find out the +location of the executable, if we’re curious. + +.... +$ which hello-flake +.... + +Once we exit that shell, the `hello-flake` command is no longer +available. + +.... +$# echo '$ exit' +$# echo '$ hello-flake' +sh: line 3: hello-flake: command not found +.... + +Actually, we can still access the command using the store path we found +earlier. That’s not particularly convenient, but it does demonstrate +that the package remains in the store for future use. + +.... +/nix/store/0xbn2hi6h1m5h4kc02vwffs2cydrbc0r-hello-flake/bin/hello-flake +.... diff --git a/source/modify-hello-flake.adoc0 b/source/modify-hello-flake.adoc0 new file mode 100644 index 0000000..ef9747e --- /dev/null +++ b/source/modify-hello-flake.adoc0 @@ -0,0 +1,201 @@ +== Modifying the flake + +Let’s make a simple modification to the script. This will give you an +opportunity to check your understanding of flakes. + +The first step is to enter a development shell. + +.... +$ cd ~/tutorial-practice/hello-flake +$ nix develop +.... + +The `flake.nix` file specifies all of the tools that are needed during +development of the package. The `nix develop` command puts us in a shell +with those tools. As it turns out, we didn’t need any extra tools +(beyond the standard environment) for development yet, but that’s +usually not the case. Also, we will soon need another tool. + +A development environment only allows you to _develop_ the package. +Don’t expect the package _outputs_ (e.g. executables) to be available +until you build them. However, our script doesn’t need to be compiled, +so can’t we just run it? + +.... +$ hello-flake +.... + +That worked before; why isn’t it working now? Earlier we used +`nix shell` to enter a _runtime_ environment where `hello-flake` was +available and on the `$PATH`. This time we entered a _development_ +environment using the `nix develop` command. Since the flake hasn’t been +built yet, the executable won’t be on the `$PATH`. We can, however, run +it by specifying the path to the script. + +.... +$ ./hello-flake +.... + +We can also build the flake using the `nix build` command, which places +the build outputs in a directory called `result`. + +.... +$ nix build +$ result/bin/hello-flake +.... + +Rather than typing the full path to the executable, it’s more convenient +to use `nix run`. + +.... +$ nix run +.... + +Here’s a summary of the more common Nix commands. + +[width="100%",cols="<17%,<83%",options="header",] +|=== +|command |Action +|`nix develop` |Enters a _development_ shell with all the required +development tools (e.g. compilers and linkers) available (as specified +by `flake.nix`). + +|`nix shell` |Enters a _runtime_ shell where the flake’s executables are +available on the `$PATH`. + +|`nix build` |Builds the flake and puts the output in a directory called +`result`. + +|`nix run` |Runs the flake’s default executable, rebuilding the package +first if needed. Specifically, it runs the version in the Nix store, not +the version in `result`. +|=== + +Now we’re ready to make the flake a little more interesting. Instead of +using the `echo` command in the script, we can use the Linux `cowsay` +command. The `sed` command below will make the necessary changes. + +.... +$ sed -i 's/echo/cowsay/' hello-flake +$ cat hello-flake +.... + +Let’s test the modified script. + +.... +$ ./hello-flake +.... + +What went wrong? Remember that we are in a _development_ shell. Since +`flake.nix` didn’t define the `devShells` variable, the development +shell only includes the Nix standard environment. In particular, the +`cowsay` command is not available. + +To fix the problem, we can modify `flake.nix`. We don’t need to add +`cowsay` to the `inputs` section because it’s included in `nixpkgs`, +which is already an input. However, we do need to indicate that we want +it available in a develoment shell. Add the following lines before the +`packages = rec {` line. + +.... + devShells = rec { + default = pkgs.mkShell { + packages = [ pkgs.cowsay ]; + }; + }; +.... + +Here is a ``diff'' showing the changes in `flake.nix`. + +.... +$# sed -i '15i\\ \ \ \ \ \ \ \ devShells = rec {\n\ \ \ \ \ \ \ \ \ \ default = pkgs.mkShell {\n\ \ \ \ \ \ \ \ \ \ \ \ packages = [ pkgs.cowsay ];\n\ \ \ \ \ \ \ \ \ \ };\n\ \ \ \ \ \ \ \ };\n' flake.nix +$ git diff flake.nix +.... + +We restart the development shell and see that the `cowsay` command is +now available and the script works. Because we’ve updated source files +but haven’t `git commit`ed the new version, we get a warning message +about it being ``dirty''. It’s just a warning, though; the script runs +correctly. + +.... +$# echo '$ nix develop' +$# nix develop --command sh +$ which cowsay # is it available now? +$ ./hello-flake +.... + +Alternatively, we could use `nix run`. + +.... +$ nix run +.... + +Note, however, that `nix run` rebuilt the package in the Nix store and +ran _that_. It did not alter the copy in the `result` directory, as +we’ll see next. + +.... +$ cat result/bin/hello-flake +.... + +If we want to update the version in `result`, we need `nix build` again. + +.... +$ nix build +$ cat result/bin/hello-flake +.... + +Let’s `git commit` the changes and verify that the warning goes away. We +don’t need to `git push` the changes until we’re ready to share them. + +.... +$ git commit hello-flake flake.nix -m 'added bovine feature' +$ nix run +.... + +=== Development workflows + +If you’re getting confused about when to use the different commands, +it’s because there’s more than one way to use Nix. I tend to think of it +as two different development workflows. + +My usual, _high-level workflow_ is quite simple. + +[arabic] +. `nix run` to re-build (if necessary) and run the executable. +. Fix any problems in `flake.nix` or the source code. +. Repeat until the package works properly. + +In the high-level workflow, I don’t use a development shell because I +don’t need to directly invoke development tools such as compilers and +linkers. Nix invokes them for me according to the output definition in +`flake.nix`. + +Occasionally I want to work at a lower level, and invoke compiler, +linkers, etc. directly. Perhaps want to work on one component without +rebuilding the entire package. Or perhaps I’m confused by some error +message, so I want to temporarily bypass Nix and ``talk'' directly to +the compiler. In this case I temporarily switch to a _low-level +workflow_. + +[arabic] +. `nix develop` to enter a development shell with any development tools +I need (e.g. compilers, linkers, documentation generators). +. Directly invoke tools such as compilers. +. Fix any problems in `flake.nix` or the source code. +. Directly invoke the executable. Note that the location of the +executable depends on the development tools – It probably isn’t +`result`! +. Repeat until the package works properly. + +I generally only use `nix build` if I just want to build the package but +not execute anything (perhaps it’s just a library). + +=== This all seems like a hassle! + +It is a bit annoying to modify `flake.nix` and ether rebuild or reload +the development environment every time you need another tool. However, +this Nix way of doing things ensures that all of your dependencies, down +to the exact versions, are captured in `flake.lock`, and that anyone +else will be able to reproduce the development environment. diff --git a/source/python-flake.adoc0 b/source/python-flake.adoc0 new file mode 100644 index 0000000..a42867e --- /dev/null +++ b/source/python-flake.adoc0 @@ -0,0 +1,194 @@ +== A new flake from scratch (Python) + +At last we are ready to create a flake from scratch! Start with an empty +directory and create a git repository. + +.... +$ cd ~/tutorial-practice +$ mkdir hello-python +$ cd hello-python +$ git init +.... + +Next, we’ll create a simple Python program. + +.... +$# curl https://codeberg.org/mhwombat/hello-flake-python/raw/branch/main/hello.py --silent --output hello.py +$ cat hello.py +.... + +Before we pachage the program, let’s verify that it runs. We’re going to +need Python. By now you’ve probably figured out that we can write a +`flake.nix` and define a development shell that includes Python. We’ll +do that shortly, but first I want to show you a handy shortcut. We can +lauch a _temporary_ shell with any Nix packages we want. This is +convenient when you just want to try out some new software and you’re +not sure if you’ll use it again. It’s also convenient when you’re not +ready to write `flake.nix` (perhaps you’re not sure what tools and +packages you need), and you want to experiment a bit first. + +The command to enter a temporary shell is + +`nix-shell -p` _packages_ + +If there are multiple packages, they should be separated by spaces. Note +that the command used here is `nix-shell` with a hyphen, not `nix shell` +with a space; those are two different commands. In fact there are +hyphenated and non-hyphenated versions of many Nix commands, and yes, +it’s confusing. The non-hyphenated commands were introduced when support +for flakes was added to Nix. I predict that eventually all hyphenated +commands will be replaced with non-hyphenated versions. Until then, a +useful rule of thumb is that non-hyphenated commands are for for working +directly with flakes; hyphenated commands are for everything else. + +Let’s enter a shell with Python so we can test the program. + +.... +$# echo '$ nix-shell -p python3' +$# nix-shell -p python3 --command sh +$ python hello.py +.... + +Next, create a Python script to build the package. We’ll use Python’s +setuptools, but you can use other build tools. For more information on +setuptools, see the +https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/[Python +Packaging User Guide], especially the section on +https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#setup-args[setup +args]. + +.... +$# curl https://codeberg.org/mhwombat/hello-flake-python/raw/branch/main/setup.py --silent --output setup.py +$ cat setup.py +.... + +We won’t write `flake.nix` just yet. First we’ll try building the +package manually. + +.... +$ python -m build +.... + +The missing module error happens because we don’t have `build` available +in the temporary shell. We can fix that by adding ``build'' to the +temporary shell. When you need support for both a language and some of +its packages, it’s best to use one of the Nix functions that are +specific to the programming language and build system. For Python, we +can use the `withPackages` function. + +.... +$# echo '$ nix-shell -p "python3.withPackages (ps: with ps; [ build ])"' +$# nix-shell -p "python3.withPackages (ps: with ps; [ build ])" --command sh +.... + +Note that we’re now inside a temporary shell inside the previous +temporary shell! To get back to the original shell, we have to `exit` +twice. Alternatively, we could have done `exit` followed by the +`nix-shell` command. + +.... +$# echo '$ python -m build' +$# python -m build > /dev/null 2>&1 +.... + +After a lot of output messages, the build succeeds. + +Now we should write `flake.nix`. We already know how to write most of +the flake from the examples we did earlier. The two parts that will be +different are the development shell and the package builder. + +Let’s start with the development shell. It seems logical to write +something like the following. + +.... + devShells = rec { + default = pkgs.mkShell { + packages = [ (python.withPackages (ps: with ps; [ build ])) ]; + }; + }; +.... + +Note that we need the parentheses to prevent `python.withPackages` and +the argument from being processed as two separate tokens. Suppose we +wanted to work with `virtualenv` and `pip` instead of `build`. We could +write something like the following. + +.... + devShells = rec { + default = pkgs.mkShell { + packages = [ + # Python plus helper tools + (python.withPackages (ps: with ps; [ + virtualenv # Virtualenv + pip # The pip installer + ])) + ]; + }; + }; +.... + +For the package builder, we can use the `buildPythonApplication` +function. + +.... + packages = rec { + hello = python.pkgs.buildPythonApplication { + name = "hello-flake-python"; + buildInputs = with python.pkgs; [ pip ]; + src = ./.; + }; + default = hello; + }; +.... + +If you put all the pieces together, your `flake.nix` should look +something like this. + +.... +$# curl https://codeberg.org/mhwombat/hello-flake-python/raw/branch/main/flake.nix --silent --output flake.nix +$ cat flake.nix +.... + +Let’s try out the new flake. + +.... +$ nix run +.... + +Why can’t it find `flake.nix`? Nix flakes only ``see'' files that are +part of the repository. We need to add all of the important files to the +repo before building or running the flake. + +.... +$ git add flake.nix setup.py hello.py +$ nix run +.... + +We’d like to share this package with others, but first we should do some +cleanup. When the package was built (automatically by the `nix run` +command), it created a `flake.lock` file. We need to add this to the +repo, and commit all important files. + +.... +$ git add flake.lock +$ git commit -a -m 'initial commit' +.... + +You can test that your package is properly configured by going to +another directory and running it from there. + +.... +$ cd ~ +$ nix run ~/tutorial-practice/hello-python +.... + +If you move the project to a public repo, anyone can run it. Recall from +the beginning of the tutorial that you were able to run `hello-flake` +directly from my repo with the following command. + +.... +nix run "git+https://codeberg.org/mhwombat/hello-flake" +.... + +Modify the URL accordingly and invite someone else to run your new +Python flake. diff --git a/source/shebangs/_include.adoc0 b/source/shebangs/_include.adoc0 new file mode 100644 index 0000000..2a4067f --- /dev/null +++ b/source/shebangs/_include.adoc0 @@ -0,0 +1,40 @@ +== Nix-shell shebangs + +You can use `nix-shell` to run scripts in arbitrary languages, providing +the necessary dependencies. This is particularly convenient for +standalone scripts because you don’t need to create a repo and write a +separate `flake.nix`. + +The script should start with two ``shebang'' (`#!`) commands. The first +should invoke `nix-shell`. The second should declares the scrpt +interpreter and any dependencies. Here are some examples. + +=== A Bash script depending on a package in the nixpkgs repo. + +Script: + +[source,bash,linenums] +.... +include::bash-with-nixpkg.sh[] +.... + +Output: + +.... +$# shebangs/bash-with-nixpkg.sh +.... + +=== A Python script depending on a package in the nixpkgs repo. + +Script: + +[source,bash,linenums] +.... +include::python-with-nixpkg.sh[] +.... + +Output: + +.... +$# shebangs/python-with-nixpkg.sh +.... diff --git a/source/shell-recipes/_include.adoc0 b/source/shell-recipes/_include.adoc0 new file mode 100644 index 0000000..8fa1530 --- /dev/null +++ b/source/shell-recipes/_include.adoc0 @@ -0,0 +1,112 @@ +== Nix shell recipes + +=== Shell with access to a package from the Nixpkgs/NixOS repo + +This shell provides access to two packages from nixpkgs: hello and +cowsay. + +[source,nix,linenums] +.... +include::0100-shell-with-nixpkgs.nix[] +.... + +Here’s a demonstration using the shell. + +.... +$ nix-shell +$ hello +Hello, world! +$ cowsay "moo" + _____ +< moo > + ----- + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || +.... + +The command-line equivalent would be `nix-shell -p hello cowsay` + +=== Shell with access to a package defined in a remote git repo + +[source,nix,linenums] +.... +include::0150-shell-with-git-nix-pkg.nix[] +.... + +Here’s a demonstration using the shell. + +.... +$ nix-shell +$ hello-nix +Hello from your nix package! +.... + +=== Shell with access to a flake + +[source,nix,linenums] +.... +include::0200-shell-with-flake.nix[] +.... + +Here’s a demonstration using the shell. + +.... +$ nix-shell +$ hello-flake +Hello from your flake! +.... + +=== Shell with access to a specific revision of a flake + +[source,nix,linenums] +.... +include::0250-shell-with-flake-rev.nix[] +.... + +Here’s a demonstration using the shell. + +.... +$ nix-shell +$ hello-flake +Hello from your flake! +.... + +=== Shell with access to a Haskell package on your local computer + +This shell provides access to three Haskell packages that are on my hard +drive. + +[source,nix,linenums] +.... +include::0300-shell-haskell-local.nix[] +.... + +=== Shell with access to a Haskell package on your local computer, with interdependencies + +This shell provides access to four Haskell packages that are on my hard +drive. The fourth package depends on the first three to build. + +[source,nix,linenums] +.... +include::0350-shell-haskell-local-deps.nix[] +.... + +=== Shell with an environment variable set + +This shell has the environment variable FOO set to ``bar'' + +[source,nix,linenums] +.... +include::0400-shell-with-env-var.nix[] +.... + +Here’s a demonstration using the shell. + +.... +$ nix-shell +$ echo $FOO +bar +....