dotfiles/nixos-config
1
0
Fork 0
mirror of https://github.com/srid/nixos-config.git synced 2026-07-16 22:01:33 +08:00
Code Issues Projects Releases Packages Wiki Activity Actions

incus-pet: shared /nix/store, restart-to-deploy, drop anywhen flake input

Browse source 
The incus-pet CLI gains three big changes that together cut per-container
disk usage by ~5GB, drop nix-rebuild + nix-copy from the deploy path, and
make `deploy` actually pull latest on every invocation. anywhen migrates
from a host-installed service to an incus-pet container in the process.

* Shared /nix/store mount. Each container gets host's /nix/store
  bind-mounted read-only (with shift=true for idmapped unprivileged
  containers). Zero per-container duplication of nixpkgs closures. The
  CLI builds toplevels on the host; the container sees them via the
  shared mount, no `nix copy` needed.

* Restart-to-deploy. switch-to-configuration is removed entirely (it
  was stalling on dbus-broker reload and hitting the same lock-
  contention bug across redeploys). New activation: update
  /nix/var/nix/profiles/system, repoint /sbin/init through the profile
  symlink, `incus restart`. The new toplevel's own init script handles
  /etc setup on boot. Trade-off: ~5s of container reboot per deploy.

* Marshaling flake exposes only nixosModules.default. No nixpkgs
  input, no nixosConfigurations entry. The CLI picks nixpkgs at build
  time (pinned to nixos-25.11 — matches the LXC image baseline, dodges
  unstable regressions) and assembles the full nixosSystem via
  `nix build --impure --expr`. Lets us bump nixpkgs without touching
  per-deploy state.

* `nix flake update --refresh` on every deploy. Was `nix flake lock`,
  which only locks first-seen inputs — re-deploying `github:srid/anywhen`
  stayed pinned at the first commit ever resolved. Now every deploy
  bumps the target input to the current tip of its ref and bypasses
  nix's fetcher cache.

* /sbin/init repoint. The NixOS LXC image hard-links /sbin/init to its
  own toplevel's init path. Without switch-to-configuration, `incus
  restart` re-execs that old init, which sets up /etc from the old
  toplevel — running services keep using the OLD anywhen binary even
  though the profile pointer moved. CLI now also does
  `ln -sfn /nix/var/nix/profiles/system/init /sbin/init` so the profile
  actually drives boot. Idempotent.

* Host gcroot via `nix-store --add-root --indirect`. Container's nix DB
  doesn't know about the path (it lives behind the shared mount), so
  without a host-side gcroot the host could nix-collect-garbage the
  running system out from under the container. Symlink at
  ~/.local/state/incus-pet/<name>/system. Only the LATEST toplevel is
  pinned per app — no rollback today (file a 5-line ring buffer when
  rollback matters).

* anywhen flake input removed. `modules/nixos/linux/anywhen.nix` is
  gone; the host config no longer imports anywhen as a flake input.
  The deploy command takes the flake ref as an argument, so locking it
  here was just bloat. `just pureintent anywhen-deploy` runs the
  current invocation.

* pureintent ops recipes. New `configurations/nixos/pureintent/mod.just`
  with anywhen-{deploy,rm,backup,restore,status,shell}. Namespaced via
  `mod pureintent` so each host's recipes live behind their own prefix.

* Drop ssh from deploy path. Bootstrap still pushes the operator's
  pubkey (for interactive debugging), but activation goes through
  `incus exec` exclusively. No NIX_SSHOPTS, no nix-copy-via-ssh,
  no host-key TOFU dance during deploys.

Tested end-to-end on pureintent: anywhen migrated from host-installed
(21-task SQLite DB at /var/lib/anywhen) to an incus-pet container with
the DB intact; hello-web deployed fresh with shared /nix/store; both
serve over the tailscale proxy at 100.122.32.106:{6111,8081}; lock
bumps + new toplevels confirmed after master advances.
This commit is contained in:
Sridhar Ratnakumar 2026-05-23 18:50:10 -04:00
parent 5997c1c848
commit 776c4425b0
9 changed files with 271 additions and 249 deletions
Download patch file Download diff file Expand all files Collapse all files

5
configurations/nixos/pureintent/default.nix
View file

@ -28,10 +28,11 @@ in
./devbox.nix ./devbox.nix
(self + /modules/nixos/linux/beszel.nix) (self + /modules/nixos/linux/beszel.nix)
(self + /modules/nixos/linux/incus) (self + /modules/nixos/linux/incus)
(self + /modules/nixos/linux/anywhen.nix)
]; ];
services.anywhen.host = "100.122.32.106"; # Tailscale IP of pureintent # anywhen runs as an incus-pet container (see modules/nixos/linux/incus/incus-pet).
# Deployed with:
# incus-pet deploy github:srid/anywhen --port 6111 --listen 100.122.32.106
# Expose the incus UI on the Tailscale interface only. # Expose the incus UI on the Tailscale interface only.
virtualisation.incus.preseed.config."core.https_address" = "100.122.32.106:8443"; virtualisation.incus.preseed.config."core.https_address" = "100.122.32.106:8443";

55
configurations/nixos/pureintent/mod.just Normal file
View file

@ -0,0 +1,55 @@
# Pureintent-specific operational recipes.
#
# Imported from the top-level Justfile. Each recipe bakes in the values
# that are stable on this host (tailscale IP, container names, port
# choices) so the day-to-day operator just types `just anywhen-deploy`,
# not the full incus-pet invocation.
LISTEN_IP := "100.122.32.106" # pureintent's tailscale IP
# ----------------------------------------------------------------------
# anywhen — github:srid/anywhen as a per-app incus-pet container.
# ----------------------------------------------------------------------
# Deploy anywhen (default: master); pass `ref` to deploy a branch/PR.
[group('anywhen')]
anywhen-deploy ref="github:srid/anywhen":
nix run .#incus-pet -- deploy {{ ref }} anywhen --port 6111 --listen {{ LISTEN_IP }}
# Drop the anywhen container entirely (state goes with it — back up first).
[group('anywhen')]
anywhen-rm:
nix run .#incus-pet -- rm anywhen
# Pull a WAL-checkpointed snapshot of the live SQLite DB to PATH.
[group('anywhen')]
anywhen-backup path="/tmp/anywhen.db":
incus exec anywhen -- systemctl stop anywhen.service
incus exec anywhen -- sh -c 'cd /var/lib/private/anywhen && sqlite3 anywhen.db "PRAGMA wal_checkpoint(TRUNCATE);" 2>/dev/null || true'
incus file pull anywhen/var/lib/private/anywhen/anywhen.db {{ path }}
incus exec anywhen -- systemctl start anywhen.service
@echo "backup -> {{ path }}"
# Push a DB file into the anywhen container (stop/copy/chown/wipe-wal/start).
[group('anywhen')]
anywhen-restore db:
incus exec anywhen -- systemctl stop anywhen.service
incus file push {{ db }} anywhen/var/lib/private/anywhen/anywhen.db --mode 644
incus exec anywhen -- sh -c 'chown $(stat -c "%u:%g" /var/lib/private/anywhen) /var/lib/private/anywhen/anywhen.db'
incus exec anywhen -- rm -f /var/lib/private/anywhen/anywhen.db-wal /var/lib/private/anywhen/anywhen.db-shm
incus exec anywhen -- systemctl start anywhen.service
@echo "restored {{ db }} -> anywhen.service active"
# Health check (curl /api/health, in-container service status, full list).
[group('anywhen')]
anywhen-status:
@echo "--- tailnet ---"
@curl -sS http://{{ LISTEN_IP }}:6111/api/health && echo
@echo "--- container ---"
@incus exec anywhen -- systemctl is-active anywhen.service
@nix run .#incus-pet -- list
# Drop into a root shell inside the anywhen container.
[group('anywhen')]
anywhen-shell:
incus exec anywhen -- bash

212
flake.lock generated
View file

@ -43,25 +43,6 @@
"type": "github" "type": "github"
} }
}, },
"anywhen": {
"inputs": {
"bun2nix": "bun2nix"
},
"locked": {
"lastModified": 1779558041,
"narHash": "sha256-qJcYSDxUfKmg7lDE68rooQs/ZEdeDYsX0DSLXG7HmXE=",
"owner": "srid",
"repo": "anywhen",
"rev": "7cb4b823332e0dce7eaf141117894819ed19d367",
"type": "github"
},
"original": {
"owner": "srid",
"ref": "abject-turn",
"repo": "anywhen",
"type": "github"
}
},
"blueprint": { "blueprint": {
"inputs": { "inputs": {
"nixpkgs": [ "nixpkgs": [
@ -88,35 +69,12 @@
} }
}, },
"bun2nix": { "bun2nix": {
"inputs": {
"flake-parts": "flake-parts",
"import-tree": "import-tree",
"nixpkgs": "nixpkgs",
"systems": "systems_2",
"treefmt-nix": "treefmt-nix"
},
"locked": {
"lastModified": 1779553062,
"narHash": "sha256-T5cVey80VVqPIkZXHX8kaIxhrqGKw97Nma6ufTfy/dY=",
"owner": "juspay",
"repo": "bun2nix",
"rev": "948386f5219e4526a2978110dacffb433cef1013",
"type": "github"
},
"original": {
"owner": "juspay",
"ref": "rawflake",
"repo": "bun2nix",
"type": "github"
}
},
"bun2nix_2": {
"inputs": { "inputs": {
"flake-parts": [ "flake-parts": [
"llm-agents", "llm-agents",
"flake-parts" "flake-parts"
], ],
"import-tree": "import-tree_2", "import-tree": "import-tree",
"nixpkgs": [ "nixpkgs": [
"llm-agents", "llm-agents",
"nixpkgs" "nixpkgs"
@ -259,11 +217,11 @@
}, },
"disc-scrape": { "disc-scrape": {
"inputs": { "inputs": {
"flake-parts": "flake-parts_2", "flake-parts": "flake-parts",
"git-hooks": "git-hooks", "git-hooks": "git-hooks",
"nixpkgs": "nixpkgs_2", "nixpkgs": "nixpkgs",
"rust-flake": "rust-flake", "rust-flake": "rust-flake",
"systems": "systems_3" "systems": "systems_2"
}, },
"locked": { "locked": {
"lastModified": 1772235986, "lastModified": 1772235986,
@ -322,14 +280,14 @@
"commonmark-wikilink": "commonmark-wikilink", "commonmark-wikilink": "commonmark-wikilink",
"ema": "ema", "ema": "ema",
"emanote-template": "emanote-template", "emanote-template": "emanote-template",
"flake-parts": "flake-parts_3", "flake-parts": "flake-parts_2",
"fourmolu-nix": "fourmolu-nix", "fourmolu-nix": "fourmolu-nix",
"git-hooks": "git-hooks_2", "git-hooks": "git-hooks_2",
"haskell-flake": "haskell-flake", "haskell-flake": "haskell-flake",
"heist-extra": "heist-extra", "heist-extra": "heist-extra",
"lvar": "lvar", "lvar": "lvar",
"nixos-unified": "nixos-unified", "nixos-unified": "nixos-unified",
"nixpkgs": "nixpkgs_3", "nixpkgs": "nixpkgs_2",
"unionmount": "unionmount" "unionmount": "unionmount"
}, },
"locked": { "locked": {
@ -363,24 +321,6 @@
} }
}, },
"flake-parts": { "flake-parts": {
"inputs": {
"nixpkgs-lib": "nixpkgs-lib"
},
"locked": {
"lastModified": 1769996383,
"narHash": "sha256-AnYjnFWgS49RlqX7LrC4uA+sCCDBj0Ry/WOJ5XWAsa0=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "57928607ea566b5db3ad13af0e57e921e6b12381",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "flake-parts",
"type": "github"
}
},
"flake-parts_2": {
"inputs": { "inputs": {
"nixpkgs-lib": [ "nixpkgs-lib": [
"disc-scrape", "disc-scrape",
@ -401,7 +341,7 @@
"type": "github" "type": "github"
} }
}, },
"flake-parts_3": { "flake-parts_2": {
"inputs": { "inputs": {
"nixpkgs-lib": [ "nixpkgs-lib": [
"emanote", "emanote",
@ -422,9 +362,9 @@
"type": "github" "type": "github"
} }
}, },
"flake-parts_4": { "flake-parts_3": {
"inputs": { "inputs": {
"nixpkgs-lib": "nixpkgs-lib_2" "nixpkgs-lib": "nixpkgs-lib"
}, },
"locked": { "locked": {
"lastModified": 1772408722, "lastModified": 1772408722,
@ -440,7 +380,7 @@
"type": "github" "type": "github"
} }
}, },
"flake-parts_5": { "flake-parts_4": {
"inputs": { "inputs": {
"nixpkgs-lib": [ "nixpkgs-lib": [
"imako", "imako",
@ -461,7 +401,7 @@
"type": "github" "type": "github"
} }
}, },
"flake-parts_6": { "flake-parts_5": {
"inputs": { "inputs": {
"nixpkgs-lib": [ "nixpkgs-lib": [
"llm-agents", "llm-agents",
@ -482,7 +422,7 @@
"type": "github" "type": "github"
} }
}, },
"flake-parts_7": { "flake-parts_6": {
"inputs": { "inputs": {
"nixpkgs-lib": [ "nixpkgs-lib": [
"vira", "vira",
@ -505,7 +445,7 @@
}, },
"flake-utils": { "flake-utils": {
"inputs": { "inputs": {
"systems": "systems_5" "systems": "systems_4"
}, },
"locked": { "locked": {
"lastModified": 1731533236, "lastModified": 1731533236,
@ -797,12 +737,12 @@
"aeson-typescript": "aeson-typescript", "aeson-typescript": "aeson-typescript",
"commonmark-simple": "commonmark-simple_2", "commonmark-simple": "commonmark-simple_2",
"commonmark-wikilink": "commonmark-wikilink_2", "commonmark-wikilink": "commonmark-wikilink_2",
"flake-parts": "flake-parts_5", "flake-parts": "flake-parts_4",
"fourmolu-nix": "fourmolu-nix_2", "fourmolu-nix": "fourmolu-nix_2",
"git-hooks": "git-hooks_4", "git-hooks": "git-hooks_4",
"haskell-flake": "haskell-flake_2", "haskell-flake": "haskell-flake_2",
"lvar": "lvar_2", "lvar": "lvar_2",
"nixpkgs": "nixpkgs_4", "nixpkgs": "nixpkgs_3",
"process-compose-flake": "process-compose-flake", "process-compose-flake": "process-compose-flake",
"unionmount": "unionmount_2" "unionmount": "unionmount_2"
}, },
@ -835,21 +775,6 @@
"type": "github" "type": "github"
} }
}, },
"import-tree_2": {
"locked": {
"lastModified": 1763762820,
"narHash": "sha256-ZvYKbFib3AEwiNMLsejb/CWs/OL/srFQ8AogkebEPF0=",
"owner": "vic",
"repo": "import-tree",
"rev": "3c23749d8013ec6daa1d7255057590e9ca726646",
"type": "github"
},
"original": {
"owner": "vic",
"repo": "import-tree",
"type": "github"
}
},
"jumphost-nix": { "jumphost-nix": {
"flake": false, "flake": false,
"locked": { "locked": {
@ -901,13 +826,13 @@
"llm-agents": { "llm-agents": {
"inputs": { "inputs": {
"blueprint": "blueprint", "blueprint": "blueprint",
"bun2nix": "bun2nix_2", "bun2nix": "bun2nix",
"flake-parts": "flake-parts_6", "flake-parts": "flake-parts_5",
"nixpkgs": [ "nixpkgs": [
"nixpkgs" "nixpkgs"
], ],
"systems": "systems_4", "systems": "systems_3",
"treefmt-nix": "treefmt-nix_2" "treefmt-nix": "treefmt-nix"
}, },
"locked": { "locked": {
"lastModified": 1775790657, "lastModified": 1775790657,
@ -1107,36 +1032,21 @@
}, },
"nixpkgs": { "nixpkgs": {
"locked": { "locked": {
"lastModified": 1770562336, "lastModified": 1772082373,
"narHash": "sha256-ub1gpAONMFsT/GU2hV6ZWJjur8rJ6kKxdm9IlCT0j84=", "narHash": "sha256-wySf8a6hvuqgFdwvvzPPTARBCMLDz7WFAufGkllD1M4=",
"owner": "NixOS", "owner": "nixos",
"repo": "nixpkgs", "repo": "nixpkgs",
"rev": "d6c71932130818840fc8fe9509cf50be8c64634f", "rev": "26eaeac4e409d7b5a6bf6f90a2a2dc223c78d915",
"type": "github" "type": "github"
}, },
"original": { "original": {
"owner": "NixOS", "owner": "nixos",
"ref": "nixos-unstable", "ref": "nixpkgs-unstable",
"repo": "nixpkgs", "repo": "nixpkgs",
"type": "github" "type": "github"
} }
}, },
"nixpkgs-lib": { "nixpkgs-lib": {
"locked": {
"lastModified": 1769909678,
"narHash": "sha256-cBEymOf4/o3FD5AZnzC3J9hLbiZ+QDT/KDuyHXVJOpM=",
"owner": "nix-community",
"repo": "nixpkgs.lib",
"rev": "72716169fe93074c333e8d0173151350670b824c",
"type": "github"
},
"original": {
"owner": "nix-community",
"repo": "nixpkgs.lib",
"type": "github"
}
},
"nixpkgs-lib_2": {
"locked": { "locked": {
"lastModified": 1772328832, "lastModified": 1772328832,
"narHash": "sha256-e+/T/pmEkLP6BHhYjx6GmwP5ivonQQn0bJdH9YrRB+Q=", "narHash": "sha256-e+/T/pmEkLP6BHhYjx6GmwP5ivonQQn0bJdH9YrRB+Q=",
@ -1152,22 +1062,6 @@
} }
}, },
"nixpkgs_2": { "nixpkgs_2": {
"locked": {
"lastModified": 1772082373,
"narHash": "sha256-wySf8a6hvuqgFdwvvzPPTARBCMLDz7WFAufGkllD1M4=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "26eaeac4e409d7b5a6bf6f90a2a2dc223c78d915",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixpkgs-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_3": {
"locked": { "locked": {
"lastModified": 1752900028, "lastModified": 1752900028,
"narHash": "sha256-dPALCtmik9Wr14MGqVXm+OQcv7vhPBXcWNIOThGnB/Q=", "narHash": "sha256-dPALCtmik9Wr14MGqVXm+OQcv7vhPBXcWNIOThGnB/Q=",
@ -1183,7 +1077,7 @@
"type": "github" "type": "github"
} }
}, },
"nixpkgs_4": { "nixpkgs_3": {
"locked": { "locked": {
"lastModified": 1770169770, "lastModified": 1770169770,
"narHash": "sha256-awR8qIwJxJJiOmcEGgP2KUqYmHG4v/z8XpL9z8FnT1A=", "narHash": "sha256-awR8qIwJxJJiOmcEGgP2KUqYmHG4v/z8XpL9z8FnT1A=",
@ -1199,7 +1093,7 @@
"type": "github" "type": "github"
} }
}, },
"nixpkgs_5": { "nixpkgs_4": {
"locked": { "locked": {
"lastModified": 1778443072, "lastModified": 1778443072,
"narHash": "sha256-zi7/fsqM/kFdNuED//4WOCUtezGtKKqRNORjMvfwjnA=", "narHash": "sha256-zi7/fsqM/kFdNuED//4WOCUtezGtKKqRNORjMvfwjnA=",
@ -1215,7 +1109,7 @@
"type": "github" "type": "github"
} }
}, },
"nixpkgs_6": { "nixpkgs_5": {
"locked": { "locked": {
"lastModified": 1765803225, "lastModified": 1765803225,
"narHash": "sha256-xwaZV/UgJ04+ixbZZfoDE8IsOWjtvQZICh9aamzPnrg=", "narHash": "sha256-xwaZV/UgJ04+ixbZZfoDE8IsOWjtvQZICh9aamzPnrg=",
@ -1231,7 +1125,7 @@
"type": "github" "type": "github"
} }
}, },
"nixpkgs_7": { "nixpkgs_6": {
"locked": { "locked": {
"lastModified": 1764635402, "lastModified": 1764635402,
"narHash": "sha256-6rYcajRLe2C5ZYnV1HYskJl+QAkhvseWTzbdQiTN9OI=", "narHash": "sha256-6rYcajRLe2C5ZYnV1HYskJl+QAkhvseWTzbdQiTN9OI=",
@ -1315,11 +1209,10 @@
"root": { "root": {
"inputs": { "inputs": {
"agenix": "agenix", "agenix": "agenix",
"anywhen": "anywhen",
"disc-scrape": "disc-scrape", "disc-scrape": "disc-scrape",
"disko": "disko", "disko": "disko",
"emanote": "emanote", "emanote": "emanote",
"flake-parts": "flake-parts_4", "flake-parts": "flake-parts_3",
"git-hooks": "git-hooks_3", "git-hooks": "git-hooks_3",
"github-nix-ci": "github-nix-ci", "github-nix-ci": "github-nix-ci",
"home-manager": "home-manager", "home-manager": "home-manager",
@ -1333,7 +1226,7 @@
"nixos-hardware": "nixos-hardware", "nixos-hardware": "nixos-hardware",
"nixos-unified": "nixos-unified_2", "nixos-unified": "nixos-unified_2",
"nixos-vscode-server": "nixos-vscode-server", "nixos-vscode-server": "nixos-vscode-server",
"nixpkgs": "nixpkgs_5", "nixpkgs": "nixpkgs_4",
"project-unknown": "project-unknown", "project-unknown": "project-unknown",
"vira": "vira", "vira": "vira",
"zmx": "zmx" "zmx": "zmx"
@ -1460,21 +1353,6 @@
"type": "github" "type": "github"
} }
}, },
"systems_5": {
"locked": {
"lastModified": 1681028828,
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
"owner": "nix-systems",
"repo": "default",
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "default",
"type": "github"
}
},
"tabler-icons-hs": { "tabler-icons-hs": {
"flake": false, "flake": false,
"locked": { "locked": {
@ -1492,28 +1370,6 @@
} }
}, },
"treefmt-nix": { "treefmt-nix": {
"inputs": {
"nixpkgs": [
"anywhen",
"bun2nix",
"nixpkgs"
]
},
"locked": {
"lastModified": 1770228511,
"narHash": "sha256-wQ6NJSuFqAEmIg2VMnLdCnUc0b7vslUohqqGGD+Fyxk=",
"owner": "numtide",
"repo": "treefmt-nix",
"rev": "337a4fe074be1042a35086f15481d763b8ddc0e7",
"type": "github"
},
"original": {
"owner": "numtide",
"repo": "treefmt-nix",
"type": "github"
}
},
"treefmt-nix_2": {
"inputs": { "inputs": {
"nixpkgs": [ "nixpkgs": [
"llm-agents", "llm-agents",
@ -1571,7 +1427,7 @@
"inputs": { "inputs": {
"co-log-effectful": "co-log-effectful", "co-log-effectful": "co-log-effectful",
"devour-flake": "devour-flake", "devour-flake": "devour-flake",
"flake-parts": "flake-parts_7", "flake-parts": "flake-parts_6",
"fourmolu-nix": "fourmolu-nix_3", "fourmolu-nix": "fourmolu-nix_3",
"git-hooks": "git-hooks_5", "git-hooks": "git-hooks_5",
"haskell-flake": "haskell-flake_3", "haskell-flake": "haskell-flake_3",
@ -1581,7 +1437,7 @@
"nix-serve-ng": "nix-serve-ng", "nix-serve-ng": "nix-serve-ng",
"nix-systems": "nix-systems", "nix-systems": "nix-systems",
"nixos-unified": "nixos-unified_3", "nixos-unified": "nixos-unified_3",
"nixpkgs": "nixpkgs_6", "nixpkgs": "nixpkgs_5",
"process-compose-flake": "process-compose-flake_2", "process-compose-flake": "process-compose-flake_2",
"record-hasfield": "record-hasfield", "record-hasfield": "record-hasfield",
"servant-event-stream": "servant-event-stream", "servant-event-stream": "servant-event-stream",
@ -1621,7 +1477,7 @@
"zig2nix": { "zig2nix": {
"inputs": { "inputs": {
"flake-utils": "flake-utils", "flake-utils": "flake-utils",
"nixpkgs": "nixpkgs_7" "nixpkgs": "nixpkgs_6"
}, },
"locked": { "locked": {
"lastModified": 1764678235, "lastModified": 1764678235,

7
flake.nix
View file

@ -38,8 +38,11 @@
# KOLU # KOLU
kolu.url = "github:juspay/kolu/feat/dock-activity-window"; kolu.url = "github:juspay/kolu/feat/dock-activity-window";
# anywhen — personal task manager (PR #2 adds NixOS module) # anywhen is NOT a flake input — it's deployed as an incus-pet
anywhen.url = "github:srid/anywhen/abject-turn"; # container, with the flake ref passed at deploy time (see
# `just pureintent anywhen-deploy`). The host config doesn't import
# anything from anywhen, so locking it here would just bloat
# flake.lock without buying us anything.
project-unknown.url = "github:juspay/project-unknown"; project-unknown.url = "github:juspay/project-unknown";
project-unknown.inputs.nixpkgs.follows = "nixpkgs"; project-unknown.inputs.nixpkgs.follows = "nixpkgs";

4
justfile
View file

@ -1,6 +1,10 @@
default: default:
@just --list @just --list
# Per-host operational recipes. Namespaced via `mod` so each host's
# recipes live behind their own prefix (e.g. `just pureintent anywhen-deploy`).
mod pureintent 'configurations/nixos/pureintent/mod.just'
# Main commands # Main commands
# -------------------------------------------------------------------------------------------------- # --------------------------------------------------------------------------------------------------

16
modules/nixos/linux/anywhen.nix
View file

@ -1,16 +0,0 @@
{ flake, pkgs, ... }:
let
inherit (flake) inputs;
in
{
imports = [
inputs.anywhen.nixosModules.default
];
services.anywhen = {
enable = true;
package = inputs.anywhen.packages.${pkgs.stdenv.hostPlatform.system}.default;
port = 6111;
};
}

33
modules/nixos/linux/incus/incus-pet/README.md
View file

@ -117,13 +117,22 @@ incus-pet deploy github:srid/anywhen --port 7700 --listen 100.122.32.106
This: This:
1. Synthesises a marshaling flake under `~/.local/state/incus-pet/anywhen/`. 1. Synthesises a marshaling flake under `~/.local/state/incus-pet/anywhen/`
2. Launches `images:nixos/25.11` as container `anywhen` (with exposing `nixosModules.default` (essentials + app's
`security.nesting=true` so `nixos-rebuild` works inside). `nixosModules.incus` + operator pubkey overlay) — no nixpkgs pin in
3. Bootstraps sshd + your pubkey via `incus exec`. per-deploy state.
4. Runs `nixos-rebuild switch --target-host root@<container-ip>`. 2. Launches `images:nixos/25.11` as container `anywhen` with
5. Records `--port` and `--listen` in container metadata. `-c security.nesting=true`.
6. Adds the `web` proxy device. 3. Pushes your `~/.ssh/id_ed25519.pub` into the container's
`/root/.ssh/authorized_keys` (the image already runs sshd via systemd
socket activation).
4. Builds the system toplevel against `github:nixos/nixpkgs/nixos-25.11`
(stable, matching the image baseline) via `nix build --impure --expr`.
5. Copies the closure with `nix copy --to ssh://root@<container-ip>
--substitute-on-destination`.
6. SSHes to set the system profile and run `switch-to-configuration switch`.
7. Records `--port` and `--listen` in container metadata.
8. Adds the `web` proxy device.
### Subsequent deploys ### Subsequent deploys
@ -165,9 +174,13 @@ container.
- **`Failed to start Firewall.` inside the container** — the launch - **`Failed to start Firewall.` inside the container** — the launch
flag `-c security.nesting=true` is supposed to handle this; if it flag `-c security.nesting=true` is supposed to handle this; if it
recurs, see `../README.md` and lxc/incus#526. recurs, see `../README.md` and lxc/incus#526.
- **SSH fails after first deploy** — the bootstrap step writes - **`switch-to-configuration` exits 4** — "some units failed to
`/etc/nixos/incus-pet-bootstrap.nix` and runs `nixos-rebuild` inside reload/restart". Treated as success: the activation itself
the container; if that fails, `incus exec <name> -- journalctl -xeu nixos-rebuild` is the place to look. completed, only a unit reload (typically dbus-broker) failed. The
service is up.
- **`switch-to-configuration` exits 11 / "Could not acquire lock"** —
a previous activation is still running (often hung). `incus exec
<name> -- pkill -9 -f switch-to-configuration` clears it.
## State ## State

10
modules/nixos/linux/incus/incus-pet/SKILL.md
View file

@ -131,6 +131,10 @@ The module must NOT set:
filters by the `flake-ref` key. filters by the `flake-ref` key.
- If `--port` is missing on the first deploy, the command fails before - If `--port` is missing on the first deploy, the command fails before
launching anything. launching anything.
- Network failures during `nixos-rebuild --target-host` leave the - Activation is two ssh steps: `nix-env --set` (atomic pointer swap on
container running its previous generation — `nixos-rebuild switch` is `/nix/var/nix/profiles/system`) then `switch-to-configuration switch`.
itself atomic. If the second fails partway, the system profile is already at the new
toplevel; running deploy again finishes the switch.
- Exit 4 from `switch-to-configuration` is treated as success (a unit
failed to reload — typically dbus-broker — but activation itself
completed).

178
modules/nixos/linux/incus/incus-pet/default.nix
View file

@ -6,9 +6,19 @@
# host-side <listen-ip>:<host-port> is unique per app and stored in # host-side <listen-ip>:<host-port> is unique per app and stored in
# container metadata so re-deploys are flagless. See ./README.md for # container metadata so re-deploys are flagless. See ./README.md for
# the network model and operator flow. # the network model and operator flow.
#
# Marshaling-flake-as-module design (vs. nixos-rebuild --target-host):
# the per-deploy state-dir flake exposes ONLY `nixosModules.default`
# (combined container essentials + app's nixosModules.incus +
# operator-key overlay). The CLI picks nixpkgs at build time (pinned
# to nixos-25.11, matching the LXC image baseline) and assembles the
# nixosSystem via `nix build --impure --expr`. Closure copy + activation
# happen via `nix copy --to ssh://…` + ssh-driven
# switch-to-configuration. Lets us dodge unstable nixpkgs regressions
# (e.g. the dbus-broker reload stall) without touching the marshaling
# flake's shape.
{ writeShellApplication { writeShellApplication
, nix , nix
, nixos-rebuild
, openssh , openssh
, jq , jq
, coreutils , coreutils
@ -24,7 +34,7 @@ in
writeShellApplication { writeShellApplication {
name = "incus-pet"; name = "incus-pet";
runtimeInputs = [ nix nixos-rebuild openssh jq coreutils gnused ]; runtimeInputs = [ nix openssh jq coreutils gnused ];
meta.description = "Deploy a NixOS service flake into a per-app incus container (pet PaaS)."; meta.description = "Deploy a NixOS service flake into a per-app incus container (pet PaaS).";
@ -35,6 +45,10 @@ writeShellApplication {
CONTAINER_PORT=8080 CONTAINER_PORT=8080
STATE_ROOT="''${XDG_STATE_HOME:-$HOME/.local/state}/incus-pet" STATE_ROOT="''${XDG_STATE_HOME:-$HOME/.local/state}/incus-pet"
INCUS_IMAGE="images:nixos/25.11" INCUS_IMAGE="images:nixos/25.11"
# Pinned to stable to match the LXC image baseline. Unstable triggers
# a dbus-broker reload stall in switch-to-configuration (~Jul 2026
# systemd/dbus-broker regression).
NIXPKGS_REF="github:nixos/nixpkgs/nixos-25.11"
log() { printf '[incus-pet] %s\n' "$*" >&2; } log() { printf '[incus-pet] %s\n' "$*" >&2; }
die() { log "error: $*"; exit 1; } die() { log "error: $*"; exit 1; }
@ -97,8 +111,12 @@ writeShellApplication {
} }
write_marshaling_flake() { write_marshaling_flake() {
local name="$1" flake_ref="$2" pubkey="$3" # Per-deploy state-dir flake. Exposes ONLY nixosModules.default —
local dir="$STATE_ROOT/$name" # the CLI assembles the full nixosSystem at build time, picking
# nixpkgs (so we can pin stable to dodge unstable regressions and
# not bake the choice into per-deploy state).
local flake_ref="$1" pubkey="$2"
local dir="$STATE_ROOT/$3" # caller-chosen subdir (probe or real name)
mkdir -p "$dir" mkdir -p "$dir"
# Copy the essentials module into the marshaling flake dir as # Copy the essentials module into the marshaling flake dir as
@ -110,19 +128,15 @@ writeShellApplication {
cat > "$dir/flake.nix" <<EOF cat > "$dir/flake.nix" <<EOF
# Generated by incus-pet — do not edit. Re-run \`incus-pet deploy\` to refresh. # Generated by incus-pet — do not edit. Re-run \`incus-pet deploy\` to refresh.
{ {
inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable"; inputs.target.url = "$flake_ref";
inputs.target.url = "$flake_ref"; outputs = { target, ... }: {
outputs = { nixpkgs, target, ... }: { nixosModules.default = { ... }: {
nixosConfigurations."$name" = nixpkgs.lib.nixosSystem { imports = [
system = "x86_64-linux";
modules = [
./container.nix ./container.nix
target.nixosModules.incus target.nixosModules.incus
({ ... }: { ];
users.users.root.openssh.authorizedKeys.keys = [ users.users.root.openssh.authorizedKeys.keys = [
"$pubkey" "$pubkey"
];
})
]; ];
}; };
}; };
@ -138,17 +152,40 @@ writeShellApplication {
fi fi
} }
eval_hostname() { # Boilerplate that assembles the full nixosSystem on demand. Pinned
local dir="$1" name="$2" # nixpkgs (NIXPKGS_REF) is fetched via builtins.getFlake; --impure
nix eval --raw \ # is required because builtins.getFlake on a URL isn't
"$dir#nixosConfigurations.\"$name\".config.incus.container.hostname" # pure-evaluable. The marshaling flake is loaded by absolute store
# path so the operator's machine doesn't re-fetch it.
sys_expr() {
local dir="$1"
printf 'let
marshaling = builtins.getFlake "path:%s";
nixpkgs = builtins.getFlake "%s";
in nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [ marshaling.nixosModules.default ];
}' "$dir" "$NIXPKGS_REF"
}
eval_config_attr() {
local dir="$1" attr="$2"
nix eval --raw --impure --expr "($(sys_expr "$dir")).config.$attr"
}
build_toplevel() {
local dir="$1"
nix build --print-out-paths --no-link --impure --expr \
"($(sys_expr "$dir")).config.system.build.toplevel"
} }
bootstrap_container() { bootstrap_container() {
# The official NixOS incus image already runs sshd via systemd # The official NixOS incus image already runs sshd via systemd
# socket activation on port 22; the only thing we need to add is # socket activation on port 22; the only thing we need to add is
# the operator's pubkey to root's authorized_keys. Idempotent — # the operator's pubkey to root's authorized_keys (for interactive
# `tee` overwrites, so a rotated key gets picked up on next deploy. # ssh debugging — the deploy path uses `incus exec`, not ssh).
# Idempotent — `tee` overwrites, so a rotated key gets picked up
# on next deploy.
local name="$1" pubkey="$2" local name="$1" pubkey="$2"
log "ensuring authorized_keys on $name" log "ensuring authorized_keys on $name"
incus exec "$name" -- mkdir -p /root/.ssh incus exec "$name" -- mkdir -p /root/.ssh
@ -157,6 +194,38 @@ writeShellApplication {
incus exec "$name" -- chmod 600 /root/.ssh/authorized_keys incus exec "$name" -- chmod 600 /root/.ssh/authorized_keys
} }
# Mount the host's /nix/store into the container as read-only —
# `shift=true` idmaps host UIDs into the unprivileged container.
# Idempotent; takes effect on next container start. With this in
# place, every container shares the host's /nix/store on disk —
# zero duplication of nixpkgs closures across containers.
ensure_nix_store_mount() {
local name="$1"
if incus config device show "$name" 2>/dev/null | grep -q '^nix-store:'; then
return 0
fi
log "adding shared /nix/store mount on $name (host's store, read-only)"
incus config device add "$name" nix-store disk \
source=/nix/store \
path=/nix/store \
readonly=true \
shift=true
}
# Pin the toplevel against host-side GC. The container's nix DB
# doesn't know about the path (it lives behind the shared mount),
# so without this the host could nix-collect-garbage the running
# system out from under the container. `nix-store --add-root
# --indirect` creates an indirect root: a user-writable symlink
# (in our state dir, no sudo) plus a tracked entry in
# /nix/var/nix/gcroots/auto/. Idempotent; overwrites prior.
register_host_gcroot() {
local name="$1" toplevel="$2"
local link="$STATE_ROOT/$name/system"
mkdir -p "$STATE_ROOT/$name"
nix-store --add-root "$link" --indirect --realise "$toplevel" >/dev/null
}
wire_proxy_device() { wire_proxy_device() {
local name="$1" listen="$2" host_port="$3" local name="$1" listen="$2" host_port="$3"
local devname="web" local devname="web"
@ -214,19 +283,29 @@ writeShellApplication {
# placeholder; we'll re-evaluate after. # placeholder; we'll re-evaluate after.
local tentative_name="''${name:-_probe}" local tentative_name="''${name:-_probe}"
local dir local dir
dir=$(write_marshaling_flake "$tentative_name" "$flake_ref" "$pubkey") dir=$(write_marshaling_flake "$flake_ref" "$pubkey" "$tentative_name")
log "locking marshaling flake at $dir" log "locking marshaling flake at $dir"
( cd "$dir" && nix flake lock ) # `nix flake update` (not `lock`) so every deploy bumps the target
# input to the current tip of its ref (otherwise re-running `deploy
# github:srid/anywhen` would stay pinned at the first commit we
# saw). `--refresh` bypasses nix's fetcher cache so the resolution
# is fresh, not a 5-min-stale entry.
( cd "$dir" && nix flake update --refresh )
if [ -z "$name" ]; then if [ -z "$name" ]; then
name=$(eval_hostname "$dir" "_probe") name=$(eval_config_attr "$dir" "incus.container.hostname")
log "container name from flake: $name" log "container name from flake: $name"
# Re-emit the marshaling flake under the real name (the # Re-emit the marshaling flake under the real name so the
# nixosConfigurations attr key must match `--flake .#<name>`). # state-dir matches what `incus-pet list` shows.
rm -rf "$STATE_ROOT/_probe" rm -rf "$STATE_ROOT/_probe"
dir=$(write_marshaling_flake "$name" "$flake_ref" "$pubkey") dir=$(write_marshaling_flake "$flake_ref" "$pubkey" "$name")
( cd "$dir" && nix flake lock ) # `nix flake update` (not `lock`) so every deploy bumps the target
# input to the current tip of its ref (otherwise re-running `deploy
# github:srid/anywhen` would stay pinned at the first commit we
# saw). `--refresh` bypasses nix's fetcher cache so the resolution
# is fresh, not a 5-min-stale entry.
( cd "$dir" && nix flake update --refresh )
fi fi
# Resolve host port (CLI flag > container metadata > error). # Resolve host port (CLI flag > container metadata > error).
@ -258,16 +337,39 @@ writeShellApplication {
bootstrap_container "$name" "$pubkey" bootstrap_container "$name" "$pubkey"
log "activating: nixos-rebuild switch --flake $dir#$name --target-host root@$ip" log "building toplevel (pinned $NIXPKGS_REF)"
# accept-new = trust-on-first-use; reject if a known host's key local toplevel
# changed. Fresh containers get a fresh host key, and the operator toplevel=$(build_toplevel "$dir")
# launched the container themselves, so TOFU is fine. log "toplevel: $toplevel"
( cd "$dir" \
&& NIX_SSHOPTS='-o StrictHostKeyChecking=accept-new' \ log "pinning host gcroot for $name → $toplevel"
nixos-rebuild switch \ register_host_gcroot "$name" "$toplevel"
--flake ".#$name" \
--target-host "root@$ip" \ # Update the container's system profile FIRST (while it can still
--use-substitutes ) # see its own /nix/store), then add the shared-mount device if
# missing, then restart so the new init picks up via the shared
# mount. Order matters: if we added the mount before updating
# the profile, the container's existing /sbin/init -> old toplevel
# would point at a path masked by the mount and the restart would
# fail.
log "setting system profile on $name → $toplevel"
incus exec "$name" -- ln -sfn "$toplevel" /nix/var/nix/profiles/system
# The NixOS LXC image hard-links /sbin/init to the image's own
# toplevel; re-pointing it through the profile symlink is what
# makes "update profile + restart" actually pick up the new
# toplevel's init (which sets up /etc, runs activation, then
# execs systemd). NixOS's switch-to-configuration normally does
# this — we have to do it ourselves since we skip s-t-c.
# Idempotent across deploys.
incus exec "$name" -- ln -sfn /nix/var/nix/profiles/system/init /sbin/init
ensure_nix_store_mount "$name"
log "restarting $name to boot via shared /nix/store"
incus restart "$name"
ip=$(container_ipv4 "$name")
log "$name back up at $ip"
# Persist host-port + listen IP for next deploy. The `key=value` # Persist host-port + listen IP for next deploy. The `key=value`
# form is required since incus 6.x; the space-separated form is # form is required since incus 6.x; the space-separated form is