mirror of
https://github.com/srid/nixos-config.git
synced 2026-07-16 22:01:33 +08:00
incus-pet: shared /nix/store, restart-to-deploy, drop anywhen flake input
The incus-pet CLI gains three big changes that together cut per-container
disk usage by ~5GB, drop nix-rebuild + nix-copy from the deploy path, and
make `deploy` actually pull latest on every invocation. anywhen migrates
from a host-installed service to an incus-pet container in the process.
* Shared /nix/store mount. Each container gets host's /nix/store
bind-mounted read-only (with shift=true for idmapped unprivileged
containers). Zero per-container duplication of nixpkgs closures. The
CLI builds toplevels on the host; the container sees them via the
shared mount, no `nix copy` needed.
* Restart-to-deploy. switch-to-configuration is removed entirely (it
was stalling on dbus-broker reload and hitting the same lock-
contention bug across redeploys). New activation: update
/nix/var/nix/profiles/system, repoint /sbin/init through the profile
symlink, `incus restart`. The new toplevel's own init script handles
/etc setup on boot. Trade-off: ~5s of container reboot per deploy.
* Marshaling flake exposes only nixosModules.default. No nixpkgs
input, no nixosConfigurations entry. The CLI picks nixpkgs at build
time (pinned to nixos-25.11 — matches the LXC image baseline, dodges
unstable regressions) and assembles the full nixosSystem via
`nix build --impure --expr`. Lets us bump nixpkgs without touching
per-deploy state.
* `nix flake update --refresh` on every deploy. Was `nix flake lock`,
which only locks first-seen inputs — re-deploying `github:srid/anywhen`
stayed pinned at the first commit ever resolved. Now every deploy
bumps the target input to the current tip of its ref and bypasses
nix's fetcher cache.
* /sbin/init repoint. The NixOS LXC image hard-links /sbin/init to its
own toplevel's init path. Without switch-to-configuration, `incus
restart` re-execs that old init, which sets up /etc from the old
toplevel — running services keep using the OLD anywhen binary even
though the profile pointer moved. CLI now also does
`ln -sfn /nix/var/nix/profiles/system/init /sbin/init` so the profile
actually drives boot. Idempotent.
* Host gcroot via `nix-store --add-root --indirect`. Container's nix DB
doesn't know about the path (it lives behind the shared mount), so
without a host-side gcroot the host could nix-collect-garbage the
running system out from under the container. Symlink at
~/.local/state/incus-pet/<name>/system. Only the LATEST toplevel is
pinned per app — no rollback today (file a 5-line ring buffer when
rollback matters).
* anywhen flake input removed. `modules/nixos/linux/anywhen.nix` is
gone; the host config no longer imports anywhen as a flake input.
The deploy command takes the flake ref as an argument, so locking it
here was just bloat. `just pureintent anywhen-deploy` runs the
current invocation.
* pureintent ops recipes. New `configurations/nixos/pureintent/mod.just`
with anywhen-{deploy,rm,backup,restore,status,shell}. Namespaced via
`mod pureintent` so each host's recipes live behind their own prefix.
* Drop ssh from deploy path. Bootstrap still pushes the operator's
pubkey (for interactive debugging), but activation goes through
`incus exec` exclusively. No NIX_SSHOPTS, no nix-copy-via-ssh,
no host-key TOFU dance during deploys.
Tested end-to-end on pureintent: anywhen migrated from host-installed
(21-task SQLite DB at /var/lib/anywhen) to an incus-pet container with
the DB intact; hello-web deployed fresh with shared /nix/store; both
serve over the tailscale proxy at 100.122.32.106:{6111,8081}; lock
bumps + new toplevels confirmed after master advances.
This commit is contained in:
Sridhar Ratnakumar
parent
5997c1c848
commit
776c4425b0
9 changed files with 271 additions and 249 deletions
|
|
@ -28,10 +28,11 @@ in
|
|||
./devbox.nix
|
||||
(self + /modules/nixos/linux/beszel.nix)
|
||||
(self + /modules/nixos/linux/incus)
|
||||
(self + /modules/nixos/linux/anywhen.nix)
|
||||
];
|
||||
|
||||
services.anywhen.host = "100.122.32.106"; # Tailscale IP of pureintent
|
||||
# anywhen runs as an incus-pet container (see modules/nixos/linux/incus/incus-pet).
|
||||
# Deployed with:
|
||||
# incus-pet deploy github:srid/anywhen --port 6111 --listen 100.122.32.106
|
||||
|
||||
# Expose the incus UI on the Tailscale interface only.
|
||||
virtualisation.incus.preseed.config."core.https_address" = "100.122.32.106:8443";
|
||||
|
|
|
|||
55
configurations/nixos/pureintent/mod.just
Normal file
55
configurations/nixos/pureintent/mod.just
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
# Pureintent-specific operational recipes.
|
||||
#
|
||||
# Imported from the top-level Justfile. Each recipe bakes in the values
|
||||
# that are stable on this host (tailscale IP, container names, port
|
||||
# choices) so the day-to-day operator just types `just anywhen-deploy`,
|
||||
# not the full incus-pet invocation.
|
||||
|
||||
LISTEN_IP := "100.122.32.106" # pureintent's tailscale IP
|
||||
|
||||
# ----------------------------------------------------------------------
|
||||
# anywhen — github:srid/anywhen as a per-app incus-pet container.
|
||||
# ----------------------------------------------------------------------
|
||||
|
||||
# Deploy anywhen (default: master); pass `ref` to deploy a branch/PR.
|
||||
[group('anywhen')]
|
||||
anywhen-deploy ref="github:srid/anywhen":
|
||||
nix run .#incus-pet -- deploy {{ ref }} anywhen --port 6111 --listen {{ LISTEN_IP }}
|
||||
|
||||
# Drop the anywhen container entirely (state goes with it — back up first).
|
||||
[group('anywhen')]
|
||||
anywhen-rm:
|
||||
nix run .#incus-pet -- rm anywhen
|
||||
|
||||
# Pull a WAL-checkpointed snapshot of the live SQLite DB to PATH.
|
||||
[group('anywhen')]
|
||||
anywhen-backup path="/tmp/anywhen.db":
|
||||
incus exec anywhen -- systemctl stop anywhen.service
|
||||
incus exec anywhen -- sh -c 'cd /var/lib/private/anywhen && sqlite3 anywhen.db "PRAGMA wal_checkpoint(TRUNCATE);" 2>/dev/null || true'
|
||||
incus file pull anywhen/var/lib/private/anywhen/anywhen.db {{ path }}
|
||||
incus exec anywhen -- systemctl start anywhen.service
|
||||
@echo "backup -> {{ path }}"
|
||||
|
||||
# Push a DB file into the anywhen container (stop/copy/chown/wipe-wal/start).
|
||||
[group('anywhen')]
|
||||
anywhen-restore db:
|
||||
incus exec anywhen -- systemctl stop anywhen.service
|
||||
incus file push {{ db }} anywhen/var/lib/private/anywhen/anywhen.db --mode 644
|
||||
incus exec anywhen -- sh -c 'chown $(stat -c "%u:%g" /var/lib/private/anywhen) /var/lib/private/anywhen/anywhen.db'
|
||||
incus exec anywhen -- rm -f /var/lib/private/anywhen/anywhen.db-wal /var/lib/private/anywhen/anywhen.db-shm
|
||||
incus exec anywhen -- systemctl start anywhen.service
|
||||
@echo "restored {{ db }} -> anywhen.service active"
|
||||
|
||||
# Health check (curl /api/health, in-container service status, full list).
|
||||
[group('anywhen')]
|
||||
anywhen-status:
|
||||
@echo "--- tailnet ---"
|
||||
@curl -sS http://{{ LISTEN_IP }}:6111/api/health && echo
|
||||
@echo "--- container ---"
|
||||
@incus exec anywhen -- systemctl is-active anywhen.service
|
||||
@nix run .#incus-pet -- list
|
||||
|
||||
# Drop into a root shell inside the anywhen container.
|
||||
[group('anywhen')]
|
||||
anywhen-shell:
|
||||
incus exec anywhen -- bash
|
||||
212
flake.lock
generated
212
flake.lock
generated
|
|
@ -43,25 +43,6 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"anywhen": {
|
||||
"inputs": {
|
||||
"bun2nix": "bun2nix"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1779558041,
|
||||
"narHash": "sha256-qJcYSDxUfKmg7lDE68rooQs/ZEdeDYsX0DSLXG7HmXE=",
|
||||
"owner": "srid",
|
||||
"repo": "anywhen",
|
||||
"rev": "7cb4b823332e0dce7eaf141117894819ed19d367",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "srid",
|
||||
"ref": "abject-turn",
|
||||
"repo": "anywhen",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"blueprint": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
|
|
@ -88,35 +69,12 @@
|
|||
}
|
||||
},
|
||||
"bun2nix": {
|
||||
"inputs": {
|
||||
"flake-parts": "flake-parts",
|
||||
"import-tree": "import-tree",
|
||||
"nixpkgs": "nixpkgs",
|
||||
"systems": "systems_2",
|
||||
"treefmt-nix": "treefmt-nix"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1779553062,
|
||||
"narHash": "sha256-T5cVey80VVqPIkZXHX8kaIxhrqGKw97Nma6ufTfy/dY=",
|
||||
"owner": "juspay",
|
||||
"repo": "bun2nix",
|
||||
"rev": "948386f5219e4526a2978110dacffb433cef1013",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "juspay",
|
||||
"ref": "rawflake",
|
||||
"repo": "bun2nix",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"bun2nix_2": {
|
||||
"inputs": {
|
||||
"flake-parts": [
|
||||
"llm-agents",
|
||||
"flake-parts"
|
||||
],
|
||||
"import-tree": "import-tree_2",
|
||||
"import-tree": "import-tree",
|
||||
"nixpkgs": [
|
||||
"llm-agents",
|
||||
"nixpkgs"
|
||||
|
|
@ -259,11 +217,11 @@
|
|||
},
|
||||
"disc-scrape": {
|
||||
"inputs": {
|
||||
"flake-parts": "flake-parts_2",
|
||||
"flake-parts": "flake-parts",
|
||||
"git-hooks": "git-hooks",
|
||||
"nixpkgs": "nixpkgs_2",
|
||||
"nixpkgs": "nixpkgs",
|
||||
"rust-flake": "rust-flake",
|
||||
"systems": "systems_3"
|
||||
"systems": "systems_2"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1772235986,
|
||||
|
|
@ -322,14 +280,14 @@
|
|||
"commonmark-wikilink": "commonmark-wikilink",
|
||||
"ema": "ema",
|
||||
"emanote-template": "emanote-template",
|
||||
"flake-parts": "flake-parts_3",
|
||||
"flake-parts": "flake-parts_2",
|
||||
"fourmolu-nix": "fourmolu-nix",
|
||||
"git-hooks": "git-hooks_2",
|
||||
"haskell-flake": "haskell-flake",
|
||||
"heist-extra": "heist-extra",
|
||||
"lvar": "lvar",
|
||||
"nixos-unified": "nixos-unified",
|
||||
"nixpkgs": "nixpkgs_3",
|
||||
"nixpkgs": "nixpkgs_2",
|
||||
"unionmount": "unionmount"
|
||||
},
|
||||
"locked": {
|
||||
|
|
@ -363,24 +321,6 @@
|
|||
}
|
||||
},
|
||||
"flake-parts": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": "nixpkgs-lib"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1769996383,
|
||||
"narHash": "sha256-AnYjnFWgS49RlqX7LrC4uA+sCCDBj0Ry/WOJ5XWAsa0=",
|
||||
"owner": "hercules-ci",
|
||||
"repo": "flake-parts",
|
||||
"rev": "57928607ea566b5db3ad13af0e57e921e6b12381",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "hercules-ci",
|
||||
"repo": "flake-parts",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_2": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": [
|
||||
"disc-scrape",
|
||||
|
|
@ -401,7 +341,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_3": {
|
||||
"flake-parts_2": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": [
|
||||
"emanote",
|
||||
|
|
@ -422,9 +362,9 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_4": {
|
||||
"flake-parts_3": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": "nixpkgs-lib_2"
|
||||
"nixpkgs-lib": "nixpkgs-lib"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1772408722,
|
||||
|
|
@ -440,7 +380,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_5": {
|
||||
"flake-parts_4": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": [
|
||||
"imako",
|
||||
|
|
@ -461,7 +401,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_6": {
|
||||
"flake-parts_5": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": [
|
||||
"llm-agents",
|
||||
|
|
@ -482,7 +422,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"flake-parts_7": {
|
||||
"flake-parts_6": {
|
||||
"inputs": {
|
||||
"nixpkgs-lib": [
|
||||
"vira",
|
||||
|
|
@ -505,7 +445,7 @@
|
|||
},
|
||||
"flake-utils": {
|
||||
"inputs": {
|
||||
"systems": "systems_5"
|
||||
"systems": "systems_4"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1731533236,
|
||||
|
|
@ -797,12 +737,12 @@
|
|||
"aeson-typescript": "aeson-typescript",
|
||||
"commonmark-simple": "commonmark-simple_2",
|
||||
"commonmark-wikilink": "commonmark-wikilink_2",
|
||||
"flake-parts": "flake-parts_5",
|
||||
"flake-parts": "flake-parts_4",
|
||||
"fourmolu-nix": "fourmolu-nix_2",
|
||||
"git-hooks": "git-hooks_4",
|
||||
"haskell-flake": "haskell-flake_2",
|
||||
"lvar": "lvar_2",
|
||||
"nixpkgs": "nixpkgs_4",
|
||||
"nixpkgs": "nixpkgs_3",
|
||||
"process-compose-flake": "process-compose-flake",
|
||||
"unionmount": "unionmount_2"
|
||||
},
|
||||
|
|
@ -835,21 +775,6 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"import-tree_2": {
|
||||
"locked": {
|
||||
"lastModified": 1763762820,
|
||||
"narHash": "sha256-ZvYKbFib3AEwiNMLsejb/CWs/OL/srFQ8AogkebEPF0=",
|
||||
"owner": "vic",
|
||||
"repo": "import-tree",
|
||||
"rev": "3c23749d8013ec6daa1d7255057590e9ca726646",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "vic",
|
||||
"repo": "import-tree",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"jumphost-nix": {
|
||||
"flake": false,
|
||||
"locked": {
|
||||
|
|
@ -901,13 +826,13 @@
|
|||
"llm-agents": {
|
||||
"inputs": {
|
||||
"blueprint": "blueprint",
|
||||
"bun2nix": "bun2nix_2",
|
||||
"flake-parts": "flake-parts_6",
|
||||
"bun2nix": "bun2nix",
|
||||
"flake-parts": "flake-parts_5",
|
||||
"nixpkgs": [
|
||||
"nixpkgs"
|
||||
],
|
||||
"systems": "systems_4",
|
||||
"treefmt-nix": "treefmt-nix_2"
|
||||
"systems": "systems_3",
|
||||
"treefmt-nix": "treefmt-nix"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1775790657,
|
||||
|
|
@ -1107,36 +1032,21 @@
|
|||
},
|
||||
"nixpkgs": {
|
||||
"locked": {
|
||||
"lastModified": 1770562336,
|
||||
"narHash": "sha256-ub1gpAONMFsT/GU2hV6ZWJjur8rJ6kKxdm9IlCT0j84=",
|
||||
"owner": "NixOS",
|
||||
"lastModified": 1772082373,
|
||||
"narHash": "sha256-wySf8a6hvuqgFdwvvzPPTARBCMLDz7WFAufGkllD1M4=",
|
||||
"owner": "nixos",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "d6c71932130818840fc8fe9509cf50be8c64634f",
|
||||
"rev": "26eaeac4e409d7b5a6bf6f90a2a2dc223c78d915",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "NixOS",
|
||||
"ref": "nixos-unstable",
|
||||
"owner": "nixos",
|
||||
"ref": "nixpkgs-unstable",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs-lib": {
|
||||
"locked": {
|
||||
"lastModified": 1769909678,
|
||||
"narHash": "sha256-cBEymOf4/o3FD5AZnzC3J9hLbiZ+QDT/KDuyHXVJOpM=",
|
||||
"owner": "nix-community",
|
||||
"repo": "nixpkgs.lib",
|
||||
"rev": "72716169fe93074c333e8d0173151350670b824c",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nix-community",
|
||||
"repo": "nixpkgs.lib",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs-lib_2": {
|
||||
"locked": {
|
||||
"lastModified": 1772328832,
|
||||
"narHash": "sha256-e+/T/pmEkLP6BHhYjx6GmwP5ivonQQn0bJdH9YrRB+Q=",
|
||||
|
|
@ -1152,22 +1062,6 @@
|
|||
}
|
||||
},
|
||||
"nixpkgs_2": {
|
||||
"locked": {
|
||||
"lastModified": 1772082373,
|
||||
"narHash": "sha256-wySf8a6hvuqgFdwvvzPPTARBCMLDz7WFAufGkllD1M4=",
|
||||
"owner": "nixos",
|
||||
"repo": "nixpkgs",
|
||||
"rev": "26eaeac4e409d7b5a6bf6f90a2a2dc223c78d915",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nixos",
|
||||
"ref": "nixpkgs-unstable",
|
||||
"repo": "nixpkgs",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_3": {
|
||||
"locked": {
|
||||
"lastModified": 1752900028,
|
||||
"narHash": "sha256-dPALCtmik9Wr14MGqVXm+OQcv7vhPBXcWNIOThGnB/Q=",
|
||||
|
|
@ -1183,7 +1077,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_4": {
|
||||
"nixpkgs_3": {
|
||||
"locked": {
|
||||
"lastModified": 1770169770,
|
||||
"narHash": "sha256-awR8qIwJxJJiOmcEGgP2KUqYmHG4v/z8XpL9z8FnT1A=",
|
||||
|
|
@ -1199,7 +1093,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_5": {
|
||||
"nixpkgs_4": {
|
||||
"locked": {
|
||||
"lastModified": 1778443072,
|
||||
"narHash": "sha256-zi7/fsqM/kFdNuED//4WOCUtezGtKKqRNORjMvfwjnA=",
|
||||
|
|
@ -1215,7 +1109,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_6": {
|
||||
"nixpkgs_5": {
|
||||
"locked": {
|
||||
"lastModified": 1765803225,
|
||||
"narHash": "sha256-xwaZV/UgJ04+ixbZZfoDE8IsOWjtvQZICh9aamzPnrg=",
|
||||
|
|
@ -1231,7 +1125,7 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"nixpkgs_7": {
|
||||
"nixpkgs_6": {
|
||||
"locked": {
|
||||
"lastModified": 1764635402,
|
||||
"narHash": "sha256-6rYcajRLe2C5ZYnV1HYskJl+QAkhvseWTzbdQiTN9OI=",
|
||||
|
|
@ -1315,11 +1209,10 @@
|
|||
"root": {
|
||||
"inputs": {
|
||||
"agenix": "agenix",
|
||||
"anywhen": "anywhen",
|
||||
"disc-scrape": "disc-scrape",
|
||||
"disko": "disko",
|
||||
"emanote": "emanote",
|
||||
"flake-parts": "flake-parts_4",
|
||||
"flake-parts": "flake-parts_3",
|
||||
"git-hooks": "git-hooks_3",
|
||||
"github-nix-ci": "github-nix-ci",
|
||||
"home-manager": "home-manager",
|
||||
|
|
@ -1333,7 +1226,7 @@
|
|||
"nixos-hardware": "nixos-hardware",
|
||||
"nixos-unified": "nixos-unified_2",
|
||||
"nixos-vscode-server": "nixos-vscode-server",
|
||||
"nixpkgs": "nixpkgs_5",
|
||||
"nixpkgs": "nixpkgs_4",
|
||||
"project-unknown": "project-unknown",
|
||||
"vira": "vira",
|
||||
"zmx": "zmx"
|
||||
|
|
@ -1460,21 +1353,6 @@
|
|||
"type": "github"
|
||||
}
|
||||
},
|
||||
"systems_5": {
|
||||
"locked": {
|
||||
"lastModified": 1681028828,
|
||||
"narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
|
||||
"owner": "nix-systems",
|
||||
"repo": "default",
|
||||
"rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "nix-systems",
|
||||
"repo": "default",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"tabler-icons-hs": {
|
||||
"flake": false,
|
||||
"locked": {
|
||||
|
|
@ -1492,28 +1370,6 @@
|
|||
}
|
||||
},
|
||||
"treefmt-nix": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
"anywhen",
|
||||
"bun2nix",
|
||||
"nixpkgs"
|
||||
]
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1770228511,
|
||||
"narHash": "sha256-wQ6NJSuFqAEmIg2VMnLdCnUc0b7vslUohqqGGD+Fyxk=",
|
||||
"owner": "numtide",
|
||||
"repo": "treefmt-nix",
|
||||
"rev": "337a4fe074be1042a35086f15481d763b8ddc0e7",
|
||||
"type": "github"
|
||||
},
|
||||
"original": {
|
||||
"owner": "numtide",
|
||||
"repo": "treefmt-nix",
|
||||
"type": "github"
|
||||
}
|
||||
},
|
||||
"treefmt-nix_2": {
|
||||
"inputs": {
|
||||
"nixpkgs": [
|
||||
"llm-agents",
|
||||
|
|
@ -1571,7 +1427,7 @@
|
|||
"inputs": {
|
||||
"co-log-effectful": "co-log-effectful",
|
||||
"devour-flake": "devour-flake",
|
||||
"flake-parts": "flake-parts_7",
|
||||
"flake-parts": "flake-parts_6",
|
||||
"fourmolu-nix": "fourmolu-nix_3",
|
||||
"git-hooks": "git-hooks_5",
|
||||
"haskell-flake": "haskell-flake_3",
|
||||
|
|
@ -1581,7 +1437,7 @@
|
|||
"nix-serve-ng": "nix-serve-ng",
|
||||
"nix-systems": "nix-systems",
|
||||
"nixos-unified": "nixos-unified_3",
|
||||
"nixpkgs": "nixpkgs_6",
|
||||
"nixpkgs": "nixpkgs_5",
|
||||
"process-compose-flake": "process-compose-flake_2",
|
||||
"record-hasfield": "record-hasfield",
|
||||
"servant-event-stream": "servant-event-stream",
|
||||
|
|
@ -1621,7 +1477,7 @@
|
|||
"zig2nix": {
|
||||
"inputs": {
|
||||
"flake-utils": "flake-utils",
|
||||
"nixpkgs": "nixpkgs_7"
|
||||
"nixpkgs": "nixpkgs_6"
|
||||
},
|
||||
"locked": {
|
||||
"lastModified": 1764678235,
|
||||
|
|
|
|||
|
|
@ -38,8 +38,11 @@
|
|||
# KOLU
|
||||
kolu.url = "github:juspay/kolu/feat/dock-activity-window";
|
||||
|
||||
# anywhen — personal task manager (PR #2 adds NixOS module)
|
||||
anywhen.url = "github:srid/anywhen/abject-turn";
|
||||
# anywhen is NOT a flake input — it's deployed as an incus-pet
|
||||
# container, with the flake ref passed at deploy time (see
|
||||
# `just pureintent anywhen-deploy`). The host config doesn't import
|
||||
# anything from anywhen, so locking it here would just bloat
|
||||
# flake.lock without buying us anything.
|
||||
|
||||
project-unknown.url = "github:juspay/project-unknown";
|
||||
project-unknown.inputs.nixpkgs.follows = "nixpkgs";
|
||||
|
|
|
|||
4
justfile
4
justfile
|
|
@ -1,6 +1,10 @@
|
|||
default:
|
||||
@just --list
|
||||
|
||||
# Per-host operational recipes. Namespaced via `mod` so each host's
|
||||
# recipes live behind their own prefix (e.g. `just pureintent anywhen-deploy`).
|
||||
mod pureintent 'configurations/nixos/pureintent/mod.just'
|
||||
|
||||
# Main commands
|
||||
# --------------------------------------------------------------------------------------------------
|
||||
|
||||
|
|
|
|||
|
|
@ -1,16 +0,0 @@
|
|||
{ flake, pkgs, ... }:
|
||||
|
||||
let
|
||||
inherit (flake) inputs;
|
||||
in
|
||||
{
|
||||
imports = [
|
||||
inputs.anywhen.nixosModules.default
|
||||
];
|
||||
|
||||
services.anywhen = {
|
||||
enable = true;
|
||||
package = inputs.anywhen.packages.${pkgs.stdenv.hostPlatform.system}.default;
|
||||
port = 6111;
|
||||
};
|
||||
}
|
||||
|
|
@ -117,13 +117,22 @@ incus-pet deploy github:srid/anywhen --port 7700 --listen 100.122.32.106
|
|||
|
||||
This:
|
||||
|
||||
1. Synthesises a marshaling flake under `~/.local/state/incus-pet/anywhen/`.
|
||||
2. Launches `images:nixos/25.11` as container `anywhen` (with
|
||||
`security.nesting=true` so `nixos-rebuild` works inside).
|
||||
3. Bootstraps sshd + your pubkey via `incus exec`.
|
||||
4. Runs `nixos-rebuild switch --target-host root@<container-ip>`.
|
||||
5. Records `--port` and `--listen` in container metadata.
|
||||
6. Adds the `web` proxy device.
|
||||
1. Synthesises a marshaling flake under `~/.local/state/incus-pet/anywhen/`
|
||||
exposing `nixosModules.default` (essentials + app's
|
||||
`nixosModules.incus` + operator pubkey overlay) — no nixpkgs pin in
|
||||
per-deploy state.
|
||||
2. Launches `images:nixos/25.11` as container `anywhen` with
|
||||
`-c security.nesting=true`.
|
||||
3. Pushes your `~/.ssh/id_ed25519.pub` into the container's
|
||||
`/root/.ssh/authorized_keys` (the image already runs sshd via systemd
|
||||
socket activation).
|
||||
4. Builds the system toplevel against `github:nixos/nixpkgs/nixos-25.11`
|
||||
(stable, matching the image baseline) via `nix build --impure --expr`.
|
||||
5. Copies the closure with `nix copy --to ssh://root@<container-ip>
|
||||
--substitute-on-destination`.
|
||||
6. SSHes to set the system profile and run `switch-to-configuration switch`.
|
||||
7. Records `--port` and `--listen` in container metadata.
|
||||
8. Adds the `web` proxy device.
|
||||
|
||||
### Subsequent deploys
|
||||
|
||||
|
|
@ -165,9 +174,13 @@ container.
|
|||
- **`Failed to start Firewall.` inside the container** — the launch
|
||||
flag `-c security.nesting=true` is supposed to handle this; if it
|
||||
recurs, see `../README.md` and lxc/incus#526.
|
||||
- **SSH fails after first deploy** — the bootstrap step writes
|
||||
`/etc/nixos/incus-pet-bootstrap.nix` and runs `nixos-rebuild` inside
|
||||
the container; if that fails, `incus exec <name> -- journalctl -xeu nixos-rebuild` is the place to look.
|
||||
- **`switch-to-configuration` exits 4** — "some units failed to
|
||||
reload/restart". Treated as success: the activation itself
|
||||
completed, only a unit reload (typically dbus-broker) failed. The
|
||||
service is up.
|
||||
- **`switch-to-configuration` exits 11 / "Could not acquire lock"** —
|
||||
a previous activation is still running (often hung). `incus exec
|
||||
<name> -- pkill -9 -f switch-to-configuration` clears it.
|
||||
|
||||
## State
|
||||
|
||||
|
|
|
|||
|
|
@ -131,6 +131,10 @@ The module must NOT set:
|
|||
filters by the `flake-ref` key.
|
||||
- If `--port` is missing on the first deploy, the command fails before
|
||||
launching anything.
|
||||
- Network failures during `nixos-rebuild --target-host` leave the
|
||||
container running its previous generation — `nixos-rebuild switch` is
|
||||
itself atomic.
|
||||
- Activation is two ssh steps: `nix-env --set` (atomic pointer swap on
|
||||
`/nix/var/nix/profiles/system`) then `switch-to-configuration switch`.
|
||||
If the second fails partway, the system profile is already at the new
|
||||
toplevel; running deploy again finishes the switch.
|
||||
- Exit 4 from `switch-to-configuration` is treated as success (a unit
|
||||
failed to reload — typically dbus-broker — but activation itself
|
||||
completed).
|
||||
|
|
|
|||
|
|
@ -6,9 +6,19 @@
|
|||
# host-side <listen-ip>:<host-port> is unique per app and stored in
|
||||
# container metadata so re-deploys are flagless. See ./README.md for
|
||||
# the network model and operator flow.
|
||||
#
|
||||
# Marshaling-flake-as-module design (vs. nixos-rebuild --target-host):
|
||||
# the per-deploy state-dir flake exposes ONLY `nixosModules.default`
|
||||
# (combined container essentials + app's nixosModules.incus +
|
||||
# operator-key overlay). The CLI picks nixpkgs at build time (pinned
|
||||
# to nixos-25.11, matching the LXC image baseline) and assembles the
|
||||
# nixosSystem via `nix build --impure --expr`. Closure copy + activation
|
||||
# happen via `nix copy --to ssh://…` + ssh-driven
|
||||
# switch-to-configuration. Lets us dodge unstable nixpkgs regressions
|
||||
# (e.g. the dbus-broker reload stall) without touching the marshaling
|
||||
# flake's shape.
|
||||
{ writeShellApplication
|
||||
, nix
|
||||
, nixos-rebuild
|
||||
, openssh
|
||||
, jq
|
||||
, coreutils
|
||||
|
|
@ -24,7 +34,7 @@ in
|
|||
writeShellApplication {
|
||||
name = "incus-pet";
|
||||
|
||||
runtimeInputs = [ nix nixos-rebuild openssh jq coreutils gnused ];
|
||||
runtimeInputs = [ nix openssh jq coreutils gnused ];
|
||||
|
||||
meta.description = "Deploy a NixOS service flake into a per-app incus container (pet PaaS).";
|
||||
|
||||
|
|
@ -35,6 +45,10 @@ writeShellApplication {
|
|||
CONTAINER_PORT=8080
|
||||
STATE_ROOT="''${XDG_STATE_HOME:-$HOME/.local/state}/incus-pet"
|
||||
INCUS_IMAGE="images:nixos/25.11"
|
||||
# Pinned to stable to match the LXC image baseline. Unstable triggers
|
||||
# a dbus-broker reload stall in switch-to-configuration (~Jul 2026
|
||||
# systemd/dbus-broker regression).
|
||||
NIXPKGS_REF="github:nixos/nixpkgs/nixos-25.11"
|
||||
|
||||
log() { printf '[incus-pet] %s\n' "$*" >&2; }
|
||||
die() { log "error: $*"; exit 1; }
|
||||
|
|
@ -97,8 +111,12 @@ writeShellApplication {
|
|||
}
|
||||
|
||||
write_marshaling_flake() {
|
||||
local name="$1" flake_ref="$2" pubkey="$3"
|
||||
local dir="$STATE_ROOT/$name"
|
||||
# Per-deploy state-dir flake. Exposes ONLY nixosModules.default —
|
||||
# the CLI assembles the full nixosSystem at build time, picking
|
||||
# nixpkgs (so we can pin stable to dodge unstable regressions and
|
||||
# not bake the choice into per-deploy state).
|
||||
local flake_ref="$1" pubkey="$2"
|
||||
local dir="$STATE_ROOT/$3" # caller-chosen subdir (probe or real name)
|
||||
mkdir -p "$dir"
|
||||
|
||||
# Copy the essentials module into the marshaling flake dir as
|
||||
|
|
@ -110,19 +128,15 @@ writeShellApplication {
|
|||
cat > "$dir/flake.nix" <<EOF
|
||||
# Generated by incus-pet — do not edit. Re-run \`incus-pet deploy\` to refresh.
|
||||
{
|
||||
inputs.nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
|
||||
inputs.target.url = "$flake_ref";
|
||||
outputs = { nixpkgs, target, ... }: {
|
||||
nixosConfigurations."$name" = nixpkgs.lib.nixosSystem {
|
||||
system = "x86_64-linux";
|
||||
modules = [
|
||||
inputs.target.url = "$flake_ref";
|
||||
outputs = { target, ... }: {
|
||||
nixosModules.default = { ... }: {
|
||||
imports = [
|
||||
./container.nix
|
||||
target.nixosModules.incus
|
||||
({ ... }: {
|
||||
users.users.root.openssh.authorizedKeys.keys = [
|
||||
"$pubkey"
|
||||
];
|
||||
})
|
||||
];
|
||||
users.users.root.openssh.authorizedKeys.keys = [
|
||||
"$pubkey"
|
||||
];
|
||||
};
|
||||
};
|
||||
|
|
@ -138,17 +152,40 @@ writeShellApplication {
|
|||
fi
|
||||
}
|
||||
|
||||
eval_hostname() {
|
||||
local dir="$1" name="$2"
|
||||
nix eval --raw \
|
||||
"$dir#nixosConfigurations.\"$name\".config.incus.container.hostname"
|
||||
# Boilerplate that assembles the full nixosSystem on demand. Pinned
|
||||
# nixpkgs (NIXPKGS_REF) is fetched via builtins.getFlake; --impure
|
||||
# is required because builtins.getFlake on a URL isn't
|
||||
# pure-evaluable. The marshaling flake is loaded by absolute store
|
||||
# path so the operator's machine doesn't re-fetch it.
|
||||
sys_expr() {
|
||||
local dir="$1"
|
||||
printf 'let
|
||||
marshaling = builtins.getFlake "path:%s";
|
||||
nixpkgs = builtins.getFlake "%s";
|
||||
in nixpkgs.lib.nixosSystem {
|
||||
system = "x86_64-linux";
|
||||
modules = [ marshaling.nixosModules.default ];
|
||||
}' "$dir" "$NIXPKGS_REF"
|
||||
}
|
||||
|
||||
eval_config_attr() {
|
||||
local dir="$1" attr="$2"
|
||||
nix eval --raw --impure --expr "($(sys_expr "$dir")).config.$attr"
|
||||
}
|
||||
|
||||
build_toplevel() {
|
||||
local dir="$1"
|
||||
nix build --print-out-paths --no-link --impure --expr \
|
||||
"($(sys_expr "$dir")).config.system.build.toplevel"
|
||||
}
|
||||
|
||||
bootstrap_container() {
|
||||
# The official NixOS incus image already runs sshd via systemd
|
||||
# socket activation on port 22; the only thing we need to add is
|
||||
# the operator's pubkey to root's authorized_keys. Idempotent —
|
||||
# `tee` overwrites, so a rotated key gets picked up on next deploy.
|
||||
# the operator's pubkey to root's authorized_keys (for interactive
|
||||
# ssh debugging — the deploy path uses `incus exec`, not ssh).
|
||||
# Idempotent — `tee` overwrites, so a rotated key gets picked up
|
||||
# on next deploy.
|
||||
local name="$1" pubkey="$2"
|
||||
log "ensuring authorized_keys on $name"
|
||||
incus exec "$name" -- mkdir -p /root/.ssh
|
||||
|
|
@ -157,6 +194,38 @@ writeShellApplication {
|
|||
incus exec "$name" -- chmod 600 /root/.ssh/authorized_keys
|
||||
}
|
||||
|
||||
# Mount the host's /nix/store into the container as read-only —
|
||||
# `shift=true` idmaps host UIDs into the unprivileged container.
|
||||
# Idempotent; takes effect on next container start. With this in
|
||||
# place, every container shares the host's /nix/store on disk —
|
||||
# zero duplication of nixpkgs closures across containers.
|
||||
ensure_nix_store_mount() {
|
||||
local name="$1"
|
||||
if incus config device show "$name" 2>/dev/null | grep -q '^nix-store:'; then
|
||||
return 0
|
||||
fi
|
||||
log "adding shared /nix/store mount on $name (host's store, read-only)"
|
||||
incus config device add "$name" nix-store disk \
|
||||
source=/nix/store \
|
||||
path=/nix/store \
|
||||
readonly=true \
|
||||
shift=true
|
||||
}
|
||||
|
||||
# Pin the toplevel against host-side GC. The container's nix DB
|
||||
# doesn't know about the path (it lives behind the shared mount),
|
||||
# so without this the host could nix-collect-garbage the running
|
||||
# system out from under the container. `nix-store --add-root
|
||||
# --indirect` creates an indirect root: a user-writable symlink
|
||||
# (in our state dir, no sudo) plus a tracked entry in
|
||||
# /nix/var/nix/gcroots/auto/. Idempotent; overwrites prior.
|
||||
register_host_gcroot() {
|
||||
local name="$1" toplevel="$2"
|
||||
local link="$STATE_ROOT/$name/system"
|
||||
mkdir -p "$STATE_ROOT/$name"
|
||||
nix-store --add-root "$link" --indirect --realise "$toplevel" >/dev/null
|
||||
}
|
||||
|
||||
wire_proxy_device() {
|
||||
local name="$1" listen="$2" host_port="$3"
|
||||
local devname="web"
|
||||
|
|
@ -214,19 +283,29 @@ writeShellApplication {
|
|||
# placeholder; we'll re-evaluate after.
|
||||
local tentative_name="''${name:-_probe}"
|
||||
local dir
|
||||
dir=$(write_marshaling_flake "$tentative_name" "$flake_ref" "$pubkey")
|
||||
dir=$(write_marshaling_flake "$flake_ref" "$pubkey" "$tentative_name")
|
||||
|
||||
log "locking marshaling flake at $dir"
|
||||
( cd "$dir" && nix flake lock )
|
||||
# `nix flake update` (not `lock`) so every deploy bumps the target
|
||||
# input to the current tip of its ref (otherwise re-running `deploy
|
||||
# github:srid/anywhen` would stay pinned at the first commit we
|
||||
# saw). `--refresh` bypasses nix's fetcher cache so the resolution
|
||||
# is fresh, not a 5-min-stale entry.
|
||||
( cd "$dir" && nix flake update --refresh )
|
||||
|
||||
if [ -z "$name" ]; then
|
||||
name=$(eval_hostname "$dir" "_probe")
|
||||
name=$(eval_config_attr "$dir" "incus.container.hostname")
|
||||
log "container name from flake: $name"
|
||||
# Re-emit the marshaling flake under the real name (the
|
||||
# nixosConfigurations attr key must match `--flake .#<name>`).
|
||||
# Re-emit the marshaling flake under the real name so the
|
||||
# state-dir matches what `incus-pet list` shows.
|
||||
rm -rf "$STATE_ROOT/_probe"
|
||||
dir=$(write_marshaling_flake "$name" "$flake_ref" "$pubkey")
|
||||
( cd "$dir" && nix flake lock )
|
||||
dir=$(write_marshaling_flake "$flake_ref" "$pubkey" "$name")
|
||||
# `nix flake update` (not `lock`) so every deploy bumps the target
|
||||
# input to the current tip of its ref (otherwise re-running `deploy
|
||||
# github:srid/anywhen` would stay pinned at the first commit we
|
||||
# saw). `--refresh` bypasses nix's fetcher cache so the resolution
|
||||
# is fresh, not a 5-min-stale entry.
|
||||
( cd "$dir" && nix flake update --refresh )
|
||||
fi
|
||||
|
||||
# Resolve host port (CLI flag > container metadata > error).
|
||||
|
|
@ -258,16 +337,39 @@ writeShellApplication {
|
|||
|
||||
bootstrap_container "$name" "$pubkey"
|
||||
|
||||
log "activating: nixos-rebuild switch --flake $dir#$name --target-host root@$ip"
|
||||
# accept-new = trust-on-first-use; reject if a known host's key
|
||||
# changed. Fresh containers get a fresh host key, and the operator
|
||||
# launched the container themselves, so TOFU is fine.
|
||||
( cd "$dir" \
|
||||
&& NIX_SSHOPTS='-o StrictHostKeyChecking=accept-new' \
|
||||
nixos-rebuild switch \
|
||||
--flake ".#$name" \
|
||||
--target-host "root@$ip" \
|
||||
--use-substitutes )
|
||||
log "building toplevel (pinned $NIXPKGS_REF)"
|
||||
local toplevel
|
||||
toplevel=$(build_toplevel "$dir")
|
||||
log "toplevel: $toplevel"
|
||||
|
||||
log "pinning host gcroot for $name → $toplevel"
|
||||
register_host_gcroot "$name" "$toplevel"
|
||||
|
||||
# Update the container's system profile FIRST (while it can still
|
||||
# see its own /nix/store), then add the shared-mount device if
|
||||
# missing, then restart so the new init picks up via the shared
|
||||
# mount. Order matters: if we added the mount before updating
|
||||
# the profile, the container's existing /sbin/init -> old toplevel
|
||||
# would point at a path masked by the mount and the restart would
|
||||
# fail.
|
||||
log "setting system profile on $name → $toplevel"
|
||||
incus exec "$name" -- ln -sfn "$toplevel" /nix/var/nix/profiles/system
|
||||
# The NixOS LXC image hard-links /sbin/init to the image's own
|
||||
# toplevel; re-pointing it through the profile symlink is what
|
||||
# makes "update profile + restart" actually pick up the new
|
||||
# toplevel's init (which sets up /etc, runs activation, then
|
||||
# execs systemd). NixOS's switch-to-configuration normally does
|
||||
# this — we have to do it ourselves since we skip s-t-c.
|
||||
# Idempotent across deploys.
|
||||
incus exec "$name" -- ln -sfn /nix/var/nix/profiles/system/init /sbin/init
|
||||
|
||||
ensure_nix_store_mount "$name"
|
||||
|
||||
log "restarting $name to boot via shared /nix/store"
|
||||
incus restart "$name"
|
||||
|
||||
ip=$(container_ipv4 "$name")
|
||||
log "$name back up at $ip"
|
||||
|
||||
# Persist host-port + listen IP for next deploy. The `key=value`
|
||||
# form is required since incus 6.x; the space-separated form is
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue