mirror of
https://github.com/srid/nixos-config.git
synced 2026-07-16 22:01:33 +08:00
b4a60aaf3a
* ralph(eval): baseline + benchmark harness pureintent cpuTime 23.17s, zest cpuTime 14.93s (median of 5). Methodology: NIX_SHOW_STATS cpuTime, eval-cache off, warm fetcher cache. * ralph(eval) cycle 1: disable NixOS options manual on pureintent documentation.nixos.enable = false. Generating the NixOS manual evaluates every option's doc string. nrThunks 20,400,114 -> 19,201,996 (-5.9%), nrFunctionCalls -6.2%, nrPrimOpCalls -7.5%. Package man pages unaffected. Metric pivoted to deterministic NIX_SHOW_STATS counters: cpuTime is unusable on this thermally-throttling machine (same eval = 11s..21s). * ralph(eval): switch home target zest -> sincereintent (per request) sincereintent home config baseline: nrThunks 5,323,428 / calls 3,227,404 / primops 1,565,120 (deterministic). * ralph(eval) cycle 2: disable home-manager options manpage manual.manpages.enable = false in shared modules/home/default.nix. HM builds home-configuration.nix(5) by default, evaluating every HM option's doc string. sincereintent nrThunks 5,323,428 -> 3,551,679 (-33.3%), calls -35%, primops -39%. * ralph(eval): record cycle-2 pureintent gain (shared HM module) pureintent nrThunks 19,201,996 -> 17,431,437 (-9.2%); cycle-2's manual.manpages.enable=false also feeds pureintent's embedded home-manager via modules/nixos/default.nix. Total pureintent -14.6% from baseline. * ralph(eval) cycle 3: profiling + dead-ends (no behaviour-preserving win) Confirmed: nixpkgs single-instantiation (no 2x trap), unused inputs are lazy (no counter impact), escapeShellArg hotspot is HM file-linking + /etc (not patchable here). Remaining eval cost is feature-bound (embedded HM, vira/kolu, google-cloud-sdk/pandoc/omnix). * ralph(eval) cycle 4: drop heavy home.packages (yt-dlp/lima/omnix/pandoc/gcloud) User-approved behaviour change. Removed from shared modules/home/cli/terminal.nix (kept hledger). pureintent nrThunks 17,431,437 -> 16,989,720 (-2.5%), sincereintent 3,551,679 -> 2,757,201 (-22.4%). Totals from baseline: pureintent -16.7%, sincereintent -48.2%. * ralph(eval): finalize report (final measurement, key findings, cost breakdown)
143 lines
8.1 KiB
Markdown
143 lines
8.1 KiB
Markdown
# Nix Eval-Time Optimization (Ralph)
|
||
|
||
Iterative measurement-driven reduction of Nix **evaluation** time for the two
|
||
configs that get evaluated most often on this machine:
|
||
|
||
- `pureintent` — the NixOS system (`nixosConfigurations.pureintent`)
|
||
- `sincereintent` — the home-manager config (`homeConfigurations."srid@sincereintent"`)
|
||
|
||
## Methodology
|
||
|
||
Primary metric: **deterministic eval-work counters** from `NIX_SHOW_STATS=1` —
|
||
`nrThunks`, `nrFunctionCalls`, `nrPrimOpCalls`. These count the actual evaluation
|
||
work and are **byte-identical across runs** (independent of CPU clock, load, GC).
|
||
|
||
> **Why not `cpuTime`/wall?** Initially planned, but rejected after measuring:
|
||
> this is a live, thermally-throttling laptop. The *same* eval (identical
|
||
> nrThunks every run) reported cpuTime climbing 11s → 21s monotonically as the
|
||
> CPU clock dropped under sustained load. Time is meaningless here; counters are
|
||
> exact. A 1% drop in nrThunks is a real 1% less work the evaluator must do, on
|
||
> any machine. Single eval per target — no median needed.
|
||
|
||
```sh
|
||
# docs/eval-bench.sh [target] target = pureintent | zest | (both)
|
||
# - persistent $HOME=/tmp/ralph-evalhome keeps the fetcher/git cache warm
|
||
# - --option eval-cache false forces a full re-eval every run
|
||
# - reports nrThunks / nrFunctionCalls / nrPrimOpCalls per target (deterministic)
|
||
nix eval --raw .#nixosConfigurations.pureintent.config.system.build.toplevel.drvPath --option eval-cache false
|
||
nix eval --raw '.#homeConfigurations."srid@sincereintent".activationPackage.drvPath' --option eval-cache false
|
||
```
|
||
|
||
No `nix store gc` between runs (per request). The eval cache is disabled rather
|
||
than cleared, so the warm fetcher cache removes input-fetch noise.
|
||
|
||
**Commit rule:** commit only if `nrThunks` improves >3% for a target with no
|
||
regression on the other. (Counters have ~zero noise, so any real reduction
|
||
counts; >3% is the bar for a "meaningful" cycle.) Behaviour may change *only* by
|
||
dropping inputs/features first confirmed unused.
|
||
|
||
Profiling tool: `nix eval … --eval-profiler flamegraph --eval-profile-file f`
|
||
then aggregate self/inclusive frames (collapsed-stack format).
|
||
|
||
## Baseline (nrThunks / nrFunctionCalls / nrPrimOpCalls)
|
||
|
||
| Target | nrThunks | nrFunctionCalls | nrPrimOpCalls |
|
||
|------------|------------|-----------------|---------------|
|
||
| pureintent | 20,400,114 | 12,975,268 | 6,507,025 |
|
||
| sincereintent | 5,323,428 | 3,227,404 | 1,565,120 |
|
||
|
||
Env: nix 2.34.7, x86_64-linux.
|
||
|
||
## Optimization log
|
||
|
||
(Δ% is on nrThunks vs baseline for the affected target.)
|
||
|
||
| Cycle | Change | pureintent thunks | sincereintent thunks | Verdict |
|
||
|-------|--------|-------------------|----------------------|---------|
|
||
| 0 | baseline | 20,400,114 | 5,323,428 | — |
|
||
| 1 | pureintent: `documentation.nixos.enable = false` (drop NixOS options manual; option doc-strings no longer evaluated) | **19,201,996 (−5.9%)** | 5,323,428 | ✅ commit |
|
||
| 2 | home: `manual.manpages.enable = false` in shared `modules/home/default.nix` (drop home-manager options manpage; evaluated every HM option's doc) | **17,431,437 (−9.2%)** | **3,551,679 (−33.3%)** | ✅ commit |
|
||
|
||
> Note: `modules/home/default.nix` is also fed to every Linux system's *embedded*
|
||
> home-manager via `modules/nixos/default.nix`, so cycle 2 reduced pureintent
|
||
> too (its HM user's manpages were previously on).
|
||
|
||
| 3 | (profiling cycle — no change) investigated nixpkgs double-instantiation, unused inputs, `escapeShellArg` hotspot | 17,431,437 | 3,551,679 | — no behaviour-preserving win found |
|
||
| 4 | home: drop heavy `home.packages` from shared `cli/terminal.nix` — `yt-dlp`, `lima`, `omnix`, `pandoc`, `google-cloud-sdk` (kept `hledger`). *Behaviour change, user-approved.* | **16,989,720 (−2.5%)** | **2,757,201 (−22.4%)** | ✅ commit |
|
||
|
||
## Dead ends
|
||
|
||
_(investigated, no improvement — recorded so we don't retry)_
|
||
|
||
- **nixpkgs is NOT double-instantiated.** `nixos-unified.lib.mkLinuxSystem` calls
|
||
`nixpkgs.lib.nixosSystem` without pinning `nixpkgs.pkgs`, but the NixOS
|
||
`nixpkgs` module imports the package set exactly once; the `pkgs/top-level/impure.nix`
|
||
frame in the profile is that single import. `home-manager.useGlobalPkgs = true`
|
||
(set by nixos-unified) already shares it with the embedded HM. No win available.
|
||
- **Dropping unused flake inputs does not reduce eval counters.** Nix input
|
||
evaluation is lazy: an input not referenced during a config's eval (e.g.
|
||
`nixos-hardware`, `git-hooks` — 0 refs) contributes ~0 thunks to that config.
|
||
Pruning them shrinks `flake.lock`/`nix flake` overhead only, not `nrThunks`.
|
||
- **`escapeShellArg` (~72% inclusive on pureintent) is not a fixable hotspot.**
|
||
It is driven by `home-manager` file-linking (`files.nix:442`, ~43% — one
|
||
escaped command per managed dotfile) and NixOS `/etc` generation
|
||
(`etc.nix:58`). Both scale with how much is managed; nixpkgs/HM own the
|
||
implementation. Not patchable from this repo.
|
||
- **`programs.command-not-found` already disabled** by the `nix-index-database`
|
||
module; nothing to gain.
|
||
|
||
## Remaining cost is feature-bound
|
||
|
||
After the two documentation wins, the dominant `derivationStrict` targets in the
|
||
pureintent eval profile are all *wanted* features (samples ≈ inclusive share):
|
||
`home-manager-generation`/`home-manager-files` (~43%, embedded HM), `/etc`
|
||
(~29%), `vira.service`+`vira-wrapped`+`vira-0.1.0.0` (Haskell app), `kolu.service`,
|
||
`pipewire`, plus heavy packages in `home.packages` (`google-cloud-sdk`, `pandoc`,
|
||
`omnix` — Haskell). Further reductions require trading a feature, not a free
|
||
structural change.
|
||
|
||
## Final measurement (median irrelevant — counters are exact)
|
||
|
||
| Target | nrThunks (base → final) | Δ | nrFunctionCalls | nrPrimOpCalls |
|
||
|--------|-------------------------|------|-----------------|---------------|
|
||
| pureintent | 20,400,114 → **16,989,720** | **−16.7%** | 12,975,268 → 10,794,555 (−16.8%) | 6,507,025 → 5,327,976 (−18.1%) |
|
||
| sincereintent | 5,323,428 → **2,757,201** | **−48.2%** | 3,227,404 → 1,631,213 (−49.5%) | 1,565,120 → 771,171 (−50.7%) |
|
||
|
||
Both configs still instantiate cleanly (`nix build --dry-run` on the system
|
||
toplevel / home activationPackage succeeds).
|
||
|
||
## Key findings
|
||
|
||
1. **On a thermally-throttling machine, time is a lie — count work instead.**
|
||
The same eval reported cpuTime from 11s to 21s as the CPU clock sagged, while
|
||
`nrThunks` was byte-identical every run. Eval-work counters (`nrThunks`,
|
||
`nrFunctionCalls`, `nrPrimOpCalls`) are deterministic, zero-noise, and
|
||
machine-independent — a far better optimization metric than wall/cpu time.
|
||
2. **Generated option documentation is the biggest free win.** The NixOS options
|
||
manual (`documentation.nixos.enable`) and the home-manager options manpage
|
||
(`manual.manpages.enable`) each evaluate the doc string of *every* option.
|
||
Disabling both: pureintent −14.6%, sincereintent −33% — with zero behaviour
|
||
cost (options are searched online).
|
||
3. **A shared module multiplies.** `manual.manpages.enable = false` in
|
||
`modules/home/default.nix` helped both the standalone home config *and* every
|
||
Linux system's embedded home-manager (via `modules/nixos/default.nix`).
|
||
4. **Eval is lazy — unused inputs are already free.** Pruning unreferenced flake
|
||
inputs does not move the counters; it only trims `flake.lock`.
|
||
5. **The remainder is feature-bound.** After the doc wins, eval cost is dominated
|
||
by embedded home-manager file-linking, `/etc` generation, and the closures of
|
||
wanted services/packages. Trimming 5 heavy `home.packages` (user-approved) took
|
||
sincereintent to −48% total; the rest are features worth their eval cost.
|
||
|
||
## Cost breakdown (per-package eval cost, standalone `pkgs.<p>.drvPath` thunks)
|
||
|
||
Shared base floor ≈ 1.09M thunks; the excess is each package's own closure-eval:
|
||
|
||
| Package | thunks | excess | removed? |
|
||
|---------|--------|--------|----------|
|
||
| yt-dlp | 2.15M | +1.05M | ✅ |
|
||
| lima | 1.78M | +0.69M | ✅ |
|
||
| omnix | 1.76M | +0.67M | ✅ |
|
||
| pandoc | 1.46M | +0.37M | ✅ |
|
||
| google-cloud-sdk | 1.39M | +0.30M | ✅ |
|
||
| hledger | 1.35M | +0.26M | kept (wanted) |
|
||
| ripgrep / fd / just / television | ~1.09M | ~0 | kept (at floor) |
|