mirror of
https://github.com/srid/nixos-config.git
synced 2026-07-16 22:01:33 +08:00
b4a60aaf3a
* ralph(eval): baseline + benchmark harness pureintent cpuTime 23.17s, zest cpuTime 14.93s (median of 5). Methodology: NIX_SHOW_STATS cpuTime, eval-cache off, warm fetcher cache. * ralph(eval) cycle 1: disable NixOS options manual on pureintent documentation.nixos.enable = false. Generating the NixOS manual evaluates every option's doc string. nrThunks 20,400,114 -> 19,201,996 (-5.9%), nrFunctionCalls -6.2%, nrPrimOpCalls -7.5%. Package man pages unaffected. Metric pivoted to deterministic NIX_SHOW_STATS counters: cpuTime is unusable on this thermally-throttling machine (same eval = 11s..21s). * ralph(eval): switch home target zest -> sincereintent (per request) sincereintent home config baseline: nrThunks 5,323,428 / calls 3,227,404 / primops 1,565,120 (deterministic). * ralph(eval) cycle 2: disable home-manager options manpage manual.manpages.enable = false in shared modules/home/default.nix. HM builds home-configuration.nix(5) by default, evaluating every HM option's doc string. sincereintent nrThunks 5,323,428 -> 3,551,679 (-33.3%), calls -35%, primops -39%. * ralph(eval): record cycle-2 pureintent gain (shared HM module) pureintent nrThunks 19,201,996 -> 17,431,437 (-9.2%); cycle-2's manual.manpages.enable=false also feeds pureintent's embedded home-manager via modules/nixos/default.nix. Total pureintent -14.6% from baseline. * ralph(eval) cycle 3: profiling + dead-ends (no behaviour-preserving win) Confirmed: nixpkgs single-instantiation (no 2x trap), unused inputs are lazy (no counter impact), escapeShellArg hotspot is HM file-linking + /etc (not patchable here). Remaining eval cost is feature-bound (embedded HM, vira/kolu, google-cloud-sdk/pandoc/omnix). * ralph(eval) cycle 4: drop heavy home.packages (yt-dlp/lima/omnix/pandoc/gcloud) User-approved behaviour change. Removed from shared modules/home/cli/terminal.nix (kept hledger). pureintent nrThunks 17,431,437 -> 16,989,720 (-2.5%), sincereintent 3,551,679 -> 2,757,201 (-22.4%). Totals from baseline: pureintent -16.7%, sincereintent -48.2%. * ralph(eval): finalize report (final measurement, key findings, cost breakdown)
32 lines
1.6 KiB
Bash
Executable file
32 lines
1.6 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
# Ralph eval-time benchmark — counter-based.
|
|
# Usage: ralph-bench.sh [target] target = pureintent | sincereintent | (both)
|
|
#
|
|
# Primary metric: deterministic eval-work counters from NIX_SHOW_STATS:
|
|
# nrThunks, nrFunctionCalls, nrPrimOpCalls.
|
|
# These are byte-identical across runs (independent of CPU clock/load), so a
|
|
# single eval gives an exact number. cpuTime is NOT used: this machine thermally
|
|
# throttles, inflating CPU-seconds for identical work (11s..21s for one eval).
|
|
# Eval cache disabled; fetcher cache warm via persistent $HOME.
|
|
set -uo pipefail
|
|
|
|
export HOME=/tmp/ralph-evalhome
|
|
mkdir -p "$HOME"
|
|
FLAKE="${FLAKE:-/home/srid/code/nixos-config/.worktrees/ralph}"
|
|
ONLY="${1:-}"
|
|
|
|
declare -A ATTR
|
|
ATTR[pureintent]="$FLAKE#nixosConfigurations.pureintent.config.system.build.toplevel.drvPath"
|
|
ATTR[sincereintent]="$FLAKE#homeConfigurations.\"srid@sincereintent\".activationPackage.drvPath"
|
|
|
|
num() { grep -o "\"$1\": [0-9.]*" /tmp/ralph-stats.json | grep -o '[0-9.]*' | head -1; }
|
|
|
|
for target in pureintent sincereintent; do
|
|
[ -n "$ONLY" ] && [ "$ONLY" != "$target" ] && continue
|
|
NIX_SHOW_STATS=1 nix eval --raw "${ATTR[$target]}" --option eval-cache false \
|
|
>/dev/null 2>/tmp/ralph-stats.json
|
|
if [ $? -ne 0 ]; then echo "$target FAILED"; tail -8 /tmp/ralph-stats.json; exit 1; fi
|
|
printf '%-11s thunks=%-10s calls=%-10s primops=%-9s | values=%s sets=%s\n' \
|
|
"$target" "$(num nrThunks)" "$(num nrFunctionCalls)" "$(num nrPrimOpCalls)" \
|
|
"$(num number)" "$(grep -o '"sets": {[^}]*"number": [0-9]*' /tmp/ralph-stats.json | grep -o '[0-9]*$')"
|
|
done
|