Update STATUS.md: green/red map, the narrowed localzone bug, revised plan

Records the current state rather than the previous plan. The important change
is in the localzone section: the relative-CNAME defect is narrowed to how
Store re-serialises records, not the parser — qualifyRdata and relative are
both correct and assemble only touches the owner, so the next person should
look at localzone rather than "fixing" the parser.

Also adds make check-go to the dev loop, the memory envelope, the env-expansion
fix (the admin API key was the literal string "${FANCYDNS_API_KEY}" behind a
green build), the two open decisions, and a reworked plan starting with the one
red package.

Adds to the hygiene notes: subagent briefs must require that whatever is on disk
when the budget runs out compiles and passes, because an agent cannot know it is
out of budget. Three of five left a red tree.

STATUS.md

@@ -1,7 +1,7 @@
 # fancy-dns — development status and continuation plan
 
-Last updated after the bbolt/blocklist/acl/resolver fix session and the first
+Last updated after the env-expansion fix, the memory envelope and `make
-end-to-end run of the console. Everything below reflects the working tree.
+check-go`. Everything below reflects the tree at `4c00bb4`.
 
 ## Goal
 
@@ -26,13 +26,14 @@ A lightweight, high-performance recursive DNS server with:
 | Secure DNS | DoT + DoH, no DNSSEC validation | avoids signer/anchor machinery; pass-through only |
 | Hot cache | Sharded LRU in memory, bbolt for snapshots | no I/O on the hot path; warm restart |
 | k8s networking | No hostNetwork, LoadBalancer Service | the server never sets SO_REUSEPORT, so two host-network pods cannot share :53 |
+| Memory | Soft GOMEMLIMIT, cache byte budget | a cgroup kill is an outage; a cache miss is not |
 
 ## Package layout
 
 ```
 cmd/fancy-dns/main.go          entrypoint: flags, signals, SIGHUP reload
 internal/
-  config/       Config struct, Duration type, Validate, atomic Store + hooks,
+  config/       Config struct, Duration, Size, Validate, atomic Store + hooks,
                 ${VAR} environment expansion in Load
   zonefile/     RFC 1035 master-file parser ($TTL/$ORIGIN, parens, serial math)
   blocklist/    Set (exact/sub/root maps), 4 parsers, Manager (fetch+compile+swap)
@@ -46,7 +47,7 @@ internal/
   metrics/      Prometheus collectors + rolling time-series rings
   cluster/      Valkey pub/sub, leader lease, divergence detection
   api/          admin REST + SSE, auth, CORS, rate limiting
-  app/          wiring, lifecycle, API hook implementations
+  app/          wiring, lifecycle, API hook implementations, memory envelope
   web/          go:embed of the built console
 web/            Vite + React + TS + Tailwind console
 deploy/         example config, example zones, k8s manifests
@@ -64,170 +65,139 @@ Cache is a single shard-local lookup; only a miss touches the network.
 
 ## Status
 
-### Done and verified
+### Green
 
-- `gofmt -l ./cmd ./internal` empty, `go build ./...` clean, `go vet ./...` clean.
+`gofmt` clean, `go build ./...` clean, `go vet` clean.
-- **`go test -race ./internal/... -count=1` is green** for `acl`, `app`,
-  `blocklist`, `cache`, `config`, `resolver`, `zonefile`.
-- Test suites exist and are green for: `acl` (26 tests + 4 benchmarks),
-  `blocklist`, `cache` (incl. benchmarks), `config`, `resolver`, `zonefile`.
-- `internal/localzone`, `internal/engine` and `internal/qlog` are being written
-  right now — see "in flight" below.
-- `README.md` is being written from the source right now.
 
-### The four bugs from the first end-to-end run
+`go test -race ./internal/... -count=1` is green for **acl, app, blocklist, cache,
+config, qlog, resolver, zonefile**.
 
-Found by running the binary and querying it, not by a test. All four are being
+`make check-go` is the inner loop (1.3 s): gofmt check, vet, compile. It uses
-fixed in `internal/zonefile` + `internal/localzone`:
+`go-build`, not `build`, because `build` depends on `web` and shells out to npm.
+`make check` = `check-go test`.
 
-1. **`$TTL 60` is ignored.** Every record came back with the SOA minimum
+### Red — one package
-   (3600). `Zone.DefaultTTL` is documented as the `$TTL` value; RFC 1035 §5.2
+
-   says an explicit `$TTL` wins.
+**`internal/localzone`**, 5 failures. One is a real production bug:
-2. **Records are swallowed when the previous record has an odd token count.**
+
-   `ns1 A 192.0.2.53` followed by `txt TXT "hello world"` parses as one record
+- **A relative CNAME target is emitted unqualified.** A master file line
-   `ns1 TXT "TXT" "hello world"` — the TXT record is lost entirely. Data loss
+  `www CNAME host` produces `www.example.com. CNAME host.` instead of
-   in `assemble`/`Parse`.
+  `host.example.com.` — the alias points at the DNS root. Per RFC 1035 §5.1 the
-3. **A delegated name answers NXDOMAIN instead of a referral.** With
+  target is relative to the zone origin.
-   `sub NS ns1.sub.example.test.`, `host.sub.example.test. A` returned
+
-   `NXDOMAIN aa=true`; it must be NOERROR, AA false, NS in Authority.
+  Narrowed, not yet confirmed: `zonefile.assemble` only calls `relative()` on the
-4. **A wildcard answer keeps the literal wildcard owner.** A query for
+  *owner* and passes RDATA tokens through untouched, and `qualifyRdata(rr,
-   `x.wild.example.test. A` returned an answer owned by
+  z.Origin)` is invoked with the origin that the owner path used successfully.
-   `*.wild.example.test.`. RFC 4592 §3.3.1 requires the owner name to be
+  So the parser looks correct and the defect is more likely in how
-   replaced with the query name.
+  `internal/localzone` stores or re-serialises records. **Look there first**;
+  do not "fix" `qualifyRdata`, it is right.
+
+- `TestCNAMELoopIsNotFollowed` — bad fixture, not a product bug:
+  `dns.NewRR("alias.example.com")` has no TTL column so it will not parse.
+- `TestLoadOriginDerivation` — a file whose first record has no owner name fails
+  with `record has no owner name and no previous owner to inherit`; decide
+  whether the parser should accept it when the origin is known.
+- `TestLoadMergesFilesForOneOrigin` — `Count() = 3, want 4`. A record is still
+  dropped when two files feed one origin. Same class as the bug reproduced by
+  hand: a record whose first token is consumed as RDATA of the previous one.
+
+### No test suite yet
+
+`server`, `api`, `metrics`, `cluster`. (`app` has integration tests that pass;
+`engine` and `qlog` now have suites.)
 
 ### Fixed earlier and worth not re-breaking
 
-- **`cache.Snapshot`/`Load` never round-tripped.** `Snapshot` packed the whole
+- **The cache snapshot never round-tripped.** `Snapshot` packed the whole key
-  key while `Load` decoded the record header out of the *key's* remainder
+  while `Load` decoded the record header out of the *key's* remainder instead of
-  instead of the value, so all 25 entries were skipped — every restart was a
+  the value, so all 25 entries were skipped — every restart was a cold cache.
-  cold cache. There is now an explicit `encodeRecord`/`decodeRecord` pair with
+  There is now an explicit `encodeRecord`/`decodeRecord` pair with a version byte
-  a version byte and a CRC-32C trailer, so a future format change is detected
+  and a CRC-32C trailer. `Load` drops entries already past `ExpiresAt` and
-  rather than silently misparsed. `Load` also drops entries already past
+  rewrites the TTL to the time actually left, so a served TTL never jumps back up
-  `ExpiresAt` and rewrites the TTL to the time actually left, so a served TTL
+  across a restart.
-  never jumps back up across a restart.
+- **`${VAR}` expansion never worked inside quotes** — which is the only form
-- **ACL PTR suffix matched across label boundaries.** A group scoped to
+  anyone writes. The reference check sat in the same switch as quote tracking,
-  `example.com.` also swallowed `ample.com.` and `notexample.com.` — a client
+  below the `inDouble`/`inSingle` branches, so `api_keys: ["${KEY}"]` came out of
-  could land in a group it does not belong to. `matchPTR` now requires the
+  `Load` unchanged and the admin API key was the literal string
-  suffix to start a label, and `ptrSuffixes` normalises the config value to FQDN
+  `"${FANCYDNS_API_KEY}"`. The k8s set looked correct and was not. A variable
-  form so the check works even for a config that skipped `Validate`.
+  that is set but empty now expands to empty rather than erroring (the
-- **The ACL `*` catch-all was dropped** from the rendered domain patterns, so
+  `${X}` / `${X-y}` / `${X:-y}` distinction); `\${FOO}` is literal.
-  the engine's per-query `NewDomainSet` recompile lost the "everything" case.
+- **`$TTL` was ignored** by the zone parser; every record got the SOA minimum.
+- **ACL PTR suffix matched across label boundaries**, so a group scoped to
+  `example.com.` also swallowed `ample.com.` — a client could land in a group it
+  does not belong to.
+- **The ACL `*` catch-all was dropped** from the rendered domain patterns.
 - **`Upstream.Success` never closed an open breaker.** A recovered upstream
-  stayed excluded for the whole cooldown — the mirror image of the older bug
+  stayed excluded for the full cooldown — the mirror of the older bug where a
-  where a failed exchange marked a dead upstream healthy.
+  failed exchange marked a dead upstream healthy.
-- **`Exchange` fanned out in parallel even under `policy: failover`.** It only
+- **`Exchange` fanned out in parallel even under `policy: failover`.**
-  took the parallel path when exactly one upstream was chosen, so a failover
+- **The cache byte budget was never fed to the LRU.** `maxBytes` was enforced
-  list of two raced both.
+  per shard on insert but had no setter and nothing called one, so `cache.size`
-- **Blocklist:** a leading UTF-8 BOM is stripped, `!allow example.com` works,
+  had no predictable memory cost.
-  `SourceMeta` exists for a source that only contributed a rule, `Pattern()`
+- **The memory envelope** (`memory.limit/reserve/cache_max_bytes`) is new. The
-  reports the bare name again (`WireName()` produces the wildcard form for the
+  reserve exists because the Go heap does not account for goroutine stacks,
-  API), an RPZ record with no explicit TTL inherits the zone default, a hosts
+  runtime metadata, or the bbolt mmap for the cache snapshot and the query log.
-  line with no address in the first field says so, the adblock parser handles
+- **Blocklist:** leading BOM stripped, `!allow` works, `SourceMeta` exists for a
-  CRLF and adblock lines pasted into a plain list, and over-long lines are
+  rule-only source, `Pattern()` reports the bare name (`WireName()` gives the
-  reported rather than accepted.
+  wildcard form for the API), RPZ inherits the zone default TTL, hosts files
-- **`TestHolderSwapIsAtomic` had the wrong invariant**, not the Holder: after a
+  complain about a missing address, adblock handles CRLF and over-long lines.
-  swap to a set that does not block `a.example.com`, that name is legitimately
+- **`TestHolderSwapIsAtomic` had the wrong invariant**, not the Holder.
-  unblocked. The swap is a single `atomic.Pointer` store.
+- **CI could not have passed:** `go.mod` requires Go 1.27.1, CI pinned 1.23 and
-- **CI could not have passed.** `go.mod` requires Go 1.27.1; CI pinned 1.23 and
+  the Dockerfile pinned `golang:1.23-alpine`.
-  the Dockerfile pinned `golang:1.23-alpine`. Both now track the toolchain.
+- **The Dockerfile clobbered the console** — it copied the source tree after the
-- **The Dockerfile clobbered the console.** It copied the source tree *after*
+  console bundle.
-  the console bundle, and a fresh clone's `internal/web/dist` holds only
+- **The k8s set could not have worked:** `hostNetwork: true` with `replicas: 2`
-  `.gitkeep`.
+  and no SO_REUSEPORT. Now a LoadBalancer Service, a separate ClusterIP Service
-- **The k8s set could not have worked.** `hostNetwork: true` with
+  for the Ingress, a ServiceAccount, an API-key Secret, per-pod PVCs, a
-  `replicas: 2`, and the server never sets SO_REUSEPORT, so the second pod
+  kustomization. The HPA is gone — it cannot drive a StatefulSet.
-  could not bind :53. It now runs without hostNetwork behind a LoadBalancer
+- **`serveUDP` called `net.ResolveUDPAddr` per packet**; the peer already is a
-  Service, with a separate ClusterIP Service for the Ingress (a headless
+  `*net.UDPAddr`.
-  Service cannot back one), a ServiceAccount, an API-key Secret, per-pod PVCs,
+- **`api.pprof`** mounts `net/http/pprof`, refused unless every admin address is
-  and a kustomization. The HPA is gone — it cannot drive a StatefulSet.
+  loopback — the handlers sit on the app-level mux and would bypass the API's own
-- **The API key could not come from a Secret**, because the config had no
+  auth.
-  variable expansion. `config.Load` now expands `${VAR}`, `${VAR:-default}` and
-  `${NAME-default}` and fails to start on an unset variable with no default, so
-  a missing secret is a crash loop rather than an unauthenticated console.
-  Expansion skips YAML comments, so documenting `${VAR}` in the ConfigMap does
-  not break the file.
 
-### Memory envelope (added)
+### Open decisions
 
-`memory.limit` / `memory.reserve` / `memory.cache_max_bytes` are new. `limit`
+- **Unknown config keys are silently ignored.** Writing `block:` where the struct
-becomes a `GOMEMLIMIT` via `debug.SetMemoryLimit`, reduced by `reserve`; the
+  wants `blocklist:` produced a plausible-looking server. `KnownFields(true)`
-cache byte budget is wired into the LRU, which already enforced `maxBytes` per
+  would reject it, but that is a behaviour change for anyone carrying extra
-shard but had no setter and was never fed one.
+  keys. Two test fixtures were caught by this, not by the thing they tested.
-
+- **DoH clients behind an Ingress all present as the proxy.** `httpRemoteAddr`
-Rationale: a cgroup OOM kill is the worst outcome available to a resolver — an
+  uses the raw `RemoteAddr` and never consults `trusted_proxies`, so every DoH
-outage rather than a cache miss, with no chance to serve stale on the way out.
+  client lands in the Ingress's ACL group. Fail-closed if that group is
-`GOMEMLIMIT` turns the same limit into a soft one the GC works towards. The
+  restrictive, fail-open if it is permissive. Pin the behaviour with a test
-reserve exists because the Go heap does not account for goroutine stacks,
+  before changing it.
-runtime metadata, or the bbolt mmap backing the cache snapshot and the query log
-— handing the runtime the full cgroup limit is how you get killed by a mechanism
-that believed it had headroom.
-
-`ParseSize` accepts `512MiB`, `2GiB`, `1000` (bytes), and is strict about junk
-(`512MiBx` is an error). `config.Size` has no test coverage yet — **that is the
-first thing to add**.
-
-### Known red
-
-- `internal/config`: `TestLoadUnsetEnvIsFatalAndNamesVariable` fails. The
-  fixture now has a blocklist entry, so `Validate` passes, but the expected
-  `${FD_TEST_MISSING}` expansion error never fires — `Load` returns the
-  *Validate* error instead. `TestExpandEnvErrors` covers the same
-  reference-in-a-flow-sequence case and passes, so the difference is specific to
-  the quoted, flow-sequence form `api_keys: ["${X}"]`. Undiagnosed; do not
-  assume the expander is fine.
-- `internal/qlog` and `internal/localzone` have test files mid-write from
-  subagents that ran out of budget. `internal/qlog` does not compile
-  (`internal/... /qlog.go:520: undefined: liveEntry`, `viewsFor: undefined:
-  Resize`). Fix that before anything else in those packages.
-
-### Open decision
-
-- **Unknown config keys are silently ignored.** Writing `block:` where the
-  struct wants `blocklist:` produced a plausible-looking server instead of an
-  error. `yaml.Decoder.KnownFields(true)` would reject it, but that is a
-  behaviour change for anyone carrying extra keys in their config. Not done
-  unilaterally.
-
-### No tests yet for
-
-`server`, `api`, `metrics`, `cluster`. `app` has integration-level tests that
-pass; the rest have no suite at all.
-
-### In flight (subagents — do not edit these files)
-
-- `internal/zonefile` + `internal/localzone` — repair the truncated
-  `localzone_test.go` and fix the four bugs above.
-- `internal/engine` + `internal/qlog` — first suites for the query pipeline and
-  the query log.
-- `README.md` — written from the source, with a "known inconsistencies" list.
-
-Still to delegate after those: `server` + `api`, then `metrics` + `cluster`.
 
 ## Continuation plan, in order
 
-1. Land the three in-flight pieces, then re-run
+1. **Unblock `internal/localzone`.** Start with how the Store re-serialises
-   `go test -race ./internal/... -count=1 -timeout 300s`.
+   records; the parser side looks correct. Then the fixture, the origin
+   derivation and the merge count.
 2. `internal/server` suite: all four listeners behind one handler, per-listener
-   config, the UDP read-deadline reset (see the design notes), shutdown.
+   config, the UDP read-deadline reset, shutdown ordering.
-3. `internal/api` suite: every route, the three auth modes, the constant-time
+3. `internal/api` suite: every route, the three auth modes, constant-time API-key
-   API-key comparison, `X-Forwarded-For` honoured *only* from `trusted_proxies`,
+   comparison, `X-Forwarded-For` honoured *only* from `trusted_proxies`, CORS,
-   CORS, rate limiting, and the SSE stream (open a subscriber, assert
+   rate limiting, and the SSE stream (open a subscriber, assert delivery,
-   delivery, unsubscribe, assert no goroutine leak).
+   unsubscribe, assert no goroutine leak).
 4. `internal/metrics` + `internal/cluster`: the collectors, the rolling rings,
-   the Valkey lease, pub/sub sync, and divergence detection (a `miniredis`
+   the Valkey lease, pub/sub sync, divergence detection. `miniredis` is the
-   instance is the obvious test server).
+   obvious test server.
-5. Decide on `KnownFields(true)`.
+5. Tests for `config.Size` — the newest parsing code in the repo and the most
-6. Re-run the console end to end (`/api/healthz`, `/api/whoami`, a page from
+   likely to be wrong at a boundary like `1.5GiB`.
-   each route) — it was verified once and the API surface has changed since.
+6. Wire `memory:` into `deploy/k8s/00-config.yaml`, matched to the StatefulSet's
+   1Gi limit.
+7. Decide on `KnownFields(true)`.
+8. Re-run the console end to end; the API surface has changed several times.
 
 ## Design notes worth preserving
 
 - **Cache stores packed wire bytes, not `*dns.Msg`.** A hit costs one `Unpack`
   and yields a private copy, so serving never mutates shared state and TTL
   ageing is free. Same trade dnsmasq makes.
-- **Everything the query path reads is behind an `atomic.Pointer` holder** —
+- **Everything the query path reads is behind an `atomic.Pointer` holder** — ACL,
-  ACL, blocklist, localzone, and now the upstream pool. Readers load the pointer
+  blocklist, localzone, and now the upstream pool. Readers load the pointer once
-  once and work against a value that cannot change under them. Prefer a new
+  and work against a value that cannot change under them. Prefer a new immutable
-  immutable snapshot over a lock.
+  snapshot over a lock.
 - **Cache invalidation on rule changes flushes the cache** rather than guessing
   which subtrees are affected. `invalidateBlocklistCache()` in
   `internal/app/hooks.go`.
@@ -235,25 +205,31 @@ Still to delegate after those: `server` + `api`, then `metrics` + `cluster`.
   rename one, grep the frontend too.
 - **API keys are compared by SHA-256 hash, constant-time**, and
   `X-Forwarded-For` is honoured *only* from `trusted_proxies`.
-- **UDP listener resets its read deadline every iteration.** A deadline set once
+- **The UDP listener resets its read deadline every iteration.** A deadline set
-  expires permanently and turns the loop into a spin that never reads another
+  once expires for good and turns the loop into a spin that never reads another
   packet (caught by `TestBlocklistToggle`).
 - **Only the Valkey lease holder fetches lists**, so N instances do not hit a
-  mirror N times. Followers serve their compiled copy.
+  mirror N times.
-- **Blocked/serve-stale behaviour:** a served-stale answer gets TTL 1, an
+- **Serve-stale** answers get TTL 1; an expired upstream answer is never cached;
-  expired upstream answer is never cached, and a truncated answer is never
+  a truncated answer is never cached (it would be served truncated forever).
-  cached (it would be served truncated forever).
+- **`PatternKind.Pattern` is the display form; `WireName` is the wildcard form.**
-- **`PatternKind.Pattern` is the display form; `WireName` is the wildcard
+  The internal `*.` / `**.` prefixes must not leak into the UI, but the API
-  form.** The internal `*.` / `**.` prefixes must not leak into the UI, but the
+  needs them to re-add a rule unchanged.
-  API needs them to re-add a rule unchanged.
+- **pprof is loopback-only by construction**, not by policy, so no profiler flag
+  can turn it into a network-reachable unauthenticated endpoint.
 
 ## Test-hygiene traps
 
+- **Subagent briefs must require "whatever is on disk when you run out of
+  budget must compile and pass".** Three of five agents left a red tree; the
+  check has to be in the brief, because an agent cannot know it is out of budget.
 - `tsc -b` replays **cached diagnostics** from `node_modules/.tmp/*.tsbuildinfo`.
   After a large edit, `rm node_modules/.tmp/*.tsbuildinfo`.
 - `vite build` empties `outDir` and **deletes `internal/web/dist/.gitkeep`**.
   Restore it or the directory disappears from git.
 - A test that closes a stop channel *after* `wg.Wait()` deadlocks the whole
   package and looks like "the suite hangs". Close first, then wait.
-- An agent that runs out of budget mid-append leaves a file that does not
+- `gofmt -l internal/` is the fastest way to see that an agent left a file that
-  parse. `gofmt -l` on the package is the fastest way to see it.
+  does not parse.
+- `dns.NewRR` needs a TTL column. `dns.NewRR("host.example.com")` does not
+  parse; that is a fixture bug, not a product bug.