Concurrent cache misses for one name each went upstream. After a restart, a
cache flush or a cluster-object change that is a burst of identical queries
exactly when the cache is empty. Identical exchanges (same name, type, class,
DO bit, pinned upstream set and forwarded client subnet) now share one: the
first query runs it on its own goroutine and context, the rest wait and get
private copies, and the leader alone writes the cache. If the leader's client
leaves, waiters take over instead of failing with its cancellation. Shared
answers are counted in fancydns_upstream_coalesced_total.
validate() accepted OpcodeUpdate, so a dynamic update was forwarded upstream as
an ordinary lookup, and every other opcode got FORMERR. A recursive resolver
implements QUERY only: anything else is now NOTIMP (RFC 1035 4.1.1), checked
before the question count since STATUS carries no question.
Prefetch only fired when a stale entry was served, so the first client after a
hot name expired got a TTL-1 stale answer. A fresh entry is now refreshed in
the background when it has been served at least twice and under a tenth of its
lifetime remains; the hit counter restarts with each stored entry, so a name
nobody comes back for stops being refreshed. Counted in
fancydns_cache_prefetch_total.
miekg/dns matches only the message ID of an upstream reply, so a forged reply
with the right ID for a different question was accepted and cached under the
name that was asked. exchangeOne now requires the reply to carry the question
that was sent (an empty section only for FORMERR, SERVFAIL, NOTIMP and
REFUSED). A mismatch fails the attempt without counting against the upstream,
so a spoofer cannot trip the circuit breaker.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
fancy-dns-console is a second binary holding the web UI, authentication and
every edit. It writes ClientGroup, BlockList and DNSZone objects through the new
kubeedit package and forwards all other /api routes to a running resolver
with its own key, dropping the browser's credentials, so it can run in the
cluster or on a workstation with a kubeconfig and nothing about the resolver's
placement matters. The resolver never imports kubeedit and the console is not on
the DNS path.
kubeedit validates every spec with the runtime's own converters
(kubesource.CheckGroup/CheckList/CheckZone) before writing, so the console
cannot store an object the resolver would reject. Zone edits keep the
file-based writer's behaviour: duplicate and below-a-delegation records are
refused, a delegation in a new zone also gives the zone its servers, and a
zone with no apex NS is never stored. Each object records the editing user in
an annotation; updates retry on conflict.
The resolver reports groups from ClientGroup objects as source "kubernetes" and
counter-only rows (the "(default)" bucket, or a group removed since) as
"history", and lists DNSZone zones as the editable ones. The console uses that
to leave the YAML's groups alone and to drop history rows, which the UI would
otherwise send back on save. Its GET /api/clients overlays the cluster's
current objects on the resolver's list, because the resolver only sees an edit
after its watch fires and a save made just after another would delete the
first.
An end-to-end test runs both halves against one fake apiserver: a group saved
through the console refuses queries, a zone built record by record answers
authoritatively, and a request with the resolver's own key is rejected by the
console.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The runtime can now take ClientGroup, BlockList and DNSZone objects
(fancydns.k6n.net/v1alpha1) from a cluster with -kube, from a pod's service
account or a kubeconfig alike. internal/kubesource watches them with dynamic
informers, converts and validates each object on its own, and publishes an
immutable snapshot. A bad object is skipped and keeps its last good version,
so one broken edit cannot remove a running policy; an invalid merged config is
dropped whole. The last snapshot is saved so a restart without apiserver access
still serves.
config.Store gains an overlay: Base() is what the YAML says, Get() is base plus
cluster objects. Everything that writes the config back (mutateConfig,
SetAdminAddr, the settings editor) now starts from Base(), otherwise cluster
objects would be copied into the file and shadow their own objects. YAML wins
every name collision. putACL and toggleBlocklist refuse to edit cluster-owned
objects. BlockList has no path field and refuses file:// URLs so a namespaced
object cannot pick files the runtime reads or writes. DNSZone is rendered
through the zone writer's own prove-by-reparse step, with the SOA serial taken
from metadata.generation so every replica renders identical bytes.
qlog: Flush reset the pending slice to [:0], so entries logged while it was
still storing the batch were appended over the same backing array. The race
detector found it through the new Kubernetes integration test. pending is now
set to nil, with a regression test that fails without the change.
CRD manifests, RBAC and examples are in deploy/k8s/crds; tests pin the schemas
to the Go spec structs so the two cannot drift.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The initial upstream exchange ignored its supplied timeout and used hardcoded client limits. Bound each attempt with a context, copy the shared client for its timeout, and share the attempt budget with UDP-to-TCP fallback. Close private connections on cancellation to interrupt reads, and exclude caller cancellation from upstream failures so parallel losers remain healthy.
The normal engine pipeline never applied server.timeouts.query, allowing failover to outlive the configured query budget. Apply a hot-reloaded overall context deadline at Handle while retaining earlier caller deadlines. Add silent UDP/TCP/TLS, cancellation, fallback, failover, and reload regressions.
The server inflight test assumed receiving a TCP reply meant its sent counter had already been incremented. Wait for both completion counters before asserting them, preserving the existing assertions while removing the scheduling race.
The query log classified every non-drop blocklist match as a block, even when an RPZ A or CNAME rule returned a successful redirect. Record redirect results with their own action so NOERROR address overrides are not presented or counted as blocked. Expose the new action in the Logs filter.
RPZ A records were parsed as address redirects, but the query pipeline treated the address as a CNAME target and returned the wrong result. Answer IP-literal redirects directly with the matching A or AAAA record, and return NODATA for other query types.
Add the requested EdgeRouter domain overrides as a local RPZ source so each bare owner covers its apex and all subdomains. Mount that source into the pods and document the address-record form. When the Zones page has no existing zone, start the Add Record form in New zone mode and tell the operator to enter its apex.
The published image is built for amd64 only, but the StatefulSet had no architecture constraint and could be scheduled on arm64 nodes. Restrict pods to Kubernetes amd64 nodes so every replica can run the image.
The Kubernetes config left blocklist.managed_zones empty, so the Zones page was read-only even though the pod has a writable persistent data volume. Configure a managed-zone directory on that volume so the console can create and update zone records; existing local_zones files remain operator-owned and read-only.
A textarea holding a master file is a text box: it says nothing about what the
resolver will answer for the zone, and a typo in a record is invisible until a
client sees the wrong answer. `/api/zones` already knows — it is compiled from
the same files — so Local zones now shows the records the daemon has actually
compiled out of the selected file, with the SOA serial and the number of
delegations, above the editor that writes it. It is the same list the Zones
page shows, which is the point: the two pages are two views of one thing, not
two ways of guessing.
The Zones page gains the way back: a zone that comes from a master file rather
than the managed directory has an "Edit the file" button that goes to Local
zones, so a read-only row is a link rather than a dead end.
The committed index.html is the shell the Go binary embeds, so it has to match
whatever the console now builds. Both zone pages are in this build: the
record-level one on /zones and the file-level one on /local-zones.
The console could not see local zones at all. `blocklist.local_zones` compiled
a store that nothing read back: there was no API for a zone, its records, its
delegations or the file a record came from, and no way to change one without
editing YAML and reloading. The Domains page is the blocklist's inline rules,
not local data, so nothing in the console covered the half of the server that
answers authoritatively from disk. There is now a Zones page and the endpoints
behind it: `GET /api/zones`, `POST`/`PUT`/`DELETE /api/zones/records`,
`POST`/`DELETE /api/zones/delegations` and `DELETE /api/zones?zone=`.
**The console owns one directory and nothing else.** `blocklist.managed_zones`
holds one `<apex>.zone` per zone, with that zone's SOA, apex NS set and the
records added there, and it is compiled by the same `localzone.Load` as the
operator's files — so a record added from the console is served on the next
query, with no reload and no restart. Files in `blocklist.local_zones` are
listed and shown read-only and are never rewritten; the key defaults to empty,
so a deployment that never sets it gets a zone browser and no zone writes.
Which file a record came from is now part of the store (`zone.files`, keyed by
the index the store already uses), so the snapshot can say "console" or "your
file" per record; the query path does not read it.
**The writer proves its own output.** A record is validated by the project's own
master-file parser, so `www` means `www.<zone>` and a bare CNAME target is
completed exactly as it would be in a zone file; the file is rendered, and the
rendering is re-parsed and compared with the records it meant to write before
the file is renamed into place. A writer bug that dropped a record would
otherwise present as a zone that silently stopped answering. A zone that cannot
answer is refused rather than created: no SOA means no NXDOMAIN or NODATA
authority, and no apex NS means nothing is delegated and no server is named. The
SOA's primary follows the apex NS set — a zone whose SOA names a server outside
its own NS set hands out a referral no resolver can follow — and the serial
moves on every write.
**A delegation takes the names under it, so a record below a cut is refused.**
The store answers those names as delegated without reading the zone file, which
would make the record data that looks alive and never answers. `DELETE
/api/zones?zone=` exists for the same reason from the other side: a zone the
console created is a real zone, so it says NXDOMAIN for what it does not hold,
and deleting a record is not what gives a name back to upstream. It is
refused for a zone that also has records outside the managed directory, because
removing that file would remove none of the zone.
**Two bugs the work turned up.** `engine.applyConfig` published local-record
metrics only when `blocklist.local_zones` was non-empty, so a zone the console
created was served but never counted; the gate is now the store's own record
count. And the record-value renderer fell back to `RR.String()` for every type
it did not know, which is the whole line rather than the RDATA — it wrote a
name and a TTL into the value column for DHCID, LOC, SSHFP and SVCB, and would
have fed that back as a record value on the next edit.
Every write recompiles the store and flushes the cache. Which names changed is
not knowable from the API, and a cached upstream answer for a name the zone now
answers is simply wrong, so the flush is blunt on purpose.
The wire shapes are type aliases in `internal/api` rather than copies, so there
is one set of json tags, and `TestZoneEndpointKeySets` pins the key set at every
level with a record and a delegation actually present — an empty list only
proves the envelope. `TestZoneWritesNeedAManagedDirectory` pins the read-only
default, and `TestZoneConsoleRoundTrip` walks the whole thing: a record added
through the API is answered authoritatively on the next query, a cut stops the
parent answering below it, and deleting the zone hands the name back.
Verified against a running instance built from this tree: records and
wildcards answer with `aa`, relative names are qualified, a record below a cut
is refused with a message that says why, and the file on disk is a zone file
an operator can read. The console page is typechecked and built; the browser was
not available in this round, so it has not been looked at.
Local zones were only editable as mounted files, so operators could not manage DNS entries from the console. Add a Local zones editor backed by a shared Kubernetes ConfigMap, validate master files before saving, scope the pod service account to that ConfigMap, and reload changed zones on every replica while flushing stale cache entries.
The post-build webhook returned success while ignoring the delivery because it omitted X-Gitea-Event: push. As a result, GitSource kept the prior revision and the published console image was never reconciled. Add the event header so the signed notification is queued only after the image has been published.
The relative Vite base made BrowserRouter pages request JavaScript and CSS under /auth/assets. The SPA fallback returned index.html for those paths, so the browser never mounted React and the sign-in page appeared blank. Emit root-relative asset URLs so every client-side route loads the embedded console bundle.
internal/server was the last package whose shutdown was tested by assumption:
four transports, three shutdown shapes, and every one of them can strand a
goroutine that a client is holding open. Both bugs below were live in the tree
as handed over and neither had a test.
An idle client could hold the whole shutdown open. The stream accept loop waits
for its connections before Stop returns, and a connection parked in ReadMsg with
no deadline in front of it only ends when the client goes away — ctx.Err() is
checked between reads, so cancelling the context cannot reach a read that is
already blocked. One connection that connected and said nothing was enough to
make Stop sit there for the caller's entire grace period (15s in main) and then
return context.DeadlineExceeded, which the app reports as "shutdown completed
with errors"; and the cache snapshot the app takes right after Stop never
happened, because the orchestrator had given up first. It is a shutdown DoS that
costs the attacker one connection. The fix is a read deadline in the past on
every accepted connection when the context is cancelled, and deliberately not a
close: a query already inside the pipeline is past its read, so its answer still
goes out, which is the invariant TestInFlightQueryIsAnsweredAcrossStop pins. The
accept loop keeps a small registry of live connections for this — the accept
path, not the query path, and no query ever takes that mutex.
A UDP query in flight at Stop was lost and nothing waited for it. The stream
paths register their connections with s.wait; servePacket did not register at
all, so a UDP query already inside the pipeline was gone twice over: nothing
waited for it, and the answer had nowhere to go once it was ready, because a
datagram is written to the very socket the query arrived on. The visible damage
was not the lost answer (the client retries) but the app, which snapshots the
cache and closes the store the moment Stop returns — a query still in the
pipeline could be writing to a store on its way out. Packets are now counted in
s.wait, and the UDP listener's close wakes the read loop with an expired
deadline and then waits, bounded by udpDrainGrace (250ms), for the loop and the
packets it spawned before closing the socket. The bound is deliberate: a
shutdown may not spend its grace period on one slow upstream, and past the bound
the answer is dropped, which is what the client would see anyway.
Stop returning did not mean the process was done, so nothing could tell a
leaking shutdown from a clean one.
TestStopLeavesNoGoroutineRunningServerCode counts goroutines whose stack still
has a frame in the package — not runtime.NumGoroutine(), which in a test binary
is net/http, crypto/tls and the race detector talking over each other — and it
found the idle-connection bug above. TestStopIsNotBlockedByAnIdleStreamConnection
sets a 60s DoT idle timeout on purpose so a config value cannot hide the
regression behind a lucky tick, and TestInFlightQueryIsAnsweredAcrossStop now
covers all four transports. harness.idleStream exists because a test that only
dials a stream transport races the accept loop: Stop can close the listener
while the connection is still in the backlog, the server never accepts it, and a
shutdown assertion that needs a live server-side goroutine passes for the wrong
reason.
AGENTS.md gains the two durable invariants (no step in the shutdown may wait for
a client; every query goroutine is registered with s.wait, UDP included) and the
two test-hygiene traps. STATUS.md drops the UDP query lost at Stop from its
pinned list, since it is now a test rather than an accident, and the plan starts
at the pinned decisions that are actually left.
internal/app/contract_test.go asserts the exact JSON key set of every admin
endpoint, and the console was reading fields that shape does not contain —
which is how a page can be entirely blank while the Go suite is green. The
types file is rewritten against the canonical shapes and every consumer is
fixed onto it.
The page bugs that came out of it:
The cache "Lookups" column read `count` and "Last seen" read `last_seen`; the
cache tracks a hit counter and a byte count per name and sends `hits`, so
there is no last-seen data to show at all. Lookups now reads hits, Last seen
is gone, and the row's bytes are shown. The "Size" card shows bytes_human,
with size as the entry-budget hint it actually is.
The Sources table read `item.enabled`, `item.rules`, `item.bytes`,
`item.updated_at` and `item.err`; those live under `item.meta`, so the table
was blank and the per-row toggle posted `name: undefined`. The refresh toast
counted the names echoed back and said "every source" for an empty list.
The dashboard read `qps` and `total_queries` where the server sends
`queries_per_sec` and `queries_total`, and rendered a `latency`/`p99` block
the dashboard response never contained. The `Partial<Dashboard>`-and-cast
that papered over this is gone. `blocked_pct` arrives as 0..100 from REST and
0..1 over SSE, so formatPercent is 0..100 only and a new formatFraction
handles the SSE frame.
Logs rendered `qtype` as a string; it is a number, with `qtype_name`
alongside, so there is a formatQtype that prefers the name. The Domains page
read `item.list` where the server sends `source`, and showed an Updated column
with no timestamp behind it.
Upstreams treated `healthcheck_interval` and `breaker_cooldown` as numbers of
seconds; they are duration strings now, so the page parses them like the
bootstrap refresh it already handled and the write payload no longer sends the
runtime health fields the server ignores. A group edit stripped the four
server-owned counters before writing, because config.Group would reject them.
`whoami.can_write` and `whoami.read_only` were fetched and never used, so a
read-only API key still showed every Save control. A single whoami fetch now
lives in a session provider behind the app shell, every write control is
gated on it, and a banner says why when the key is read-only. It fails open:
an unreachable probe must not lock an operator out of their own console.
Two server fixes the console work exposed, both pinned by the contract test:
POST /api/blocklists/refresh with a body that omits `names` answered
{"names": null}, and the console counts the names it gets back to build its
toast — so "refresh everything" said "0 sources". A nil slice now becomes an
empty one and the key set does not depend on the request.
blocklist.SourceMeta.duration was a raw time.Duration, so it went out as a
nanosecond count and the console divided by 1e6 to compensate — the only
duration on the wire that was not a duration string. It is a config.Duration
now, and formatDuration parses it like the others.
The contract test grew to cover what the first pass left out: the domain
check in all three of its shapes (match, miss, and an error delivered as 200),
the log clear, the settings validate in both the valid and the malformed case,
the settings save and reload, and the first SSE frame.
The console's TypeScript types are hand-written and apiFetch ends in
`JSON.parse(text) as T`, so nothing in the toolchain compared them with the
Go structs. A field that lost its json tag, or a response a struct never had,
showed up as an empty table in a browser and nowhere else: the Go suite was
green and the console was wrong. internal/app/contract_test.go now asserts the
exact JSON key set of every admin endpoint, driven through the real router
with the real hook implementations, plus a test that drives the two write
endpoints with the exact payload web/src builds.
The drift it found, all four fixed here:
blocklist.Source and SourceMeta had no json tags at all, so GET /api/blocklists
sent Go field names ("URL"/"Format") where the console expects snake_case: the
Sources table rendered blank rows and its per-row toggle posted
`name: undefined`. Tagged to match ParseStats.
acl.GroupSummary was flat where the console reads a nested `match` object — the
same shape config.Group has — so every group card read "matches nothing". The
four match fields are nested now, under a new acl.Match.
PUT /api/clients answered 400 `json: unknown field "ptr_suffixes"` to the
console's payload, because config.Group and config.Match carried yaml tags only
and the decoder runs with DisallowUnknownFields. Both structs have json tags
now; the same applies to config.UpstreamServer and UpstreamBootstrap.
PUT /api/upstreams answered 400 `json: unknown field "health"`, because the
console round-trips the whole server object it read, runtime health fields
included. They are accepted and ignored, because the server owns them, and they
are `yaml:"-"` so settings.yaml does not gain runtime state.
Also fixed, each with a test:
A duration on the wire was a nanosecond count. config.Duration had a
MarshalYAML but no MarshalJSON, so bootstraps[].refresh arrived as 30000000000
while the console typed the field as a string, and healthcheck_interval was a
float of seconds in one place and a string in another. Durations on the wire
are duration strings everywhere now — the same spelling config.yaml uses — and a
bare JSON number is still accepted and means seconds, not nanoseconds.
TestDurationJSONForm used to pin the old behaviour and now pins the new one,
saying in its comment that the old behaviour was the bug.
A log row's key set depended on the row. api.LogEntry was omitempty on
client_group, response, upstream and list, so a query with no ACL group
arrived without the key and the console rendered "undefined" where the column
should be empty. The contract test can only pin one shape if the server emits
one, so nothing on that struct is omitempty any more.
A dead peer stayed in the cluster peer list forever. Cluster.seen was only ever
added to and deleted from on an explicit `bye`, so a node that crashed — or
whose Valkey connection died — was reported as a live peer for the life of the
process, and its stale blocklist digest was flagged as divergence forever.
Peers() now reaps peers unheard for peer_ttl*4 and stamps each state with the
receiver's own clock, so clock skew between two machines can neither reap a
healthy peer nor keep a dead one. Discover() ages out the same way on the
publisher's own last_seen stamp, because the keyed Valkey record outlives a
crash: a live node keeps refreshing the key's TTL, so a dead node's field stayed
readable for the life of the fleet.
peer_ttl: 0 would have crashed the process. time.NewTicker(PeerTTL/2) panics on
a non-positive period and `EXPIRE key 0` deletes the heartbeat key, so a config
that left peer_ttl unset would have taken the process down at Start, or made
every instance believe it is alone in the fleet. Both go through peerTTL() and
peerDeadline(), which fall back to the default.
Two cluster tests raced on the wire, not on the product, and one was
environment-dependent rather than wrong. A node that joined after another's
heartbeat was in flight hears about it on the next tick, so "b does not see a"
read immediately after "a sees b" is a coin flip; the divergence test waited for
a digest on the wrong node's view of the fleet; and 127.0.0.1:1 is only
unbindable outside a container that sets
net.ipv4.ip_unprivileged_port_start=0, where it binds and the partial-bind test
fails with both transports up. That test now occupies a port with a live
listener in the test process, which is a deterministic EADDRINUSE on every
machine and every uid.
internal/server/counters_test.go covers the counter-key layer that no other
test touched: the "<transport>.<field>" keyspace, that the aggregate folds only
the known fields, and that the snapshot is safe while counters move.
Two pinned behaviours changed on purpose, each with a test that says so:
An RPZ record did not cover the name it names. rpzRule gave a bare owner name
the subdomain form (*.name), so analytics.example.org. in a policy zone blocked
cdn.analytics.example.org. and left analytics.example.org. resolving normally —
and an rpz-passthru. record did not allowlist its own domain either. RPZ
semantics say the record applies to the domain and everything under it, so a
bare owner now takes the apex form **.name; an owner written with an explicit *.
is still strict, so a zone that wants subdomain-only behaviour can still ask for
it. This changes which queries get blocked for an existing policy zone, which is
why it wanted a decision. TestExampleRPZZoneCoversTheApexItNames pins both
halves and the example zone now says which half it relies on.
metrics.OutcomeLocal had no producer. The engine reported a local-zone answer as
qlog.ActionAllow, which outcomeFor mapped to OutcomeUpstream, so every locally
answered query was counted as an upstream one and the outcome="local" series was
never published — the dashboard's local column was permanently empty.
Response.Local is now set on the local-zone path and outcomeFor consults it
first. That moves published counters, so the README's metrics section says so
too.
The container limit is 1Gi, so declare the envelope next to it: a 1GiB soft
heap limit, a 256MiB reserve for what the Go heap does not account for, and an
explicit 384MiB cache byte budget. cache.size is an entry count, so the byte
budget is what actually bounds the cache now that ParseSize applies its units.
STATUS.md records the round: the fixes, the new suites (engine, config.Size,
metrics, and the api suite that landed separately), three findings left
unfixed -- the login rate limiter never applies to the login endpoint, auth
configuration is frozen at New, and api.constantTimeEqual is dead -- and the
continuation plan with items 1, 2, 5 and 6 struck.
internal/engine, internal/config and internal/metrics had no test file at all,
and writing them turned up five real bugs.
internal/engine (23 tests, new harness_test.go): a wired engine over a fake
upstream on loopback, a local zone, a blocklist, a cache, a query log and an
injectable clock. It pins the pipeline order end to end -- ACL deny before the
local zone, local zone before blocklist, blocklist before cache, drop sends
nothing, the concurrency and rate-limit refusals, serve-stale served with TTL 1
and refreshed once per key, truncation against a 512-byte client, and exactly
one metrics record and one query-log entry per query.
Five fixes:
- config.ParseSize ignored the unit multiplier, so 512MiB parsed as 512 bytes
and the whole memory envelope was inert for anyone who wrote a unit.
- finalize (EDNS0 and the UDP size) ran only on the cache-hit path, so a large
uncached answer left oversized for a 512-byte client. It now runs on the
upstream, local-zone and blocklist paths too.
- The upstream UDP client read buffer was hard-coded to 1232 while we advertise
dns.max_udp_size, so with a larger buffer configured every answer above 1232
bytes was read truncated and became SERVFAIL. The pool takes the size now.
- LRU.prefetchGate was a 64-slot global cap, not the single flight its comment
promised: a burst of stale hits launched up to 64 duplicate refreshes of one
name. It is a per-key latch now; distinct names still refresh in parallel.
- metrics.RecordMiss and RecordStaleHit moved only the Prometheus collector, so
the Cache page's misses and stale hits read 0 for the life of the process.
AGENTS.md gains the trap that cost the most time: this version of miekg/dns
reports a character-string over 255 bytes as zero length, so such a message
silently fails to pack, and the write failure looks like a client timeout.
internal/api is the admin REST + SSE surface and had no test file at all,
so nothing pinned the auth modes, the X-Forwarded-For trust boundary, the
rate limiter or the fact that a read route must not mutate anything. This
adds a local harness (a Server over fake hooks on an httptest listener,
with a real config file on disk) and three test files.
What it pins:
* Routing. A table over every route in Server.routes: each one answers 200
and reaches the hook it is supposed to, so a new route without a case
here is a route nothing tests. Plus method-not-allowed / 404 / outside
the base path, a base path other than /api, and a read-route sweep that
asserts no mutating hook was reached, the config generation did not move
and the file on disk is byte-identical afterwards. A panicking component
becomes a 500 and the server keeps serving, because the admin API is the
only way to fix a broken instance.
* Auth. All three modes. apikey: the right key always works, and a prefix,
a suffix, a case-folded key, a wrong basic password, an unknown bearer
token, an unsupported scheme and a malformed Authorization header never
do; curl -u any:KEY is accepted because that is what clients send.
basic: bad username and bad password refused, a successful login issues a
bearer token, the token works for the whole API, basic auth also works
per request, an API key is not a credential in this mode, logout revokes
the token, and a token past its session_ttl stops working. none: nothing
is required and the caller runs as a writable loopback principal. trust
_loopback: a local client with no credentials is let through, and a
forwarded non-loopback client is not. Every mutating route refuses a
read-only key with 403 without reaching its hook or touching the file
(and the same request with a read-write key succeeds, so the 403 is about
the key and not the payload), and allow_config_write: false refuses the
settings editor with a valid key while the other mutations still work.
The hashing is pinned on the state, not on timing: the authenticator's
maps are keyed on the SHA-256 of the credential and hold no plaintext key
or password, so a copied server struct cannot leak one.
* Product code is untouched; the suite is `package api` with its own setup.
AGENTS.md is the part that should not churn: the settled decisions with their
reasons, the invariants that hold the design together, the working rules, and
the security-sensitive areas that have been wrong before. A new agent should
read it and STATUS.md and know what not to re-litigate.
Deliberately excluded: anything about how to search, read or patch. Those are
being automated per round, and writing advice about them into the repo would
rot. What is here instead is project-specific and cannot be automated away —
for example that a PTR suffix must match on a label boundary, that a truncated
answer must never be cached, and that the UDP read deadline has to be reset every
iteration or the loop spins.
The working rules are the part worth arguing for. Never leave the tree red,
because nobody can detect the budget running out: three of five subagents left
broken files, and each cost a round to discover. Report honestly, and when a test
fails, decide first whether the test, the implementation or the specification is
wrong — all three happen here, and the decision belongs in the commit message.
STATUS.md now records a fully green tree: nine packages with suites, all passing
under -race. The plan starts with internal/engine, which is the largest
uncovered surface and the one every packet goes through.
It asserted that a reader had to observe both of the swapped-in upstream
addresses, which is a demand about *which* list a reader happens to see. A
reader that finished its sampling before the rebind loop got there satisfied
every real property and still failed, so this failed roughly 2 runs in 10 and
would have turned CI red on nothing.
What the test should pin is the copy-on-write invariant itself: a reader must
only ever see a whole, published list. Each reader now compares every observed
snapshot against the three lists this test publishes, and fails immediately on
a half-applied one. That is checked on every iteration rather than inferred
afterwards, and it makes no demand about which list was seen, so it cannot
flake. 15 consecutive -race runs, no failure.
A domain name in the RDATA section written without a trailing dot is
relative to the zone origin (RFC 1035 §5.1), so "www CNAME host" in
example.com. means host.example.com. The qualifier ran on the record
after dns.NewRR had already resolved every RDATA name against the root,
which turns "host" into "host." and makes the two indistinguishable --
relative() saw the trailing dot and left it alone. Every relative CNAME,
NS, MX, PTR, SRV, NAPTR, SOA and SVCB target therefore pointed at the
DNS root, which showed up as a local zone answering a CNAME with
"host." and the alias target holding no local answer. Qualification now
happens in assemble on the token stream, before the record is built,
using a table of the RDATA positions that hold a name per type; names
that already end in a dot stay absolute, so the RPZ targets "." and
"rpz-drop." and the null MX targets are unaffected.
Fix three wrong expectations in the localzone tests while here. The loop
fixture was built with dns.NewRR on "alias.example.com", which cannot
parse because the RR library wants a TTL and class column, so the
expected record now spells them out. The origin-derivation fixture
used "a" as an owner name, and a leading token that names an RR type
reads as an omitted owner, which has no previous owner to inherit on
the first line of a file; the owner is now a word. The merge test
counted the SOA, but it is zone metadata rather than a queryable
record, so two files contribute three records. The holder race test also
looked up a name from the base zone while the holder was serving a store
compiled from other.test., which no answer could satisfy; it now
accepts either store's data.
Records the current state rather than the previous plan. The important change
is in the localzone section: the relative-CNAME defect is narrowed to how
Store re-serialises records, not the parser — qualifyRdata and relative are
both correct and assemble only touches the owner, so the next person should
look at localzone rather than "fixing" the parser.
Also adds make check-go to the dev loop, the memory envelope, the env-expansion
fix (the admin API key was the literal string "${FANCYDNS_API_KEY}" behind a
green build), the two open decisions, and a reworked plan starting with the one
red package.
Adds to the hygiene notes: subagent briefs must require that whatever is on disk
when the budget runs out compiles and passes, because an agent cannot know it is
out of budget. Three of five left a red tree.
The zone parser ignored an explicit $TTL and stamped every record with the SOA
minimum instead, so a 60s zone served 3600s and DefaultTTL was wrong too.
RFC 1035 §5.2: an explicit $TTL wins. qualifyRdata was also added to apply
RFC 1035 §5.1 relative-name expansion to RDATA, which is correct but has one
outstanding defect noted below.
make check-go is the tight inner loop: gofmt check, go vet, compile. It uses
go-build rather than build, because build depends on web and shells out to npm
to rebuild a console bundle that rarely changes. check now depends on it, so
the fast path and the CI path cannot drift apart.
Outstanding, not fixed here: a relative CNAME target is still emitted as
"host." instead of "host.example.com.". qualifyRdata is called with the right
origin, and relative() is correct in isolation, so the target is almost
certainly being dot-terminated in Zone.assemble before dns.NewRR ever sees it,
which makes relative() treat it as already absolute. Narrowed to that, not
confirmed — see the localzone test failures.
The `${VAR}` support added with the Kubernetes API-key Secret was silently
useless in the only form anyone would write it. The reference check sat in the
same switch as the quote-state handling, below the `inDouble`/`inSingle`
branches, so both of those branches consumed the bytes and a reference inside a
quoted scalar was copied through verbatim. `api_keys: ["${KEY}"]` — the exact
form deploy/k8s/00-config.yaml uses — came out of Load unchanged, so the key
arrived as the literal string "${FANCYDNS_API_KEY}" and the admin API was
protected by nothing. Everything looked fine because no test asserted on the
quoted form at the Load boundary until now.
Two further corrections while in there:
- A variable that is set but empty now expands to empty instead of being an
error. `${X}`, `${X-y}` and `${X:-y}` differ precisely on that point, and
getting it wrong turns an intentionally blank secret into a startup failure.
- A reference is skipped when it follows a backslash, so "\${FOO}" is literal.
Also fixes two test fixtures that had an `inline:` key, which is not a field —
the struct spells it block_entries — and were failing Validate for a reason
that had nothing to do with what they were testing. That is the same
silent-ignore-unknown-keys behaviour flagged in STATUS.md, caught here by a
test that thought it was testing environment expansion.
A cgroup OOM kill is the worst outcome available to a resolver: an outage
rather than a cache miss, with no chance to serve stale on the way out. So:
- memory.limit / memory.reserve / memory.cache_max_bytes, a new config section.
limit becomes a GOMEMLIMIT through debug.SetMemoryLimit, reduced by the
reserve. The reserve exists because the Go heap does not account for
goroutine stacks, runtime metadata, or the bbolt mmap backing the cache
snapshot and the query log — handing the runtime the full cgroup limit is how
you get killed by a mechanism that believed it had headroom.
- config.Size, so a budget reads as "512MiB" rather than a byte count converted
by hand. Strict about junk: "512MiBx" is an error.
- The cache byte budget is now actually fed to the LRU. It already enforced
maxBytes per shard on insert, but had no setter and nothing ever called one,
so the footprint of cache.size was unbounded: one cached ANY or TXT answer
can be tens of kilobytes, so 500k entries is anywhere from 150MB to several GB
depending on what the traffic asked for.
- api.pprof mounts net/http/pprof on the admin listener. The pprof handlers sit
on the app-level mux, not inside the API's own handler, so they would bypass
the auth middleware entirely; the option is therefore refused unless every
admin address is loopback. Profiling means port-forwarding, so this is safe
by construction rather than by policy, and cmdline cannot leak an argument
that happens to be a key.
- serveUDP was calling net.ResolveUDPAddr per packet. The peer already is a
*net.UDPAddr; that call builds a string, re-parses the port and allocates on
the busiest path in the process.
Also fixes a flaky test: TestParallelPolicyAnswersFromTheFastestUpstream
asserted the slow upstreams were hit, but the fan-out cancels on the first
answer, so whether they were dispatched before cancellation is a real race. It
now asserts the answer came from the fast upstream and that nothing waited.
Records the four bugs the first end-to-end run surfaced, what is green, what
is still without a suite, and what is in flight, so the next session does not
start from the previous plan.
The cache snapshot never loaded, so every restart was a cold cache. Snapshot
packed the whole key while Load decoded the record header out of the key's
remainder instead of the value, so every entry was skipped. The two sides are
now an explicit encodeRecord/decodeRecord pair with a version byte and a CRC-32C
trailer, so a future format change is detected instead of silently misparsed.
Load also drops entries that are already past ExpiresAt and rewrites the TTL to
the time actually left, so a served TTL never jumps back up across a restart.
Blocklist, all six gaps the failing tests were pointing at: a leading UTF-8 BOM
is stripped, "!allow example.com" works, SourceMeta exists for a source that
only ever contributed a rule, Pattern() reports the bare name again (the new
WireName() produces the wildcard form for the API), an RPZ record with no
explicit TTL inherits the zone default instead of whatever the reader left in
the RR, a hosts line with no address in the first field says so, and the
adblock parser handles CRLF and adblock-flavoured lines pasted into a plain
list. TestHolderSwapIsAtomic was asserting the wrong invariant — the swap was
always atomic, a name blocked by the old set is legitimately unblocked by the
new one.
Tests added for acl and resolver, both of which turned up real bugs:
- a PTR suffix matched across label boundaries, so a group scoped to
example.com. also swallowed ample.com. and notexample.com. — a client could
land in a group it does not belong to;
- the "*" catch-all was dropped from the rendered domain patterns, so the
engine's per-query recompile lost the "everything" case;
- Upstream.Success never closed an open breaker, so a recovered upstream stayed
excluded for the full cooldown;
- Exchange fanned out in parallel even under policy: failover.
Deploy set: go.mod requires Go 1.27.1 but CI pinned 1.23 and the Dockerfile
pinned golang:1.23-alpine, so neither could have built. The Dockerfile also
copied the source tree after the console bundle, which clobbered it.
The k8s set asked for hostNetwork with two replicas, but the server never sets
SO_REUSEPORT, so the second pod could not bind :53 and would crash-loop. It now
runs without hostNetwork behind a LoadBalancer Service, with a separate
ClusterIP Service for the Ingress (a headless Service cannot back one), a
ServiceAccount, an API-key Secret, per-pod PVCs, and a kustomization. The HPA
is gone: it cannot drive a StatefulSet. To let the API key come from that
Secret, config.Load now expands ${VAR} / ${VAR:-default} / ${NAME-default} and
fails to start on an unset variable with no default, so a missing secret is a
crash loop rather than an unauthenticated console. Expansion skips YAML
comments, so documenting ${VAR} in the ConfigMap does not break the file.
Verified: gofmt, go build, go vet clean; go test -race ./internal/... green
(acl, app, blocklist, cache, config, resolver, zonefile).
- cache.Entry.Hits and LRU.perShard become atomics; SetCapacity now evicts
down to the new budget; packKey fully qualifies names before packing
- resolver.Pool rewritten copy-on-write (atomic.Pointer to an immutable
state); Upstream is immutable with shared *upstreamStats, so health
survives a Rebind and the query path stops racing the config hook
- Pool.Exchange no longer marks a dead upstream healthy: Success was
called before the error check, so the breaker never tripped
- qlog live ring publishes through atomic pointers
- blocklist.Holder.Get nil-safe; Builder lists each source once
- web: fix toast(string) misuse that rendered empty mutation toasts
- add unit tests for cache (with benchmarks) and zonefile
- STATUS.md: green -race integration suite, remaining failures and the
broken bbolt snapshot round trip documented
Blocklists, Clients, Domains and Upstreams each carried a ToastLike shim
that duck-typed a react-hot-toast style API. The console uses its own
Toast context, so notify() was calling toast('some message') with a string
where ToastOptions is expected: every mutation toast rendered an empty
title. They now take the toast function from useToast() and send a real
{ title, tone } payload, mapping 'fail' onto the danger tone.
matsandClaude Sonnet 5.5