16 Commits

Include renames

Author	SHA1	Message	Date
mats	66f36cfac6	more stuff	2026-10-16 14:49:50 +02:00
mats	a3d1614dca	rework	2026-10-16 11:13:13 +02:00
mats	340b4eaf80	Enforce upstream attempt and overall query deadlines ... The initial upstream exchange ignored its supplied timeout and used hardcoded client limits. Bound each attempt with a context, copy the shared client for its timeout, and share the attempt budget with UDP-to-TCP fallback. Close private connections on cancellation to interrupt reads, and exclude caller cancellation from upstream failures so parallel losers remain healthy. The normal engine pipeline never applied server.timeouts.query, allowing failover to outlive the configured query budget. Apply a hot-reloaded overall context deadline at Handle while retaining earlier caller deadlines. Add silent UDP/TCP/TLS, cancellation, fallback, failover, and reload regressions. The server inflight test assumed receiving a TCP reply meant its sent counter had already been incremented. Wait for both completion counters before asserting them, preserving the existing assertions while removing the scheduling race.	2026-10-03 19:06:57 +02:00
mats	d48c1863bf	more config	2026-09-28 21:34:59 +02:00
developer	504512f247	Add a Zones console: create and manage local zones, records and delegations ... The console could not see local zones at all. `blocklist.local_zones` compiled a store that nothing read back: there was no API for a zone, its records, its delegations or the file a record came from, and no way to change one without editing YAML and reloading. The Domains page is the blocklist's inline rules, not local data, so nothing in the console covered the half of the server that answers authoritatively from disk. There is now a Zones page and the endpoints behind it: `GET /api/zones`, `POST`/`PUT`/`DELETE /api/zones/records`, `POST`/`DELETE /api/zones/delegations` and `DELETE /api/zones?zone=`. **The console owns one directory and nothing else.** `blocklist.managed_zones` holds one `<apex>.zone` per zone, with that zone's SOA, apex NS set and the records added there, and it is compiled by the same `localzone.Load` as the operator's files — so a record added from the console is served on the next query, with no reload and no restart. Files in `blocklist.local_zones` are listed and shown read-only and are never rewritten; the key defaults to empty, so a deployment that never sets it gets a zone browser and no zone writes. Which file a record came from is now part of the store (`zone.files`, keyed by the index the store already uses), so the snapshot can say "console" or "your file" per record; the query path does not read it. **The writer proves its own output.** A record is validated by the project's own master-file parser, so `www` means `www.<zone>` and a bare CNAME target is completed exactly as it would be in a zone file; the file is rendered, and the rendering is re-parsed and compared with the records it meant to write before the file is renamed into place. A writer bug that dropped a record would otherwise present as a zone that silently stopped answering. A zone that cannot answer is refused rather than created: no SOA means no NXDOMAIN or NODATA authority, and no apex NS means nothing is delegated and no server is named. The SOA's primary follows the apex NS set — a zone whose SOA names a server outside its own NS set hands out a referral no resolver can follow — and the serial moves on every write. **A delegation takes the names under it, so a record below a cut is refused.** The store answers those names as delegated without reading the zone file, which would make the record data that looks alive and never answers. `DELETE /api/zones?zone=` exists for the same reason from the other side: a zone the console created is a real zone, so it says NXDOMAIN for what it does not hold, and deleting a record is not what gives a name back to upstream. It is refused for a zone that also has records outside the managed directory, because removing that file would remove none of the zone. **Two bugs the work turned up.** `engine.applyConfig` published local-record metrics only when `blocklist.local_zones` was non-empty, so a zone the console created was served but never counted; the gate is now the store's own record count. And the record-value renderer fell back to `RR.String()` for every type it did not know, which is the whole line rather than the RDATA — it wrote a name and a TTL into the value column for DHCID, LOC, SSHFP and SVCB, and would have fed that back as a record value on the next edit. Every write recompiles the store and flushes the cache. Which names changed is not knowable from the API, and a cached upstream answer for a name the zone now answers is simply wrong, so the flush is blunt on purpose. The wire shapes are type aliases in `internal/api` rather than copies, so there is one set of json tags, and `TestZoneEndpointKeySets` pins the key set at every level with a record and a delegation actually present — an empty list only proves the envelope. `TestZoneWritesNeedAManagedDirectory` pins the read-only default, and `TestZoneConsoleRoundTrip` walks the whole thing: a record added through the API is answered authoritatively on the next query, a cut stops the parent answering below it, and deleting the zone hands the name back. Verified against a running instance built from this tree: records and wildcards answer with `aa`, relative names are qualified, a record below a cut is refused with a message that says why, and the file on disk is a zone file an operator can read. The console page is typechecked and built; the browser was not available in this round, so it has not been looked at.	2026-09-28 07:45:16 +00:00
developer	b6d70a50fe	Finish the server shutdown tests; fix the two bugs they found ... internal/server was the last package whose shutdown was tested by assumption: four transports, three shutdown shapes, and every one of them can strand a goroutine that a client is holding open. Both bugs below were live in the tree as handed over and neither had a test. An idle client could hold the whole shutdown open. The stream accept loop waits for its connections before Stop returns, and a connection parked in ReadMsg with no deadline in front of it only ends when the client goes away — ctx.Err() is checked between reads, so cancelling the context cannot reach a read that is already blocked. One connection that connected and said nothing was enough to make Stop sit there for the caller's entire grace period (15s in main) and then return context.DeadlineExceeded, which the app reports as "shutdown completed with errors"; and the cache snapshot the app takes right after Stop never happened, because the orchestrator had given up first. It is a shutdown DoS that costs the attacker one connection. The fix is a read deadline in the past on every accepted connection when the context is cancelled, and deliberately not a close: a query already inside the pipeline is past its read, so its answer still goes out, which is the invariant TestInFlightQueryIsAnsweredAcrossStop pins. The accept loop keeps a small registry of live connections for this — the accept path, not the query path, and no query ever takes that mutex. A UDP query in flight at Stop was lost and nothing waited for it. The stream paths register their connections with s.wait; servePacket did not register at all, so a UDP query already inside the pipeline was gone twice over: nothing waited for it, and the answer had nowhere to go once it was ready, because a datagram is written to the very socket the query arrived on. The visible damage was not the lost answer (the client retries) but the app, which snapshots the cache and closes the store the moment Stop returns — a query still in the pipeline could be writing to a store on its way out. Packets are now counted in s.wait, and the UDP listener's close wakes the read loop with an expired deadline and then waits, bounded by udpDrainGrace (250ms), for the loop and the packets it spawned before closing the socket. The bound is deliberate: a shutdown may not spend its grace period on one slow upstream, and past the bound the answer is dropped, which is what the client would see anyway. Stop returning did not mean the process was done, so nothing could tell a leaking shutdown from a clean one. TestStopLeavesNoGoroutineRunningServerCode counts goroutines whose stack still has a frame in the package — not runtime.NumGoroutine(), which in a test binary is net/http, crypto/tls and the race detector talking over each other — and it found the idle-connection bug above. TestStopIsNotBlockedByAnIdleStreamConnection sets a 60s DoT idle timeout on purpose so a config value cannot hide the regression behind a lucky tick, and TestInFlightQueryIsAnsweredAcrossStop now covers all four transports. harness.idleStream exists because a test that only dials a stream transport races the accept loop: Stop can close the listener while the connection is still in the backlog, the server never accepts it, and a shutdown assertion that needs a live server-side goroutine passes for the wrong reason. AGENTS.md gains the two durable invariants (no step in the shutdown may wait for a client; every query goroutine is registered with s.wait, UDP included) and the two test-hygiene traps. STATUS.md drops the UDP query lost at Stop from its pinned list, since it is now a test rather than an accident, and the plan starts at the pinned decisions that are actually left.	2026-09-27 19:28:40 +00:00
developer	9881d2a095	Make the API response contract a test; fix what it found ... The console's TypeScript types are hand-written and apiFetch ends in `JSON.parse(text) as T`, so nothing in the toolchain compared them with the Go structs. A field that lost its json tag, or a response a struct never had, showed up as an empty table in a browser and nowhere else: the Go suite was green and the console was wrong. internal/app/contract_test.go now asserts the exact JSON key set of every admin endpoint, driven through the real router with the real hook implementations, plus a test that drives the two write endpoints with the exact payload web/src builds. The drift it found, all four fixed here: blocklist.Source and SourceMeta had no json tags at all, so GET /api/blocklists sent Go field names ("URL"/"Format") where the console expects snake_case: the Sources table rendered blank rows and its per-row toggle posted `name: undefined`. Tagged to match ParseStats. acl.GroupSummary was flat where the console reads a nested `match` object — the same shape config.Group has — so every group card read "matches nothing". The four match fields are nested now, under a new acl.Match. PUT /api/clients answered 400 `json: unknown field "ptr_suffixes"` to the console's payload, because config.Group and config.Match carried yaml tags only and the decoder runs with DisallowUnknownFields. Both structs have json tags now; the same applies to config.UpstreamServer and UpstreamBootstrap. PUT /api/upstreams answered 400 `json: unknown field "health"`, because the console round-trips the whole server object it read, runtime health fields included. They are accepted and ignored, because the server owns them, and they are `yaml:"-"` so settings.yaml does not gain runtime state. Also fixed, each with a test: A duration on the wire was a nanosecond count. config.Duration had a MarshalYAML but no MarshalJSON, so bootstraps[].refresh arrived as 30000000000 while the console typed the field as a string, and healthcheck_interval was a float of seconds in one place and a string in another. Durations on the wire are duration strings everywhere now — the same spelling config.yaml uses — and a bare JSON number is still accepted and means seconds, not nanoseconds. TestDurationJSONForm used to pin the old behaviour and now pins the new one, saying in its comment that the old behaviour was the bug. A log row's key set depended on the row. api.LogEntry was omitempty on client_group, response, upstream and list, so a query with no ACL group arrived without the key and the console rendered "undefined" where the column should be empty. The contract test can only pin one shape if the server emits one, so nothing on that struct is omitempty any more. A dead peer stayed in the cluster peer list forever. Cluster.seen was only ever added to and deleted from on an explicit `bye`, so a node that crashed — or whose Valkey connection died — was reported as a live peer for the life of the process, and its stale blocklist digest was flagged as divergence forever. Peers() now reaps peers unheard for peer_ttl*4 and stamps each state with the receiver's own clock, so clock skew between two machines can neither reap a healthy peer nor keep a dead one. Discover() ages out the same way on the publisher's own last_seen stamp, because the keyed Valkey record outlives a crash: a live node keeps refreshing the key's TTL, so a dead node's field stayed readable for the life of the fleet. peer_ttl: 0 would have crashed the process. time.NewTicker(PeerTTL/2) panics on a non-positive period and `EXPIRE key 0` deletes the heartbeat key, so a config that left peer_ttl unset would have taken the process down at Start, or made every instance believe it is alone in the fleet. Both go through peerTTL() and peerDeadline(), which fall back to the default. Two cluster tests raced on the wire, not on the product, and one was environment-dependent rather than wrong. A node that joined after another's heartbeat was in flight hears about it on the next tick, so "b does not see a" read immediately after "a sees b" is a coin flip; the divergence test waited for a digest on the wrong node's view of the fleet; and 127.0.0.1:1 is only unbindable outside a container that sets net.ipv4.ip_unprivileged_port_start=0, where it binds and the partial-bind test fails with both transports up. That test now occupies a port with a live listener in the test process, which is a deterministic EADDRINUSE on every machine and every uid. internal/server/counters_test.go covers the counter-key layer that no other test touched: the "<transport>.<field>" keyspace, that the aggregate folds only the known fields, and that the snapshot is safe while counters move. Two pinned behaviours changed on purpose, each with a test that says so: An RPZ record did not cover the name it names. rpzRule gave a bare owner name the subdomain form (*.name), so analytics.example.org. in a policy zone blocked cdn.analytics.example.org. and left analytics.example.org. resolving normally — and an rpz-passthru. record did not allowlist its own domain either. RPZ semantics say the record applies to the domain and everything under it, so a bare owner now takes the apex form **.name; an owner written with an explicit *. is still strict, so a zone that wants subdomain-only behaviour can still ask for it. This changes which queries get blocked for an existing policy zone, which is why it wanted a decision. TestExampleRPZZoneCoversTheApexItNames pins both halves and the example zone now says which half it relies on. metrics.OutcomeLocal had no producer. The engine reported a local-zone answer as qlog.ActionAllow, which outcomeFor mapped to OutcomeUpstream, so every locally answered query was counted as an upstream one and the outcome="local" series was never published — the dashboard's local column was permanently empty. Response.Local is now set on the local-zone path and outcomeFor consults it first. That moves published counters, so the README's metrics section says so too.	2026-09-27 18:42:59 +00:00
developer	e8c5b414d9	half broken	2026-09-27 17:17:53 +00:00
developer	cc7466823b	Wire the memory envelope into the k8s ConfigMap; bring STATUS.md up to date ... The container limit is 1Gi, so declare the envelope next to it: a 1GiB soft heap limit, a 256MiB reserve for what the Go heap does not account for, and an explicit 384MiB cache byte budget. cache.size is an entry count, so the byte budget is what actually bounds the cache now that ParseSize applies its units. STATUS.md records the round: the fixes, the new suites (engine, config.Size, metrics, and the api suite that landed separately), three findings left unfixed -- the login rate limiter never applies to the login endpoint, auth configuration is frozen at New, and api.constantTimeEqual is dead -- and the continuation plan with items 1, 2, 5 and 6 struck.	2026-09-27 16:03:25 +00:00
developer	bbd8692bab	Add AGENTS.md; bring STATUS.md up to a green tree ... AGENTS.md is the part that should not churn: the settled decisions with their reasons, the invariants that hold the design together, the working rules, and the security-sensitive areas that have been wrong before. A new agent should read it and STATUS.md and know what not to re-litigate. Deliberately excluded: anything about how to search, read or patch. Those are being automated per round, and writing advice about them into the repo would rot. What is here instead is project-specific and cannot be automated away — for example that a PTR suffix must match on a label boundary, that a truncated answer must never be cached, and that the UDP read deadline has to be reset every iteration or the loop spins. The working rules are the part worth arguing for. Never leave the tree red, because nobody can detect the budget running out: three of five subagents left broken files, and each cost a round to discover. Report honestly, and when a test fails, decide first whether the test, the implementation or the specification is wrong — all three happen here, and the decision belongs in the commit message. STATUS.md now records a fully green tree: nine packages with suites, all passing under -race. The plan starts with internal/engine, which is the largest uncovered surface and the one every packet goes through.	2026-09-27 14:17:14 +00:00
developer	ad9989b28c	Update STATUS.md: green/red map, the narrowed localzone bug, revised plan ... Records the current state rather than the previous plan. The important change is in the localzone section: the relative-CNAME defect is narrowed to how Store re-serialises records, not the parser — qualifyRdata and relative are both correct and assemble only touches the owner, so the next person should look at localzone rather than "fixing" the parser. Also adds make check-go to the dev loop, the memory envelope, the env-expansion fix (the admin API key was the literal string "${FANCYDNS_API_KEY}" behind a green build), the two open decisions, and a reworked plan starting with the one red package. Adds to the hygiene notes: subagent briefs must require that whatever is on disk when the budget runs out compiles and passes, because an agent cannot know it is out of budget. Three of five left a red tree.	2026-09-27 14:01:47 +00:00
developer	b0e6b3bb92	Add the memory envelope, pprof, and a per-packet fix on the UDP path ... A cgroup OOM kill is the worst outcome available to a resolver: an outage rather than a cache miss, with no chance to serve stale on the way out. So: - memory.limit / memory.reserve / memory.cache_max_bytes, a new config section. limit becomes a GOMEMLIMIT through debug.SetMemoryLimit, reduced by the reserve. The reserve exists because the Go heap does not account for goroutine stacks, runtime metadata, or the bbolt mmap backing the cache snapshot and the query log — handing the runtime the full cgroup limit is how you get killed by a mechanism that believed it had headroom. - config.Size, so a budget reads as "512MiB" rather than a byte count converted by hand. Strict about junk: "512MiBx" is an error. - The cache byte budget is now actually fed to the LRU. It already enforced maxBytes per shard on insert, but had no setter and nothing ever called one, so the footprint of cache.size was unbounded: one cached ANY or TXT answer can be tens of kilobytes, so 500k entries is anywhere from 150MB to several GB depending on what the traffic asked for. - api.pprof mounts net/http/pprof on the admin listener. The pprof handlers sit on the app-level mux, not inside the API's own handler, so they would bypass the auth middleware entirely; the option is therefore refused unless every admin address is loopback. Profiling means port-forwarding, so this is safe by construction rather than by policy, and cmdline cannot leak an argument that happens to be a key. - serveUDP was calling net.ResolveUDPAddr per packet. The peer already is a *net.UDPAddr; that call builds a string, re-parses the port and allocates on the busiest path in the process. Also fixes a flaky test: TestParallelPolicyAnswersFromTheFastestUpstream asserted the slow upstreams were hit, but the fan-out cancels on the first answer, so whether they were dispatched before cancellation is a real race. It now asserts the answer came from the fast upstream and that nothing waited.	2026-09-27 13:41:48 +00:00
developer	2b84474228	Rewrite STATUS.md to match the tree after the bbolt/blocklist/acl/resolver fixes ... Records the four bugs the first end-to-end run surfaced, what is green, what is still without a suite, and what is in flight, so the next session does not start from the previous plan.	2026-09-27 13:32:17 +00:00
developer	728374dfda	STATUS.md: correct the deliverables section ... Makefile, Dockerfile, .gitignore, CI workflow and two k8s manifests do exist; only README.md is empty and the k8s set is partial.	2026-09-27 12:59:27 +00:00
developer	aa12b03eb9	fix data races in cache, resolver, qlog; add cache/zonefile/blocklist tests; update STATUS ... - cache.Entry.Hits and LRU.perShard become atomics; SetCapacity now evicts down to the new budget; packKey fully qualifies names before packing - resolver.Pool rewritten copy-on-write (atomic.Pointer to an immutable state); Upstream is immutable with shared *upstreamStats, so health survives a Rebind and the query path stops racing the config hook - Pool.Exchange no longer marks a dead upstream healthy: Success was called before the error check, so the breaker never tripped - qlog live ring publishes through atomic pointers - blocklist.Holder.Get nil-safe; Builder lists each source once - web: fix toast(string) misuse that rendered empty mutation toasts - add unit tests for cache (with benchmarks) and zonefile - STATUS.md: green -race integration suite, remaining failures and the broken bbolt snapshot round trip documented	2026-09-27 12:59:02 +00:00
developer	85453852d5	initial commit	2026-09-27 11:20:34 +00:00