Make the resolver pure DNS: JSON access log on stdout instead of a log store

The resolver kept its own query-log store (a bbolt ring, a live ring with SSE
fanout, rolling per-domain counters and dashboard time-series) for the console.
In a multi-node deployment each pod held a different slice of that, so every
view disagreed with itself, and the cost sat on the query path. Charts belong
in Grafana and log search in the log collector, so the store is gone.

internal/accesslog writes one JSON line per notable query to stdout. The default
is what was blocked and what failed (blocked, redirected, SERVFAIL, dropped,
refused); log_cached and sample_rate add ordinary answers for a debugging
session. Lines go through a bounded buffer to a single writer goroutine and are
dropped, and counted in fancydns_access_log_dropped_total, when the stream
cannot keep up, so a stalled log pipe can never stall a query. The config key is
access_log (the old qlog block is ignored, like any unknown key).

The metrics gain what the dashboard used to compute: fancydns_group_queries_total
by ACL group and outcome and fancydns_responses_total by RCODE. Removed with the
store: /api/dashboard, /api/logs, /api/logs/stream, /api/logs/clear,
/api/cache/top, the series rings, and the cache's per-name hit table, which took
a mutex and built a string on every cache hit.

Tests that read the old log now read the access-log lines; the contract test
drops the removed endpoints. The web console still calls them and is changed in
the next commit.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
matsandClaude Sonnet 5.5 committed 2026-10-16 09:53:04 +02:00
1 parent dd1117fd91
commit c6c5ca4ff7
31 files changed
+818 -3389

No files matched your search

+7 -6
View File
@@ -11,7 +11,7 @@ the hard way.
A recursive DNS server in Go with an embedded React console. Config is YAML plus
RFC 1035 master files, hot-reloaded over SIGHUP. Blocklists in four formats, a
sharded LRU cache with a bbolt snapshot for warm restart, per-client ACLs,
query logging, Prometheus metrics, and optional Valkey/Redis for cross-instance
an access log on stdout, Prometheus metrics, and optional Valkey/Redis for cross-instance
sync.
## Layout
@@ -29,7 +29,7 @@ internal/
localzone/ authoritative data from master files, wildcards, delegations
engine/ THE QUERY PIPELINE, reverse-PTR cache
server/ UDP, TCP, DoT, DoH listeners; all funnel into one handler
qlog/ bbolt query-log ring + live in-memory ring + SSE fanout
accesslog/ async JSON-lines access log to stdout (blocked/errors by default)
metrics/ Prometheus collectors + rolling time-series rings
cluster/ Valkey pub/sub, leader lease, divergence detection
kubeedit/ the console's writes to ClientGroup/BlockList/DNSZone objects
@@ -48,7 +48,7 @@ deploy/ example config, example zones, k8s manifests
```
validate → ACL deny → per-client concurrency → global concurrency → rate limit
→ group allow/deny domains → local zone → blocklist → cache → upstream
→ cache put → finalize (EDNS0, UDP truncation) → metrics + qlog
→ cache put → finalize (EDNS0, UDP truncation) → metrics + access log
```
Cache is a single shard-local lookup; only a miss touches the network.
@@ -57,11 +57,12 @@ Cache is a single shard-local lookup; only a miss touches the network.
| Decision | Choice | Why |
|---|---|---|
| Scope | Full production | clustering, metrics, ACLs, query logs, hot reload, Docker, k8s, CI |
| Scope | Full production | clustering, metrics, ACLs, access log, hot reload, Docker, k8s, CI |
| Config store | YAML + DNS master files | reviewable/diffable in git |
| Console-written zone data | Its own directory, `blocklist.managed_zones`, one file per apex | the console is a real operator surface, so it needs somewhere to write; a separate directory keeps the operator's master files read-only to it, and an unset key keeps the console read-only by default |
| Cluster sync | Valkey (Redis protocol) | simple, mature Go client, easy local dev |
| Secure DNS | DoT + DoH, **no DNSSEC validation** | deliberately out of scope; AD/CD/extended-RCODE are passed through only |
| Observability | Prometheus metrics + a JSON access log on stdout; **no in-app dashboard, log store or live tail** | the resolver is DNS only. Charts belong in Grafana, search in the log collector, and per-pod state in a multi-node deployment only disagrees with itself. The access log is blocked/errors by default, goes through a bounded buffer that drops (and counts) rather than ever blocking a query, and nothing may be added to the query path to feed a UI |
| Hot cache | Sharded LRU in memory, bbolt for snapshots | no I/O on the hot path; warm restart |
| k8s networking | **No hostNetwork**, LoadBalancer Service | the server never sets SO_REUSEPORT, so two host-network pods cannot share :53 |
| Memory | Soft GOMEMLIMIT + cache byte budget | a cgroup kill is an outage; a cache miss is not |
@@ -137,8 +138,8 @@ Break these and you break the design, not just the test.
The internal `*.` / `**.` prefixes must not leak into the UI, but the API needs
them to re-add a rule unchanged.
- **The Go heap does not account for goroutine stacks, runtime metadata, or the
bbolt mmap.** `memory.reserve` exists to cover them; the cache snapshot and
query log are mmapped and live outside the heap entirely.
bbolt mmap.** `memory.reserve` exists to cover them; the cache snapshot is
mmapped and lives outside the heap entirely.
- **The zone writer proves its own output.** Every record is validated by the
project's own master-file parser, the file is rendered, and the rendering is
re-parsed and compared with the records it meant to write *before* the file