Concurrent cache misses for one name each went upstream. After a restart, a
cache flush or a cluster-object change that is a burst of identical queries
exactly when the cache is empty. Identical exchanges (same name, type, class,
DO bit, pinned upstream set and forwarded client subnet) now share one: the
first query runs it on its own goroutine and context, the rest wait and get
private copies, and the leader alone writes the cache. If the leader's client
leaves, waiters take over instead of failing with its cancellation. Shared
answers are counted in fancydns_upstream_coalesced_total.
validate() accepted OpcodeUpdate, so a dynamic update was forwarded upstream as
an ordinary lookup, and every other opcode got FORMERR. A recursive resolver
implements QUERY only: anything else is now NOTIMP (RFC 1035 4.1.1), checked
before the question count since STATUS carries no question.
Prefetch only fired when a stale entry was served, so the first client after a
hot name expired got a TTL-1 stale answer. A fresh entry is now refreshed in
the background when it has been served at least twice and under a tenth of its
lifetime remains; the hit counter restarts with each stored entry, so a name
nobody comes back for stops being refreshed. Counted in
fancydns_cache_prefetch_total.
miekg/dns matches only the message ID of an upstream reply, so a forged reply
with the right ID for a different question was accepted and cached under the
name that was asked. exchangeOne now requires the reply to carry the question
that was sent (an empty section only for FORMERR, SERVFAIL, NOTIMP and
REFUSED). A mismatch fails the attempt without counting against the upstream,
so a spoofer cannot trip the circuit breaker.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
internal/engine, internal/config and internal/metrics had no test file at all,
and writing them turned up five real bugs.
internal/engine (23 tests, new harness_test.go): a wired engine over a fake
upstream on loopback, a local zone, a blocklist, a cache, a query log and an
injectable clock. It pins the pipeline order end to end -- ACL deny before the
local zone, local zone before blocklist, blocklist before cache, drop sends
nothing, the concurrency and rate-limit refusals, serve-stale served with TTL 1
and refreshed once per key, truncation against a 512-byte client, and exactly
one metrics record and one query-log entry per query.
Five fixes:
- config.ParseSize ignored the unit multiplier, so 512MiB parsed as 512 bytes
and the whole memory envelope was inert for anyone who wrote a unit.
- finalize (EDNS0 and the UDP size) ran only on the cache-hit path, so a large
uncached answer left oversized for a 512-byte client. It now runs on the
upstream, local-zone and blocklist paths too.
- The upstream UDP client read buffer was hard-coded to 1232 while we advertise
dns.max_udp_size, so with a larger buffer configured every answer above 1232
bytes was read truncated and became SERVFAIL. The pool takes the size now.
- LRU.prefetchGate was a 64-slot global cap, not the single flight its comment
promised: a burst of stale hits launched up to 64 duplicate refreshes of one
name. It is a per-key latch now; distinct names still refresh in parallel.
- metrics.RecordMiss and RecordStaleHit moved only the Prometheus collector, so
the Cache page's misses and stale hits read 0 for the life of the process.
AGENTS.md gains the trap that cost the most time: this version of miekg/dns
reports a character-string over 255 bytes as zero length, so such a message
silently fails to pack, and the write failure looks like a client timeout.
A cgroup OOM kill is the worst outcome available to a resolver: an outage
rather than a cache miss, with no chance to serve stale on the way out. So:
- memory.limit / memory.reserve / memory.cache_max_bytes, a new config section.
limit becomes a GOMEMLIMIT through debug.SetMemoryLimit, reduced by the
reserve. The reserve exists because the Go heap does not account for
goroutine stacks, runtime metadata, or the bbolt mmap backing the cache
snapshot and the query log — handing the runtime the full cgroup limit is how
you get killed by a mechanism that believed it had headroom.
- config.Size, so a budget reads as "512MiB" rather than a byte count converted
by hand. Strict about junk: "512MiBx" is an error.
- The cache byte budget is now actually fed to the LRU. It already enforced
maxBytes per shard on insert, but had no setter and nothing ever called one,
so the footprint of cache.size was unbounded: one cached ANY or TXT answer
can be tens of kilobytes, so 500k entries is anywhere from 150MB to several GB
depending on what the traffic asked for.
- api.pprof mounts net/http/pprof on the admin listener. The pprof handlers sit
on the app-level mux, not inside the API's own handler, so they would bypass
the auth middleware entirely; the option is therefore refused unless every
admin address is loopback. Profiling means port-forwarding, so this is safe
by construction rather than by policy, and cmdline cannot leak an argument
that happens to be a key.
- serveUDP was calling net.ResolveUDPAddr per packet. The peer already is a
*net.UDPAddr; that call builds a string, re-parses the port and allocates on
the busiest path in the process.
Also fixes a flaky test: TestParallelPolicyAnswersFromTheFastestUpstream
asserted the slow upstreams were hit, but the fan-out cancels on the first
answer, so whether they were dispatched before cancellation is a real race. It
now asserts the answer came from the fast upstream and that nothing waited.
matsandClaude Sonnet 5.5