bdcdbaf23afb6efb814168f0171f2df5306c9fa2
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  matsandClaude Sonnet 5.5
|
c6c5ca4ff7 |
Make the resolver pure DNS: JSON access log on stdout instead of a log store
The resolver kept its own query-log store (a bbolt ring, a live ring with SSE fanout, rolling per-domain counters and dashboard time-series) for the console. In a multi-node deployment each pod held a different slice of that, so every view disagreed with itself, and the cost sat on the query path. Charts belong in Grafana and log search in the log collector, so the store is gone. internal/accesslog writes one JSON line per notable query to stdout. The default is what was blocked and what failed (blocked, redirected, SERVFAIL, dropped, refused); log_cached and sample_rate add ordinary answers for a debugging session. Lines go through a bounded buffer to a single writer goroutine and are dropped, and counted in fancydns_access_log_dropped_total, when the stream cannot keep up, so a stalled log pipe can never stall a query. The config key is access_log (the old qlog block is ignored, like any unknown key). The metrics gain what the dashboard used to compute: fancydns_group_queries_total by ACL group and outcome and fancydns_responses_total by RCODE. Removed with the store: /api/dashboard, /api/logs, /api/logs/stream, /api/logs/clear, /api/cache/top, the series rings, and the cache's per-name hit table, which took a mutex and built a string on every cache hit. Tests that read the old log now read the access-log lines; the contract test drops the removed endpoints. The web console still calls them and is changed in the next commit. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
matsandClaude Sonnet 5.5
|
dd1117fd91 |
Coalesce identical misses, refuse non-QUERY opcodes, refresh hot entries early, check the answered question
Concurrent cache misses for one name each went upstream. After a restart, a cache flush or a cluster-object change that is a burst of identical queries exactly when the cache is empty. Identical exchanges (same name, type, class, DO bit, pinned upstream set and forwarded client subnet) now share one: the first query runs it on its own goroutine and context, the rest wait and get private copies, and the leader alone writes the cache. If the leader's client leaves, waiters take over instead of failing with its cancellation. Shared answers are counted in fancydns_upstream_coalesced_total. validate() accepted OpcodeUpdate, so a dynamic update was forwarded upstream as an ordinary lookup, and every other opcode got FORMERR. A recursive resolver implements QUERY only: anything else is now NOTIMP (RFC 1035 4.1.1), checked before the question count since STATUS carries no question. Prefetch only fired when a stale entry was served, so the first client after a hot name expired got a TTL-1 stale answer. A fresh entry is now refreshed in the background when it has been served at least twice and under a tenth of its lifetime remains; the hit counter restarts with each stored entry, so a name nobody comes back for stops being refreshed. Counted in fancydns_cache_prefetch_total. miekg/dns matches only the message ID of an upstream reply, so a forged reply with the right ID for a different question was accepted and cached under the name that was asked. exchangeOne now requires the reply to carry the question that was sent (an empty section only for FORMERR, SERVFAIL, NOTIMP and REFUSED). A mismatch fails the attempt without counting against the upstream, so a spoofer cannot trip the circuit breaker. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
matsandClaude Sonnet 5.5
|
1a536e7d0e |
Split the console from the resolver: fancy-dns-console writes the cluster
fancy-dns-console is a second binary holding the web UI, authentication and every edit. It writes ClientGroup, BlockList and DNSZone objects through the new kubeedit package and forwards all other /api routes to a running resolver with its own key, dropping the browser's credentials, so it can run in the cluster or on a workstation with a kubeconfig and nothing about the resolver's placement matters. The resolver never imports kubeedit and the console is not on the DNS path. kubeedit validates every spec with the runtime's own converters (kubesource.CheckGroup/CheckList/CheckZone) before writing, so the console cannot store an object the resolver would reject. Zone edits keep the file-based writer's behaviour: duplicate and below-a-delegation records are refused, a delegation in a new zone also gives the zone its servers, and a zone with no apex NS is never stored. Each object records the editing user in an annotation; updates retry on conflict. The resolver reports groups from ClientGroup objects as source "kubernetes" and counter-only rows (the "(default)" bucket, or a group removed since) as "history", and lists DNSZone zones as the editable ones. The console uses that to leave the YAML's groups alone and to drop history rows, which the UI would otherwise send back on save. Its GET /api/clients overlays the cluster's current objects on the resolver's list, because the resolver only sees an edit after its watch fires and a save made just after another would delete the first. An end-to-end test runs both halves against one fake apiserver: a group saved through the console refuses queries, a zone built record by record answers authoritatively, and a request with the resolver's own key is rejected by the console. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
mats
|
340b4eaf80 |
Enforce upstream attempt and overall query deadlines
build / build-push (push) Successful in 35s
The initial upstream exchange ignored its supplied timeout and used hardcoded client limits. Bound each attempt with a context, copy the shared client for its timeout, and share the attempt budget with UDP-to-TCP fallback. Close private connections on cancellation to interrupt reads, and exclude caller cancellation from upstream failures so parallel losers remain healthy. The normal engine pipeline never applied server.timeouts.query, allowing failover to outlive the configured query budget. Apply a hot-reloaded overall context deadline at Handle while retaining earlier caller deadlines. Add silent UDP/TCP/TLS, cancellation, fallback, failover, and reload regressions. The server inflight test assumed receiving a TCP reply meant its sent counter had already been incremented. Wait for both completion counters before asserting them, preserving the existing assertions while removing the scheduling race. |
||
|
mats
|
98b65945f6 |
Label DNS policy redirects accurately
build / build-push (push) Successful in 30s
The query log classified every non-drop blocklist match as a block, even when an RPZ A or CNAME rule returned a successful redirect. Record redirect results with their own action so NOERROR address overrides are not presented or counted as blocked. Expose the new action in the Logs filter. |
||
|
mats
|
87b6946ab7 |
Support RPZ address overrides for local DNS
build / build-push (push) Successful in 30s
RPZ A records were parsed as address redirects, but the query pipeline treated the address as a CNAME target and returned the wrong result. Answer IP-literal redirects directly with the matching A or AAAA record, and return NODATA for other query types. Add the requested EdgeRouter domain overrides as a local RPZ source so each bare owner covers its apex and all subdomains. Mount that source into the pods and document the address-record form. When the Zones page has no existing zone, start the Add Record form in New zone mode and tell the operator to enter its apex. |
||
|
developer
|
504512f247 |
Add a Zones console: create and manage local zones, records and delegations
The console could not see local zones at all. `blocklist.local_zones` compiled a store that nothing read back: there was no API for a zone, its records, its delegations or the file a record came from, and no way to change one without editing YAML and reloading. The Domains page is the blocklist's inline rules, not local data, so nothing in the console covered the half of the server that answers authoritatively from disk. There is now a Zones page and the endpoints behind it: `GET /api/zones`, `POST`/`PUT`/`DELETE /api/zones/records`, `POST`/`DELETE /api/zones/delegations` and `DELETE /api/zones?zone=`. **The console owns one directory and nothing else.** `blocklist.managed_zones` holds one `<apex>.zone` per zone, with that zone's SOA, apex NS set and the records added there, and it is compiled by the same `localzone.Load` as the operator's files — so a record added from the console is served on the next query, with no reload and no restart. Files in `blocklist.local_zones` are listed and shown read-only and are never rewritten; the key defaults to empty, so a deployment that never sets it gets a zone browser and no zone writes. Which file a record came from is now part of the store (`zone.files`, keyed by the index the store already uses), so the snapshot can say "console" or "your file" per record; the query path does not read it. **The writer proves its own output.** A record is validated by the project's own master-file parser, so `www` means `www.<zone>` and a bare CNAME target is completed exactly as it would be in a zone file; the file is rendered, and the rendering is re-parsed and compared with the records it meant to write before the file is renamed into place. A writer bug that dropped a record would otherwise present as a zone that silently stopped answering. A zone that cannot answer is refused rather than created: no SOA means no NXDOMAIN or NODATA authority, and no apex NS means nothing is delegated and no server is named. The SOA's primary follows the apex NS set — a zone whose SOA names a server outside its own NS set hands out a referral no resolver can follow — and the serial moves on every write. **A delegation takes the names under it, so a record below a cut is refused.** The store answers those names as delegated without reading the zone file, which would make the record data that looks alive and never answers. `DELETE /api/zones?zone=` exists for the same reason from the other side: a zone the console created is a real zone, so it says NXDOMAIN for what it does not hold, and deleting a record is not what gives a name back to upstream. It is refused for a zone that also has records outside the managed directory, because removing that file would remove none of the zone. **Two bugs the work turned up.** `engine.applyConfig` published local-record metrics only when `blocklist.local_zones` was non-empty, so a zone the console created was served but never counted; the gate is now the store's own record count. And the record-value renderer fell back to `RR.String()` for every type it did not know, which is the whole line rather than the RDATA — it wrote a name and a TTL into the value column for DHCID, LOC, SSHFP and SVCB, and would have fed that back as a record value on the next edit. Every write recompiles the store and flushes the cache. Which names changed is not knowable from the API, and a cached upstream answer for a name the zone now answers is simply wrong, so the flush is blunt on purpose. The wire shapes are type aliases in `internal/api` rather than copies, so there is one set of json tags, and `TestZoneEndpointKeySets` pins the key set at every level with a record and a delegation actually present — an empty list only proves the envelope. `TestZoneWritesNeedAManagedDirectory` pins the read-only default, and `TestZoneConsoleRoundTrip` walks the whole thing: a record added through the API is answered authoritatively on the next query, a cut stops the parent answering below it, and deleting the zone hands the name back. Verified against a running instance built from this tree: records and wildcards answer with `aa`, relative names are qualified, a record below a cut is refused with a message that says why, and the file on disk is a zone file an operator can read. The console page is typechecked and built; the browser was not available in this round, so it has not been looked at. |
||
|
developer
|
9881d2a095 |
Make the API response contract a test; fix what it found
The console's TypeScript types are hand-written and apiFetch ends in
`JSON.parse(text) as T`, so nothing in the toolchain compared them with the
Go structs. A field that lost its json tag, or a response a struct never had,
showed up as an empty table in a browser and nowhere else: the Go suite was
green and the console was wrong. internal/app/contract_test.go now asserts the
exact JSON key set of every admin endpoint, driven through the real router
with the real hook implementations, plus a test that drives the two write
endpoints with the exact payload web/src builds.
The drift it found, all four fixed here:
blocklist.Source and SourceMeta had no json tags at all, so GET /api/blocklists
sent Go field names ("URL"/"Format") where the console expects snake_case: the
Sources table rendered blank rows and its per-row toggle posted
`name: undefined`. Tagged to match ParseStats.
acl.GroupSummary was flat where the console reads a nested `match` object — the
same shape config.Group has — so every group card read "matches nothing". The
four match fields are nested now, under a new acl.Match.
PUT /api/clients answered 400 `json: unknown field "ptr_suffixes"` to the
console's payload, because config.Group and config.Match carried yaml tags only
and the decoder runs with DisallowUnknownFields. Both structs have json tags
now; the same applies to config.UpstreamServer and UpstreamBootstrap.
PUT /api/upstreams answered 400 `json: unknown field "health"`, because the
console round-trips the whole server object it read, runtime health fields
included. They are accepted and ignored, because the server owns them, and they
are `yaml:"-"` so settings.yaml does not gain runtime state.
Also fixed, each with a test:
A duration on the wire was a nanosecond count. config.Duration had a
MarshalYAML but no MarshalJSON, so bootstraps[].refresh arrived as 30000000000
while the console typed the field as a string, and healthcheck_interval was a
float of seconds in one place and a string in another. Durations on the wire
are duration strings everywhere now — the same spelling config.yaml uses — and a
bare JSON number is still accepted and means seconds, not nanoseconds.
TestDurationJSONForm used to pin the old behaviour and now pins the new one,
saying in its comment that the old behaviour was the bug.
A log row's key set depended on the row. api.LogEntry was omitempty on
client_group, response, upstream and list, so a query with no ACL group
arrived without the key and the console rendered "undefined" where the column
should be empty. The contract test can only pin one shape if the server emits
one, so nothing on that struct is omitempty any more.
A dead peer stayed in the cluster peer list forever. Cluster.seen was only ever
added to and deleted from on an explicit `bye`, so a node that crashed — or
whose Valkey connection died — was reported as a live peer for the life of the
process, and its stale blocklist digest was flagged as divergence forever.
Peers() now reaps peers unheard for peer_ttl*4 and stamps each state with the
receiver's own clock, so clock skew between two machines can neither reap a
healthy peer nor keep a dead one. Discover() ages out the same way on the
publisher's own last_seen stamp, because the keyed Valkey record outlives a
crash: a live node keeps refreshing the key's TTL, so a dead node's field stayed
readable for the life of the fleet.
peer_ttl: 0 would have crashed the process. time.NewTicker(PeerTTL/2) panics on
a non-positive period and `EXPIRE key 0` deletes the heartbeat key, so a config
that left peer_ttl unset would have taken the process down at Start, or made
every instance believe it is alone in the fleet. Both go through peerTTL() and
peerDeadline(), which fall back to the default.
Two cluster tests raced on the wire, not on the product, and one was
environment-dependent rather than wrong. A node that joined after another's
heartbeat was in flight hears about it on the next tick, so "b does not see a"
read immediately after "a sees b" is a coin flip; the divergence test waited for
a digest on the wrong node's view of the fleet; and 127.0.0.1:1 is only
unbindable outside a container that sets
net.ipv4.ip_unprivileged_port_start=0, where it binds and the partial-bind test
fails with both transports up. That test now occupies a port with a live
listener in the test process, which is a deterministic EADDRINUSE on every
machine and every uid.
internal/server/counters_test.go covers the counter-key layer that no other
test touched: the "<transport>.<field>" keyspace, that the aggregate folds only
the known fields, and that the snapshot is safe while counters move.
Two pinned behaviours changed on purpose, each with a test that says so:
An RPZ record did not cover the name it names. rpzRule gave a bare owner name
the subdomain form (*.name), so analytics.example.org. in a policy zone blocked
cdn.analytics.example.org. and left analytics.example.org. resolving normally —
and an rpz-passthru. record did not allowlist its own domain either. RPZ
semantics say the record applies to the domain and everything under it, so a
bare owner now takes the apex form **.name; an owner written with an explicit *.
is still strict, so a zone that wants subdomain-only behaviour can still ask for
it. This changes which queries get blocked for an existing policy zone, which is
why it wanted a decision. TestExampleRPZZoneCoversTheApexItNames pins both
halves and the example zone now says which half it relies on.
metrics.OutcomeLocal had no producer. The engine reported a local-zone answer as
qlog.ActionAllow, which outcomeFor mapped to OutcomeUpstream, so every locally
answered query was counted as an upstream one and the outcome="local" series was
never published — the dashboard's local column was permanently empty.
Response.Local is now set on the local-zone path and outcomeFor consults it
first. That moves published counters, so the README's metrics section says so
too.
|
||
|
developer
|
e8c5b414d9 |
half broken
build / build-push (push) Failing after 23s
|
||
|
developer
|
4f49033f1b |
Cover the engine, config.Size and metrics; fix the five bugs they found
internal/engine, internal/config and internal/metrics had no test file at all, and writing them turned up five real bugs. internal/engine (23 tests, new harness_test.go): a wired engine over a fake upstream on loopback, a local zone, a blocklist, a cache, a query log and an injectable clock. It pins the pipeline order end to end -- ACL deny before the local zone, local zone before blocklist, blocklist before cache, drop sends nothing, the concurrency and rate-limit refusals, serve-stale served with TTL 1 and refreshed once per key, truncation against a 512-byte client, and exactly one metrics record and one query-log entry per query. Five fixes: - config.ParseSize ignored the unit multiplier, so 512MiB parsed as 512 bytes and the whole memory envelope was inert for anyone who wrote a unit. - finalize (EDNS0 and the UDP size) ran only on the cache-hit path, so a large uncached answer left oversized for a 512-byte client. It now runs on the upstream, local-zone and blocklist paths too. - The upstream UDP client read buffer was hard-coded to 1232 while we advertise dns.max_udp_size, so with a larger buffer configured every answer above 1232 bytes was read truncated and became SERVFAIL. The pool takes the size now. - LRU.prefetchGate was a 64-slot global cap, not the single flight its comment promised: a burst of stale hits launched up to 64 duplicate refreshes of one name. It is a per-key latch now; distinct names still refresh in parallel. - metrics.RecordMiss and RecordStaleHit moved only the Prometheus collector, so the Cache page's misses and stale hits read 0 for the life of the process. AGENTS.md gains the trap that cost the most time: this version of miekg/dns reports a character-string over 255 bytes as zero length, so such a message silently fails to pack, and the write failure looks like a client timeout. |
||
|
developer
|
85453852d5 | initial commit |