initial commit
This commit is contained in:
developer committed
1 parent
5b935301e5
commit
85453852d5
105 files changed
+24846
No files matched your search
@@ -0,0 +1,207 @@
|
||||
# fancy-dns — development status and continuation plan
|
||||
|
||||
Last updated: session checkpoint. Everything below reflects the state of the
|
||||
working tree at the time of writing, including known-failing tests and the
|
||||
exact next command to run.
|
||||
|
||||
## Goal
|
||||
|
||||
A lightweight, high-performance recursive DNS server with:
|
||||
|
||||
- **Go backend** — sharded LRU cache backed by bbolt, blocklist replication from
|
||||
multiple formats, per-client ACLs, query logging, Prometheus metrics.
|
||||
- **Secure DNS** — DNS-over-TLS (RFC 7858) and DNS-over-HTTPS (RFC 8484).
|
||||
DNSSEC validation was explicitly scoped **out**; AD/CD/extended-RCODE are
|
||||
passed through only.
|
||||
- **React + Tailwind admin console** embedded in the binary.
|
||||
- **Config as zone files + YAML**, hot-reloadable, with **Valkey/Redis** for
|
||||
cross-instance sync (chosen over AMQP).
|
||||
|
||||
## Architecture decisions (settled with the user)
|
||||
|
||||
| Decision | Choice | Why |
|
||||
|---|---|---|
|
||||
| Scope | Full production | clustering, metrics, ACLs, query logs, hot reload, Docker, k8s, CI |
|
||||
| Config store | YAML + DNS master files | reviewable/diffable in git |
|
||||
| Cluster sync | Valkey (Redis protocol) | simple, mature Go client, easy local dev |
|
||||
| Secure DNS | DoT + DoH, no DNSSEC validation | avoids signer/anchor machinery; pass-through only |
|
||||
| Hot cache | Sharded LRU in memory, bbolt for snapshots | no I/O on the hot path; warm restart |
|
||||
|
||||
## Package layout
|
||||
|
||||
```
|
||||
cmd/fancy-dns/main.go entrypoint: flags, signals, SIGHUP reload
|
||||
internal/
|
||||
config/ Config struct, Duration type, Validate, atomic Store + hooks
|
||||
zonefile/ RFC 1035 master-file parser ($TTL/$ORIGIN, parens, serial math)
|
||||
blocklist/ Set (exact/sub/root maps), 4 parsers, Manager (fetch+compile+swap)
|
||||
cache/ Key, Entry, sharded LRU, TTL policy, serve-stale, bbolt Store
|
||||
resolver/ Upstream pool: health, circuit breakers, parallel/failover/random
|
||||
acl/ compiled groups, CIDR/PTR/tag matching, per-group domain policy
|
||||
localzone/ authoritative data from master files, wildcards, delegations
|
||||
engine/ THE QUERY PIPELINE (see below), reverse-PTR cache
|
||||
server/ UDP, TCP, DoT, DoH listeners; all funnel into one handler
|
||||
qlog/ bbolt query-log ring + live in-memory ring + SSE fanout
|
||||
metrics/ Prometheus collectors + rolling time-series rings
|
||||
cluster/ Valkey pub/sub, leader lease, divergence detection
|
||||
api/ admin REST + SSE, auth, CORS, rate limiting
|
||||
app/ wiring, lifecycle, API hook implementations
|
||||
web/ go:embed of the built console
|
||||
web/ Vite + React + TS + Tailwind console (subagent-built)
|
||||
deploy/ example config, example zones
|
||||
```
|
||||
|
||||
### Query pipeline order (engine.Handle)
|
||||
|
||||
```
|
||||
validate → ACL deny → per-client concurrency → global concurrency → rate limit
|
||||
→ group allow/deny domains → local zone → blocklist → cache → upstream
|
||||
→ cache put → finalize (EDNS0, UDP truncation) → metrics + qlog
|
||||
```
|
||||
|
||||
Cache is a single shard-local lookup; only a miss touches the network.
|
||||
|
||||
## Status
|
||||
|
||||
### Done and verified
|
||||
|
||||
- All packages compile; `go vet ./...` clean.
|
||||
- `go test ./internal/app/` **passes** (integration suite, ~20 tests) with a fake
|
||||
upstream on loopback: resolve/caching, all blocklist shapes, allow-over-block,
|
||||
local zones (exact/wildcard/NODATA/NXDOMAIN), ACL deny + allow_domains +
|
||||
skip_blocklist, all four block actions, malformed query FORMERR, TCP parity,
|
||||
API auth (401/403/200), domain add→blocked→delete round trip, ACL update,
|
||||
upstream update, settings validate/save + rejected-write-leaves-file-alone,
|
||||
blocklist toggle both directions, dashboard/status shape, logs, cache
|
||||
endpoints, reload failure keeps serving + success clears error, SSE stream
|
||||
emits `ready` and `entry`, metrics protected/unprotected, concurrency smoke.
|
||||
- Verified live end-to-end against the real binary on :15353/:18080 with dig.
|
||||
- Subagent-verified working examples of `/api/domains/check` for hosts-file,
|
||||
RPZ-wildcard and inline rules.
|
||||
|
||||
### Known failures (fix these first)
|
||||
|
||||
1. **`go test -race ./internal/app/` still reports DATA RACE and two failures.**
|
||||
- `TestDomainAddDeleteRoundTrip` and `TestUpstreamUpdate` fail **only under
|
||||
`-race`** (they pass without it), which points at timing, not logic.
|
||||
- I had just started fixing races by moving `engine.sem`, `engine.semSize`
|
||||
and `engine.upstreamTimeout` behind atomics. The `applyConfig` hook now
|
||||
writes `e.sem.Store(&sem)`; `Handle` does `sem := e.sem.Load()`. **This
|
||||
refactor is mid-flight** — re-read `internal/engine/engine.go` before
|
||||
touching it. Remaining races are almost certainly in
|
||||
`config.Config` field access from `applyConfig` (e.g. `a.pool.Rebind`
|
||||
called from the swap hook while queries read `cfg`), and in
|
||||
`harness_test.go` reading the config file the server rewrites (partially
|
||||
fixed by snapshotting `doc` before `Start` — verify).
|
||||
- `internal/app/dbg_test.go` was deleted; do not reintroduce it.
|
||||
2. **Frontend `web/` is incomplete.** The foundation/pages subagents each ran
|
||||
out of budget. `src/lib/types.ts`, `format.ts`, `cn.ts`, `vite.config.ts`
|
||||
exist; `src/index.css` is still the purple Vite template; `src/lib/api.ts`,
|
||||
`src/hooks/*`, `src/components/ui/*`, `src/components/layout/*` and all of
|
||||
`src/pages/*` are missing. `npm run build` and `npx tsc -b --noEmit` have
|
||||
never passed. `internal/web/dist/index.html` is a placeholder so the binary
|
||||
compiles.
|
||||
3. **No unit tests outside `internal/app/`.** `blocklist`, `cache`, `zonefile`,
|
||||
`acl`, `resolver`, `localzone` have no direct tests.
|
||||
4. **Missing deliverables:** `Makefile`, `.gitignore`, `Dockerfile`, k8s
|
||||
manifests, CI workflow, `README.md`.
|
||||
|
||||
## Continuation plan, in order
|
||||
|
||||
### Step 1 — green tests (highest priority)
|
||||
|
||||
```bash
|
||||
go build ./... && go vet ./...
|
||||
go test -race ./internal/app/ -count=1 2>&1 | grep -A20 "DATA RACE" | head -60
|
||||
```
|
||||
|
||||
Expected races to fix, in likelihood order:
|
||||
- `config.Store` swap hook running `app.onConfigSwap` → `pool.Rebind` and
|
||||
`reloadLocalZones` while the query path reads the old `*config.Config`.
|
||||
Fix direction: make `Pool.Rebind` build a new slice and swap an
|
||||
`atomic.Pointer` rather than mutating in place under `sync.RWMutex` while
|
||||
readers hold `RLock` on a *different* field.
|
||||
- `blocklist.Manager` `m.running` map vs `RefreshAll` goroutine + API handler.
|
||||
Already mutex-guarded; confirm it is the *Stats* pointer that races.
|
||||
- `qlog.Logger.live` ring: `Log` writes `l.live[pos%len]` from many goroutines
|
||||
while `Recent` reads. Needs a seqlock or a per-slot atomic pointer.
|
||||
- Harness reading the config file path from a test goroutine.
|
||||
|
||||
For each race, prefer removing shared mutable state over adding locks: the
|
||||
design already uses `atomic.Pointer` holders for ACL, blocklist, localzone and
|
||||
pool; extend that pattern.
|
||||
|
||||
### Step 2 — unit tests for the hot packages
|
||||
|
||||
Priority order (highest bug-density first):
|
||||
- `blocklist`: `Set.Match` for `*.x` (subdomains only) vs `**.x` (apex too)
|
||||
vs exact; allow-over-block priority; hosts/adblock/plain/rpz parser cases;
|
||||
`DetectFormat` sniffing; duplicate handling.
|
||||
- `cache`: LRU eviction, TTL clamping, `TTLFor` on NXDOMAIN/NODATA/SERVFAIL,
|
||||
serve-stale window, key pack/unpack round-trip, bbolt snapshot→load.
|
||||
- `zonefile`: `$TTL` inheritance, omitted owner/TTL/class, parens, `@`,
|
||||
`$ORIGIN` relative vs absolute names, `ParseTTL`/`SerialNext`.
|
||||
- `acl`: first-match-wins, most-specific CIDR, PTR suffix, `DomainSet.Match`.
|
||||
- `localzone`: delegation cut, closest-wildcard (RFC 1034 §4.3.3), NODATA.
|
||||
- `resolver`: breaker open/close, parallel race, TCP fallback on truncation.
|
||||
- Add `BenchmarkLookup` for `Set.Match` and `cache.LRU.Get` to substantiate
|
||||
the "great performance" claim.
|
||||
|
||||
### Step 3 — finish the frontend
|
||||
|
||||
Respawn two subagents with disjoint file ownership (this worked structurally;
|
||||
they just need more budget):
|
||||
|
||||
- **Foundation agent** owns `src/index.css` (replace the purple template with
|
||||
dark-first CSS vars + one emerald accent), `src/lib/api.ts`, `src/hooks/*`,
|
||||
`src/components/ui/*`, `src/components/layout/*`. The exact prop contract I
|
||||
gave them is in the transcript and must be kept.
|
||||
- **Pages agent** owns `src/pages/*` only, coding against the same contract.
|
||||
|
||||
Then wire `src/App.tsx` + `src/main.tsx` yourself (react-router, `AppShell`,
|
||||
`ToastProvider`, theme init) and run `npm run build`. Once `web/dist` is real,
|
||||
the `//go:embed all:dist` in `internal/web/web.go` picks it up with no Go
|
||||
change; delete the placeholder `internal/web/dist/index.html`.
|
||||
|
||||
API contract for the frontend is fixed and implemented — see
|
||||
`internal/api/handlers.go` route table and `internal/app/hooks.go` payload
|
||||
shapes. `/api/logs/stream` is SSE with events `ready`, `entry`, `stats`, and
|
||||
`X-Accel-Buffering: no`.
|
||||
|
||||
### Step 4 — deploy + docs
|
||||
|
||||
- `Makefile`: `build`, `test`, `race`, `vet`, `web`, `lint`, `run`, `image`.
|
||||
- `Dockerfile`: multi-stage — node build for `web/`, then Go build with
|
||||
`-ldflags "-X ...app.Version=..."`; distroless/scratch; `EXPOSE 53/udp 53/tcp
|
||||
853/tcp 443/tcp 8080/tcp`; non-root; `VOLUME /var/lib/fancy-dns`.
|
||||
- k8s: Deployment (hostNetwork or a NodePort Service for :53), Service,
|
||||
ConfigMap for the YAML, PVC for the bbolt files, readiness probe on
|
||||
`/api/healthz`, optional Ingress for the console.
|
||||
- CI: `go vet`, `go test -race`, `gofmt -l`, `npm ci && tsc -b && npm run build`.
|
||||
- `README.md`: quickstart, config reference, zone-file semantics, API
|
||||
reference, performance notes, the security caveats that actually matter
|
||||
(`trusted_proxies` and `X-Forwarded-For`; `metrics.protect`; the
|
||||
`trust_loopback` + `auth.mode` matrix).
|
||||
|
||||
## Design notes worth preserving
|
||||
|
||||
- **Cache stores packed wire bytes, not `*dns.Msg`.** A hit costs one
|
||||
`Unpack` and yields a private copy, so serving never mutates shared state and
|
||||
TTL ageing is free. Same trade dnsmasq makes.
|
||||
- **Cache invalidation on rule changes flushes the cache** rather than
|
||||
guessing which subtrees are affected. A newly blocked name must not keep
|
||||
serving its cached good answer, and a new allow rule can rescue a name many
|
||||
lists blocked. `invalidateBlocklistCache()` in `internal/app/hooks.go`.
|
||||
- **API responses share the field names in `internal/app/hooks.go`.** If you
|
||||
rename one, grep the frontend too.
|
||||
- **API keys are compared by SHA-256 hash, constant-time**, and
|
||||
`X-Forwarded-For` is honoured *only* from `trusted_proxies` — otherwise a
|
||||
client could spoof its identity past the ACL and the rate limiter.
|
||||
- **UDP listener resets its read deadline every iteration.** A deadline set
|
||||
once expires permanently and turns the loop into a spin that never reads
|
||||
another packet (this was a real bug caught by `TestBlocklistToggle`).
|
||||
- **Only the Valkey lease holder fetches lists**, so N instances do not hit a
|
||||
mirror N times. Followers serve their compiled copy.
|
||||
- **Blocked/serve-stale behaviour:** a served-stale answer gets TTL 1, an
|
||||
expired upstream answer is never cached, and a truncated answer is never
|
||||
cached (it would be served truncated forever).
|
||||
Reference in new issue
Block a user