Fix the bbolt round trip, the blocklist gaps and the deploy set
ci / Console typecheck and build (push) Successful in 13s
ci / Go build, vet and test (push) Failing after 22s
ci / Binary with the console embedded (push) Skipped
ci / Container image (push) Skipped

The cache snapshot never loaded, so every restart was a cold cache. Snapshot
packed the whole key while Load decoded the record header out of the key's
remainder instead of the value, so every entry was skipped. The two sides are
now an explicit encodeRecord/decodeRecord pair with a version byte and a CRC-32C
trailer, so a future format change is detected instead of silently misparsed.
Load also drops entries that are already past ExpiresAt and rewrites the TTL to
the time actually left, so a served TTL never jumps back up across a restart.

Blocklist, all six gaps the failing tests were pointing at: a leading UTF-8 BOM
is stripped, "!allow example.com" works, SourceMeta exists for a source that
only ever contributed a rule, Pattern() reports the bare name again (the new
WireName() produces the wildcard form for the API), an RPZ record with no
explicit TTL inherits the zone default instead of whatever the reader left in
the RR, a hosts line with no address in the first field says so, and the
adblock parser handles CRLF and adblock-flavoured lines pasted into a plain
list. TestHolderSwapIsAtomic was asserting the wrong invariant — the swap was
always atomic, a name blocked by the old set is legitimately unblocked by the
new one.

Tests added for acl and resolver, both of which turned up real bugs:

- a PTR suffix matched across label boundaries, so a group scoped to
  example.com. also swallowed ample.com. and notexample.com. — a client could
  land in a group it does not belong to;
- the "*" catch-all was dropped from the rendered domain patterns, so the
  engine's per-query recompile lost the "everything" case;
- Upstream.Success never closed an open breaker, so a recovered upstream stayed
  excluded for the full cooldown;
- Exchange fanned out in parallel even under policy: failover.

Deploy set: go.mod requires Go 1.27.1 but CI pinned 1.23 and the Dockerfile
pinned golang:1.23-alpine, so neither could have built. The Dockerfile also
copied the source tree after the console bundle, which clobbered it.

The k8s set asked for hostNetwork with two replicas, but the server never sets
SO_REUSEPORT, so the second pod could not bind :53 and would crash-loop. It now
runs without hostNetwork behind a LoadBalancer Service, with a separate
ClusterIP Service for the Ingress (a headless Service cannot back one), a
ServiceAccount, an API-key Secret, per-pod PVCs, and a kustomization. The HPA
is gone: it cannot drive a StatefulSet. To let the API key come from that
Secret, config.Load now expands ${VAR} / ${VAR:-default} / ${NAME-default} and
fails to start on an unset variable with no default, so a missing secret is a
crash loop rather than an unauthenticated console. Expansion skips YAML
comments, so documenting ${VAR} in the ConfigMap does not break the file.

Verified: gofmt, go build, go vet clean; go test -race ./internal/... green
(acl, app, blocklist, cache, config, resolver, zonefile).
This commit is contained in:
developer committed 2026-09-27 13:30:43 +00:00
1 parent 3bcc10a029
commit 26134ba437
22 files changed
+3106 -225

No files matched your search

+88 -13
View File
@@ -56,7 +56,7 @@ type ParseStats struct {
// DetectFormat sniffs a list's syntax from its first meaningful lines.
func DetectFormat(data []byte) Format {
sc := bufio.NewScanner(bytes.NewReader(data))
sc := bufio.NewScanner(bytes.NewReader(stripBOM(data)))
sc.Buffer(make([]byte, 0, 64*1024), 1024*1024)
sawRecord := false
for i := 0; sc.Scan() && i < 200; i++ {
@@ -104,11 +104,23 @@ func allDigitsOrName(s string) bool {
return false
}
// utf8BOM is the byte-order mark a UTF-8 file may start with. Editors on
// Windows add it, and it would otherwise glue itself to the first rule's
// owner name, silently dropping the first entry of the list.
var utf8BOM = []byte{0xef, 0xbb, 0xbf}
// stripBOM drops a leading UTF-8 byte-order mark. It is applied once, at the
// front door of the parse path, so every parser sees the same bytes.
func stripBOM(data []byte) []byte {
return bytes.TrimPrefix(data, utf8BOM)
}
// Parse converts list data into rules using the given format (FormatAuto
// sniffs). It never returns partial garbage: on a hard error the rules
// accumulated so far are returned alongside it so the caller can decide
// whether to keep the last good set or adopt the partial one.
func Parse(format Format, data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
data = stripBOM(data)
if opt.DefaultTTL == 0 {
opt.DefaultTTL = 60
}
@@ -165,9 +177,13 @@ func parseHosts(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
addr := fields[0]
if !isAddr(addr) && !strings.EqualFold(addr, "localhost") {
// Not an address: this is not a hosts file after all, so treat
// the whole line as a domain list.
stats.Notices = appendNotice(stats.Notices, "no address in first field, falling back to a plain domain list")
return parsePlain(data, opt)
// the whole line as a domain list. The notice has to travel with
// the stats the plain parser returns, or the operator sees the
// format switch with no explanation for it.
const notice = "no address in first field, falling back to a plain domain list"
rules, pstats, perr := parsePlain(data, opt)
pstats.Notices = appendNotice(pstats.Notices, notice)
return rules, pstats, perr
}
names := fields[1:]
if isLoopbackName(names) {
@@ -346,13 +362,21 @@ func parsePlain(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
}
continue
}
// Support inline allow markers: "!allow example.com".
// Support inline allow markers: the documented "!allow example.com"
// and the bare "!example.com", plus the adblock exception form
// "@@||example.com^" operators paste into plain lists too.
allow := false
if strings.HasPrefix(line, "!") {
if strings.HasPrefix(line, "@@") {
allow = true
line = strings.TrimSpace(line[1:])
line = strings.TrimPrefix(line, "@@")
} else if strings.HasPrefix(line, "!") {
allow = true
line = trimAllowWord(strings.TrimSpace(line[1:]))
}
line = strings.TrimPrefix(line, "||")
// A plain list often carries a few adblock lines; drop the marker and
// everything from the first adblock separator on, so they survive
// instead of being counted as malformed.
line = cutAdblockTail(strings.TrimPrefix(line, "||"))
if !validDomain(line) {
stats.Malformed++
continue
@@ -374,6 +398,45 @@ func parsePlain(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
return rules, stats, nil
}
// cutAdblockTail drops everything from the first adblock separator onwards, so
// a plain list that someone pasted adblock lines into still yields the domain
// instead of a name with "^$|*//" decoration glued to it. A leading "*." is a
// wildcard prefix rather than a separator, and a lone "$" separator is only
// decoration when something follows it.
func cutAdblockTail(s string) string {
for i := 0; i < len(s); i++ {
switch s[i] {
case '^', '|', '/':
return s[:i]
case '$':
if i+1 < len(s) {
return s[:i]
}
case '*':
if i > 0 {
return s[:i]
}
}
}
return s
}
// trimAllowWord drops a leading allow keyword that is separated from the
// domain by whitespace, so the documented "!allow example.com" and the bare
// "!example.com" mean the same thing, while a domain that merely starts with
// the word (allow.example.com) is left alone.
func trimAllowWord(s string) string {
fields := strings.Fields(s)
if len(fields) < 2 {
return s
}
switch strings.ToLower(fields[0]) {
case "allow", "allowlist", "whitelist":
return strings.TrimSpace(s[len(fields[0]):])
}
return s
}
func parseRPZ(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
var rules []Rule
stats := ParseStats{Bytes: int64(len(data)), Format: FormatRPZ}
@@ -381,11 +444,18 @@ func parseRPZ(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
if err != nil {
return nil, stats, err
}
// A record line with no TTL column comes back from the zone reader
// carrying its own placeholder, which would shadow the TTL the caller
// asked for, so the zone's own default wins for those records.
defaultTTL := z.DefaultTTL
if defaultTTL == 0 {
defaultTTL = opt.DefaultTTL
}
seen := make(map[string]bool, len(z.Records))
for _, rec := range z.Records {
ttl := rec.TTL
if ttl == 0 && opt.DefaultTTL == 0 {
ttl = opt.DefaultTTL
if ttl == 0 || (ttl == implicitRRTTL && defaultTTL != implicitRRTTL) {
ttl = defaultTTL
}
rule, ok := rpzRule(rec, ttl, opt)
if !ok {
@@ -403,19 +473,24 @@ func parseRPZ(data []byte, opt ParseOptions) ([]Rule, ParseStats, error) {
return rules, stats, nil
}
// implicitRRTTL is the TTL the zone reader stamps on a record whose line
// carries no TTL column. It is a placeholder rather than the file's intent,
// which is why parseRPZ replaces it with the zone's own default.
const implicitRRTTL = 3600
// rpzRule maps one policy-zone record onto a rule (RFC draft-ietf-dnsop-rpz).
// RPZ encodes the response in the rdata: a CNAME to "." means NXDOMAIN, to
// "rpz-drop." means drop, to "rpz-passthru." means resolve normally, and
// A/AAAA records return that address verbatim.
func rpzRule(rec zonefile.Record, ttl uint32, opt ParseOptions) (Rule, bool) {
pattern := rec.Name
if !strings.HasPrefix(pattern, "*") && opt.Subdomains {
pattern = "*." + pattern
}
target, action, resp := rpzTarget(rec.RR)
if target == "" {
return Rule{}, false
}
if !strings.HasPrefix(pattern, "*") && opt.Subdomains {
pattern = "*." + pattern
}
return Rule{Name: pattern, Action: action, Response: resp, TTL: ttl}, true
}