Add the Python variant, agreeing with Go byte for byte

A second, independent implementation of the same contract, in the same repo:

- swehockey/dom.py builds the tree on html.parser, so the variant has no
  runtime dependencies; the only HTML5 behaviour it depends on - stray <tr>
  landing in an implicit <tbody> - is handled explicitly by table_rows().
- parse.py mirrors the Go layout profiles, dual-render checks, carry-forward and
  invariants; a DriftError is raised in the same places with the same advice.
- cli.py offers the same three commands, flags and exit codes as the Go main.
- tests/ assert against the Go golden files, so the suite is also a
  cross-implementation check. 23 offline tests plus an opt-in live test.

Verified against the live site: JSON and CSV are byte-identical to the Go
output (364 games, 102536 bytes / 365 lines), and the full archive crawl gives
the same 27 seasons, 2119 games and 1779 played.

Also split the DriftError hint out in Go, so a golden mismatch no longer tells
the reader to extend layouts[].
This commit is contained in:
Mats Tornberg committed 2026-09-27 08:53:23 +00:00
1 parent 6b701819ca
commit 47e1811d9e
15 files changed
+1821 -3

No files matched your search

+44
View File
@@ -90,6 +90,50 @@ $ ./swehockey crawl -league 20961 -on-drift record | jq -s '[.[] | .games | leng
2119
```
## Two implementations, one contract
The crawler exists twice, in Go and in Python, and that is a feature: two
independent parsers of the same page that have to agree.
| | Go | Python |
| --- | --- | --- |
| entry point | `./swehockey schedule -season 20961` | `python3 -m swehockey.cli schedule --season 20961` |
| tests | `go test ./...` (offline) | `python3 -m unittest discover -s tests -t .` (offline) |
| live test | `SWEHOCKEY_LIVE=1 go test -run TestLive` | `SWEHOCKEY_LIVE=1 python3 -m unittest tests.test_swehockey.TestLive` |
| dependencies | `golang.org/x/net/html` | none — the tree is built on `html.parser` |
Both share `testdata/`: the Python suite asserts its output against the **Go
golden files**, so a change in one implementation that the other does not share
fails a test. Verified on the live page as well:
```
$ ./swehockey schedule -season 20961 -out go.json
$ python3 -m swehockey.cli schedule --season 20961 --out py.json
$ cmp go.json py.json # 364 games, 102536 bytes, identical
$ ./swehockey schedule -season 20961 -format csv -out go.csv
$ python3 -m swehockey.cli schedule --season 20961 --format csv --out py.csv
$ cmp go.csv py.csv # 365 lines, identical
$ python3 -m swehockey.cli crawl --league 20961 | jq -s '[.[].games|length]|add'
2119 # same as the Go crawl
```
Same layout profiles, same invariants, same canonical order, same exit codes
(`0` ok, `1` error, `2` usage, `3` drift, `4` timeout). The only intentional
difference is in the tree builder: Python treats the site's empty `<th … />`
cells as empty elements instead of leaving them open the way a browser does,
which is what makes the two trees agree. See `python/README.md`.
## Repository layout
```
main.go parse.go seasons.go fetch.go output.go the Go crawler
parse_test.go the Go tests
python/swehockey/ the Python crawler
python/tests/ the Python tests
testdata/ HTML fixtures and golden JSON, shared by both
README.md this file
```
## Commands
| command | what it does |