Add the Python variant, agreeing with Go byte for byte

A second, independent implementation of the same contract, in the same repo:

- swehockey/dom.py builds the tree on html.parser, so the variant has no
  runtime dependencies; the only HTML5 behaviour it depends on - stray <tr>
  landing in an implicit <tbody> - is handled explicitly by table_rows().
- parse.py mirrors the Go layout profiles, dual-render checks, carry-forward and
  invariants; a DriftError is raised in the same places with the same advice.
- cli.py offers the same three commands, flags and exit codes as the Go main.
- tests/ assert against the Go golden files, so the suite is also a
  cross-implementation check. 23 offline tests plus an opt-in live test.

Verified against the live site: JSON and CSV are byte-identical to the Go
output (364 games, 102536 bytes / 365 lines), and the full archive crawl gives
the same 27 seasons, 2119 games and 1779 played.

Also split the DriftError hint out in Go, so a golden mismatch no longer tells
the reader to extend layouts[].
This commit is contained in:
Mats Tornberg committed 2026-09-27 08:53:23 +00:00
1 parent 6b701819ca
commit 47e1811d9e
15 files changed
+1821 -3

No files matched your search

+70
View File
@@ -0,0 +1,70 @@
# swehockey-schedule (Python)
The Python twin of the Go crawler in the parent directory. Same layout profiles,
same invariants, same canonical order — and, on purpose, **the same bytes**.
```console
$ python3 -m swehockey.cli schedule --season 20961 --format csv | head -2
season_id,game_id,event_id,date,time,home,away,home_goals,away_goals,periods,spectators,venue
20961,90001001,1109941,2026-09-19,15:15,Frölunda HC,Växjö Lakers HC,5,4,1:1-1 2:2-1 3:2-2,12044,Scandinavium
$ python3 -m swehockey.cli crawl --league 20961 # every season, as NDJSON
$ python3 -m swehockey.cli leagues # 119 competitions
$ python3 -m unittest discover -s tests -t . # 23 offline tests
$ SWEHOCKEY_LIVE=1 python3 -m unittest discover -s tests -t . # + the site
```
With `pip install -e .` the same CLI is on `$PATH` as `swehockey-schedule`.
## Why it can agree byte for byte
- Same `LAYOUTS` profiles, keyed on the header row; same per-field shape
checks; same dual-render agreement for the modern layout; same carry-forward
for the date heading and the round number; same invariants (periods sum to
the score, ids unique, played ⇔ event link).
- Same canonical sort by `(date, time, game_id)`, because the site orders rows
by round and puts re-scheduled games where their round says.
- The output carries only page-derived data — no fetch timestamp, no unordered
iteration, fixed key order — and `json.dumps(..., indent=2, ensure_ascii=False)`
is byte-compatible with Go's `json.Encoder`.
- The tests read the **Go golden files** in `../testdata`, so the suite doubles
as a cross-implementation check.
Verified on the live page, not just on fixtures:
```
$ python3 -m swehockey.cli schedule --season 20961 --out py.json
$ ../swehockey schedule -season 20961 -out go.json
$ cmp py.json go.json && echo identical
identical # 364 games, 102536 bytes
# and the same for CSV: 365 identical lines
# and for the whole archive: 27 seasons, 2119 games, 1779 played
```
## No runtime dependencies
The tree is built by a ~120-line builder on `html.parser`, because the only part
of the HTML5 tree construction algorithm this crawler relies on is that stray
`<tr>` elements land in an implicit `<tbody>` — which `table_rows()` handles
explicitly, exactly as in Go.
One deliberate difference from a browser: the site writes empty cells as
`<th class="…" />`, and HTML5 ignores that slash, so a real browser leaves the
cell open and swallows its siblings. This builder treats the tag as an empty
element, which is what the author meant and what keeps `child_elements()` honest.
That is also why the Go and Python trees agree on the header row (7 `th`, 6
non-empty labels) even though the Go parser closes the tag implicitly.
## Modules
| module | what it does |
| ----------------- | --------------------------------------------------------- |
| `dom.py` | the HTML tree builder and the node helpers |
| `parse.py` | layout profiles, the game reader, the invariants |
| `seasons.py` | league and season discovery |
| `fetch.py` | one GET per page, charset guard, skip-unchanged state file |
| `output.py` | JSON, NDJSON, CSV, golden-file checking |
| `cli.py` | `schedule`, `crawl`, `leagues` and the exit codes |
Exit codes match the Go CLI: `0` ok, `1` error, `2` usage, `3` markup drift or
golden mismatch, `4` timeout.