Mats Tornberg
47e1811d9e
A second, independent implementation of the same contract, in the same repo: - swehockey/dom.py builds the tree on html.parser, so the variant has no runtime dependencies; the only HTML5 behaviour it depends on - stray <tr> landing in an implicit <tbody> - is handled explicitly by table_rows(). - parse.py mirrors the Go layout profiles, dual-render checks, carry-forward and invariants; a DriftError is raised in the same places with the same advice. - cli.py offers the same three commands, flags and exit codes as the Go main. - tests/ assert against the Go golden files, so the suite is also a cross-implementation check. 23 offline tests plus an opt-in live test. Verified against the live site: JSON and CSV are byte-identical to the Go output (364 games, 102536 bytes / 365 lines), and the full archive crawl gives the same 27 seasons, 2119 games and 1779 played. Also split the DriftError hint out in Go, so a golden mismatch no longer tells the reader to extend layouts[].
3.5 KiB
swehockey-schedule (Python)
The Python twin of the Go crawler in the parent directory. Same layout profiles, same invariants, same canonical order — and, on purpose, the same bytes.
$ python3 -m swehockey.cli schedule --season 20961 --format csv | head -2
season_id,game_id,event_id,date,time,home,away,home_goals,away_goals,periods,spectators,venue
20961,90001001,1109941,2026-09-19,15:15,Frölunda HC,Växjö Lakers HC,5,4,1:1-1 2:2-1 3:2-2,12044,Scandinavium
$ python3 -m swehockey.cli crawl --league 20961 # every season, as NDJSON
$ python3 -m swehockey.cli leagues # 119 competitions
$ python3 -m unittest discover -s tests -t . # 23 offline tests
$ SWEHOCKEY_LIVE=1 python3 -m unittest discover -s tests -t . # + the site
With pip install -e . the same CLI is on $PATH as swehockey-schedule.
Why it can agree byte for byte
- Same
LAYOUTSprofiles, keyed on the header row; same per-field shape checks; same dual-render agreement for the modern layout; same carry-forward for the date heading and the round number; same invariants (periods sum to the score, ids unique, played ⇔ event link). - Same canonical sort by
(date, time, game_id), because the site orders rows by round and puts re-scheduled games where their round says. - The output carries only page-derived data — no fetch timestamp, no unordered
iteration, fixed key order — and
json.dumps(..., indent=2, ensure_ascii=False)is byte-compatible with Go'sjson.Encoder. - The tests read the Go golden files in
../testdata, so the suite doubles as a cross-implementation check.
Verified on the live page, not just on fixtures:
$ python3 -m swehockey.cli schedule --season 20961 --out py.json
$ ../swehockey schedule -season 20961 -out go.json
$ cmp py.json go.json && echo identical
identical # 364 games, 102536 bytes
# and the same for CSV: 365 identical lines
# and for the whole archive: 27 seasons, 2119 games, 1779 played
No runtime dependencies
The tree is built by a ~120-line builder on html.parser, because the only part
of the HTML5 tree construction algorithm this crawler relies on is that stray
<tr> elements land in an implicit <tbody> — which table_rows() handles
explicitly, exactly as in Go.
One deliberate difference from a browser: the site writes empty cells as
<th class="…" />, and HTML5 ignores that slash, so a real browser leaves the
cell open and swallows its siblings. This builder treats the tag as an empty
element, which is what the author meant and what keeps child_elements() honest.
That is also why the Go and Python trees agree on the header row (7 th, 6
non-empty labels) even though the Go parser closes the tag implicitly.
Modules
| module | what it does |
|---|---|
dom.py |
the HTML tree builder and the node helpers |
parse.py |
layout profiles, the game reader, the invariants |
seasons.py |
league and season discovery |
fetch.py |
one GET per page, charset guard, skip-unchanged state file |
output.py |
JSON, NDJSON, CSV, golden-file checking |
cli.py |
schedule, crawl, leagues and the exit codes |
Exit codes match the Go CLI: 0 ok, 1 error, 2 usage, 3 markup drift or
golden mismatch, 4 timeout.