0 Fork 0

Files

T

History

Mats Tornberg 47e1811d9e Add the Python variant, agreeing with Go byte for byte

A second, independent implementation of the same contract, in the same repo:

- swehockey/dom.py builds the tree on html.parser, so the variant has no
  runtime dependencies; the only HTML5 behaviour it depends on - stray <tr>
  landing in an implicit <tbody> - is handled explicitly by table_rows().
- parse.py mirrors the Go layout profiles, dual-render checks, carry-forward and
  invariants; a DriftError is raised in the same places with the same advice.
- cli.py offers the same three commands, flags and exit codes as the Go main.
- tests/ assert against the Go golden files, so the suite is also a
  cross-implementation check. 23 offline tests plus an opt-in live test.

Verified against the live site: JSON and CSV are byte-identical to the Go
output (364 games, 102536 bytes / 365 lines), and the full archive crawl gives
the same 27 seasons, 2119 games and 1779 played.

Also split the DriftError hint out in Go, so a golden mismatch no longer tells
the reader to extend layouts[].

2026-09-27 08:53:23 +00:00

swehockey

Add the Python variant, agreeing with Go byte for byte

2026-09-27 08:53:23 +00:00

tests

Add the Python variant, agreeing with Go byte for byte

2026-09-27 08:53:23 +00:00

pyproject.toml

Add the Python variant, agreeing with Go byte for byte

2026-09-27 08:53:23 +00:00

README.md

Add the Python variant, agreeing with Go byte for byte

2026-09-27 08:53:23 +00:00

README.md

swehockey-schedule (Python)

The Python twin of the Go crawler in the parent directory. Same layout profiles, same invariants, same canonical order — and, on purpose, the same bytes.

$ python3 -m swehockey.cli schedule --season 20961 --format csv | head -2
season_id,game_id,event_id,date,time,home,away,home_goals,away_goals,periods,spectators,venue
20961,90001001,1109941,2026-09-19,15:15,Frölunda HC,Växjö Lakers HC,5,4,1:1-1 2:2-1 3:2-2,12044,Scandinavium

$ python3 -m swehockey.cli crawl --league 20961     # every season, as NDJSON
$ python3 -m swehockey.cli leagues                  # 119 competitions
$ python3 -m unittest discover -s tests -t .        # 23 offline tests
$ SWEHOCKEY_LIVE=1 python3 -m unittest discover -s tests -t .   # + the site

With pip install -e . the same CLI is on $PATH as swehockey-schedule.

Why it can agree byte for byte

Same LAYOUTS profiles, keyed on the header row; same per-field shape checks; same dual-render agreement for the modern layout; same carry-forward for the date heading and the round number; same invariants (periods sum to the score, ids unique, played ⇔ event link).
Same canonical sort by (date, time, game_id), because the site orders rows by round and puts re-scheduled games where their round says.
The output carries only page-derived data — no fetch timestamp, no unordered iteration, fixed key order — and json.dumps(..., indent=2, ensure_ascii=False) is byte-compatible with Go's json.Encoder.
The tests read the Go golden files in ../testdata, so the suite doubles as a cross-implementation check.

Verified on the live page, not just on fixtures:

$ python3 -m swehockey.cli schedule --season 20961 --out py.json
$ ../swehockey schedule        -season 20961 -out go.json
$ cmp py.json go.json && echo identical
identical          # 364 games, 102536 bytes
                   # and the same for CSV: 365 identical lines
                   # and for the whole archive: 27 seasons, 2119 games, 1779 played

No runtime dependencies

The tree is built by a ~120-line builder on html.parser, because the only part of the HTML5 tree construction algorithm this crawler relies on is that stray <tr> elements land in an implicit <tbody> — which table_rows() handles explicitly, exactly as in Go.

One deliberate difference from a browser: the site writes empty cells as <th class="…" />, and HTML5 ignores that slash, so a real browser leaves the cell open and swallows its siblings. This builder treats the tag as an empty element, which is what the author meant and what keeps child_elements() honest. That is also why the Go and Python trees agree on the header row (7 th, 6 non-empty labels) even though the Go parser closes the tag implicitly.

Modules

module	what it does
`dom.py`	the HTML tree builder and the node helpers
`parse.py`	layout profiles, the game reader, the invariants
`seasons.py`	league and season discovery
`fetch.py`	one GET per page, charset guard, skip-unchanged state file
`output.py`	JSON, NDJSON, CSV, golden-file checking
`cli.py`	`schedule`, `crawl`, `leagues` and the exit codes

Exit codes match the Go CLI: 0 ok, 1 error, 2 usage, 3 markup drift or golden mismatch, 4 timeout.