swehockey/README.md

swehockey-schedule

A deterministic crawler for the Schedule and Results table on stats.swehockey.se — one plain HTTP GET per season, no browser, no JavaScript, no waiting, and a byte-identical output for identical input.

$ go build -o swehockey .
$ ./swehockey schedule -season 20961 -format csv | head -3
season_id,game_id,event_id,date,time,home,away,home_goals,away_goals,periods,spectators,venue
20961,90001001,1109941,2026-09-19,15:15,Frölunda HC,Växjö Lakers HC,5,4,1:1-1 2:2-1 3:2-2,12044,Scandinavium
20961,90001002,1109942,2026-09-19,15:15,HV 71,IF Malmö Redhawks,4,5,1:1-0 2:1-1 3:1-3 4:1-1,6800,Husqvarna Garden

$ ./swehockey crawl -league 20961            # every season of the SHL, as NDJSON
$ ./swehockey leagues                        # 119 competitions and their ids
$ ./swehockey schedule -fixture testdata/schedule_20961.html   # offline

How determinism is achieved

The site's page is server-rendered: curl returns all 364 rows of the current season in one response. That removes the usual sources of flakiness (timers, waitForSelector, lazy rendering) and leaves exactly one risk — that the markup changes under you. The rest of the design is about that risk.

1. One GET, no retries, no cookies, no state. Same HTML in, same bytes out. The output contains only page-derived data: no fetch timestamp, no map iteration, fixed field order. TestRenderIsPure and the golden files enforce this.
2. The table is found by its header signature, not by a CSS path or a position in the document. Ads, navigation and layout changes cannot move it out from under the crawler.
3. A declared contract per layout, not a pile of indices. The site has shipped more than one version of this table, so each version is a profile in layouts[] (parse.go), identified by the header row it declares. An unknown shape fails with the header it actually found.
4. Every field is shape-checked, so a renamed column or a new cell is an error naming the game and the cell, never a silently empty column.
5. Facts the site renders twice must agree. In the modern layout the date is printed in a desktop cell and as a mobile <bold> group heading, and the start time appears in the tooltip span and in the mobile cell. The parser reads both and rejects the row if they disagree. Values the site prints once per group (the date heading, the round number) are carried forward, and the first row of a group must carry the value to inherit.
6. Cross-field invariants: the period scores must sum to the final score, a played game always has an event link and an unplayed one never does, game ids are unique, dates parse, venues are non-empty.
7. Canonical order, imposed by us. The site lists games in round order, and re-scheduled or awarded games keep the round they belong to, so the page's own date order is not monotonic (2011-12 has September games after October ones). The output is sorted by date, time and id, so it is a function of the set of rows rather than of the site's row order — TestOrderIndependence swaps two rows and requires identical bytes.
8. Fixtures, not the network, in the tests. Real HTML slices live in testdata/ next to golden JSON. CI is offline and hermetic; a site change surfaces as a failing test with a readable message. SWEHOCKEY_LIVE=1 go test -run TestLive is the one opt-in test that talks to the site.
9. Drift is a distinct failure mode, not a crash: *DriftError and exit code 3. Callers can alert on it without parsing a stack trace.
10. Idempotent cron. -state <file> skips the run when the page body hash is unchanged, and -check <golden> exits 3 when the output no longer matches a committed file, so an upstream change becomes a reviewable diff.

The two layouts on the site today

profile	header	cells	seasons
modern	Date Time Game Result … Spectators Venue	8	2018-19 onwards
legacy	Round Date Game Result … Spectators Venue	7	up to 2017-18

They differ in more than columns:

• modern renders every fact twice, for a desktop and a mobile table, and prints the date only on the first row of each date. The time lives in span.lnkTooltip[title], which is also the row's stable id.
• legacy renders it once. The round is in its own column, the date and the time share one cell, and old team cells end with an <i> note (Eighth final 1) that must not be swallowed into a team name.
• A played game has 3, 4, 5 or 6 period scores depending on the era's rules, so the count is deliberately not pinned; the sum check is what protects the data.

Verified on the whole SHL archive: 27 of 27 seasons, 2119 games, 1779 played.

$ ./swehockey crawl -league 20961 -on-drift record | jq -s '[.[] | .games | length] | add'
2119

Commands

command	what it does
schedule	one season, found by -season (default 20961)
crawl	every season of -league, in ascending id order, one document each
leagues	the competitions linked from the front page

Flags: -format json|csv|ndjson, -out, -fixture, -state, -check, -timeout, -quiet, plus -limit, -delay and -on-drift for crawl.

crawl -on-drift record emits {"season_id":…,"error":…} in place of a season that breaks the contract, so a bulk run neither dies on one bad page nor hides the hole.

Exit codes: 0 ok, 1 error, 2 usage, 3 markup drift or golden mismatch, 4 timeout.

Output

{
  "season_id": 20961,
  "layout": "modern",
  "title": "Schedule and Results",
  "last_update": "2026-09-26 22:41",
  "games": [
    {
      "game_id": "90001001",
      "event_id": "1109941",
      "date": "2026-09-19",
      "time": "15:15",
      "home": "Frölunda HC",
      "away": "Växjö Lakers HC",
      "home_goals": 5,
      "away_goals": 4,
      "periods": [{"label": "1", "home": 1, "away": 1}],
      "spectators": 12044,
      "venue": "Scandinavium"
    }
  ]
}

game_id is the site's own stable per-fixture id and is present for scheduled as well as played games; event_id comes from the /Game/Events/<id> link and appears only once a game has been played. home_goals is null, not 0, for a game that has not been played. last_update is the site's own clock, verbatim and unparsed, so it can be compared across runs without trusting a timezone conversion.

Tests and fixtures

testdata/schedule_20961.html              2026-27, 3 date groups, OT + shootout, unplayed games
testdata/schedule_legacy.html             2018-19, round carry-forward, a 6-period game
testdata/schedule_legacy_playoffs.html    2013-14, <i>notes</i> in the team cells
testdata/seasons_20961.html               the season selector and league links
testdata/*.json                           golden output for the fixtures above

TestDrift mutates a fixture once per way the page can change shape — renamed header, extra column, score that no longer matches the periods, a removed date group marker, a duplicated id, a missing timestamp, an empty table — and requires a named, actionable error for each.

The fixtures are cut from real pages; the only thing the cutter changes is which <tr> elements it keeps, so the markup around them stays real, including the nested wrapper tables and the ad div.

Known limits

• Old seasons are pruned by the site: 2018-19 serves its last 45 games, 2024-25 only its 6 playoff games. The crawler reports exactly what the page contains.
• A handful of season pages serve more slowly than the default 30s timeout (they stream a large table); -timeout covers it.
• Rows with no result and no round appear in a few old pages; those are the reason -on-drift record exists.