Deterministic swehockey.se schedule crawler
One HTTP GET per season, no browser: the page is server-rendered, so every row arrives in one response and the extraction becomes a pure function of the HTML. - layouts[] declares one profile per version of the site's table (modern and legacy), each identified by its header row, so a new version of the page is a new profile plus a fixture instead of a guess. - The table is located by header signature, not by a CSS path. - Every cell is shape-checked, and facts the site renders twice (desktop and mobile date, time in the tooltip) must agree. - Cross-field invariants: periods sum to the score, ids are unique, played games always have an event link and unplayed ones never do. - Output is sorted by date, time and id, so it does not inherit the site's row order, and it carries no fetch timestamp: same HTML, same bytes. - Drift is a typed error and exit code 3, with the header it actually found. - Tests run on committed HTML fixtures and golden JSON; TestLive is opt-in. Verified over the full SHL archive: 27/27 seasons, 2119 games, 1779 played.
This commit is contained in:
commit
6b701819ca
20 files changed
+2876
No files matched your search
+163
@@ -0,0 +1,163 @@
|
||||
{
|
||||
"season_id": 20961,
|
||||
"layout": "legacy",
|
||||
"title": "Schedule and Results",
|
||||
"last_update": "2014-04-14 14:19",
|
||||
"games": [
|
||||
{
|
||||
"game_id": "90011001",
|
||||
"event_id": "205038",
|
||||
"round": "1",
|
||||
"date": "2014-03-10",
|
||||
"time": "19:00",
|
||||
"home": "HV 71",
|
||||
"away": "Leksands IF",
|
||||
"home_goals": 4,
|
||||
"away_goals": 2,
|
||||
"periods": [
|
||||
{
|
||||
"label": "1",
|
||||
"home": 1,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "2",
|
||||
"home": 1,
|
||||
"away": 1
|
||||
},
|
||||
{
|
||||
"label": "3",
|
||||
"home": 2,
|
||||
"away": 1
|
||||
}
|
||||
],
|
||||
"spectators": 6923,
|
||||
"venue": "Kinnarps Arena"
|
||||
},
|
||||
{
|
||||
"game_id": "90011002",
|
||||
"event_id": "205039",
|
||||
"round": "1",
|
||||
"date": "2014-03-10",
|
||||
"time": "19:00",
|
||||
"home": "Linköpings HC",
|
||||
"away": "MODO Hockey",
|
||||
"home_goals": 4,
|
||||
"away_goals": 0,
|
||||
"periods": [
|
||||
{
|
||||
"label": "1",
|
||||
"home": 1,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "2",
|
||||
"home": 3,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "3",
|
||||
"home": 0,
|
||||
"away": 0
|
||||
}
|
||||
],
|
||||
"spectators": 5853,
|
||||
"venue": "Cloetta Center"
|
||||
},
|
||||
{
|
||||
"game_id": "90011003",
|
||||
"event_id": "205040",
|
||||
"round": "2",
|
||||
"date": "2014-03-12",
|
||||
"time": "19:00",
|
||||
"home": "Leksands IF",
|
||||
"away": "HV 71",
|
||||
"home_goals": 2,
|
||||
"away_goals": 1,
|
||||
"periods": [
|
||||
{
|
||||
"label": "1",
|
||||
"home": 0,
|
||||
"away": 1
|
||||
},
|
||||
{
|
||||
"label": "2",
|
||||
"home": 1,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "3",
|
||||
"home": 1,
|
||||
"away": 0
|
||||
}
|
||||
],
|
||||
"spectators": 6018,
|
||||
"venue": "Tegera Arena"
|
||||
},
|
||||
{
|
||||
"game_id": "90011004",
|
||||
"event_id": "205041",
|
||||
"round": "2",
|
||||
"date": "2014-03-12",
|
||||
"time": "19:00",
|
||||
"home": "MODO Hockey",
|
||||
"away": "Linköpings HC",
|
||||
"home_goals": 4,
|
||||
"away_goals": 5,
|
||||
"periods": [
|
||||
{
|
||||
"label": "1",
|
||||
"home": 2,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "2",
|
||||
"home": 2,
|
||||
"away": 1
|
||||
},
|
||||
{
|
||||
"label": "3",
|
||||
"home": 0,
|
||||
"away": 3
|
||||
},
|
||||
{
|
||||
"label": "4",
|
||||
"home": 0,
|
||||
"away": 1
|
||||
}
|
||||
],
|
||||
"spectators": 5526,
|
||||
"venue": "Fjällräven Center"
|
||||
},
|
||||
{
|
||||
"game_id": "90011005",
|
||||
"event_id": "205042",
|
||||
"round": "3",
|
||||
"date": "2014-03-13",
|
||||
"time": "19:00",
|
||||
"home": "Leksands IF",
|
||||
"away": "HV 71",
|
||||
"home_goals": 2,
|
||||
"away_goals": 4,
|
||||
"periods": [
|
||||
{
|
||||
"label": "1",
|
||||
"home": 0,
|
||||
"away": 3
|
||||
},
|
||||
{
|
||||
"label": "2",
|
||||
"home": 0,
|
||||
"away": 0
|
||||
},
|
||||
{
|
||||
"label": "3",
|
||||
"home": 2,
|
||||
"away": 1
|
||||
}
|
||||
],
|
||||
"spectators": 6462,
|
||||
"venue": "Tegera Arena"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user