Accuracy

The reader is measured on games it never trained on. Its labels come from a language model, so most figures here are agreement with those labels; the figures checked against a person are marked as such.

Against the alternatives

458 claims from the ten held-out games, twenty per subject. Every row answers only where it is sure enough and is scored on what it answers. The answers are in reference/frontier-key.json.

On all 5,080 held-out claims as they come, the reader answers 91.5% and agrees on 81.5% of those (macro F1 0.707).

Claude Opus 5 does better, at about 800 tokens a claim; a large game holds millions of claims, which the reader reads on one graphics card, offline. The labels were written by Claude Fable 5.1 and, from 27 September 2026, Claude Opus 5.5, so the model it is compared with wrote none of them.

Each size

The app uses the largest size the graphics card has room for, and without one starts with the small one and offers the standard one.

• Accuracy at a coverage: 458 claims from ten games held out of training, labelled under the current category sheet. Each reader answers the given share of claims it is most confident about, scored on DirectML against reference/frontier-key.json (frontier.py reader).
• GPU memory and one large game on the GPU: one read of game 920210 (117,664 claims) through the desktop app on DirectML, on an otherwise idle NVIDIA GeForce RTX 4090. Memory is the peak during the read minus what the GPU held before.
• One small game on the CPU: 1,416 claims of game 1888930 on the CPU only, loading included, on an AMD Ryzen 9 5950X with other work running.

Checked by a person

One person answered 200 claims on the page steamgauge gold --serve writes: 169 drawn blind, in English, from the held-out games, and 30 the two labellers disagreed about.

A claim can fairly sit under two subjects, so the person also judged whether the reader's subject, where it differed, would do as well. The reader's figures count the set as filed after the person read each difference again beside the sheet's rules.

In each language

40,327 claims, each answered by a reader that never trained on its game, over every claim whether or not the reader would answer it. The last column keeps only the claims the labeller was sure of.

The five in bold read below English. The reader keeps its three-in-four promise in each by answering less in them: a Korean or Spanish claim has to clear a line three times the English one. A language with fewer than 100 labelled claims declines rather than borrowing English's line.

How the labels are made

• sample-claims draws reviews by a seed, seven in ten English, and splits them into claims. Every claim of a drawn review is labelled.
• A labeller sees each claim inside its review and nothing else: not the game, the rating or the reader's answer. The one exception is whether the store lists the game as VR-only, which store-facts records and the reader is told as well.
• Every labeller works from one sheet, which brief writes from the taxonomy.
• ingest-claims refuses a returned set that does not cover the draw exactly.
• second-opinion and compare-labels have a second labeller read part of every set, and every held-out game in full: 21,449 of 33,615 claims. They agree on the subject 88.6% of the time (Cohen's kappa 0.87) and on praise or complaint 94.9% (0.92).
• declined and rare-claims draw teaching sets where the reader is weakest. They are marked as such and never measured on.
• gold and ingest-gold take a person's answers; measure-claims and ceiling score readings.
• Whole games are held out, chosen by a hash of their app ID, so two claims from one review never sit on both sides.

Limits

• English by default. Figures count English reviews unless you choose otherwise, and on some games English is under half of what was written. Every figure names the language it counts, and switching recounts from the same download.
• Unfiled claims. A rate is a share of the claims the reader would answer. It usually declines about 10.9% of claims; a game that declines 1.2 times that and five points more is flagged, with the languages behind it.
• Labels from a model. Most figures here are agreement with a language model's labels; only the person's are accuracy.
• Mechanical splitting. Reviews are split into claims by rules, and 9.1% of 30,414 randomly drawn claims are still cut where a labeller said they should not be. Training does not fix that.
• Contested is partly the labeller. Two labellers marked 27.4% and 43.9% of the same claims as genuinely ambiguous, so one game's contested rate is not compared with another's.
• Thin subjects stay thin. A subject a game barely discusses is read less reliably in that game; labelled examples from games that do discuss it are the only fix.
• Unreachable reviews. Reviews by deleted and private accounts count in Valve's total and cannot be fetched. Each game shows the coverage it reached.
• A download is a snapshot. Bringing a game up to date catches new and edited reviews, not deleted ones or changed vote counts; downloading it again does.
• Irony is not read across a game. A rating that disagrees with its text, "0/10, haven't slept in three days", is flagged on labelled reviews only.
• Counts are a summary. A count says where to read, not what was meant; every figure opens onto the reviews behind it.