Every DFS tool says its projections are the best. Almost none of those claims can be checked: there's no shared scoreboard, no common rules, and no record of what each tool said before the games started. DFS Benchmark is that scoreboard.
What we grade
On every graded slate, we capture each tool's full projections, its ownership projections and its #1 lineup at the same minute before lock, on default settings. After the games, every tool is graded against DraftKings results on one shared player pool.
Eight headline benchmarks rank the tools:
- Beat-the-Baseline: how much less error a tool makes than each player's recent-form average.
- Points Accuracy: average DK points missed per player.
- Rank Accuracy: whether players were ordered correctly, within each position.
- Value Finder: how many of the slate's 10 best values were in its top 20.
- Ownership Accuracy: average miss in projected ownership.
- Chalk Finder: how many of the 5 most-owned players it saw coming.
- % of Perfect: the points its highest-projected lineup scored, as a share of the best possible lineup's.
- Cash-Line Rate: how often that lineup would have cashed.
Beat-the-Baseline leads the table. A tool that can't beat a simple average of each player's recent games isn't adding much.
Rules before data
The rulebook is public before the first capture. Every captured file gets a SHA-256 fingerprint before games start, and the fingerprints are posted publicly, so nothing can be swapped after the fact. Late swap isn't graded: the capture is final.
Why nothing is ranked yet
One slate proves very little. DFS results are noisy, so every number on the leaderboard carries a 95% range, and a tool only leads a row when its range clears the next tool's. No row is ranked before 20 graded NBA slates. Until then, results post to the results log after each slate, marked provisional.
Who's tested
Version 1 covers SaberSim, Stokastic, RotoGrinders and FantasyLabs. Each is graded only after the company agrees. Claude is graded in its own AI Division, with the same fixed prompt every slate, web search on and the salary file attached. Two simple baselines, recent form and salary, set the floor every tool should clear.
Who runs it
DFS Benchmark is owned and operated by the founders of The DFS Kitchen. The Kitchen appears as a labeled house entry, graded under the same rules as every other tool. Its files are captured and fingerprinted before any other tool's files are downloaded each slate, and data from other companies never reaches it. We don't call ourselves independent. The rules are public so anyone can check the work.
What's next
- October 20: NBA opening night, the first graded slate.
- After every slate: results in the results log, including the slate's perfect lineup and Claude's full answer.
- Mid-November: the first ranked leaderboard, after 20 graded slates.
- Every Monday: the weekly email, starting after the first week of grading.