Paper

Status

A formal paper is not published yet. The site is the public benchmark surface for the active release.

Current release includes

Public leaderboard, September 8 model roster, per-track tables, statement sample, score schema, and submission preview.

Planned artifacts

Paper draft, harness notes, submission format, reviewer protocol, and clearer release/version history.

Reproducibility note

Current release is operator-run. Submission review and reruns are batch-managed while the public flow is being formalized.

Draft outline

• 1. Task definition and the six tracks
• 2. Statement pack, gold labeling, and adjudication
• 3. Harness, normalization, and the three-pass protocol
• 4. Results: models vs. accounting agents, Jun 29 and Sep 8 rosters
• 5. Error taxonomy from 1,140 reviewed misses
• 6. Limitations: single statement shape, USD only, public rows scored

Timeline

Cite the release

Until the preprint is out, cite the site and pack version:

@misc{evaltax2026,
  title  = {eval.tax: AI Accounting Arena},
  author = {{eval.tax}},
  year   = {2026},
  month  = sep,
  note   = {Pack v1.2, harness 1.2.3, roster of September 8, 2026},
  url    = {https://eval.tax/}
}

Data availability