News Summary

• Computer scientist Moritz Hardt calls benchmarks “the mistake that made machine learning.” He’s studied where they went wrong, and why they’ve managed to drive progress anyway.
• Hardt is fascinated by how benchmarks paradoxically helped start both a 25-year AI winter in the late 1980s and a deep learning revolution in the 2010s.
• The first benchmark, in fact, was created in 1959, meant to test how well different scanners could recognize written characters.