My Computer Is an Honor Student — but How Intelligent Is It? Standardized Tests as a Measure of AI

Authors

  • Peter Clark Allen Institute for Artificial Intelligence
  • Oren Etzioni Allen Institute for Artificial Intelligence

DOI:

https://doi.org/10.1609/aimag.v37i1.2636

Abstract

Given the well-known limitations of the Turing Test, there is a need for objective tests to both focus attention on, and measure progress towards, the goals of AI. In this paper we argue that machine performance on standardized tests should be a key component of any new measure of AI, because attaining a high level of performance requires solving significant AI problems involving language understanding and world modeling - critical skills for any machine that lays claim to intelligence. In addition, standardized tests have all the basic requirements of a practical test: they are accessible, easily comprehensible, clearly measurable, and offer a graduated progression from simple tasks to those requiring deep understanding of the world. Here we propose this task as a challenge problem for the community, summarize our state-of-the-art results on math and science tests, and provide supporting datasets

Downloads

Published

2016-04-13

How to Cite

Clark, P., & Etzioni, O. (2016). My Computer Is an Honor Student — but How Intelligent Is It? Standardized Tests as a Measure of AI. AI Magazine, 37(1), 5-12. https://doi.org/10.1609/aimag.v37i1.2636

Issue

Section

Articles