LOS ANGELES Before the Central Council for Education undertakes the formidable task of revising Japans university entrance exam, it needs a refresher course in assessment. Thats because standardized tests are poorly understood.
To begin with, there is a fundamental difference between an aptitude test and an achievement test. Although scores sometimes correlate, the former is designed to predict how well a test-taker is likely to perform in a future setting. In contrast, the latter is designed to measure the knowledge and skills a test-taker possesses in a given subject.
By indicating that the current unified exam for admission to public universities in Japan places too much emphasis on mere academic knowledge, an education ministry council implies that the new instrument should be an aptitude test. But the history of the SAT in the U.S. shows that changing the focus produces confusion and resentment.
In 1926 when the test was conceived by Carl Brigham as an instrument to promote greater meritocracy, it was called the Scholastic Aptitude Test in the assumption that it measured innate ability. By 1994, however, the College Board, which oversees the SAT, was having second thoughts.
In response to concern that the original designation was associated too much with eugenics, it changed the name to the Scholastic Assessment Test. When criticism still failed to subside, the College Board again altered the name in 1997 simply to the SAT, which ironically stands for nothing.
Despite the additions and subtractions over the years, the test has been shown to have poor predictive value. In 1984, Bates College made submission of SAT scores optional for students seeking admission. In 2004, the college announced that its 20-year study had found virtually no difference in the four-year academic performance and on-time graduation rates of 7,000 submitters and nonsubmitters.
Since then, more than 850 colleges and universities that have followed the same policy have reported similar results, calling into question the indispensability of the SAT.
Theyve found that high school grades and courses taken are a far more reliable indicator of success.
Japan now has a tough choice to make. If it decides to ignore the experience of the United States and instead rely heavily on interviews, essays and group debates, it will run into a logistic nightmare in assessing the roughly 550,000 university applicants. Rubrics will need to be established and staffs will have to be expanded and trained. Moreover, the policy will leave itself open to criticism about subjectivity.
The most important question is to first determine what an admissions test is attempting to measure. Its one thing to proclaim that the goal is to identify applicants who possess the ability to think critically on their own and quite another to develop an instrument that actually does that.
The rest is here: