The Graduate Record Examination (GRE) does little to predict who will do well in graduate school for psychology and quite likely in other fields as well, according to a new study by Cornell and Yale universities.
Of the three subtests of the GRE (verbal, quantitative and analytical) and the GRE advanced test in psychology, only the analytical subtest predicted any aspect of graduate success beyond the first-year grade point average (GPA), and this prediction held for men only. The verbal subtest and psychology test predicted first-year GPA, but this prediction vanished by the second year's GPA.
"With these exceptions, the GRE scores were not useful as predictors of various aspects of graduate performance, including ratings by primary advisers of analytical, creative, practical, research and teaching abilities by primary advisers and ratings of dissertation quality by independent faculty readers," said Wendy M. Williams, associate professor of human development at Cornell.
Williams and her colleague, Robert J. Sternberg of Yale University both experts on measures and theories of intelligence reported their findings in the June issue of American Psychologist.
The researchers strongly suspect that the GREs may prove to lack validity in predicting performance in other fields as well.
"We know from other researchers' work that the GREs also have failed to predict success in the field of physics, and we suspect that the GREs will fail to prove predictive for the humanities as well," Williams said.
"Instead of relying so heavily on the GREs and many applicants aren't even considered if their GRE scores are not in the top group we need to develop and use tests that measure meaningful performances in specific areas. The GREs, including the one specifically for psychology, do not assess many of the types of abilities required for succeeding as a professional psychologist," Williams said.
She also pointed out that applicants from less privileged backgrounds, who are not as likely to do as well on the GRE as applicants from good preparatory schools, lose out even though they may have the appropriate skills for the profession they desire. "Graduate programs rely so heavily on GREs to make their initial cuts, many well-qualified applicants who are strong in the appropriate areas aren't even being considered. This is a huge disservice to the applicants, the graduate programs and society at large."
The researchers set out to test the validity of the GRE, working within the broader framework of the triarchic theory of human intelligence. The triarchic theory distinguishes academic or analytical abilities from creative and practical abilities.
"Academic-analytical abilities are used when one analyzes, compares and contrasts, evaluates, judges or critiques," said Sternberg, who has published widely on the theories of intelligence. "Creative abilities are used when one invents, discovers, supposes, hypothesizes or theorizes. Practical abilities are used when one applies, uses or implements."
To assess the validity of GREs in predicting success or failure of graduate students, the researchers asked 40 faculty members of psychology at Yale to provide ratings on five scales of the 166 graduate students they had had since 1980. In addition, the researchers looked at GPAs of students in their first and second years of graduate training and overall evaluations of dissertations by outside, independent raters.
"This study suggests the need to reflect on the use of tests before they become firmly and, as it sometimes seems, irrevocably entrenched. Too often, we believe, the use of a test becomes self-perpetuating, without serious attempts to verify its effectiveness," the psychologists wrote. "We believe that our results underscore the need for serious validation studies of the GRE, not to mention other admissions indexes, against measures of consequential performances, whether of students or of professionals."
Next, Williams hopes to look at GRE scores of men and women in the social sciences, natural sciences and humanities. The study was supported in part by the U.S. Department of Education.