294. What Does IQ Really Measure?

By Michael Balter, ScienceNow, April 25, 2011

Kids who score higher on IQ tests will, on average, go on to do better in conventional measures of success in life: academic achievement, economic success, even greater health, and longevity. Is that because they are more intelligent? Not necessarily. New research concludes that IQ scores are partly a measure of how motivated a child is to do well on the test. And harnessing that motivation might be as important to later success as so-called native intelligence.

Researchers have long debated what IQ tests actually measure, and whether average differences in IQ scores--such as those between different ethnic groups--reflect differences in intelligence, social and economic factors, or both. The debate moved heavily into the public arena with the 1994 publication of The Bell Curve by Richard Herrnstein and Charles Murray, which suggested that the lower average IQ scores of some ethnic groups, such as African-Americans and Hispanics, were due in large part to genetic differences between them and Caucasian groups. That view has been challenged by many scientists. For example, in his 2009 book "Intelligence and How to Get It," Richard Nisbett, a psychologist at the University of Michigan, Ann Arbor, argued that differences in IQ scores largely disappear when researchers control for social and economic factors.

New work, led by Angela Lee Duckworth, a psychologist at the University of Pennsylvania, and reported online today in the Proceedings of the National Academy of Sciences explores the effect of motivation on how well people perform on IQ tests. While subjects taking such tests are usually instructed to try as hard as they can, previous research has shown that not everyone makes the maximum effort. A number of studies have found that subjects who are promised monetary rewards for doing well on IQ and other cognitive tests score significantly higher.

To further examine the role of motivation on both IQ test scores and the ability of IQ tests to predict life success, Duckworth and her team carried out two studies, both reported in today's paper. First, they conducted a "meta-analysis" that combined the results of 46 previous studies of the effect of monetary incentives on IQ scores, representing a total of more than 2000 test-taking subjects. The financial rewards ranged from less than $1 to $10 or more. The team calculated a statistical parameter called Hedge's g to indicate how big an effect the incentives had on IQ scores; g values of less than 0.2 are considered small, 0.5 are moderate, and 0.7 or higher are large.

Duckworth's team found that the average effect was 0.64 (which is equivalent to nearly 10 points on the IQ scale of 100), and remained higher than 0.5 even when three studies with unusually high g values were thrown out. Moreover, the effect of financial rewards on IQ scores increased dramatically the higher the reward: Thus rewards higher than $10 produced g values of more than 1.6 (roughly equivalent to more than 20 IQ points), whereas rewards of less than $1 were only one-tenth as effective.

In the second study, Duckworth and her colleagues analyzed data from an earlier study of more than 500 boys from Pittsburgh, Pennsylvania, whose IQs were tested in the late 1980s by a team from the University of Wisconsin, Madison. During the IQ test, the boys, whose average age was 12.5 years, were videotaped; then observers trained to detect signs of boredom and lack of motivation (such as yawning, laying their heads on the table, or looking often around the room) viewed the videos and assigned motivation scores.

Researchers followed the boys over time, and when the boys reached early adulthood (average age 24), 251 of them agreed to a series of interviews about their educational and job achievements (there were no differences in IQ or other key factors between those boys who participated and those who didn't.)

Duckworth's team analyzed the results of these earlier studies to see what they said about the relationship between motivation, IQ scores, and life success. By constructing a series of computer models of the data, the team found that higher motivation accounted for a significant amount of the differences in IQ scores and also in how well IQ predicted later success in life. For example, differences in motivation levels accounted for up to 84% of the differences between the boys in how many years of school they had completed or whether they had been able to find a job. On the other hand, motivation differences accounted for about only 25% of the differences in how well they had done in school as teenagers. According to the researchers, that suggests that native intelligence does still play an important role in both IQ scores and academic achievement.

Nevertheless, the Duckworth team concludes that IQ tests are measuring much more than just raw intelligence--they also measure how badly subjects want to succeed both on the test and later in life. Yet Duckworth and her colleagues caution that motivation isn't everything: The lower role for motivation in academic achievement, they write, suggests that "earning a high IQ score requires high intelligence in addition to high motivation."

The study has important social policy implications, Duckworth says. "I hope that social scientists, educators, and policy makers turn a more critical eye to any kind of measure, intelligence or otherwise," she says, adding that how hard people try "could be as important to success in life as intellectual ability itself." Duckworth suggests that admissions to programs for "gifted and talented" children should not be based on IQ scores alone, but also on "who wants to do the work."

Nisbett agrees that the study is "tremendously important in its implications." Motivation, along with self-discipline, "are crucial," Nisbett says. "A high IQ and a subway token will only get you into town."

Lex Borghans, an economist at the Maastricht University in the Netherlands, who has also studied the relationship between intelligence tests and economic success, says the new report shows that "both intelligence and personality matter." Even if native intelligence cannot be increased, Borghans says, "there might be other routes to success."