Meritocracy: Dangerous Cancer Statistics
About the only detailed public criticism of my Meritocracy article by an academic has come from Prof. Janet Mertz, a Wisconsin cancer researcher. Since her analysis draws so heavily upon her own 2008 academic paper on top performing math students, I decided that paper warranted a close examination.
The primary focus of her article was a worldwide gender analysis of top performing math students aimed at refuting the controversial speculations of former Harvard President Larry Summers, who had suggested that men might be better at math than women, at least at the very high end of math ability. She and her co-authors therefore examined the previous twenty years of the International Math Olympiad, determining the exact number of male and female participants from all the leading countries. They provided their findings in Table 6 (p. 1252), which I am summarizing below in terms of the male percentages for the aggregate years 1988-2008:
China, 96% male
India, 97% male
Iran, 98% male
Israel, 98% male
Japan, 98% male
Kazakhstan, 99% male
South Korea, 93% male
Taiwan, 95% male
Turkey, 96% male
Vietnam, 97% male
Belarus, 94% male
Bulgaria, 91% male
Czech Republic, 96% male
Slovakia, 88% male
France 97% male
Germany, 94% male
Hungary, 94% male
Poland, 99% male
Romania, 94% male
Russia/USSR, 88% male
Serbia and Montenegro, 80% male
Ukraine, 93% male
United Kingdom, 93% male
Australia, 94% male
Brazil, 96% male
Canada, 90% male
USA, 96% male
INTERNATIONAL AVERAGE, 94.4% male
Now to an untutored eye such as my own, Mertz’s discovery that the top math students of virtually every country have been around 95% male for decades might seem to somewhat confirm rather than refute the distasteful speculations of President Summers; but Mertz had a very different perspective. Surrounding these basic quantitative facts by 10,000 words of often complex verbiage, she concluded that math performance differences between males and females were overwhelmingly due to culture rather biology, and that at the very high end, women had just as much math ability as men.
She later cited these same research results to support her equal-ability gender claims in subsequent published papers, with her most recent 2012 paper bearing the descriptive title “Debunking Myths about Gender and Mathematics Performance.” Most importantly, she claimed in her media interviews that her research had demonstrated that men and women had equal innate ability in mathematics, and that any current differences in performance were due to culture or bias. Therefore, the press reported that Mertz and her allies had proven Summers wrong, and women had just as much talent in math as men.
One obvious possibility was that I was missing something in Mertz’s research, and somehow misunderstanding her apparent result that 95% of all top math students have always been male and just 5% female. Perhaps a careful researcher such as Mertz was correct and I was just failing to comprehend her analysis. Fortunately, there are others with far greater statistical expertise, much better able to judge such matters.
Consider, for example, Prof. Andrew Gelman, an award-winning Ivy League statistics professor. Over the last month or so, Prof. Gelman has heavily promoted Prof. Mertz’s research on his blogsite, repeatedly pointing to her strong academic credentials in sharp contrast to what he describes as my own background as a “political activist” with “sloppy” research methods.
I therefore dropped Prof. Gelman a respectful note, asking him what he thought of the conclusions that Prof. Mertz drew from her 2008 paper, and whether they seemed warranted by her underlying data. He replied that he had merely “skimmed” her paper and had no particular opinion on whether she was right or wrong. With his permission, I am publishing our brief exchange.
Although I am certainly pleased that Gelman now seems to be backpedaling from his criticism of my work—he argues that “sloppy” was never meant to be an insulting adjective—I really wonder what this indicates about his own scholarly methods. After all, not only had he written four or five separate columns and numerous comments—probably totaling over 15,000 words—promoting Mertz, but he had also repeatedly cited or linked to her 2008 paper. Shouldn’t he have actually *read* rather than merely “skimmed” her paper and even investigated her use of statistics before he repeatedly used it to denounce my own research? Perhaps this raises questions about whether he bothered reading my own article before criticizing it. One suspects that something more than mere dispassionate scholarly interest explains the rapidity with which Gelman wholeheartedly endorsed Mertz’s accusations without apparently bothering to investigate them.
We must ask ourselves what it means for our society when an academic such as Mertz can determine that 95% of all top math students have always been male and then immediately announce that she has proven that men and women have equal mathematical ability at the high end, thereby producing headlines in Science Daily and on popular websites. Mertz may simply be an agitated ideologue, but those lazy or biased journalists who eagerly promote her absurd claims are just as guilty, and that goes double for seemingly-reputable academics such as Prof. Gelman who lend their names to similar nonsense.
I suspect that Mertz’s tendency to wildly mischaracterize statistical data is confined to her ideological math-feminism and similar matters, but perhaps that I am mistaken. Her field of professional expertise is cancer research, in which misuse of statistics may have life-or-death consequences.
Suppose Mertz had conducted a study of two different cancer treatments, tested in trials across two dozen countries around the world. Suppose also the patient death rate for one of those treatments averaged twenty times greater than the other, with the death ratio for nearly all the countries falling in the range between 15-to-1 and 50-to-1. If Mertz then summarized her results by reporting that the two cancer treatments seemed very similar in effectiveness, there might be very serious consequences for human health. I do hope that someone of Prof. Gelman’s statistical expertise is keeping close watch to ensure that Mertz’s statistical misfortunes are confined solely to ideological matters and do not contaminate her life-determining medical research.
Finally, on a different matter, NYT columnist David Brooks had been so surprised and impressed with some of my findings that he gave my article a “Sidney award” as among the best of the year. One of my findings had been the collapse of Jewish academic achievement in recent decades, and I had speculated that the exponential growth of the academically unimpressive ultra-Orthodox community might be a major reason. Brooks has now investigated this “Orthodox Surge” in greater detail, and written a column about it.
I also yesterday participated in a well-attended DC Aspen Institute panel on raising the minimum wage. More about that in a future column.