Here’s a really interesting post on Language Log about those surveys that consistently reveal how ignorant Americans are, especially about public affairs. Arthur Lupia of the University of Michigan has discovered that there are some issues, to put it gently, with how such surveys are scored.

Some years ago, when William Rehnquist was the Chief Justice of the United States, a typical survey asked people what job William Rehnquist held. Here are some of the answers that were marked wrong:

Supreme Court justice. The main one.
He’s the senior judge on the Supreme Court.
He is the Supreme Court justice in charge.
He’s the head of the Supreme Court.
He’s top man in the Supreme Court.
Supreme Court justice, head.
Supreme Court justice. The head guy.
Head of Supreme Court.
Supreme Court justice head honcho.

In fact, even the most technically correct answer — Chief Justice of the United States — was marked wrong, because the only answers deemed correct contained two phrases, “Chief Justice” and “Supreme Court.”

If this kind of measurement is indeed common in such surveys, as Lupia suggests, then we really don’t know anything about what we don’t know. Might be time to fix that.