Many observers express concern about low levels of “political knowledge” among citizens. The
basis for such concerns is responses to “knowledge questions” on political surveys. It is well known that
many respondents do badly at answering these questions (Delli Carpini and Keeter 1996, Luskin 1990).
In this paper, we examine whether such surveys offer an unbiased view of what citizens know. We
find that they do not. Indeed, we find that unusual aspects of the context in which most survey interviews
are conducted inhibit effective performance on political knowledge questions.
Most political surveys are conducted by telephone. Some survey firms give respondents advance
warning (i.e., a letter in the mail) that they will be contacted. Many other firms give no such notice. Of
those respondents who receive advance notice, few – if any – are given detailed information about the
content of the survey. And few, if any, are told that the survey will include questions about factual aspects
of politics. Therefore, it is likely that political knowledge questions catch respondents by surprise.
Are such means a fair way to assess respondents’ political knowledge? Several factors work against
this conclusion. In many cases where citizens make important political decisions, such as whether to
turnout for a particular election or choose which candidate to vote for, circumstances are different. The
need to make such decisions does not appear “out of the blue.” Elections, particularly in countries where
election dates are fixed and known for years in advance. Even in parliamentary countries where
legislatures set election dates with less notice, citizens have time to prepare and some have an incentive to
research relevant issues. Citizens are much more likely to have advance warning that a decision is at hand
than respondents in political surveys.
Differences between common attributes of the survey context and attributes of the contexts in which
people make important political decisions matter because opportunity and motivation should affect
respondents’ abilities to answer political knowledge questions. Indeed, survey respondents may perform
poorly on political knowledge tests not because they are incapable of answering the questions, but
because they are caught unprepared and unmotivated to perform well. To see why, note that the survey
respondent’s situation is akin to you being asked how to get from your home to an unfamiliar restaurant.
If asked for an immediate response and given no prior warning that the question is coming, you may not
1
Convention