Correctional Service Canada
www.csc-scc.gc.ca
Common menu bar links
FORUM on Corrections Research
Public Opinion and Public Policy
|
"Surveys show that most Canadians believe" has become a universal phrase in contemporary social
analysis and public policy. Opinion surveys are big business and getting bigger. The latest survey
findings regularly make the national headlines and in significant ways shape the social policy agenda.
Pollsters and survey experts are becoming influential policy advisors. As the use of opinion surveys
becomes increasingly pervasive and as their influence grows, we need more than ever to understand their
limits and dangers. Some of the most important issues are not empirical, not questions of sampling or
accuracy but rather conceptual. That is, we must ask what such a technology can contribute to policy and
program development irrespective of technical improvements. Proponents often point out that public opinion surveys are an example of a uniquely democratic technology. If democracy is to serve "the people", then we must have some way to hear the "people's voice". Surveys, the argument goes, have allowed us to tap public needs and concerns. Without them, we hear only organized interest groups or depend on our best guess of the public mood. Critics, who have become more adamant over the past few years, argue that our opinion surveys give us at best a partial glimpse and at worst a distortion of the public view. According to this perspective, survey methodology is ill-equipped to get at people's complex, nuanced and shifting perceptions and opinions. Further, they argue, such surveys promote 'quick fix" policy solutions, "quick change politics". According to the critics, the proliferation of surveys has resulted in confusion between opinion and judgment informed by experience and research on what works. Some have gone so far as to suggest a moratorium on public opinion surveying and a return to more fundamental research. But these surveys are here to stay and, as practitioners become increasingly sophisticated in their use, they can become a valuable instrument for public policy - if we know their limits and their dangers. The Problem of Public Public opinion surveys are limited by the assumption that often underlies them, that there is a singular homogeneous public whose opinions can be recorded. This assumption influences the design of surveys and the analysis of survey data. The very notion of close-ended questions, where the responses are predetermined by the researchers, presumes that the researchers already know, more or less, the range of opinions they are tapping. The set questions and answers, regardless of how often they are pre-tested, constrain the range of responses possible. Surveys may, therefore, altogether miss those answers that do not fit the researchers' preconceived notions of what most people think. Imagine, for example, that you are asked to rank a list of issues in order of how much they concern you. The list might include issues such as the deficit, inflation, interest rates, unemployment, Canadian unity, violence, crime, aboriginal self-government, sexual equality, cultural sovereignty, and the environment. No doubt you could rank these issues. How different would your answer be if the question were open-ended, if there had been no list? Research shows that there can be significant differences.(1) For example, respondents may rank "crime" relatively high on a list but are far less likely to volunteer it when there is no list. Generally, it is more expensive to use open-ended questions. They are more difficult to code, quantify, compare and track over time, but they allow respondents to tell it their way. Closed questions inevitably constrain respondents and therefore limit the capacity of surveys to uncover the unexpected. Closed questions have a self-fulfilling effect. Some critics have gone further to suggest that the way these results are often presented has similar consequences on the opinions of Canadians. Imagine, for example, that you read that most Canadians are concerned about crime and corrections. Might you wonder whether you should rethink your own views? What do they know that you don't know? The bandwagon effect may operate not only in politics but also in public policy. Public opinion surveys, then, can become an instrument for shaping rather than measuring public opinion. Thus, we read newspaper articles about what "most Canadians" or the "average Canadian" think(s) about crime, corrections, conditional release, or what have you. We read that the average Canadian is concerned about crime and lacks confidence in our system of corrections and conditional release. Sometimes we read that the average Canadian wants a tougher system, or at least one that gives unambiguous emphasis to public protection. All of this is, of course, very important to know for policy and program development, but it represents only some subset of public opinion. Who is this average Canadian? The average Canadian is a myth. Real people have a particular gender, age, ethnicity, level of education; they also live and work in communities, which shape their views. They draw their views from their experiences and interactions, and as their experiences and interactions vary, so do their views. For policy purposes, it is generally less important to know what most people think than to know how particular groups of people with shared interests and experiences think. The "quick and dirty" survey cannot capture the views of the many publics that must be taken into account in policy development. While sophisticated sampling techniques may allow us to measure within a few percentage points Canadian opinions on any issue, by surveying fewer than 1,000, they may not give policy makers the information they need, particularly in a country as diverse as ours. Of course larger surveys may be able to take into account some of the key socio-demographic factors that differentiate publics -gender, age, urban/rural residence, education level, etc. - but even these analyses are limited. First, unless the surveys are very large and therefore expensive, they can only get at a select few of these factors. Selecting the key factors, then, depends on the ingenuity and insight of the researcher in defining the important groupings of opinion. Second, and more important, these techniques are rarely able to tackle the more subtle bases for opinion formation. If we are to understand public opinion, we must know something of how these opinions were formed. Were they a result of direct experience, opinion leaders, the media? In our field, for example, we will want to understand the views of those who have been victimized and those who have not, of those who have had contact with the criminal justice system and those who have not, of the informed and uninformed, etc. It is important for us to know, for example, whether concern about corrections is more common among the uninformed than among those who understand our correctional system. We would imagine a very different policy direction if we found that people became more satisfied with our programs as they learned more about them than if the reverse were the case or if we found that information made no difference at all. In response, survey researchers are developing more varied, and sometimes more expensive, techniques to allow targeting of particular groups, and to provide some understanding of the various publics - opinion groupings -that may form around any issue. Some may complement their survey findings with more subjective qualitative (open-ended) research, focus groups and the like.(2) In any case, the value of the survey data is not simply a function of the sophistication of the methodology; it is dependent on the skill, knowledge and ingenuity of the researchers and particularly their understanding of the "structure of the Canadian public". The Problem of Opinion The second problem resides in the very idea of "opinion". Let's assume we have asked a number of respondents to give us their opinion on whether corrections programs are effective. What have we learned if the answer is "no, corrections is not effective"? Some may be telling us that they are generally cynical or distrustful about government and government institutions. Their answer may reflect their attitudes toward government programs in general. Some may be saying little more than "I've never thought about this before, but now that you ask, I guess not." Such opinions are not likely strongly held or even very meaningful. Others may be saying that they don't believe anything can work to reform criminals. Their opinion, then, reflects their view of human nature. Yet others may be saying that their judgment, based on their knowledge, is that we're not as good as we should or could be. The possibilities are virtually limitless. Few surveys help us to know whether the so-called opinions we have tapped reflect enduring attitudes, firmly held beliefs, top-of-the-mind views, judgment based on experience and knowledge, or simply an answer created on the spot in order to fill out the questionnaire. For policy purposes, we must know whether,. for example, some measured public lack of confidence in criminal justice is episodic and transitory, perhaps the result of a particular incident or series of incidents, or enduring and reflective of deeply held concerns. We must know what underlies this apparent lack of confidence. Is it the perception that crime rates are growing, or that we are not doing enough, or that nothing works anyway? Is it the perception that traditional community values are crumbling and the sense of insecurity about what this means? Perhaps most important, we must know whether these views are based on an accurate understanding of crime and corrections or are based on myths and misperceptions. Pollsters increasingly recognize the limits of one-time snapshots of opinion and argue that only through tracking public opinion over time can we know whether we are getting at views that are enduring and meaningful. In addition, some survey researchers have tried to link people's opinions on specific issues to their general views on the state and future of the nation, the government and their own lives. For policy purposes, it is also crucial to link opinions to level of knowledge and understanding of issues. But the problems run deeper. You have probably all completed a questionnaire or responded to an interviewer on some issue or other. You were probably not always sure about your answers or your commitment to them. No doubt you sometimes treated the exercise lightly, sometimes seriously. Perhaps you were not always truthful. While researchers have tried to develop technical solutions to these kinds of problems, they can never fully address the difficulty in determining just what we are measuring when we measure opinion. Psychoanalysts, psychologists and sociologists have developed countless techniques to try to get at people's subjective worlds, their views about themselves and others and their concerns and preferences. Social scientists like to argue about which techniques are most suitable but they do tend to agree that no technique allows us to know with confidence people's inner experiences. Opinion surveys are most useful when they provide information on our various publics; are at least to some extent open-ended; track people's opinions over time; link people's opinions on specific issues to their general views of the world; and link people's opinions to their level of knowledge and understanding of the issues. Even when they are technically sound and methodologically sophisticated, the quality of survey data will always be limited by the researchers' skills and their understanding of the issues. We often have difficulty in articulating our own views and opinions. We know how difficult it is to penetrate the inner experience of others. The results of the best public opinion surveys can never be more than an approximation. Closing the gap between these approximations and "reality" generally requires more intensive, often multidisciplinary approaches. Researchers investigating public opinion on capital punishment know how important it can be to close the gap. In poll after poll, the majority of Canadians (between 70% and 80%) appear to support capital punishment in some case or other. When they are given specific case scenarios and asked if they would like to see capital punishment imposed in these cases, we find far less support. Many Canadians support capital punishment in the abstract but not when confronted with real people in real situations. Similarly, most Canadians support some version of community corrections - but not necessarily in their community. Nevertheless, opinion surveys can help to ensure that the policy process does not lose touch with the publics the policies aim to serve. They can be useful for getting some sense of public concerns and preferences. They can therefore help to shape public policy objectives and to track the effects of social action (or inaction). They may be less helpful in the really hard decisions. If, for example, public opinion challenges our policies or programs, should we change our thinking and develop new policy directions or should we inform and educate the public about what we do and what is possible? Opinion surveys help us to understand the environment in which policy development must occur - what is demanded or expected, what will be resisted or rejected. The surveys may help us to establish our policy and communications objectives and priorities but they are less useful for developing the specific policies and programs, the means, for achieving our objectives. Alex Himelfarb, formerly the Director General, Planning and Systems with the Solicitor General Secretariat, is currently Director General of Policy, Planning and Research with the National Parole Board of Canada. Following completion of his Ph.D. at the University of Toronto, Dr. Himelfarb spent 10 years as an academic with the University of New Brunswick and in various research posts in the United States. (1)Davidson, W.P., Leiserson, A. (1972). International Encyclopedia of Social Sciences, Crowell, Collier and MacMillan Inc. (2)Public Agenda Foundation (1987). Crime and Punishment: The Public's View, New York: Edna McConnell Clark Foundation. |
