FORUM on Corrections Research

Buyer beware: A consumer's guide to reading and understanding correctional research (Part III)

The "buyer beware" series of articles has attempted to make correctional research more understandable to the average reader (the typical research consumer) by looking at some important, but often neglected, questions that should be asked and answered when examining or using research results. Just like when you're buying a new car or stereo, a little background information can make the product much more understandable and useful to the consumer.

Part II of the series emphasized the theoretical importance (and reality) of questions in social science research. In this, the series' final article, we step back a little further and expore the problems faced by researchers and research consumers in asking and answering questions. Questions, questions, questions... Research consumers must understand that a good study often raises more questions than it answers. Otherwise, research results will often seem like one step forward that leaves us three steps back. The recognition that a complete answer may require several studies is crucial because an important factor in a study's ability to achieve concrete results is funding.

Research is usually funded by organizations (governments, corporations or both) that have an agenda. This agenda is often directed by a need to find once-and-for-all answers and this leads to time-limited, highly focused research. Therefore, if a study raises some important questions, finding answers to the new questions depends not only on the researcher's ability, but also on the funding agency's agenda.

Correctional research is extremely susceptible to this problem because of close links between correctional and government policy. A government's political concerns are often rapidly translated into research priorities and equally relevant research may, as a result, be shelved indefinitely, if not permanently. Recognizing agendas Often, very different political agendas underlie different streams of research. These agendas can be spotted in the questions asked by researchers. For example, Stephen Jay Gould's book, The Mismeasure of Man,⁽²⁾ describes how IQ testing was misused in the United States in the earlier part of this century to limit the immigration of certain ethnic groups.

Consider the following two questions:

• Is Race A less intelligent than Race B?
• Why does Race B perform differently than Race A on this test?

The first question assumes that intelligence is what intelligence tests measure. The second question, on the other hand, leaves the door open to examining whether differences in results arise from cultural biases built into the test. If one wishes to "prove" that Race A is more intelligent than Race B, the possibility of cultural bias is simply ignored. If, on the other hand, we only want to understand why the groups differ, then we must remain open to the possibility that the test is biased.

In correctional research, we might consider the difference between the following two questions:

• Why are aboriginal offenders so likely to reoffend?
• Why are aboriginal offenders so likely to be reconvicted?

The noticeable difference between these two questions cuts to the heart of this issue. Is the recidivism rate a result of the innate "criminality" of a particular group? Or, is it the result of law-enforcement and judicial practices in the district(s) where that group tends to be concentrated?

Both are fair questions, but the second question is more likely to be answered fairly. It leaves the door open to the possibility that there is something particular about the group in question, but does not close the door to the possibility that other factors may be at work.

The questions that researchers choose to ask and answer may, therefore, tell us a great deal about their view of the world - more, perhaps, than their results will tell us about how the world works. Models of the world We all operate from a set of assumptions. Our actions, therefore, make sense to others to the extent that those people share our assumptions. For example, if we as a group assume that a certain type of offender cannot be rehabilitated, nobody (apart from the offenders themselves) will question the denial of parole to such individuals.

The qualification "apart from the offenders themselves" was not just a humorous aside. It points out that the individuals denied parole will not understand why the decision was made.

If an offender feels that "fair" means receiving what everybody else gets, then a serial killer might feel that justice has not been served if a kidnapper serving an equal sentence gets parole after 12 years, and the serial killer does not. Although most people can see some justice in the imbalance, it simply may not be clear to the serial killer.

Why is there such a difference in perception? The differences arise from different ideas about justice, which arise, in turn, from conflicting "models of the world." Our hypothetical serial killer's model of the way things work is based on a simple interpretation of justice: "He got life, I got life. I should get out at the same time he does. It's only fair." A more complex model of justice (the one applied by most people) goes further. It weighs the severity of the crimes and the likelihood of a repeat episode.

Different models of the world can also divide researchers, as can be seen in the literature on the success of rehabilitation programs. Paul Gendreau and Robert Ross recently reviewed the literature,⁽³⁾ and it is clear that there are two basic streams of thought - rehabilitation either is, or is not, thought to be possible.

Given the evidence, it appears that the authors are justified in asserting that "it is downright ridiculous to say nothing works... The principles underlying effective rehabilitation generalize across far too many intervention strategies and offender samples to be dismissed as trivial."⁽⁴⁾

Nevertheless, others remain unconvinced. As Gendreau and Ross observe, "we are prone to becoming inextricably bound up in ideologies. All too often, in the face of all contrary empirical evidence, we adhere to theories for political or ideological reasons ... or cavalierly switch ideologies depending upon transient political developments..."⁽⁵⁾

In short, being aware of what is going on behind the scenes (as a research consumer) is the best way to avoid being led to the same conclusions as those dictated by a particular researcher's model of the world. What to look for in a model Social-scientific models have important features that are not always stated clearly in research reports. Three of the most important are: assumptions, internal consistency and implications.

Obviously, if an assumption is wrong, or there is some logical inconsistency, the model itself is wrong. For example, a sex offender treatment program designed on the assumption that this group of offenders cannot be rehabilitated would be both unworkable and absurd (not to mention inconsistent with itself).

Similarly, research results may not be consistent with the researcher's conclusions. For instance, if we know that childhood sexual abuse often leads to depression and sexual dysfunction, that is one thing. However, we cannot infer automatically that a person complaining of depression and sexual dysfunction was abused as a child. There are many other causes of these symptoms.

Consider an extreme example. Ingesting cyanide invariably causes death. But if somebody dies, we cannot infer automatically that he or she was killed by cyanide. A model relying on this type of illogic would obviously be quite flawed.

Finally, predictions are also vital. If a model says something should happen and it doesn't (or vice versa), the implication is that the model is either wrong or incomplete. For example, in the Gendreau and Ross article, the "rehabilitation doesn't work" model is contradicted by many successful rehabilitation programs. Question the answers to questions Questions and answers are the essence of research. When we read research reports as research consumers, we must ask our own questions and, perhaps most important, we must evaluate the answers for ourselves. Sometimes authors will leave something out (inadvertently or otherwise) - it is up to the consumer to be aware that something is missing and find it.



⁽¹⁾Department of Psychology, Carleton University, Ottawa,
⁽²⁾S.J. Gould, The Mismeasure of Man (New York: Norton, 1981).
⁽³⁾P. Gendreau and R. Ross, "Revivification of Rehabilitation: Evidence from the 1980's," Justice Quarterly, 4, 3 (September, 1987).
⁽⁴⁾Gendreau and Ross, Revivification of Rehabilitation: Evidence from the 1980's.
⁽⁵⁾Gendreau and Ross, Revivification of Rehabilitation: Evidence from the 1980's.