FORUM on Corrections Research

Reporting on results: The Correctional Service of Canada experience

Gerry Hooper¹
Performance Assurance, Correctional Service of Canada

Canadians expect it, the Government of Canada has promised it, legislators need it and taxpayers are entitled to know what they are getting for the money that is spent on their behalf. This is the basic message behind the question “why report on results?” However, it is only the beginning of the answer. Reporting on results serves the needs of a variety of people. Sometimes to help them do their jobs, sometimes to inform them, to provide direction for further research, to show new opportunities, to explode myths or to give the satisfaction of knowing what came about as a result of hard-work. In short, results information helps shape intelligent debate and focussed action...at least for those interested in knowing the truth, or willing to be confused by the facts.

This means that citizens, managers, legislators, critics, members of the media, editorialists ... all of them can use information to do their jobs better and with more integrity.

At the Correctional Service of Canada, we are rather proud of the fact that we began reporting and monitoring some years ago, in some ways long before the current emphasis on the subject. Closer inspection suggests that these earlier efforts were not as focussed on results as they should have been. They were more designed to report on activities, incidents and processes, than on outputs and outcomes. This is not an insignificant distinction; there are at least two cautionary notes.

First, there is an off-quoted cliché “what gets measured gets managed”... if for “measured” you substitute “counted” then you risk treading on dangerous ground. The risk is obvious: effort is focussed on improving the number, on climbing or falling on some scale, not on contributing to a quality output or outcome. We have experienced this; it is often easier to count than to measure. We have derived false comfort from numbers that said compliance to time standards was increasing only to learn, through more in-depth review, that quality, the real contribution to achieving our Mission and legislative obligations, was being sacrificed.

Second, unless you clearly understand how events and activities are linked to outcomes, then reporting on them really tells you nothing about results; it merely tells you “more” vs. “less.” In the business of corrections, we feel this acutely. “More offenders in the community” can be used to inflame strong feelings both positive and negative. But it may be the result of nothing more than a coincidence of release dates, a change in sentence lengths or better compliance with the law of the land. Similarly, we can derive false comfort from increases in offender programming. More offenders in more programs sound encouraging. What we need to know is who successfully completes programs, what skills or insights that program(s) gave them, which in turn, is likely to lead to safer reintegration into society. Even then we must be careful. If an offender does or does not return to criminal behaviour we should never assume that the credit, or the blame, are entirely ours. There are too many variables at play.

If we fail to heed these apparently obvious cautions, we do indeed run serious risks, whether you are a practitioner, a manager, an advocate or a critic. As an old mentor used to caution “be sure you know what you really know.”

From some ambitious early beginnings, we have tried to learn and refine our reporting on results. We have moved our focus from the “obvious”, the “common sense” and the “intuitive” and are trying to work backwards, in our reporting strategy, from our Mission and Legislative mandate. We have a long way to go. We ask ourselves, in short, “if this is what we are supposed to do, what will we report on that will tell us, directly or by proxy what we are achieving or what contribution are we making to those eventual achievements.”

In this we are lucky, we think, in having very clear legislative requirements and very emphatic statements in our Mission, core values and strategic objectives. The Corrections and Conditional Release Act provides guiding principles that give clear areas against which to report, for example:

• Protection of society be the paramount consideration
  
• All relevant information be used in the corrections process
  
• Least restrictive measures consistent with safety be used

The Mission gives similar fertile ground for monitoring and reporting, if we wish to manage with integrity.

“... respecting the rule of law, contributes to the protection of society by actively encouraging and assisting offenders to become law abiding citizens, while exercising safe, secure and humane control.”

Both of these documents provide the basis for our reporting efforts, whether we report on compliance (rule of law), actions by offenders while incarcerated or on conditional release (protection of society, safe, secure humane control) or program interventions (actively encourage and assist).

Indeed we do use these principles as the basis for our reporting, whether through audits, investigations, evaluations, or more particularly for this discussion, our regular presentations of data to our managers.

Building around the Act and our Mission we have created hundreds of reports for the management cadre of the Service ... even the Auditor General refers to it as a “vast array” of information. This material, the “Corporate Results” is presented at every meeting of the Service’s Executive Committee and is extensively discussed. It is not an easy session as reasons are offered, excuses dismissed, corrective actions proposed and objectives for improvement are set. But that is “what it is all about”. In order to manage in an informed focussed way, to manage with integrity, we accept that you have to have the information, assess the results and confront what they tell you.

It also helps, to be fair, to deal with perceptions and myths that abound in our profession. To have information, to have data, to have results helps respond to those who through lack of knowledge, fear or who knows what other motives, form inaccurate impressions. It helps both us and Canadians move closer together on what needs to be done in criminal justice. We know, and we can show, that people genuinely interested in knowing more about our business learn, change attitudes and contribute insights when the data and the results are known and shared.

Today we enjoy the secondary benefits of a significant investment in technology. Over the last decade an “electronic file” has been created on each offender, a file that is the official offender file. This system, with its electronic files and structured data fields, gave us an unrivalled opportunity to access vast amounts of information, in millions of combinations. A real bonanza, to be sure, but one that brought yet another set of problems.

Our early efforts at reporting set off a series of seemingly endless, often acrimonious debates that, in crude summary said “your numbers are different from ours ... ours are right, yours are wrong.” Frustration reigned.

In many ways this was inevitable; the database is huge, with the same or similar information recorded in many different fields. The chances of going to different fields for similar or similar sounding reports were, and are, great. There were data quality problems too, particularly as old file data were converted en-masse to a new medium. Early errors were rampant, however, we worked through it.

In an ideal world, nothing would have been produced until we had consensus on sources, definitions and “interpretation rules” for data and reports, but that is not reality. Organizations constantly need information on how they are doing and they need it in increasing detail and complexity. Finally we caught up, though this particular race never ends, and have built consensus around existing reports and processes to decide on the need for and the content of new ones. We invite anyone contemplating moving into this field to avoid our mistakes! And as anyone can see who reads newspapers, the debate over definition and interpretation goes on, and on.

Finally, we had to mature and to discipline ourselves. It is too easy to use results to decide who is right and who is wrong, who won and who lost. This may be a predictable human reaction, but it is not the reaction of mature managers in what aspires to be a learning organization. Nothing stifles creativity, contribution and progress like the fear of reports that will be used to inflict pain and punishment rather than to make progress. This is a constant challenge. As in all things, there is another side to reporting on results. We have talked a little about what we have done and what we do. As noted earlier, much remains to be incorporated before results can be reported in a truly broad and balanced fashion.

We need to access the outcome data for program interventions and participation and link it to correctional decisions and outcomes in a routine way. We need to have virtually seamless links to financial data, so we can add the critical managerial insight into cost per “unit” and cost per outcome. We need to add information about our staff and their utilization to help with costing, training and recruitment strategies. Perhaps the biggest lesson of reporting on results is that the biggest product is more questions ... and more reports.

This does not mean we have misg ivings about the effort, even when our reports generate still more work or are used to criticise us. That is part of being open and accountable. Reporting openly and often has let us understand what we need to do. It has let us focus on things that will improve outcomes, and shift resources. Reporting, as said at the outset, leads to more structured and informed decision making. That makes it worthwhile.

---

1. 340 Laurier Avenue West, Ottawa, Ontario K1A 0P9.