The evaluation addressed three components of the community-based sex offender program (CSOP): administration; treatment content, delivery and outcome; and the therapists' service to parole. The first and third elements were included because treatment per se is only one part of sex offender programming in the area of corrections.

First, the CSOP is one of thousands of programs provided by the federal government and, as such, administration at every level is accountable for its rationale, administration, content and outcome.⁽²⁾ Thus, evaluation of administrative aspects of the program is crucial.

Second, the parolees are being treated for behaviour that is viewed very negatively, and a great burden is placed both on the therapist and on corrections to provide treatment that will reduce or prevent reoffending. Therefore, treatment content, delivery and outcome must be evaluated.

Third, treatment can be seen as a service to parole staff because the therapists must provide any information about the parolee's behaviour that could lead to reoffending.⁽³⁾ Parole staff are to use this information as an integral part of ongoing parolee supervision and monitoring. As well, therapists' immediate contracting authority is parole services - that is who they work for. Thus, this service to parole must be evaluated.

The evaluation methodology was designed to capture both qualitative and quantitative data on the three main components of the program. The evaluation data-collection strategy was very complicated because of the program's size and complexity and the necessity of an analysis of recidivism.

The evaluators elicited data from several perspectives on any given aspect of the program - an extended triangulation approach to data collection. In order to learn more about administrative processes, evaluators interviewed key senior administrators at the regional, district and local office levels about their management approach and, particularly, about the contracting and supervisory functions they performed for the CSOP. All contracts and related documents for each district's program were intensively reviewed. Also, interviews with the service-providing contractors, senior administrators and local parole officers included questions about their views on the administrative process.

To learn about program content, delivery and outcome, the evaluators first reviewed the few documents that discussed the history of the initiative or described local or district program goals and content. Interviews with senior administrators included questions about program content, delivery and outcome. The contractors themselves were interviewed by the forensic therapist about their background in the field, their treatment philosophy, treatment goals and techniques, their record-keeping and reporting activities and their assessment of their own efficacy. They were also asked for suggestions for improving the service.

The forensic therapist and another senior team member observed at least one group therapy session delivered by each therapist. (Since not all districts offered group therapy, this was not always an option.) At the end of these sessions, the contractor therapist left, and the evaluators conducted focus-group discussions with participants about the nature and usefulness of the individual and group therapy they received.

Recidivism rates are an important measurement of treatment success. The review of recidivism among CSOP and non-CSOP sex offenders was the single, most demanding aspect of the evaluation because no one data source at the national or regional level identifies sex offenders or combines information on offence, criminal history and treatment history. Given the differing information needs of each agency or department, the lack of such a single source is not surprising, even if it is frustrating for the evaluator.

The sources reviewed for information on recidivism were the Correctional Service of Canada's Offender Information System (OIS); Canadian Police Information Centre (CPIC) data; contractors' lists of those in treatment; parole files (archival and current); and an array of other lists used to cross-check whether a parolee was in fact a sex offender, was ever in treatment or was currently in treatment. These latter sources included parole office supervision lists, duty officer log books, police reporting lists, CSOP attendance lists and Correctional Service of Canada incarceration/supervision lists.

Various respondents were often surprised to learn that parole files did not contain the above information, and that other sources had to be tapped for information on recidivism. But to those experienced in conducting this type of research, it comes as no surprise. It was not even possible to identify every person who was in the CSOP over the years of an individual contractor's service, or over the life of a local or district program, because many of the contractors did not have an accurate list of those served. One or two were nearly complete, but some were woefully inadequate.

There were several cases where a person listed on a contractor's treatment list was not in police or Correctional Service of Canada records as an offender. There were instances, too, of a person on a treatment list being identified from Correctional Service of Canada records as incarcerated at the time of treatment. The evaluators took this to be a problem in record keeping rather than something more disturbing, but the dilemmas for data collection on recidivism and time-at-risk cannot be overemphasized.

Because of these identification and data-retrieval obstacles, fully one third of evaluation resources were allotted to the recidivism segment of the study.⁽⁴⁾ However, it was ultimately possible to build a verified data base of 449 CSOP participants and 181 non-CSOP individuals (a modest control group). This was the remainder of an initial count of 997 sex offenders either in the CSOP at any time in the life of the programs or convicted of a sex offence in the province since 1984. (At the time of the evaluation, 226 were still in prison and there was insufficient information on 141 individuals to allow inclusion in the study.)

To evaluate the contractors' services to parole, much of the data was drawn from in-depth interviews with the parole officers. Those with at least two sex offenders on their caseload were interviewed about how they believed the CSOP service helped prevent reoffending, and about the nature and usefulness of the therapist's service to parole. Treatment effectiveness was one element of that service. Another element was whether the therapist's written and verbal reporting to respective parole officers was useful to the officers in their monitoring and supervision of parolees. The evaluators explored this in some detail.

Senior administrators in each local and district office were asked similar questions about the service of therapists to parole. The parolees were also asked about the nature of communication between their therapists and their parole officers, and in what ways they were included in the interaction.

The numbers and categories of key respondents interviewed in person or in focus groups are as follows:

Regional level: 3
District/local office levels:

• Senior management, contracting officers: 11
• Contractors/subcontractors: 11
• Group therapy observations: 11
• Parole officers: 32
• Parolees (as an average of six per focus group): 66
• Other (police liaison officer): 1

The relationship of the evaluation questions, data-collection strategies and subsequent analysis are summarized in the table.

The full evaluation report dealt with a broad range of issues. Because of limited space, this article must limit its focus to the evaluation of treatment content and delivery. Administration and service to parole can be dealt with only very briefly here.

Though the evaluation report itself presented findings at the level of individual contractor performance, and of administrative and parole issues at a local and district level, this overview will report on the results at a broader district and regional level. Individual districts or contractors will not be identified here.

The presentation of findings on each evaluation component begins with an overview of the criteria against which each was compared. The evaluators were well aware of the difficulties of implementing a region-wide program for sex offender treatment. Thus, care was taken that the criteria reflect standards that were real, not ideal. The program was compared with what could realistically be accomplished in the present context, particularly with reference to the levels of treatment that could be expected for a community-based program for sex offenders on parole.

The evaluative criteria called for comparable program standards among all districts; adequate supervision and monitoring of all aspects of the program; special attention to comprehensive and open contracting processes that would define treatment content, delivery and standards of record keeping; and reporting by contractors and administrators.

No comparability of program standards or of contracts across the region was found. Neither was there continuous, thorough supervision of the CSOP services at the local office, district or regional levels.

At the regional and district levels, the contracting process did not meet the evaluative criteria - a condition which correlates highly, the evaluators believe, with the findings on problematic areas of program content and delivery to be discussed below. Contracts were generally not tendered. Though it is recognized that there may not be a pool of qualified or interested therapists to choose from in outlying districts, the weight of habit and custom appeared to figure very heavily in administrators' not making efforts to put contracts out for tender.

Contract content was very vague, and specificity diminished over time. It would be impossible to discern from the contracts across the region what treatment approach the contractor was expected (and, indeed, required) to take, how it might differ in group and individual therapy, whether and how therapists would assess participants, and what progress the parolee made in treatment. Record-keeping and reporting requirements were also extremely vague. This was true both for recording and reporting of treatment activities and for the contractors' own financial administration of their services.

Formal evaluation of contractor performance was not built into the contract, and it did not occur. Many front-line parole staff members were frustrated at the lack of a formal mechanism for providing feedback on the service and at the lack of consultation on the contracting process, from participation in defining service needs, through input on contractors' treatment approach, to the reporting activities of contractors.

The evaluators' recommendations for improvement of administration were numerous and detailed, emphasizing ways to use the entire contracting process to improve the quality of programming.

The evaluative criteria for this component addressed assessment timing and techniques, treatment planning, treatment content and modes of delivery, ground rules for therapists' relationships with parolees and with parole services, and standards for contractors.

It may be useful to describe these criteria in more detail, not only because it may provide a better understanding of the context of the evaluative assessments, but also because this evaluative model could be implemented elsewhere in similar circumstances. Furthermore, the fact that one of the districts does largely meet the above criteria speaks to the practicality of the model.

The evaluative criteria for treatment included these elements of the therapist's approach to service:

a) Treatment approaches should be primarily cognitive-behavioural but eclectic enough to incorporate additional approaches (e.g., educational, psychodynamic).

b) Treatment content should include, but not be restricted to:

• the identification of "deviant sexual drive" or "deviant sexual arousal patterns" that lead to offences;
• the identification of the "crime cycle" of the offence;
• the delineation of the "pathway to the offence";
• the provision of ways for the offender to stop or divert himself from the pathway to a reoffence and the teaching and practising of various coping techniques;
• the provision of relapse-prevention training;
• awareness and therapeutic response to disorders and lifestyle characteristics that may contribute to risk of reoffending.

The delivery was expected to be tough-minded and confrontational yet able to build trust on the part of the parolee.

The criteria further stated that:

c) There must be a strong focus on countering denial, rationalization, and minimization of responsibility for the offence.

• group therapy must be available wherever numbers warrant;
• individual therapy should be offered for all program participants if group therapy is not feasible and as an essential supplement to group therapy where this is offered;
• individual therapy is an ideal opportunity for the treatment of the deviant drive through using behavioural modification techniques (i.e., covert sensitization).

For the therapeutic relationship, the therapists would be expected to make clear to participants the nature and limits of confidentiality in this particular setting, and to provide parole staff promptly with information on treatment progress and on the parolee's risk of reoffending.

In terms of selection standards for therapists, only those with experience in providing treatment to sex offenders should be considered. They must also be able to administer the program efficiently and keep detailed treatment and financial records.

Based on these criteria for treatment, contractor eligibility and contractor service administration, it was clear that only one of the four districts ("District I," for convenience's sake) met the majority of the evaluation standards.⁽⁵⁾

It was a reassuring confirmation of the triangulation research methodology that the evaluators' perspectives tended strongly to reinforce each other. Furthermore, the recidivism results tended to confirm the interview and observational data.

In District I, it was clear that, in each of the groups observed, denial simply was not tolerated. The group began with members stating their name, sentence and offence. Therapists were offence-oriented in their approach, and every opportunity was taken to link discussion to understanding offending behaviour and to learning how to avoid it. Participants had homework to do and were expected to do it.

No district had a very elaborate assessment of parolees or extensive case planning, but the contractors in District I did prepare a pretreatment assessment and plan, and did tend to update it with parolees over time.

Even though this clearly was the most demanding of the district programs, parolees expressed respect for the therapists and generally seemed to feel that they were getting a square deal. They were fully informed of their rights, the plan for their treatment and the limitations of confidentiality.

Senior staff members had close, regular contact with the contractors and were quite well informed about the treatment delivered and the status of parolees in treatment. Senior staff expressed reasonable confidence in contractors, and front-line staff also tended to rate their performance positively.

The contractors were weak in their reporting. Some parole officers would have preferred more detailed reports, but the levels were nonetheless seen as adequate. The record keeping of District I contractors was the most complete and accurate overall.

This treatment approach and reporting activity may seem to be quite basic and even modest as a treatment program, but the difference between District I and the other three districts was striking. In no other group was there a treatment plan - for the group or individuals. Contractors were not tough-minded in their approach, and denial was extremely common. Discussions were not offence-oriented, and elements of the crime cycle were not identified or followed up. Behaviour in the groups was often disruptive, and considerable disrespect was expressed toward some therapists, even during the sessions.

The evaluators reviewed the records of participants to see whether any one district had offenders that could be considered to be more resistant to treatment, with possible indicators in such factors as type of crime or length of sentence. There were no significant indicators of this sort.

In the other three districts, a significant proportion of staff, especially the parole officers themselves, expressed little confidence in the service. Assessments of the treatment approach were generally neutral to negative. Parole officers were generally very dissatisfied with the level of service provided to them in their supervision and monitoring of parolees. Contractors' record keeping and reporting to parole staff were also generally very poor.

An aspect of the treatment programs that overlaps heavily with the administrative aspect is cost. Over the years, District I has had the most modest costs per client of each of the four districts. Furthermore, invoicing by District I contractors related directly to the slight increase or decrease in the number of offenders they served over a given contract year.

In contrast, the other three districts have had dramatic increases in invoicing by contractors though there has not been a concomitant increase in numbers of parolees in the service. There has been some increase in group and individual therapy sessions in some cases, but no appreciable increase in numbers of those served.

This leads full circle to the issue of the program, as a whole, needing clearer contracts, closer supervision by administration and more careful record keeping and reporting of all service activities by contractors and the Correctional Service of Canada administration.

Recommendations were that future programming be designed to meet these evaluative criteria, with appropriate modifications to meet local conditions. Monitoring of progress at every level of administration would be essential to long-range goal achievement.

It is not possible in this article to go into sufficient detail about the strengths and limitations of the recidivism methodology. Two factors give rise to a conservative approach to the findings: the number of participants in individual contractors' programs ranged from three to more than 200, and only three years of time-at-risk could be reviewed for both the CSOP and non-CSOP cases.

However, the finding that does help assess the effectiveness of the various districts' programs is that the recidivism rate in District I, calculated by combining the rates of the district contractors, was 4%. This is only about a quarter of the rates in each of the other districts.⁽⁶⁾ When the rates of contractors in each of the other districts are calculated and combined, we find average recidivism rates of 12%, 10% and 13% (figures rounded up). The highest recidivism rate of an individual contractor in the region was 14.06% (with more than 60 cases). In District I, though, two of the four contractors had no recidivists (with a combined total of 18 cases between these two).

Since there is no significant difference between programs in type of parolee served and little difference in the intensity of administrative awareness and control over the local programs, the evaluators have concluded that the appreciably lower recidivism rate in District I is most likely accounted for by treatment content and delivery.

It appears then that a rigorous, tough-minded approach has a measurable positive effect on recidivism rates. Furthermore, District I provides this type of service with the lowest cost per parolee and with any changes in cost linked directly to numbers served.

The key evaluative criteria for service to parole were twofold: comprehensive, up-to-date record keeping on all aspects of treatment and on all financial aspects of services delivered; and timely, sufficiently detailed oral and, especially, written reporting to parole officers on parolee status as it relates to risk of reoffending.

These areas of contractor performance tended to be sources of frustration in all districts, but it was much less intense in District I. In District I, the parole staff reported that they received written reports at expected intervals and that the information was adequate. They found the contractors to be generally accessible for informal consultation.

This was much less the case in the other districts where it was common for a parole officer not to have received written reports for many months or even years.

It was recommended that the contract be used to clarify the expectations for reporting and to encourage administration to insist that these standards be met.

The Pacific region's Community Sex Offender Program is a large one, of which much has been expected. The evaluators have concluded that most expectations have not been met.

At the same time, it is clear that it is indeed possible to provide an effective program, both modest in scope and demanding in approach. Furthermore, the program can be effective in terms of outcome, cost and level of service to parole.

The evaluation therefore does not suggest that the region, or the Correctional Service of Canada, take a whole new tack in community-based sex offender programming. Rather, it is a matter of affirming the approach which appears to be effective and working to extend this type of programming throughout the region.