FORUM on Corrections Research

Improving program performance among offenders

Ralph Serin¹
Research Branch, Correctional Service of Canada

At the extreme negative end of the program performancespectrum is offenders who fail to complete a prescribed correctional program. Relative to program completers, these dropouts have markedly higher post-release failure rates.² The actual differences vary according to the type of outcome (general, sexual or violent recidivism) and type of offender (sex offender versus non-sex offender), because of different base rates. Furthermore, rates of attrition (typically calculated as the proportion of offenders who did not complete a program)³ vary according to setting and type of program. For the Correctional Service of Canada, across all core programs, the dropout rate has been estimated at 18%, although there are some program differences.⁴ Further, programs for highly resistant offenders report attrition rates of approximately 20%.⁵ Finally, program attrition rates in the community appear to be significantly higher (by a factor of 3.5).⁶ Intuitively, then, program performance seems important in considering the broader issue of program effectiveness. Accordingly, program performance is critical to effective correctional practice, including safe and timely release.

With respect to program performance, the question is whether offenders who do better in a program, have relatively better outcomes. Several issues have impeded our ability to answer this fundamental and apparently straightforward question regarding program performance. A discussion of these issues forms the basis of this article.

Referral issues

One measurement issue that might improve our understanding of program performance is that of program referral criteria. Having an offender in the wrong program, or one that is not required, will clearly interfere with program performance and demonstration of a treatment effect. Determining how well an offender has done in a program will be impeded if the offender is poor in motivation⁷ or if the program is a poor match, either in terms of program content or intensity.⁸ Although increased motivation may be a legitimate treatment target, alone it may be insufficient to yield improved outcome. For these reasons, explicit program referral guidelines are required for accredited programs.⁹ Nonetheless, strategies for enhancing treatment response such as motivational interviewing,¹⁰ the use of a decisional balance,¹¹ and the use of cost benefit analyses¹² can lead to improved program performance.¹³

Classification/typology

One approach to gaining a better understanding regarding treatment response is the use of typological or classification schemes.14 In this manner, it is possible to generate hypotheses regarding the likely response by particular offenders who share certain traits, characteristics and needs. Moreover, it may help account for apparently discrepant findings.

Assessment

Increasingly, program staff recognize the limitations of over-reliance on offender self-reports regarding the measurement of program participation and performance. Whether due to self-deception or exaggerated insight, the situational demands on offenders to present favourably are substantial. This is particularly noteworthy, as correctional programming has gained increased prominence in correctional planning and discretionary release decision-making. Offenders likely believe that while a positive post-treatment report does not guarantee parole release, a negative report is a significant obstacle to overcome. This is not to say that consumer satisfaction reports of a program cannot be informative, but offender self-reports are typically highly correlated with measures of social desirability.¹⁵ Further, in some cases, offender self-reports and clinician independent ratings of performance are discrepant. This finding implies that discretion is required when interpreting post-treatment comments by offenders. This state of affairs does not fare well within a decision model already suspect about clinical ratings.¹⁶ For the past several years the Research Branch has been investigating the utility of an assessment protocol related to program performance. This is a structured rating scale with behavioural anchors regarding aspects of program performance or participation.¹⁷ As well, other structured ratings of program performance are emerging that warrant investigation.¹⁸

In a related vein, quality assurance evaluations across programs provide information about program drift. That is, poor adherence to standards erodes the quality of the service, which in turn results in poor program performance, which negatively impacts on program effectiveness.¹⁹

Threshold versus change scores

In addition to overall or global ratings regarding an offender’s participation and gain, another index of performance relates to change scores on the pre and post-treatment test battery. The test battery should reflect those needs addressed by the program, such that change scores should represent gain.²⁰ In principle, then, those offenders who make the greatest change might be considered to have the best performance. Alternatively, the post-treatment score could be considered a threshold score. Thus, of interest is not how far an offender has travelled, i.e., degree of change, but whether she/he reached the final destination, i.e., mastery or competence or a skill. One possible caveat may be in cases where the offender begins the program with very poor skills and despite substantial effort and motivation, achieves relatively modest post-treatment skill levels. These case-specific (within subject) analyses may be important to avoid diluting an overall treatment effect (some offenders do well, others do not, and the result may be a null effect).

Intermediate measures

In addition to considering global ratings of performance and pre-post-treatment test results, another index of performance could be intermediate measures.²⁰ Again, these could be offence-specific and directly related to program content.²² For instance, in the case of violent offenders these could include reduced incidence or severity of institutional misconducts or fewer verbal confrontations (assuming there is a baseline and reliable monitoring strategy). Although such research is preliminary compared to using recidivism, as the penultimate index of program performance, the expectation is that there should be a modest relation between intermediate measures and outcome.

Role of staff

Increasingly, there is evidence regarding the impact of staff on program performance and effectiveness.²³ As the field moves away from strict reliance on offender self-reports of program performance, staff issues become increasingly important. Again, program accreditation guidelines provide structure regarding the requisite staff qualities, characteristics, and skills. These guidelines also describe initial training and ongoing professional development requirements. These should greatly facilitate the reliable use of clinical, behavioural rating scales of program performance, which in turn should yield improved predictive validity.

Differentiated outcome

Presently, there is evidence that certain factors, such as risk, influence differential outcome. Specifically, greater effects are noted in higher risk offenders.²⁴ An important task, then, is to conduct systematic, prospective research utilizing multi-method assessments to investigate the influence of program performance on outcome. Further, this research should also consider the contribution of other independent variables such as type and intensity of program, type of offender, and programming sequence.

Model for incorporating performance into risk management

The final impediment to measures of program performance contributing to reintegration potential and post-release adjustment, relates to the absence of a systematic model to incorporate such information into post-treatment risk appraisals.²⁵ For instance, in the case of high-risk offenders, how much gain is sufficient to warrant release? Alternatively, in the case of low-risk offenders, how marginal a performance is sufficient to withhold discretionary release? Presently the correctional strategy provides for an assessment of an offender’s needs at intake and the development of a correctional treatment plan that prescribes programs. The assumption is that these programs will address criminogenic needs (dynamic factors) and provide for improved risk management.²⁶ This is different than stating that effective correctional programs reduce risk. In those circumstances where risk is based on static factors, programming cannot alter the estimate. Positive program performance can, however, provide for a case-differentiated risk management strategy. This strategy considers other factors such as risk level, release plan, community support, and programming aftercare to ensure that a particular offender’s release does not constitute an undue risk to public safety.

---

1. 340 Laurier Avenue West, Ottawa, Ontario K1A 0P9.

2. Marques, J. K., Day, D. M., Nelson, C., & West, M. A. (1994). Effects of cognitive-behavioral treatment on sex offender recidivism: Preliminary results of a longitudinal study. Criminal Justice and Behavior, 21, 28-54. See also Marques, J. K., & Day, D. M. (1998). Sex offender treatment and evaluation project: Progress report. Department of Mental Health, CA. And see Dowden, C., & Serin, R. C. (2000). A Follow-up Evaluation ofthe CSC Anger Management Program for Federal Male Inmate Dropouts. Research Report (in preparation). Ottawa, ON: Correctional Service of Canada.

3. Program refusers likely represent a group presumably distinct from drop outs, but the reasons for program attrition are not reliably available in many evaluations. Marques et al. (1994) is a notable exemption.

4. Brideau, M. (2000). Community programming. Presentation at the International Community Corrections Association conference. September, Ottawa: ON.

5. Marques, & Day, (1998). See also Preston, D.L., & Serin R. C. (1999).
Case file: Persistently violent (non-sexual) offender treatment program. Ottawa, ON: Correctional Service of Canada.

6. Op. Cit., Brideau (2000).

7. Murphy, C. M., & Baxter, V. A. (1997). Motivating batterers to change in the treatment context. Journal of Interpersonal Violence, 12, 607-619.

8. Kennedy, S. M., & Serin, R. C. (1999). Examining offender readiness to change and the impact on treatment outcome. In P. M. Harris (Ed.).
Research to results: Effective community corrections. Lanham, MD: ACA.

9. Thurber, A. (2000). Accreditation of correctional programs: Recognizing and maintaining effective program interventions for offenders. Monograph. International Community Corrections Association, WI.

10. Miller, W. R., & Rollnick, S. (1991). Motivational interviewing: Preparing people to change addictive behavior. New York, NY: Guilford Press.

11.Prochaska, J. O., Velicer, W. F., Rossi, J. S., Goldstein, M. G., Marcus, B. H., Rakowski, W., Fiori, C., Harlow, L. L., Redding, C. A., Rosenbloom, D., & Rossi, S. R. (1994). Stages of change and decisional balance for twelve problem behaviors. Health Psychology, 13, 39-46.

12. Preston, D. L., & Murphy, S. (1997). Motivating treatment-resistant clients in therapy. Forum on Corrections Research, 9(2), 39-43.

13. Preston, D. L. Addressing treatment resistance in corrections, Compendium 2000 on Effective Correctional Programming. Ottawa, ON: Correctional Service of Canada (in progress).

14. Serin, R. C., & Preston, D. L. (2000). Managing and treating violent offenders. In J.B. Ashford, B. D. Sales, & W. Reid (Eds.). Treating adult and juvenile offenders with special needs (pp. 249-272). Washington, DC: American Psychological Association.

15. Dowden, C., Blanchette, K., & Serin, R. C. (1999). Anger management programming for federal male inmates: An effective intervention. Research Report R-82. Ottawa, ON: Correctional Service of Canada.

16. Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychology, Public Policy, and Law, 2, 293-323.

17. Serin, R. C., & Kennedy, S. (1998). Treatment readiness, responsivity and gain: Assessment protocol. Unpublished paper. Ottawa, ON: Correctional Service of Canada.

18. Stewart, L. (2000). National Family Violence Standards. Reintegration Programs. Ottawa, ON: Correctional Service of Canada. See also Wong, S., & Gordon, A. (1999). The Violence Risk Scale, Version 2. Ottawa, ON: Correctional Service of Canada.

19. Agee, V. (2000). CCS Mastery Achievement Process Sample Standards and Checklists, personal communication.

20. In some cases, negative scores may reflect gain due to diminished need. In other cases, post-treatment scores reflect increased needs due to increased disclosure. See Serin, R. C., & Kuriychuk, M. (1994). Social and cognitive processing deficits in violent offenders: Implications for treatment.International Journal of Law and Psychiatry, 17, 431-441.

21. VanVoorhis, P., Cullen, F. T., & Applegate, B. (1995). Evaluating interventions with violent offenders: A guide for practitioners and policymakers.Federal Probation, 59, 17-28.

22. McDougall, C., Clark, D., & Fisher, M. (1994). The assessment of violent behaviour. In M. McMurran & J. Hodge (Eds), The assessment of criminal behaviour of clients in secure settings. (pp. 68-93). Jessica Kingsley Publishers: London.

23. Fernandez, Y. M., Serran, G., & Marshall, W. L. (1999). The Reliable Identification of Therapist Features in the Treatment of Sexual Offenders.
Paper presented at the Association for Treatment of Sexual Abusers annual conference. October, Orlando, FL.

24. Dowden, Blanchette & Serin (1999).

25. Serin, R. C. (1998). Treatment responsivity, intervention, and reintegration: A conceptual model. Forum on Corrections Research,
10 (1), 29-32.

26. Motiuk, L. L., & Serin, R. C. (1998). Situating risk assessment into a reintegration framework. Forum on Corrections Research, 10(1), 19-22.