Research Brief

Effectiveness of the Cognitive Skills Training Program: From Pilot to National Implementation

No. B-07

Prepared By:
The Research and Statistics Branch
Correctional Service of Canada

May 1991

The points of view expressed in this research report are those of the authors and do not necessarily reflect the views of the Correctional Service of Canada.

Acknowledgements

Correctional Service of Canada Research Briefs are prepared by the staff of the Research and Statistics Branch. The points of view expressed are those of the authors and do not necessarily reflect the views or policies of the Correctional Service of Canada. This brief was prepared by David Robinson, Marcy Grossman, and Frank Porporino.

Acknowledgements in research reports are usually perfunctory. In this case we want to very sincerely thank Liz Fabiano for her unfaltering support for this research project. As well, all of the Cognitive Skills Training coaches played an invaluable role in securing the data. We also thank the regional research assistants for completing the file reviews, and all institutional and community staff who assisted with the location of files.

This research brief is also available in French. Ce sommaire de recherche est également disponible en français. It is available from the Communications and Corporate Development Sector, Correctional Service of Canada, 340 Laurier Avenue West, Ottawa, Ontario K1A 0P9.

Summary

Investigations of two samples of offenders who were participants in the Cognitive Skills Training Program revealed that, in comparison to non-participants, participants were less likely to recidivate following release. The evidence suggests that program completers have lower rates of readmissions for new convictions and that the program effects on recidivism appear to be most beneficial for high risk offenders. Additional data indicate that referral and screening procedures operating for Cognitive Skills Training favour the selection of offenders who are at high risk of recidivism. Psychometric data also indicate that Cognitive Skills participants improve on a number of attitudinal and cognitive skill dimensions during their exposure to the program. The latter finding suggests that the program is effective in producing positive changes on the intermediate targets which are associated with recidivism.

Introduction

In this brief report we examine the effectiveness of the Cognitive Skills Training Program in relation to:

• post-release outcomes of offenders - reduction of recidivism
• selection of appropriate offenders for participation in intensive rehabilitative programming - identification of high risk offenders
• pre/post test changes made by participants on a variety of attitudinal and cognitive skills factors - positive effects on targeted dimensions

The effectiveness of the Cognitive Skills Training Program is assessed using a sample of participants from the 1988-1989 pilot project, and a larger sample of offenders who participated in the program since the national implementation began in 1990.

The issue of the program's effects on recidivism is examined using post-release follow-up data for offenders who participated in the initial Cognitive Skills Training Pilot Project. The readmission rate for participants is compared with the rate for a group of offenders who were selected for participation but did not receive programming.

The selection of high risk offenders is addressed by examining the characteristics of offenders who were selected following the national implementation of the Cognitive Skills Training Program. Finally, the program`s effectiveness in producing intended changes on the attitudinal and cognitive dimensions which the program addresses is investigated using pre/post test psychometric data which has been gathered for the national implementation sample.

A description of the Cognitive Skills Training Program is available in the initial report on the pilot implementation of the program in the Correctional Service of Canada (Fabiano, Robinson & Porporino, 1990). Also, a lengthy discussion on the theoretical basis for the program appears in Ross and Fabiano (1985).

Evidence for Effects on Recidivism

Post-release outcome data is now available for 63 of the 73 (86.3%) offenders who were included in the original Cognitive Skills pilot sample. The program was piloted in two sites in the Atlantic Region and two sites in the Pacific Region in late 1988 and early 1989. The pilot sample included 47 program participants (treatment group), and 26 non-participants (comparison group) who volunteered and met the criteria for the program but for a variety of reasons did not participate (e.g., transfers to other institutions, conditional release). The treatment and comparison groups did not differ significantly on a variety of characteristics including criminal history, risk (as measured by the Statistical Information on Recidivism (SIR) Scale), and a set of relevant demographic dimensions and cognitive measures that were examined. The pilot sample is described fully in an earlier report on the pilot implementation of the Cognitive Skills Training Program (Fabiano, Robinson & Porporino, 1990).

Preliminary results of the pilot study indicated that offenders in the treatment group made positive gains on various attitudinal and cognitive indices. In addition, after an average of six months post-release follow-up, the rate of return to federal institutions for the offenders from the treatment group was lower than the return rate of the comparison group. Program participants also returned at a lower rate than would have been expected on the basis of their pre-program risk levels calculated with the SIR Scale.

For the present report, we re-examined rates of recidivism after an additional one year follow-up period on a larger number of offenders who were released. Only 10 of the original 73 offenders in the pilot sample had not been granted released by March 31, 1991. The current sample includes 40 offenders from the treatment group and 23 offenders from the comparison group who were followed-up for a mean of 19.7 months after community release (S.D. = 6.3 months). Although the follow-up times differ across offenders in the sample, two-thirds of all released offenders were followed-up for a minimum of 18 months, and none were monitored for less than five months following release. It should also be noted that there was no significant difference in the average follow-up time for the treatment group (19.0 months) and the comparison group (20.8 months).

Forty-eight percent of the sample had been initially released on Day Parole, 11% on Full Parole, and 41% on Mandatory Supervision. The distribution of release types did not differ across the treatment and comparison groups.

Post-Release Outcomes

The post-release outcomes of the treatment and comparison group are displayed in Table 1. The figures show that offenders in the treatment group were readmitted for new convictions at a lower rate than the comparison group during the follow-up period. Specifically, only 20% of the treatment group were readmitted for new convictions compared to 30% of the offenders in the comparison group. Although our current sample size was too small to test appropriately for statistical significance, the trend in the data suggests clearly that the Cognitive Skills Training Program is effective in reducing the number of offenders who are returned with convictions for new offenses.

Table 1
Cognitive Skills Training Program:
Post Release Outcome Status For Treatment And Comparison Groups

	Treatment	Comparison
Readmissions with new convictions	20.0% (8/40)	30.4% (7/23)
Readmissions without new convictions	25.0% (10/40)	21.7% (5/23)
No readmissions	55.0% (22/40)	47.9% (11/23)

It is interesting to note that the number of offenders who were returned to prison without new convictions (e.g., technical violations, day-parole terminations) is similar yet marginally larger in the treatment group. It is possible that the Cognitive Skills Training participants may be subjected to closer monitoring because of expectations regarding the program.

The results of the pilot follow-up can also be compared to expected outcomes that can be predicted from knowledge of the SIR scores of the released offenders (see Figure 1). The average probability of reconviction for both treatment and comparison groups is 52% based on available SIR score data (n=46). This base rate would be expected after a post-release follow-up period of 2.5 years on either full-parole or mandatory supervision. Although the follow-up period for our sample is shorter, it is known that most recidivists are readmitted within the first year of release (for example, see Hann & Harman, 1988). It should also be noted that our sample includes offenders who were released on Day-Parole. Since day-parolees are not included in SIR recidivism predictions, we can assume that the expected base rate would be even higher for our sample.

In actuality, the estimated base rate is considerably higher than the reconviction rate we have observed for the treatment group (20%). This again suggests, quite convincingly that the program is effective in reducing recidivism. The actual reconviction rate for the comparison group is also lower (30%) than the expected base rate, suggesting that volunteering for programming may have beneficial effects in itself or that motivation for treatment may be influential in post-release success.

Figure 1
Comparison Of Readmission Rates for all Release types

(New convictions only)

A second sample of 64 offenders from the initial pilot sites also received exposure to Cognitive Skills Training during 1989, prior to the implementation of the program on a national basis. We were also able to examine the post-release outcomes of this group, although no comparison group had been established for the purpose of comparing the recidivism rates of participants and non-participants. The number of offenders from this "extended pilot sample" who had been released (Day Parole, Full Parole or Mandatory Supervision) by March 31, 1991 included 42 offenders who had completed the program and 12 who had initially participated but did not complete the program.

After an average follow-up period of 12 months, only 2 of the 42 program completers (4.7%) had been readmitted for a new conviction. Assuming that this group of offenders was similar to the original pilot sample with respect to risk of recidivism, the reconviction rate of 4.7% after one year is remarkably lower than would be expected from base rates. As noted above, the expected base rate after a 2.5 year period would be approximately 50%.

It is also interesting to note that the Cognitive Skills completers had superior post-release outcomes than the 12 released offenders who did not complete the program. Among these offenders, 33.3% had been returned with new convictions during the one year follow-up period. Although this group may have possessed different characteristics from the treatment group, completion of the program does appear to be associated with better post-release outcomes

Treatment Effectiveness and Risk Level

An important question concerns the extent to which Cognitive Skills Training is more effective among different groups of offenders. For example, research evidence regarding the risk principle of case classification (Andrews, Bonta & Hoge, 1990) suggests that intensive programming is most effective among offenders who are at higher risk of recidivism. Hence, we hypothesized that Cognitive Skills Training would be most beneficial among offenders who possessed higher SIR Scale scores. We were able to test this hypothesis using outcome data from the original pilot sample. Although our sample size is small due to missing values for the SIR scale, the trends that were observed support the risk principle.

We classified offenders who scored in the "Fair to Poor" to "Poor" risk range on the SIR score as "High Risk" offenders (n=29) and offenders who scored in the "Very Good" to "Fair" range as "Low Risk" offenders (n=17). We then compared the recidivism outcomes of offenders who fell in the two risk levels in both the treatment and comparison group. The outcomes for the various groups are presented in Figure 2. This figure also shows the effect of Cognitive Skills Training for the total sample. The difference between the outcomes of the treatment and comparison group is most dramatic among the High Risk offenders. While only 18% of the high risk treated group were readmitted for new convictions, 42% of the comparison group were readmitted for new convictions.

Figure 2 also shows that the outcomes for the Low Risk offenders in this sample is opposite to the effect reported for the High Risk offenders. While the reconvictions for Low Risk treated offenders remained low, the rate of reconviction for the untreated comparison group was even lower. Although this type of "deterioration" effect has been associated with the use of treatment for low risk offenders in other studies (Andrews, Bonta & Hoge, 1990), the number of offenders in the low risk group in our sample is too small to provide good evidence of this type of effect. However, the number of high risk offenders in our sample is large enough to provide encouraging evidence that the program is more beneficial with high risk offenders.

Figure 2
Pilot Sample Readmission Rates for All Release Types

 

Research Component of the National Implementation of the Cognitive Skills Training Program

The National implementation of the Cognitive Skills Training Program was initiated in January 1990. Since the adoption of this program as a primary initiative in personal development programming within the Correctional Service of Canada, it has been offered in a total of 17 sites across four of the five national regions. Data collection for this phase of implementation included an extensive assessment battery for measuring pre/post test status on a variety of attitudinal and cognitive skills scales (described below) and case file reviews to obtain data on numerous demographic, criminal history, risk and need for programming indices.

The current report highlights the assessment and file review data collected for the national phase of implementation up until the end of March 1991. The sample consisted of 208 offenders, including treatment participants and a non-participant control group. The treatment and control groups were established through random assignment procedures. Case management referrals to the program, which were judged to be appropriate by program coaches, were randomly drawn for treatment participation. The number of cases selected for treatment corresponded to the number of spaces available in the program (usually 8 offenders), leaving the remaining referrals for the control group condition. Control group members were placed on a "waiting list" and were permitted to enter the next available treatment group if they remained interested in the program.

Among the sample of cases available for analysis up to the end of March 1991, there were 146 program participants and 62 offenders in the control group. Eight of the 62 offenders on the waiting-list subsequently completed the program, reducing the control group size to 54.

Targeting Higher Risk Offenders for Participation

The appropriate selection of participants for Cognitive Skills Programming was regarded as an important objective in the national implementation of the Cognitive Skills Training Program. Consistent with the risk principle reviewed above, the selection procedures were designed to target the program for high risk offenders.

An Inmate Selection Criteria Checklist was developed specifically for this program and was used by institutional Case Management staff to screen all offenders who were 1 to 2 years away from some form of conditional release. This checklist evaluated basic cognitive abilities and identified "high need" offenders. Offenders with high levels of cognitive skill deficits were referred for an interview with the program coach. During this interview, the coach completed an assessment battery and determined the offender's level of motivation.

Our analysis suggests that Cognitive Skills Training referrals, who were judged to be appropriately in need of the program, appeared to represent high risk offenders according to a number of indices that were reviewed. For example, the majority (70%) had failed on previous community supervision, only 13% had no history of alcohol or drug problems, most had poor ratings on pre-arrest "street stability", and most had offenses that were classified as "serious" or "major". A detailed description of the characteristics of both treatment and comparison offenders is included in Appendix A. It should be noted that the two groups did not differ on any of the characteristics that were examined.

The table in Appendix A also shows the pre-treatment risk levels of the current national sample based on classifications made using SIR Scale scores. These data are also displayed in Figure 3 which shows the distribution of risk levels for the national implementation sample, the initial pilot sample, and a normative sample of offenders reported earlier by Hann and Harman (1988).

The figure shows very clearly that referrals to the Cognitive Skills Training Program are drawn from the higher risk offender population. The current data also indicate that the screening procedures designed to select high risk offenders, as established in the pilot project, has been maintained during the national implementation. Close to 60% of the offenders in the national implementation sample fell within the two highest risk groups according to the SIR scale

Program Effects on Attitudes, Behaviour and Cognition

Prior to the beginning of Cognitive Skills Training (pre-test), all potential program participants were administered a battery of measures designed to assess their level of cognitive skills and their attitudes toward criminal behaviour. Following completion of the program, offenders were reassessed (post-test) using the same instruments so that any changes could be measured.

Specifically, the pre- and post-test assessment batteries included a semi-structured interview to assess cognitive ability; corresponding behaviour-rating scales completed by the coach (the Cognitive Ability Behaviour Rating Scale); a measure of attitudes toward the criminal justice system, identification with antisocial peers and tolerance toward law violations (The Criminal Sentiments Scale: Andrews, 1985); and a measure of impulsivity, empathy and risk-taking (The Eysenck Impulsiveness Questionnaire: Eysenck, Easting, Pearson & Allsop, 1985). For a detailed description of the research design and assessment battery of the Cognitive Skills Training Program, please refer to a Research Brief prepared by the Research and Statistics Branch (1990).

Tables 2 and 3 reflect the statistically significant changes for the treatment and control groups on all of the psychometric measures. Except for the Venturesomeness scale (Table 2) and the Motivation scale (Table 3), the treatment group made statistically significant improvements on all of the other measures. Although the control group made statistically significant improvements on some of these measures, the treatment group made consistently greater changes.

One important finding was that the treatment group made significant improvements on all of the sub-scales of the Criminal Sentiments Scale (Table 2). Offenders who completed the Cognitive Skills program showed more pro-social attitudes toward the law, the courts and the police following program completion. They also expressed less identification with criminal peers and showed less tolerance for law violations. The findings are meaningful since changes in these types of attitudes are predictive of actual changes in criminal behaviour (Andrews, 1985).

Figure 3
Distrubution of SIR Risk Groups:
Current Cognitive Skills Participants Compared with Pilot and Normative Sample

The pre/post-test coach ratings indicated significantly greater changes among the treated offenders on 9 of the 10 skills that were rated. The coach ratings were based on the information collected on offenders during semi-structured interviews as well as in-program observations of the treatment group. As displayed in Table 3, the coach ratings focus on impulsivity, egocentricity, motivation and a variety of problem solving skills. Motivation was the only measure which did not demonstrate significant movement. Perhaps this reflects a ceiling effect in that all offenders were required to show a minimum level of motivation before they were considered for the program.

Another notable finding was that offenders who completed Cognitive Skills Training made significantly greater improvements on the self-report Impulsiveness scale. This finding is important since impulsiveness was one of the primary targets of the program, and it too has been linked with changes in criminal behaviour. These results are substantiated by coach perceptions of improvements on the impulse control of program participants.

Table 2
Statistically Significant Changes On
Cognitive Skills Measures

	Significant Changes		Significantly Greater Changes
	Treatment	Comparison
Attitudes toward the law	*
Attitudes toward the courts	*	*
Attitudes toward the police	*
Law, Courts, Police (TOTAL)	*		T
Identification with criminal others	*		T
Tolerance for law violations	*		T
Impulsiveness	*	*	T
Empathy	*	*
Venturesomeness

Table 3
Statistically Significant Changes On The
Cognitive Abilities Rating Scale

	Significant Changes		Significantly Greater Changes
	Treatment	Comparison
COGNITIVE ABILITIES RATING SCALE TOTAL	*	*	T
Problem recognition	*		T
Problem solving ability	*	*	T
Developing alternatives	*	*	T
Awareness of consequences	*	*	T
Setting goals	*	*	T
Egocentricity	*		T
Social perspective taking	*		T
Impulsivity	*	*	T
Cognitive style	*		T
Motivation

Conclusions

Several indicators of program success provide encouraging support for the effectiveness of the Cognitive Skills Training Program. An examination of recidivism rates and some intermediate target measures revealed that in comparison to the non-completers, program participants recidivated at a lower rate and made substantially greater improvements on the psychometric measures.

Examination of post-release success in the community reflected an interesting pattern of results. First, there were substantial reductions in the rates of reconvictions of program participants. This was true when the reconviction rates of program participants were compared to those of non-completers and when the reconviction rates of program participants were compared to normative base rates that would be expected given their particular SIR scores. Although the non-completers did not achieve as much success as the program participants, they also recidivated at a lower than the expected base rate. This finding may be reflective of the therapeutic effects of volunteering for treatment (i.e., offenders motivated to participate in programming are already at lower risk to reoffend).

It was also apparent that reductions in recidivism were associated with the types of offenders being targeted for the program. As hypothesized, the Cognitive Skills Training Program was most beneficial to high risk offenders. The categorization of offenders into high and low risk groups indicated that the high risk program participants had fewer post-release reconvictions compared to the non-completers, and they had fewer reconvictions in comparison to the low risk offenders. This is a positive finding since we have been targeting high risk offenders for programming in the national implementation sample. We believe that future investigation of recidivism rates for this sample will yield findings which substantiate the risk hypotheses.

The community follow-up period for the national implementation sample has been approximately three months. This period is considered too brief for an analysis of success in the community. Recidivism was therefore not examined for the current sample. However, changes were assessed on a number of intermediate target measures which have been linked to recidivism. The program completers made significantly greater improvements on their attitudes toward crime and the criminal justice system, on their levels of self-control, and a variety of cognitive skills as rated by their coaches.

Appendix A

Characteristics Of The Treatment And Comparison Groups

	Treatmentn = 144 Mean Percent		Comparisonn = 89 Mean Percent
Age	30.0		29.2
Aggregate Sentence (years)	5.5		5.0
Number of Current Offences	4.1		4.0
Prior Supervision		86.1		89.8
Security Level of Offenses
Minor		1.5		0.0
Moderate		39.1		49.1
Serious		41.3		35.0
Major		18.0		15.7
SIR Risk Level
Very Good		14.5		11.3
Good		10.6		20.7
Fair		14.5		11.3
Fair to Poor		25.9		22.6
Poor		34.3		33.9
Substance Abuse History
None		11.8		16.9
Alcohol Only		26.3		27.1
Drug Only		18.7		8.4
Both Alcohol and Drug		43.0		47.4
Prior Street Stability
Previous full-time employment		18.4		29.1
Previous part-time employment		28.6		35.5
Receiving Welfare		17.7		24.0
Receiving UIC		7.2		3.9
Residence with spouse		34.2		24.4
Stable family unit		47.1		49.1
Criminal associates		65.9		59.2

References

Andrews, D.A., Bonta, J., & Hoge, R. (1990). Classification for effective rehabilitation: Rediscovering psychology. Criminal Justice and Behavior, 17, 19-52.

Eysenck, S.B.G., Pearson, P.R., Easting, G.,& Allsop, J.F. (1985). Age norms for impulsiveness, venturesomeness, and empathy in adults. Personality and Individual Differences, 6, 613-619.

Fabiano, E., Robinson, D., & Porporino, F. (1990). A preliminary assessment of the Cognitive Skills Training Program: A component of Living Skills programming: Program description, research findings and implementation strategy. Ottawa: Correctional Service of Canada.

Hann, R.G., & Harman, W.G. (1988). Release risk prediction: A test of the Nuffield scoring system. A report of the Parole Decision-Making and Release Risk Assessment Project. Ottawa: Ministry of the Solicitor General.

Research and Statistics Branch (1990). Research Design of the Cognitive Skills Training Program. Ottawa: Correctional Service of Canada.

Ross, R. R., & Fabiano, E. (1985). Time to Think; A cognitive model of delinquency prevention and offender rehabilitation. Tennesse: Institute of Social Sciences and Arts, Inc.