Correctional Service Canada
www.csc-scc.gc.ca
Common menu bar links
Research
This Web page has been archived on the Web.
Estimating Risk of Dropout and Expulsion from Correctional Programs
2006 No R-177
Kevin L. Nunes
&
Franca Cortoni
Correctional Service of Canada
April 2006
ACKNOWLEDGEMENTS
The authors thank Phil Chitty for sharing his expertise in programming issues in the Offender Management System (OMS) and Colette Cousineau for retrieving and organizing the data required for this project. In addition, thanks go to Shelley Brown and Ben Vuong for sharing their procedural knowledge and research skills. We also wish to thank Amey Bell and Kelly Taylor for reviewing an earlier draft of this report.
EXECUTIVE SUMMARY
There is an abundance of evidence that correctional treatment is associated with reduced recidivism (Andrews et al., 1990; Wilson, Bouffard, & MacKenzie, 2005). By targeting criminogenic needs (also known as dynamic risk factors), such as criminal attitudes and employment skills, treatment can have a positive impact on offenders. Dropout and expulsion from correctional programs, however, hinder the goal of the Correctional Service Canada (CSC) to safely reintegrate offenders into the community. Research has shown that offenders who dropout or are expelled from programs recidivate at higher rates than offenders who complete their programs.
The goal of the present research was to determine whether it would be possible to estimate the risk of dropout or expulsion with a reasonable degree of accuracy. A screening measure was created to assist in identifying offenders at risk of dropping out of, or being expelled from, correctional programs. Given that criminal risk factors and treatment readiness have been found to be important predictors of dropout/expulsion in past research, we considered risk, criminogenic need, and motivation for intervention for inclusion in the screening measure. A number of other variables, such as offender type and program type, were also considered for inclusion. In an attempt to create a low-cost measure that would require minimal additional resources if applied in the field, all potential predictors were drawn from data routinely collected and readily available in CSC's automated Offender Management System (OMS).
Participants were 6,316 federally sentenced male offenders who had participated in at least one correctional program between April 2002 and March 2004. Aboriginal offenders made up 16.7% of the sample.
The identification of predictors of dropout/expulsion was accomplished through a multi-step process. Offenders were first randomly divided into two roughly equal-sized groups: a development sample and a cross-validation sample. In the development sample, individual predictors were identified, weighted based on their relationship with dropout/expulsion, and combined into a composite measure. This composite measure was then applied to the cross-validation sample in order to determine its predictive accuracy. Separate measures were created for the non-Aboriginal offenders and the Aboriginal offenders.
For the non-Aboriginal offenders, this process yielded the Dropout Risk Screen (DRS). The DRS consists of five predictors: the Statistical Information on Recidivism - Revised 1 (SIR-R1) scale, age, marital/family need, prosocial attitudes, and motivation for intervention. The DRS predicted dropout/expulsion with a moderate and statistically significant degree of accuracy in the development sample (AUC = .72; n = 2,617) and the cross-validation sample (AUC = .70; n = 2,630). Scores on the DRS were grouped into three risk categories: low (-5 to 0), moderate (1 to 2), and high (3 to 6). There was a linear increase in dropout/expulsion with each successive DRS category in the cross-validation sample: in the lowest category only 5% dropped out, 13% in the moderate category dropped out, and in the highest category 26% of the offenders dropped out.
The measure developed with the Aboriginal offenders was called the Aboriginal Dropout Risk Screen (ADRS). The ADRS consists of three variables: age, community functioning, and motivation for intervention. Unlike the measure designed for the non-Aboriginal offenders, the ADRS achieved only a low degree of predictive accuracy in the development sample (AUC = .67; N = 544) and in the cross-validation sample (AUC = .61; N = 504).
Whereas the ADRS performed poorly and is not yet appropriate for use in the field, the DRS demonstrated that it would be of practical value to screen non-Aboriginal offenders for risk of dropout. The predictive accuracy of the DRS in the non-Aboriginal cross-validation sample (AUC = .70) was comparable to the level of accuracy achieved by many validated instruments designed to estimate risk of recidivism.
The accuracy of a measure like the DRS is complemented by its low cost of administration. Little effort would be required to score this measure because all the necessary data are readily available in OMS. In a system where high priorities compete for scarce resources, measures like the DRS may provide a low cost means of identifying those non-Aboriginal male offenders who are at risk for dropping out of, or being expelled from, correctional programs. It must be noted that a clear misuse of the DRS, however, would be to use it as an indicator of "treatability". It would be incorrect and inappropriate to interpret a high risk score on the DRS as indicating that an offender is resistant to treatment or cannot successfully be treated. Similarly, it would be incorrect and inappropriate to refuse an offender entry into a program based on his score on the DRS. Rather, prior to commencement of a treatment program, the offenders at higher risk for dropout/expulsion could be more thoroughly assessed and, when necessary, targeted with pre-treatment efforts to increase their motivation and general readiness for treatment.
TABLE OF CONTENTS
LIST OF TABLES
- Table 1: Racial Composition of Sample
- Table 2: Program Participation
- Table 3: Program Type, Intensity, and Setting
- Table 4: Stepwise Logistic Regression: Predictors of Dropout/Expulsion Among Non-Aboriginal Men
- Table 5: Program Dropout/Expulsion (%) by Predictor Variables - Non-Aboriginal Men
- Table 6: Dropout Risk Screen (DRS)
- Table 7: DRS Means (M), Standard Deviations (SD), and Medians (Mdn) for Non-Aboriginal Dropouts/Expulsions and Completers
- Table 8: Accuracy of Predicting Program Dropout/Expulsion with DRS Among Non-Aboriginal Male Offenders
- Table 9: DRS Scores and Dropout/Expulsion (%) in Non-Aboriginal Development Sample
- Table 10: Dropout/Expulsion Rates by DRS Categories for Non-Aboriginal Male Offenders
- Table 11: Stepwise Logistic Regression: Predictors of Dropout/Expulsion Among Aboriginal Men
- Table 12: Program Dropout/Expulsion (%) by Predictor Variables - Aboriginal Men
- Table 13: Aboriginal Dropout Risk Screen (ADRS)
- Table 14: ADRS Means (M), Standard Deviations (SD), and Medians (Mdn) for Aboriginal Dropouts/ExpulsionsOffenders and Completers
- Table 15: Accuracy of Predicting Program Dropout/Expulsion with ADRS Among Aboriginal Male Offenders
- Table 16: ADRS Scores and Dropout/Expulsion (%) in Aboriginal Development Sample
- Table 17: Dropout/Expulsion Rates by ADRS Categories for Aboriginal Male Offenders
INTRODUCTION
There is an abundance of evidence that correctional treatment is associated with reduced recidivism (Andrews et al., 1990; Wilson, Bouffard, & MacKenzie, 2005). By targeting criminogenic needs (also known as dynamic risk factors), such as criminal attitudes, treatment can have a positive impact. Dropout and expulsion from correctional programs, however, hinder the goal of the Correctional Service Canada (CSC) to safely reintegrate offenders into the community. If offenders dropout of, or are expelled from, treatment prematurely, they cannot benefit from the associated reductions in risk. A number of researchers have attempted to identify reliable predictors of dropout/expulsion from correctional programs. Generally, the evidence suggests that both criminal risk and motivational factors are associated with higher likelihood of dropout/expulsion. The purpose of the present research was to determine whether it would be possible to predict dropout or expulsion with a reasonable degree of accuracy. A screening measure was created by combining easily scored predictors that could be used to identify offenders at risk for program dropout or expulsion. The performance of the predictive scheme was then evaluated with a new group of offenders (cross-validation sample).
There is a robust relationship between failure to complete correctional programs and recidivism. Hanson et al. (2002) conducted a meta-analysis of the effectiveness of sex offender treatment. In addition to concluding that sex offender treatment was associated with reductions in both sexual and general recidivism, they also found that the odds of sexual and general recidivism were twice as high for those who failed to complete treatment than for completers. Wormith and Olver (2002) found similar results with a sample of 93 violent federally sentenced offenders who participated in the Aggressive Behaviour Control (ABC) program at the Regional Psychiatric Centre (RPC; Saskatoon). They compared offenders who had completed the program to those who had failed to complete the program either because they withdrew or because they were expelled. The dropouts/expulsions were more likely than the completers to generally recidivate. Thus, offenders who fail to complete treatment may often be those who most need it.
At least two explanations of the relationship between treatment non-completion and recidivism are possible based on currently available evidence. The first explanation is that the factors that are responsible for recidivism may also increase the likelihood of non-completion of treatment (Seager, Jellicoe, & Dhaliwal, 2004; Proulx et al., 2004; Wormith & Olver, 2002). In the second explanation, it is posited that failure to complete treatment, in and of itself, may increase the likelihood of recidivism (Langton, Barbaree, Harkins, & Peacock, 2003). To date, however, there is no unequivocal support for one explanation over the other (Hanson et al., 2002).
Although it remains unclear whether there is a causal link between non-completion and recidivism, researchers have generally found that recidivism and dropout/expulsion are predicted by many of the same variables. With regard to predictors of dropout/expulsion, Wormith and Olver (2002) found that, compared to program completers, dropouts/expulsions were more likely to be unemployed when in the community, less educated, and at higher risk as measured by the Statistical Information on Recidivism (SIR) Scale. Nunes and Cortoni (2006) also found that, compared to completers, dropouts/expulsions were higher in risk and criminogenic need. Similarly, in their review of the literature on predictors of dropout from batterers' programs, Daly and Pelowski (2000) concluded that unemployment, less education, criminal history, and substance abuse were associated with a greater likelihood of dropout.
Similarly, criminal history, employment, education, and substance abuse have been found to be important recidivism risk factors (Gendreau, Little, & Goggin, 1996; Hanson & Bussière, 1998; Hanson & Morton-Bourgon, 2004; Wormith & Olver, 2002). The SIR scale is also a good predictor of recidivism (Hanson & Morton-Bourgon, 2004; Nafekh & Motiuk, 2002). Thus, both dropout/expulsion and recidivism appear to be predicted by many of the same static risk factors and criminogenic needs. Static risk factors comprise those predictors that are relatively unchanging, such as criminal history. Measures like the SIR Scale consist primarily of static risk factors. Criminogenic needs, which are also referred to as dynamic risk factors (Andrews & Bonta, 2003), are potentially changeable predictors, such as criminal attitudes, employment instability, education deficits, and substance abuse.
In addition to risk and need, responsivity factors also appear to predict dropout/expulsion. Risk and need speak to propensity for criminal behaviour and targets for treatment, whereas responsivity refers to the degree to which an offender will be receptive to treatment (Andrews & Bonta, 2003). Even a well designed program that targets criminogenic needs will not be effective if the offender is not responsive to the mode of treatment; in other words, if the treatment does not get through to the offender.
Motivation for treatment is a responsivity issue because low motivation would be expected to interfere with one's ability to benefit from treatment (Serin & Kennedy, 1997). Consistent with this expectation, researchers have found that lower motivation for treatment was associated with higher rates of dropout/expulsion across different types of treatment and different types of clients. Scott (2004) investigated whether motivation for change predicted dropout from a batterers' program. Scott conceptualized motivation as stage of change within the transtheoretical model. The transtheoretical model of change, formulated by Prochaska and DiClemente (1982; Prochaska, DiClemente, & Norcross, 1992), proposed that people cycle through different stages of motivation or readiness to change problematic behaviour. At one extreme, the precontemplation stage is characterized by no motivation to change, whereas, at a more advanced level, the action stage involves high motivation and active management of the problem. Scott found that dropout was significantly predicted by stage of change even after accounting for demographic factors, such as history of arrest and age; lower motivation for change was predictive of dropout. Motivation for change has also been found to predict dropout/expulsion among correctional samples (Beyko & Wong, 2005; Krawczyk, Witte, Gordon, Wong, & Wormith, 2002; Mckenzie, Witte, Beyko, Wong, Olver, & Wormith, 2002; Nunes & Cortoni, 2006; Wormith & Olver, 2002). Taken together, this evidence suggests that motivation for treatment, in addition to risk and need, predicts dropout/expulsion.
A very important yet rarely addressed issue in the identification of predictors of treatment attrition is how attrition itself is defined. There are many reasons why an offender may not complete treatment, only some of which would be expected to involve risk, need, and motivation (Wormith & Olver, 2002). There are many situations in which an offender would not complete a program, but not all of these situations would be accurately defined as dropout/expulsion nor would they necessarily reflect poorly on the offender's performance in the program. For example, non-completion could also be due to inter-institution transfers, conditional release, or other issues outside of the program. In contrast to dropout or expulsion, one would expect that some of these reasons for non-completion would actually be associated with lower risk and lower rates of recidivism. For example, an offender who did not complete a program because he was granted parole would not necessarily be expected to be a higher recidivism risk than an offender who completed the same program.
Nunes and Cortoni (2006) recently compared offenders who had completed or failed to complete correctional programs and found, as did Wormith and Olver (2002), that dropouts/expulsions were higher in risk and need, and lower in motivation for treatment, than completers. Offenders who did not complete programs for administrative reasons (e.g., transferred, paroled) or due to personal circumstances (e.g., hospital stay, placed in segregation), however, occupied a middle ground between the two extremes of completers and dropouts/expulsions. If the different types of non-completers do in fact constitute heterogeneous groups, researching and managing all non-completers as one homogeneous group may be counterproductive. Whereas individual differences, which can be targeted by staff, may be largely responsible for dropout or expulsion, perhaps other types of non-completion are best dealt with through administrative channels (cf. Wormith & Olver, 2002). It follows that the comparisons that would be most informative may be those between offenders who complete a correctional program and those who dropout or are expelled because of unsatisfactory or unacceptable program performance.
The goal of the present research was to determine whether dropout or expulsion could be predicted with a reasonable degree of accuracy. A screening measure was created that could assist in identifying offenders at risk of dropping out of, or being expelled from, correctional programs. Given that criminal risk factors and motivation have been found to be important predictors of dropout/expulsion in past research, we considered risk, criminogenic need, and motivation for intervention for inclusion in the screening measure. A number of other variables, such as offender type and program type, were also considered for inclusion. In an attempt to create a low-cost measure that would require minimal additional resources if applied in the field, all potential predictors were drawn from data routinely collected and readily available in CSC's automated Offender Management System (OMS).
METHOD
Participants
Participants were 6,316 federally sentenced male offenders who had participated in at least one high-, medium-, or low-intensity correctional program between April 2002 and March 2004, which started no earlier than January 1, 2002. These offenders were drawn from a sample examined by Nunes and Cortoni (2006). The average age of the offenders at the start of the program was 35.01 years (standard deviation [SD] = 10.77) and ranged from 18 to 83.
As shown in Table 1, the majority of offenders were White. Because there were so few offenders in some of the race categories, they were grouped into two more broadly defined categories: non-Aboriginal (83.3%), which included White and all remaining non-Aboriginal races; and Aboriginal (16.7%), which included Inuit, Métis, and First Nation offenders.
In the present study, an offender was classified as (a) a sex offender if he had any current sex convictions or (b) a non-sex offender if he had no current sex convictions. A minority of offenders (18.6%; n = 1,174) were sex offenders, and 81.4% (n = 5,140) were non-sex offenders. A distinction was not made between violent and non-violent offenders in the current study. Closer inspection of the files for a randomly selected subsample revealed that many of the offenders with a current non-violent offence had previously committed violence offences.
With regard to current sentence length, 8.3% (524/6,316) were serving indeterminate sentences (e.g., life). For those with determinate sentences (n = 5,792), the mean current sentence length was 4.67 years (SD = 3.99).
Women offenders were not included in the current study because the number of women offenders was too low to adequately address our research questions. Specifically, a total of 148 women offenders had participated in correctional programs during the timeframe examined. Of these, 94.6% (n = 140) completed the program and 5.4% (n = 8) dropped out.
Table 1 Racial Composition of Sample
| Race | % |
|---|---|
| Arab/West Asian | 0.7 |
| Asiatic | 0.3 |
| Black | 6.5 |
| White | 71.8 |
| East Indian | 0.1 |
| Hispanic | 0.1 |
| Inuit | 0.5 |
| Métis | 4.9 |
| First Nation | 11.3 |
| Other | 1.2 |
| Chinese | 0.3 |
| Filipino | 0.2 |
| Japanese | 0.02 |
| Korean | 0.05 |
| Latin American | 0.5 |
| South Asian | 0.5 |
| South East Asian | 0.9 |
Note. N = 6,292. Race data were missing for some offenders.
Measures
Risk was assessed with the Statistical Information on Recidivism - Revised 1 (SIR-R1) scale (for non-Aboriginal offenders only) and the Level of Intervention Based on Static Factors rating (for both non-Aboriginal and Aboriginal offenders). Criminogenic needs were measured by the summary ratings from the Case Needs Identification and Analysis. Finally, motivation was defined as the Motivation for Intervention rating.
Statistical Information on Recidivism - Revised 1 (SIR-R1). The Statistical Information on Recidivism - Revised 1 (SIR-R1) scale (Nafekh & Motiuk, 2002; Standard Operating Practices [SOP] 700-04, 2004) is an actuarial instrument designed to estimate risk for recidivism. The original scale, the General Statistical Information on Recidivism Scale, was designed by Nuffield (1982). The SIR-R1 incorporates several individual demographic and criminal history variables, which were weighted according to their association with recidivism. Scores for these individual items are summed for a total score. Almost all of the SIR-R1 items are static in nature. Lower scores are indicative of higher risk for recidivism. Scores are grouped into five categories: very good (least likely to recidivate), good, fair, poor, and very poor (most likely to recidivate). The SIR-R1 is currently used only with non-Aboriginal male offenders under federal jurisdiction. The measure has demonstrated good reliability (internal consistency) and predictive validity for general, violent, and sexual recidivism in a variety of samples (Hanson & Morton-Bourgon, 2004; Nafekh & Motiuk, 2002).
Static Risk. Another indication of static risk is the Level of Intervention Based on Static Factors rating (Motiuk, 1997; SOP 700-04). At intake a rating of high, medium, or low is assigned to an offender based on criminal history, offence severity, and, for male non-Aboriginal offenders, the SIR-R1. Considerable involvement with the criminal justice system, many sex offences, extremely severe offences, and, when applicable, a SIR-R1 score indicative of high risk would warrant a rating of high. In contrast, an offender would be rated as low risk if he had little involvement with the criminal justice system, very few sex offences, low offence severity, and, when applicable, a SIR-R1 score indicative of low risk. Given that the SIR-R1 is only administered to non-Aboriginal male offenders, it contributes to the level of intervention based on static factors only for these offenders. This variable is referred to as static risk throughout the remainder of the report.
Criminogenic Needs. Criminogenic needs (i.e., dynamic risk factors) are also assessed at intake and at various points throughout an offender's sentence (Motiuk, 1997; SOP 700-04, paragraph 78). Seven domains are assessed: employment, marital/family, associates/social interaction, substance abuse, community functioning, personal/emotional orientation, and attitude. Employment refers to the value placed on work and the role of work in one's life. Marital/family concerns the value placed on being with family and the support one derives from them. Associations/social interaction involves the value placed on non-criminal associates and the opportunity for positive social interaction. Substance abuse refers to the value placed on living without reliance on alcohol and/or drugs. Community functioning concerns the value placed on having the knowledge and necessary skills for daily living. Personal/emotional orientation involves the value placed on being in control of one's life. Finally, attitude refers to the value placed on living in law-abiding ways.
Based on interviews and collateral information, each domain, with the exception of substance abuse and personal/emotional orientation, is rated on a four-point scale from factor seen as an asset to community adjustment to considerable need for improvement. Substance abuse and personal/emotional orientation are rated on a 3-point scale from no immediate need for improvement to considerable need for improvement.
Motivation for Intervention. Motivation for intervention (SOP 700-04) is assessed at intake and again at various points in an offender's sentence. This rating reflects the degree to which an offender recognizes that he or she has a problem, is willing to change, has the ability to change, and has demonstrated positive change in the past. An offender is rated as being high in motivation if he or she is self-motivated and will actively address problem areas. A rating of medium motivation is given for an offender who may not fully accept the overall assessment but will still participate in recommended programs or other interventions. Finally, an offender is rated as being low in motivation if he or she strongly rejects the need for change or is unwilling to participate in recommended programs or other interventions.
Procedure
Data on offenders who had participated in at least one program between April 2002 and March 2004 were gathered from CSC's Offender Management System (OMS). On average, offenders participated in 1.43 programs (SD = 0.71, median [Mdn] = 1) during the 2-year data collection period. As shown in Table 2, the majority of offenders (67.2%) participated in only one program during the 2-year data collection period, whereas fewer offenders participated in more than one program.
Only 13.0% of offenders dropped out of or were expelled from their first program during the two-year data collection period. For 47.5% of offenders, the first program in the data collection period was also the first correctional program in which they had participated during their current sentence.
Table 2 Program Participation
| Number of programs | % |
|---|---|
| 1 | 67.2 |
| 2 | 24.8 |
| 3 | 6.2 |
| 4 | 1.5 |
| 5 | 0.2 |
| 6 | 0.1 |
We focused almost exclusively on dropout/expulsion from the first correctional program in which offenders participated during the data collection period. We decided to focus on the first program for two reasons. First, the majority of offenders participated in only one program during the data collection period. Second, by focusing on the first program rather than any program during the data collection period, we avoided confounding the number of attempts with the likelihood of dropout/expulsion. It is conceivable, for example, that an offender may be found to be more likely to dropout or be expelled from a program if he were observed over six programs compared to one program (i.e., more programs equal more opportunities for dropout/expulsion to occur). Throughout the remainder of this report, reference to program participation concerns only the first correctional program in which an offender participated during the data collection period, unless otherwise stated.
Definitions of Completion and Dropout/Expulsion. Program status is entered into OMS for each offender indicating whether a program has been completed. In cases of non-completion, the status indicates general reasons for this outcome. In the current study, program status entries were organized into two groups. Program outcome was coded as completed if the status was successful completion (n = 5,312), attended all sessions (n = 182), or unsuccessful completion (n = 1). These entries all indicate that all or most sessions were attended. Successful completion indicates that an offender was compliant and successful in the program, whereas attended all sessions and unsuccessful completion indicate that an offender completed the program but was unproductive or failed to fully meet the program requirements.
Program outcome was coded as dropout/expulsion if the status was suspension (n = 821). This category is assigned to offenders who dropped out or were expelled from a program. Expulsion would most typically be for unacceptable behaviour or performance within the program. Offenders who neither completed nor dropped out of a program, as defined above, were excluded from the study.
The data used in the analyses below were drawn from different points in the offenders' sentence. The static variables, such as the SIR-R1 scale and risk ratings, were taken from data collected during the Offender Intake Assessment. Other variables, such as criminogenic needs, motivation level, and program setting, were taken from the most recent data available at the beginning of the program. In all cases, the variables considered preceded the treatment outcome. For example, the data on criminogenic need used in the current study were measured and reported in OMS prior to the commencement of the program in question. Hence, the study design was predictive.
RESULTS
Program data are presented in Table 3. The first program in which offenders participated was most likely to have been a living skills program (47.6%), a moderate-intensity program (67.3%), and a program delivered in a medium-security institution (49.9%).
In light of evidence that Aboriginal offenders are more likely to dropout or be expelled from programs than are non-Aboriginal offenders (Nunes & Cortoni, 2006; Wormith & Olver, 2002), these groups were examined separately. The first set of analyses concerned the non-Aboriginal offenders and the second set focused on the Aboriginal offenders.
Table 3 Program Type, Intensity, and Setting
| % | |
| Type of first program | |
| Violent offender | 4.6 |
| Sex offender | 9.5 |
| Substance abuse | 12.7 |
| Family violence | 16.7 |
| Living skills | 47.6 |
| Anger management | 8.8 |
| Intensity of first program | |
| High | 7.1 |
| Moderate | 67.3 |
| Low | 25.6 |
| Program setting | |
| Multi-level institution | 3.2 |
| Maximum-security institution | 16.6 |
| Medium-security institution | 49.9 |
| Minimum-security institution | 11.6 |
| Community | 18.7 |
Non-Aboriginal Male Offenders
The identification of predictors of dropout/expulsion was accomplished through a multi-step process. Non-Aboriginal male offenders were randomly divided into two roughly equal-sized groups, the development sample (n = 2,617) and the cross-validation sample (n = 2,630). In the development sample, individual predictors were identified, weighted based on their relationship with dropout/expulsion, and combined into a composite measure, which was named the Dropout Risk Screen (DRS). The DRS was then applied to the cross-validation sample. A more detailed account of this procedure is provided below.
To identify potential predictors of dropout/expulsion, Cohen's d was computed for the association between dropout/expulsion and a number of variables in the development sample. The initial pool of variables included race, offence type, age, risk, need, motivation level, program type, program intensity, program setting, and whether it was the first program in their current sentence. A decision rule was established, in which a d equal to or greater than 0.20 was considered indicative of a meaningful association. By convention, a d of around 0.20 is considered a small effect size (Cohen, 1992). Similarly sized effects are commonly found and considered meaningful in forensic psychology. For example, in their meta-analysis, Gendreau, Little, and Goggin (1996) found small to medium effect sizes for many predictors of criminal recidivism, such as age, criminal history, gender, and criminal companions. Similarly, the variables of a validated risk assessment instrument for sexual offenders were individually associated to a small to medium degree with sexual recidivism (Hanson, 1997). Hence, a small effect size is often indicative of a meaningful association for researchers and clinicians working with offender populations (see Meyer et al., 2003, for a compilation of effect sizes found in a broader range of research areas).
For 15 variables from the initial pool, d was at 0.20 or greater. Compared to completers, dropouts/expulsions were higher risk, as measured by the SIR-R1 categories, d = 0.55, and static risk, d = 0.24. Dropouts/expulsions were also younger, d = -0.39 and had lower motivation for intervention, d = -0.46. Dropout/expulsions had greater criminogenic need in the areas of: employment, d = 0.36; marital/family, d = 0.25; associates, d = 0.26; substance abuse, d = 0.26; community functioning, d = 0.31; personal/emotional orientation, d = 0.20; and criminal attitudes, d = 0.40. Dropout/expulsion was less likely in family violence programs than in other types of programs, d = -0.36, and more likely in living skills programs than in other types of programs, d = 0.24. Dropouts/expulsions were more likely in higher intensity programs, d = 0.21. Finally, sex offenders were less likely than other offenders to dropout or be expelled, d = -0.21.
With the exception of program type and intensity, the variables above were entered into a stepwise logistic regression to eliminate redundant predictors of dropout/expulsion. A redundant predictor is a variable that does not increase predictive accuracy once other predictors have been considered. We excluded program type (family violence and living skills) and intensity because such information would not be as readily available at intake as the other variables. For example, at intake assessors may not know the type and intensity of program in which an offender will participate once he is placed in his home institution. These variables would therefore limit the flexibility of a measure. Incidentally, even when these variables were retained, the predictive accuracy achieved was not significantly different from when they were excluded.
The results of the logistic regression are reported in Table 4. As indicated by the odds ratios and their 95% confidence intervals, 5 of the 12 variables remained as significant predictors of dropout/expulsion. The odds ratio can be interpreted as the increase in the odds of dropout/expulsion that corresponded to an increase of one point on the predictor. For example, the odds ratio for the SIR-R1 (1.28) indicates that for an increase of one risk level on the SIR-R1 (e.g., from fair to poor), the odds of dropout/expulsion increased by 28%. An odds ratio of 1.00 would reflect no relationship between the predictor and the outcome; that is, the odds of dropout/expulsion would be equal at all SIR-R1 categories.
The confidence interval about the odds ratio provides an estimate of the range of values within which the odds ratio for the population of offenders would be expected to fall 95% of the time. For example, the odds ratio for the SIR-R1 was 1.28 but it would be expected that if the population from which this sample was drawn were sampled 100 times, in 95 of those samples the odds ratio would fall between 1.16 and 1.40. If the confidence interval does not contain include 1.00, the variable significantly predicts dropout/expulsion (p < .05).
Table 4 Stepwise Logistic Regression: Predictors of Dropout/Expulsion Among Non-Aboriginal Men
| Predictor | B | SE B | Odds Ratio | 95% CI | |
|---|---|---|---|---|---|
| Lower | Upper | ||||
| SIR-R1 Category | 0.24 | .05 | 1.28* | 1.16 | 1.40 |
| Age | -0.04 | .01 | 0.96* | 0.95 | 0.98 |
| Motivation for intervention | -0.59 | .12 | 0.56* | 0.44 | 0.71 |
| Marital / family | 0.26 | .08 | 1.30* | 1.11 | 1.53 |
| Attitude | 0.25 | .09 | 1.28* | 1.08 | 1.52 |
Note. SE = Standard error. CI = confidence interval.
* p < .01.
The dropout/expulsion rates at the various levels of each variable are presented in Table 5. As shown in the table, of the 723 offenders whose SIR-RI scores placed them in the very good category, only 3.7% dropped out or were expelled. In contrast, 17.6% of the 700 offenders in the very poor category dropped out. Age was transformed from a continuous variable into a categorical one by grouping together ages that appeared to have similar rates of dropout/expulsion. The dropout/expulsion rate was fairly steady from 18 to 25 years of age, but then suddenly dropped off. Accordingly, two age groups were created: 18 to 25 and 26 or older.
Table 5 Program Dropout/Expulsion (%) by Predictor Variables - Non-Aboriginal Men
| Predictor | N | Dropout/expulsion (%) |
|---|---|---|
| SIR-R1 | ||
| Very good | 723 | 3.7 |
| Good | 376 | 10.9 |
| Fair | 414 | 8.9 |
| Poor | 377 | 17.0 |
| Very poor | 700 | 17.6 |
| Age | ||
| 18-25 | 558 | 18.5 |
| 26 + | 2,059 | 9.4 |
| Motivation level | ||
| Low | 362 | 22.4 |
| Medium | 1,470 | 11.7 |
| High | 452 | 5.8 |
| Marital/family | ||
| Factor seen as an asset | 125 | 4.8 |
| No current difficulty | 1,375 | 9.6 |
| Some difficulty | 674 | 14.7 |
| Considerable difficulty | 420 | 14.3 |
| Attitude | ||
| Factor seen as an asset | 88 | 4.5 |
| No current difficulty | 941 | 7.5 |
| Some difficulty | 820 | 11.5 |
| Considerable difficulty | 745 | 17.2 |
Note. For some variables, N sums to less than 2,617 because data were missing.
These five variables were next combined to create a measure for estimating likelihood of dropout/expulsion. We called this measure the Dropout Risk Screen (DRS). Following a procedure employed by some researchers in creating recidivism risk assessment instruments (Nuffield, 1982; Quinsey, Harris, Rice, & Cormier, 1998), the variables were combined by weighting them based on their corresponding departures from the dropout/expulsion base rate. The dropout/expulsion base rate refers to the proportion of offenders who dropped out of or were expelled. In the development sample, the dropout/expulsion base rate was 11.3%.
The weighting procedure was as follows. Each value of each variable was first examined for its fluctuation from the dropout/expulsion base rate. To illustrate, consider the motivation for intervention variable. As shown in Table 5, offenders in the low motivation category had a higher dropout/expulsion rate (22.4%) than did the entire development sample (base rate = 11.3%), which is a difference of 11.1% (22.4% - 11.3% = 11.1%). One point was assigned for every 5% departure from the base rate. If the departure was a 5% increase, one point was added, whereas if the departure was a 5% decrease, one point was subtracted. Thus, the increased dropout/expulsion rate resulted in assigning a score of +2 to low motivation for intervention (11.1%/5 = 2.22, rounded down to 2). Each of the five variables was weighted following the same procedure (see Table 6). The weighted scores from the five variables were then summed for each offender to create a DRS total score that could range from -5 to 6. Higher values reflect higher likelihood of dropout/expulsion. Means, standard deviations, and medians are presented separately for the dropouts/expulsions and the completers in Table 7 for both the development sample and the cross-validation sample.
Table 6 Dropout Risk Screen (DRS)
| Item | Value | Scoring |
|---|---|---|
| SIR-R1 | Very good | -2 |
| Good or Fair | 0 | |
| Poor or Very poor | 1 | |
| Age | 18-25 | 1 |
| 26 + | 0 | |
| Motivation for intervention | Low | 2 |
| Medium | 0 | |
| High | -1 | |
| Marital/family | Factor seen as an asset | -1 |
| No current difficulty | 0 | |
| Some difficulty or Considerable difficulty | 1 | |
| Attitude | Factor seen as an asset or No current difficulty | -1 |
| Some difficulty | 0 | |
| Considerable difficulty | 1 | |
| Total (-5 to 6) | ||
Table 7 DRS Means (M), Standard Deviations (SD), and Medians (Mdn) for Non-Aboriginal Dropouts / Expulsions and Completers
| Dropouts/Expulsions | Completers | |||||||
| Sample | N | M | SD | Mdn | N | M | SD | Mdn |
| Development | 297 | 1.95 | 1.88 | 2.00 | 2320 | 0.24 | 2.19 | 0.00 |
| Cross-validation | 303 | 1.78 | 1.89 | 2.00 | 2327 | 0.22 | 2.18 | 0.00 |
| Sex offenders | 34 | 1.68 | 1.85 | 2.00 | 421 | -0.47 | 2.20 | -1.00 |
| Non-sex offenders | 269 | 1.80 | 1.90 | 2.00 | 1906 | 0.38 | 2.15 | 1.00 |
The ability of the DRS to identify dropouts/expulsions was first examined in the development sample. Raw scores on the DRS were entered for all analyses in Table 8. The DRS significantly predicted dropout/expulsion and demonstrated a moderate level of predictive accuracy. As shown in Table 8, two different types of statistics were computed; both provide an indication of the magnitude of the association between the DRS and dropout/expulsion. These statistics were the odds ratio and the area under the curve (AUC) of the receiver operating characteristic (ROC). The AUC is commonly used to evaluate the predictive validity of risk assessment instruments (e.g., Hanson & Thornton, 2000; Quinsey, Harris, Rice, & Cormier, 1998). An area of .50 indicates a chance level of predictive accuracy, whereas an area of 1.00 indicates perfect prediction. The AUC can be interpreted as the likelihood that a randomly selected dropout/expulsion would score higher than a randomly selected completer on the DRS. For example, an AUC of .72 indicates that 72% of the time, a randomly selected dropout/expulsion would have a higher score on the DRS than a randomly selected completer. The confidence intervals for the AUC are interpreted in the same way as the confidence intervals about the odds ratio.
The 95% confidence intervals about the odds ratios and the AUCs, which are also reported in Table 8, can be used to determine statistical significance. Specifically, if the confidence interval about the odds ratio does not include 1, the association is statistically significant (p < .05). Similarly, if the confidence interval about the AUC does not include .50, the association is statistically significant (p < .05). In addition, whether two values are statistically different from one another can be determined from the 95% confidence intervals. If one value falls outside the confidence intervals associated with the other value, the difference between the values is statistically significant (p < .05). For example, an AUC of .77 with a lower 95% confidence interval of .70 and an upper confidence interval of .84 would be significantly larger than an AUC of .69 with confidence intervals of .66 to .72. This is because the AUC of .77 falls outside of the other AUC's confidence intervals of .66 to .72.
Table 8 Accuracy of Predicting Program Dropout/Expulsion with DRS Among Non-Aboriginal Male Offenders
| Sample | N | OR | 95% CI | AUC | 95% CI | ||
|---|---|---|---|---|---|---|---|
| Lower | Upper | Lower | Upper | ||||
| Development | 2617 | 1.49* | 1.39 | 1.59 | .72* | .68 | .74 |
| Cross-validation | 2630 | 1.43* | 1.34 | 1.52 | .70* | .67 | .73 |
| Sex offenders | 455 | 1.53* | 1.29 | 1.80 | .77* | .70 | .84 |
| Non-sex offenders | 2175 | 1.40* | 1.31 | 1.50 | .69* | .66 | .72 |
Note. N = number of offenders in each sample. OR = odds ratio. CI = confidence interval. AUC = area under the curve of the receiver operating characteristic (ROC).
* p < .05.
Table 9 DRS Scores and Dropout/Expulsion (%) in Non-Aboriginal Developmental Sample
| DRS score | N | Dropout / Expulsion (%) |
|---|---|---|
| -5 | 17 | 0.0 |
| -4 | 78 | 1.3 |
| -3 | 226 | 2.2 |
| -2 | 250 | 4.8 |
| -1 | 274 | 4.4 |
| 0 | 408 | 5.4 |
| 1 | 459 | 12.2 |
| 2 | 447 | 17.2 |
| 3 | 271 | 20.7 |
| 4 | 112 | 26.8 |
| 5 | 69 | 30.4 |
| 6 | 6 | 83.3 |
| Total | 2,617 | 11.3 |
In addition to the strength of association between the DRS and dropout/expulsion, decision-makers in applied settings would also likely be interested in the dropout/expulsion rate by score on the DRS. Dropout/expulsion rates for each DRS score are presented in Table 9. The rate of dropout/expulsion generally increased as DRS score increased.
Scores on the DRS were next grouped into three risk categories: low (-5 to 0), moderate (1 to 2), and high (3 to 6). These groupings were guided by visual inspection of the dropout/expulsion rates for each score. Scores that were judged to have similar dropout/expulsion rates were grouped together. As shown in Table 10, there was a linear increase in dropout/expulsion with each successive DRS category in the development sample. In the lowest category only 4% dropped out, whereas in the highest category 25% of the offenders dropped out. The total dropout/expulsion base rate for the development sample (11.3%) is also presented in Table 10. Demonstration of the stability of the DRS would support the utility of this tool for identifying offenders at risk for dropout/expulsion.
Table 10 Dropout/Expulsion Rates by DRS Categories for Non-Aboriginal Male Offenders
| DRS | ||||
|---|---|---|---|---|
| Low | Moderate | High | ||
| Total* | -5 to 0 | 1 to 2 | 3 to 6 | |
| Sample | % (n) dropout or expulsion | |||
| Development | 11.3 (297/2,617) | 4.2 (52/1,253) | 14.7 (133/906) | 24.5 (112/458) |
| Cross-validation | 11.5 (303/2,630) | 5.4 (70/1,296) | 13.1(116/885) | 26.1 (117/449) |
| Sex offenders | 7.5 (34/455) | 3.1 (9/293) | 11.1 (12/108) | 24.1 (13/54) |
| Non-sex offenders | 12.4 (269/2,175) | 6.1 (61/1,003) | 13.4 (104/777) | 26.3 (104/395) |
*Dropout/expulsion base rate for sample.
Although the results obtained with the development sample were a positive indication of the potential utility of the DRS, these findings alone were not sufficient evidence of its predictive validity. Because the DRS was created based on the specifications of the development sample, it was possible that the association between the DRS and dropout would decrease when applied to a new group of offenders. In other words, it was not surprising that the DRS was predictive of dropout/expulsion in the development sample because the variables and their weights were both derived from the development sample itself. It was therefore possible that the relationship between the predictors and dropout/expulsion in the development sample, on which we capitalized, would not be found in other samples. Thus, the performance of the DRS with the development sample was more of a demonstration that dropout/expulsion risk can be reliably estimated (a necessary and important first step) than it was a validation of the DRS.
To adequately assess the validity of the DRS, it was necessary to apply it to the cross-validation sample, which did not overlap at all with the development sample. As shown in Table 8, the DRS performed similarly in the cross-validation sample (AUC = .70) and in the development sample (AUC = .71). Again, dropout/expulsion was predicted with a moderate degree of accuracy by the DRS. Often, shrinkage of effects is a concern as one moves from the development sample to a new sample. In this case, however, virtually identical levels of accuracy were seen in both samples. In terms of the pattern of dropout/expulsion across scores on the DRS, the rates reported in Table 10 were also virtually identical in the cross-validation and the development sample. Again, as the DRS category increased, so did the dropout/expulsion rate, with 26% dropout/expulsion in the highest risk category.
Given the heterogeneity of federal offenders, it would be helpful to know if the DRS performed as well with specific groups of offenders as it did with the combined groups. Offence type may moderate the relationship between the DRS and dropout/expulsion, which could make the DRS a good predictor of dropout/expulsion for some groups but not for others. To address this possibility, the accuracy of the DRS was assessed by offence type in the cross-validation sample. As shown in Tables 8 and 10, the results were generally consistent with the initial findings. The DRS significantly predicted dropout/expulsion among both sex offenders and non-sex offenders in the cross-validation sample.
Although the DRS demonstrated at least small to moderate levels of predictive accuracy in all three groups, the degree of accuracy achieved was not equivalent. Specifically, accuracy was significantly (p < .05) greater with sex offenders (AUC = .77) than with non-sex offenders (AUC = .69).
Creation of a composite dropout/expulsion predictor was also approached following an alternate procedure. Specifically, we developed measures separately with the sex and non-sex offenders in the development sample. When these measures were applied to their respective groups in the cross-validation sample, however, their accuracy was no better than that reported for the DRS, which was developed with the mixed group of offenders in the development sample. Given that there was no improvement in accuracy, the DRS was presented rather than the measures specific to each offender type.
Aboriginal Male Offenders
Aboriginal offenders were examined separately from the non-Aboriginal offenders for three reasons: their dropout/expulsion rates are generally higher (Nunes & Cortoni, 2006; Wormith & Olver, 2002); they are not administered all the same risk measures (e.g., SIR-R1); and, perhaps more importantly, it would be preferable to develop a measure specific to Aboriginal offenders to increase the confidence with which it could be used with this group. The procedure for identifying predictors of dropout/expulsion for the Aboriginal offenders and combining them into a composite measure was identical to the procedure used with the non-Aboriginal male offenders. The composite measure, in this case, was called the Aboriginal Dropout Risk Screen (ADRS).
Although the ADRS is similar in name and purpose to the DRS, it was developed independently with an Aboriginal male sample. Aboriginal male offenders were randomly divided into two roughly equal-sized groups, the development sample (n = 544) and the cross-validation sample (n = 504). Six variables from the initial pool of potential predictors were associated with dropout/expulsion at d = 0.20 or greater.
In the development sample, dropouts/expulsions were younger, d = -0.47; had lower levels of motivation for intervention, d = -0.34; and had greater criminogenic needs in the areas of employment, d = 0.40; associates, d = 0.40; community functioning, d = 0.36; and attitudes, d = 0.20. Dropout/expulsion was also more likely to occur in Living Skills programs than in other types of programs, d = 0.32.
With the exception of program type, these six variables were entered into a stepwise logistic regression to eliminate redundant predictors of dropout/expulsion. As in the development of the DRS, program type information was excluded to maintain the flexibility of the measure. As shown in Table 11, only age, community functioning, and motivation for intervention remained as significant predictors in the logistic regression.
Table 11 Stepwise Logistic Regression: Predictors of Dropout / Expulsion Among Aboriginal Men
| Predictor | B | SE B | Odds Ratio | 95% CI | |
| Lower | Upper | ||||
| Age | -0.06 | .01 | 0.94* | 0.92 | 0.97 |
| Community functioning | 0.50 | .16 | 1.65* | 1.20 | 2.28 |
| Motivation for intervention | -0.57 | .19 | 0.57* | 0.39 | 0.83 |
Note. SE = Standard error. CI = confidence interval.
* p < .05.
The dropout/expulsion rates at the various levels of each variable are presented in Table 12. Age was transformed from a continuous variable into a categorical one by grouping together ages that appeared to have similar rates of dropout/expulsion. The dropout/expulsion rate was fairly steady from 18 to 30 years of age, but then suddenly dropped off. Following this procedure, two age groups were created: 18 to 30 and 31 or older.
Table 12 Program Dropout / Expulsion (%) by Predictor Variables - Aboriginal Male Offenders
| Predictor | N | Dropout/Expulsion (%) |
|---|---|---|
| Age (years) | ||
| 18 to 30 | 262 | 29.0 |
| 31+ | 282 | 14.5 |
| Community functioning | ||
| Factor seen as an asset | 6 | 16.7 |
| No current difficulty | 337 | 17.8 |
| Some difficulty | 162 | 24.7 |
| Considerable difficulty | 38 | 42.1 |
| Motivation for intervention | ||
| Low | 99 | 26.3 |
| Medium | 333 | 24.6 |
| High | 83 | 6.0 |
Note. For some variables, N sums to less than 544 because data were missing.
Age, community functioning, and motivation were next combined into the ADRS by weighting them based on their corresponding departures from the dropout/expulsion base rate. The dropout/expulsion base rate was 21.5% in the development sample of Aboriginal male offenders. The weighted scores from these three variables were then summed for each offender to create an ADRS total score that could range from -5 to 7 (see Table 13). Higher values reflect higher likelihood of dropout/expulsion. Means, standard deviations, and median scores for dropouts/expulsions and completers are presented in Table 14 for both the development and cross-validation sample.
Table 13 Aboriginal Dropout Risk Screen (ADRS)
| Item | Value | Scoring |
|---|---|---|
| Age | 18 to 30 | 2 |
| 31+ | -1 | |
| Community functioning | Factor seen as an asset or No current difficulty | -1 |
| Some difficulty | 1 | |
| Considerable difficulty | 4 | |
| Motivation for intervention | Low or Medium | 1 |
| High | -3 | |
| Total (-5 to 7) | ||
Table 14 ADRS Means (M), Standard Deviations (SD), and Medians (Mdn) for Aboriginal Dropouts/Expulsions and Completers
| Dropouts / Expulsions | Completers | |||||||
|---|---|---|---|---|---|---|---|---|
| Sample | N | M | SD | Mdn | N | M | SD | Mdn |
| Development | 117 | 2.11 | 2.49 | 2.00 | 427 | 0.35 | 2.57 | 1.00 |
| Cross-validation | 102 | 1.54 | 2.21 | 2.00 | 402 | 0.45 | 2.66 | 1.00 |
| Sex offenders | 14 | 0.64 | 1.95 | 0.5 | 93 | -0.20 | 2.56 | -1.00 |
| Non-sex offenders | 88 | 1.68 | 2.23 | 2.00 | 309 | 0.64 | 2.66 | 1.00 |
Performance of the ADRS with the Aboriginal male development sample was low to moderate (AUC = .67; see Table 15). Dropout/expulsion rates for each ADRS score are presented in Table 16. There was a general increase in the rate of dropout/expulsion as ADRS score increased. Scores on the ADRS were next grouped into three risk categories: low (-5 to -1), moderate (0 to 3), and high (4 to 7). As shown in Table 17, over one third (40%) of the offenders in the high-risk ADRS category dropped out.
When the ADRS was applied to the cross-validation sample, predictive accuracy was significantly greater than chance. Performance in the cross-validation sample, however, was significantly less accurate than in the development sample (AUC = .61 vs. AUC = .67, p < .05). Different types of offenders were next drawn from the cross-validation Aboriginal male sample. As shown in Table 15, a significant level of accuracy was observed for the non-sex offenders but not for the sex offenders (p < .10). Although the AUCs for both groups are identical, statistical significance was not reached in the sex offender sample because of its smaller size. In addition, as presented in Table 17, a linear increase in dropout/expulsion rates was observed across the ADRS categories for the non-sex offenders but not for the sex offenders. Overall, the predictive accuracy of the ADRS with Aboriginal offenders was low.
We were unable to create alternate measures for each offender type, as was done with the non-Aboriginal offenders, because the number of Aboriginal offenders in our sample was too low.
Table 15 Accuracy of Predicting Program Dropout/Expulsion with ADRS Among Aboriginal Male Offenders
| Sample | N | OR | 95% CI | AUC | 95% CI | ||
|---|---|---|---|---|---|---|---|
| Lower | Upper | Lower | Upper | ||||
| Development | 544 | 1.32* | 1.21 | 1.45 | .67* | .62 | .73 |
| Cross-validation | 504 | 1.19* | 1.08 | 1.30 | .61* | .55 | .67 |
| Sex offenders | 107 | 1.15 | 0.91 | 1.43 | .60 | .46 | .75 |
| Non-sex offenders | 397 | 1.18* | 1.07 | 1.30 | .60* | .54 .67 | |
Note. n = number of offenders in each sample. OR = odds ratio. CI = confidence interval. AUC = area under the curve of the receiver operating characteristic.
* p < .05.
Table 16 ADRS Scores and Dropout/Expulsion (%) in Aboriginal Developmental Sample
| ADRS score | N | Dropout / Expulsion (%) |
|---|---|---|
| -5 | 33 | 0 |
| -4 | 0 | -- |
| -3 | 17 | 0 |
| -2 | 34 | 14.7 |
| -1 | 137 | 16.8 |
| 0 | 11 | 18.2 |
| 1 | 69 | 17.4 |
| 2 | 132 | 25.0 |
| 3 | 11 | 18.2 |
| 4 | 83 | 33.7 |
| 5 | 0 | -- |
| 6 | 0 | -- |
| 7 | 17 | 70.6 |
| Total | 544 | 21.5 |
Table 17 Dropout/Expulsion Rates by ADRS Categories for Aboriginal Male Offenders
| ADRS | ||||
|---|---|---|---|---|
| Low | Moderate | High | ||
| Total* | -5 to -1 | 0 to 3 | 4 to 7 | |
| Sample | % (n) dropout or expulsion | |||
| Development | 21.5 (117/544) | 12.7 (28/221) | 22.0 (49/223) | 40.0 (40/100) |
| Cross-validation | 20.2 (102/504) | 13.1 (26/198) | 21.8 (48/220) | 32.6 (28/86) |
| Sex offenders | 13.1 (14/107) | 9.7 (6/62) | 18.2 (6/33) | 16.7 (2/12) |
| Non-sex offenders | 22.2 (88/397) | 14.7 (20/136) | 22.5 (42/187) | 35.1 (26/74) |
*Dropout/expulsion base rate for sample.
DISCUSSION
The goal of the present research was to determine whether it would be possible to estimate risk of dropout or expulsion from correctional programs with a reasonable degree of accuracy. Two screening measures, the DRS and ADRS, were developed by combining variables found to predict dropout/expulsion in both the current and previous research. The DRS, which was developed and designed for use with non-Aboriginal male offenders, demonstrated a moderate level of predictive accuracy in a cross-validation sample. The ADRS, which was developed with Aboriginal male offenders, demonstrated low predictive accuracy in a cross-validation sample.
As in previous research, static risk, criminogenic need, and motivation were associated with dropout/expulsion. Among non-Aboriginal offenders, higher rates of dropout/expulsion were associated with greater risk on the SIR-R1, younger age, criminogenic need in the areas of marital/family and attitude, and lower motivation for intervention. Similarly, with the Aboriginal offenders, younger age, criminogenic need in the area of community functioning, and lower motivation were associated with higher dropout/expulsion rates. That risk, need, and motivation were associated with dropout/expulsion is consistent with previous research (Daly & Pelowski, 2000; Wormith & Olver, 2002).
These variables were combined to create a measure with which to estimate risk for dropout/expulsion. Variables were weighted based on their associated dropout/expulsion rates in the development sample. The total score on the composite measure was the sum of these weights. Two measures were created in this way: the DRS, for use with non-Aboriginal male offenders, and the ADRS, for use with Aboriginal male offenders. This approach to constructing the screening measures was modeled, in part, on the procedures used to create many widely applied risk assessment instruments, such as the Rapid Risk Assessment for Sexual Offense Recidivism (RRASOR; Hanson, 1997), the Statistical Information on Recidivism (SIR) scale (Nuffield, 1982), and the Violence Risk Appraisal Guide (VRAG; Quinsey, Harris, Rice, & Cormier, 1998).
Moderate predictive validity was demonstrated for the DRS among the non-Aboriginal male offenders. The same level of predictive accuracy was seen in both the developmental and cross-validation samples, which suggests that the DRS may be a robust predictor of dropout/expulsion. When the offenders in the cross-validation sample were grouped by offence type, the DRS was more accurate for the sex offenders than for the non-sex offenders. Even with the non-sex offenders, however, the DRS demonstrated a low to moderate level of accuracy. Thus, the DRS was a reasonably good predictor of dropout/expulsion and its performance was relatively stable across different samples.
In contrast to the moderate predictive validity demonstrated for the DRS, the measure designed to estimate risk of dropout/expulsion among Aboriginal offenders, the ADRS, achieved only a low level of accuracy in predicting dropout/expulsion in the cross-validation sample. In addition, the accuracy found for the ADRS with the cross-validation sample was significantly lower than with the development sample. The ADRS did not appear to be as stable across different samples as did the DRS. Overall, the performance of the ADRS was lacklustre, and we would not expect it to be of great practical value to evaluators in the field.
There are a number of reasons why our attempts to estimate dropout/expulsion risk may have been more successful with non-Aboriginal than Aboriginal offenders. First, it is possible that we failed to consider certain variables that would have been better predictors of dropout/expulsion among Aboriginal offenders. For example, some risk factors may be unique to Aboriginal offenders (Ellerby & MacPhersen, 2002), but because of cultural differences and the relatively small number of Aboriginal offenders, we may not yet be aware of these unique predictors or have the ability to address them.
A second related possibility is that valid measures are simply not as available for use with Aboriginal offenders as for non-Aboriginal offenders. For example, the SIR-R1 scale has good validity as a risk measure for non-Aboriginal male federal offenders. The SIR-R1 is not administered to Aboriginal offenders (SOP 700-04), however, and a comparable measure is not routinely used with Aboriginal offenders under federal jurisdiction. Individual variables with greater validity would be expected to create a composite measure of greater validity.
A third possibility, of a methodological nature, is that the smaller size of the Aboriginal sample may have been less suited to our procedure for developing the measures. For example, the observed relationships between the predictors and dropout/expulsion in the development sample were likely caused, to some degree, by chance. The extent to which this was the case would be affected by sample size. The larger the sample, the less likely the observed relationships were due to chance. Conversely, the smaller the sample, the greater the possibility that the observed relationships were due to chance. Thus, the association between the predictors and dropout/expulsion may have had more to do with chance in the Aboriginal sample than in the non-Aboriginal sample. This would explain why the ADRS appeared somewhat less stable than the DRS when applied to their respective cross-validation samples.
Whereas the ADRS performed poorly and does not seem appropriate for use in the field, the DRS or a similar type of measure could be of practical value. The predictive accuracy of the DRS in the non-Aboriginal cross-validation sample (AUC = .70) was comparable to the level of accuracy achieved by many instruments designed to estimate risk of recidivism. For example, Nafekh and Motiuk (2002) reported an AUC of .71, demonstrating moderate predictive accuracy of the SIR-R1 in predicting violent recidivism. Hanson and Thornton (2000) reported the same level of accuracy (AUC = .71) for the Static-99 in predicting sexual recidivism.
The accuracy of a measure like the DRS is complemented by its low cost of administration. Little effort would be required to score a measure like the DRS because all of the necessary data are readily available in OMS. Conceivably, the scoring could even be automated and integrated within OMS. In a system where high priorities compete for scarce resources, such measures may provide a low cost means of identifying those non-Aboriginal male offenders who may be at risk for dropping out of, or being expelled from, correctional programs. Prior to commencement of a program, the offenders at higher risk for dropout/expulsion could be more thoroughly assessed and, if warranted, targeted with pre-treatment efforts to increase their motivation and general readiness for treatment.
It must be emphasized that this measure was designed as a screening tool, which has implications for its application. The false positive rate, or the proportion of offenders who were classified as high risk but who did not actually dropout or get expelled, was very high for both the DRS and ADRS. For example, only 28% of the men in the cross-validation sample who were classified as high risk on the DRS dropped out. Thus, although higher scores on the DRS corresponded to higher dropout/expulsion rates, even offenders in the high-risk category were still more likely than not to complete their correctional programs. For screening purposes, this high false-positive rate is acceptable because the goal is to be over-inclusive. Offenders designated as high risk on the DRS could then be assessed more carefully (e.g., Serin & Kennedy, 1997), at which point only those still judged to be at risk of dropout/expulsion could be targeted, possibly with some form of pre-treatment. Because the DRS can be so easily scored, it would provide a low-effort and low-cost way to identify those offenders who may require further attention. This would permit a more efficient use of limited resources by directing them only to those who are most likely to benefit.
In contrast, a clear misuse of the DRS would be as an indicator of "treatability". It would be incorrect to interpret a high-risk score on the DRS as indicating that an offender is resistant to treatment or cannot successfully be treated. Given the high false-positive rate, it would be indefensible to deny treatment to an offender or to judge that an offender cannot benefit from treatment simply on the basis of his DRS score. As a screening measure, however, the DRS appears to have some utility.
Certain limitations of the current study should be noted. The generality of the risk, need, and motivation variables, which make up the DRS and ADRS, may have limited the predictive accuracy of these measures. Although we would speculate that dropout/expulsion is best predicted by general criminal rather than offence-specific risk factors, and we found that general risk, need, and motivation predicted dropout/expulsion more accurately among sex offenders than non-sex offenders, it is possible that predictive accuracy could have been improved if we had included more specific measures tailored to different types of offenders and programs. For example, perhaps using measures of risk and need specific to sexual recidivism, and motivation to participate in sex offender programs, would have made for a better predictor of dropout/expulsion among sex offenders in sex offender programs than would the DRS. This is an empirical issue and certainly warrants further research but was beyond the scope of the present study.
The purpose of the current research was to develop a screening measure that could be applied to as many offenders in as many circumstances as possible, regardless of their offence history or location. Despite our goal of developing a measure that could be used as widely as possible, however, we could not develop a measure of dropout/expulsion risk for women offenders. Data was available from too few women offenders to create such a measure for this group following the procedures used to develop the DRS and ADRS.
There are a number of areas that remain to be addressed. In the future, we will explore the extent to which dropout/expulsion risk is related to offence-specific risk and need versus general risk and need. Given the higher rates of dropout/expulsion generally found among Aboriginal offenders, research efforts should be directed at identifying at-risk Aboriginal offenders. Accurate dropout/expulsion risk assessment instruments could assist in the development and implementation of more effective strategies for retaining higher risk offenders in programs, and, ultimately, to greater reductions in recidivism. In addition, such measures could be used to improve the internal validity of treatment studies.
We developed two measures designed to predict dropout/expulsion from correctional programs among federally sentenced male offenders. One measure, the DRS, was designed for non-Aboriginal male offenders, and a second measure, the ADRS, for Aboriginal male offenders. We approached development and validation of the DRS and ADRS using generally accepted techniques commonly used in the creation of actuarial risk assessment instruments (Hanson, 1997; Nuffield, 1982; Quinsey, Harris, Rice, & Cormier, 1998). The measures drew from information that can be easily retrieved from OMS; thus, scoring these measures would not require a significant investment of resources. The DRS demonstrated moderate accuracy in estimating risk of dropout/expulsion among non-Aboriginal male offenders. The measure designed for Aboriginal male offenders (ADRS), however, did not perform well. At least in the case of the DRS, we demonstrated that it is possible to create a relatively simple screening measure to assist in identifying offenders who may be at risk for dropout/expulsion. The potential benefits of such a measure would likely outweigh the minimal costs associated with its use as a screening measure to identify offenders for whom more careful assessment of dropout/expulsion risk is warranted.
REFERENCES
Andrews, D. A., & Bonta, J. (2003). The psychology of criminal conduct (3rd ed.). Cincinnati, OH: Anderson.
Andrews, D. A., Zinger, I., Hoge, R. D., Bonta, J., Gendreau, P., Cullen, F. T., et al. (1990). Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology, 28, 369-404.
Beyko, M. J., & Wong, S. C. P. (2005). Predictors of treatment attrition as indicators for program improvement not offender shortcomings: A study of sex offender treatment attrition. Sexual Abuse: A Journal of Research and Treatment, 17, 375-389.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
Daly, J. E., & Pelowski, S. (2000). Predictors of dropout among men who batter: A review of studies with implications for research and practice. Violence and Victims, 15, 137-160.
Ellerby, L. A., & MacPhersen, P. (2002). Contrasting Aboriginal and non-Aboriginal sexual offenders to determine unique client characteristics and potential implications for sex offender assessment and treatment strategies (Research Report No. R-122). Ottawa, Canada: Correctional Service of Canada.
Gendreau, P., Little, T., & Goggin, C. (1996). A meta-analysis of the predictors of adult offender recidivism: What works! Criminology, 34, 575-607.
Hanson, R. K. (1997). The development of a brief actuarial risk scale for sexual offense recidivism. (User Report 97-04). Ottawa: Department of the Solicitor General of Canada.
Hanson, R. K., & Bussière, M. T. (1998). Predicting relapse: A meta-analysis of sexual offender recidivism studies. Journal of Consulting and Clinical Psychology, 66, 348-362.
Hanson, R. K., Gordon, A., Harris, A. J. R., Marques, J. K., Murphy, W., Quinsey, V. L., & Seto, M. D. (2002). First report of the collaborative outcome data project on the effectiveness of psychological treatment for sex offenders. Sexual Abuse: A Journal of Research and Treatment, 14, 169-194.
Hanson, R. K., & Morton-Bourgon, K. (2004). Predictors of sexual recidivism: An updated meta-analysis. Ottawa: Public Safety and Emergency Preparedness Canada.
Hanson, R. K., & Thornton, D. (2000). Improving risk assessments for sex offenders: A comparison of three actuarial scales. Law and Human Behavior, 24, 119-136.
Krawczyk, S. C., Witte, T., Gordon, A., Wong, S., & Wormith, J. S. (2002, May). Treatment attrition and institutional offending in violent offenders. Poster session presented at the Canadian Psychological Association's 63rd Annual Convention, Vancouver, B.C.
Langton, C. M., Barbaree, H. E., Harkins, L., & Peacock, E. J. (2003, October). Failure to complete sexual offender treatment and sexual recidivism: Does actuarial risk or psychopathy account for the association? In H. Barbaree (Chair), Alternative approaches to evaluating treatment efficacy with sexual offenders. Symposium conducted at the 22nd Annual Research and Treatment Conference of the Association for the Treatment of Sexual Abusers, St. Louis, MO.
McKenzie, K., Witte, T., Beyko, M., Wong, S., Olver, M., & Wormith, J. S. (2002, May). Predictors of attrition in a sex offender program. Poster session presented at the Canadian Psychological Association's 63rd Annual Convention, Vancouver, B.C.
Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., Eisman, E. J., Kubiszyn, T. W., & Reed, G. M. (2003). Psychological testing and psychological assessment: A review of evidence and issues. In A. E. Kazdin (Ed.), Methodological issues and strategies in clinical research (3rd ed.) (pp. 265-345). Washington, DC: American Psychological Association.
Motiuk, L.L. (1997). Classification for correctional programming: The Offender Intake Assessment (OIA) process. Forum on Corrections Research, 9 (1), 18-22.
Nafekh, M., & Motiuk, L. L. (2002). The Statistical Information on Recidivism - Revised 1 (SIR-R1) Scale: A psychometric examination (Research Report No. R-126). Ottawa, Canada: Correctional Service Canada.
Nuffield, J. (1982). Parole decision making in Canada: Research towards decision guidelines. Ottawa: Solicitor General of Canada.
Nunes, K. L., & Cortoni, F. A. (2006). The heterogeneity of treatment non-completers (Research Report No. R-176). Ottawa, Canada: Correctional Service Canada.
Prochaska, J. O., & DiClemente, C. C. (1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research and Practice, 19, 276-288.
Prochaska, J. O., DiClemente, C. C., & Norcross, J. C. (1992). In search of how people change: Applications to the addictive behaviors. American Psychologist, 47, 1102-1114.
Proulx, J., Brien, T., Ciampi, A., Allaier, J-F., McDonald, M., & Chouinard, A. (2004, October). Treatment attrition in sexual aggressors. Paper presented at the 23rd Annual Research and Treatment Conference of the Association for the Treatment of Sexual Abusers, Albuquerque, NM.
Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998). Violent offenders: Appraising and managing risk. Washington, DC: American Psychological Association.
Rice, M. E., & Harris, G. T. (1995). Violent recidivism: Assessing predictive validity. Journal of Consulting and Clinical Psychology, 63, 737-748.
Scott, K. L. (2004). Stage of change as a predictor of attrition among men in a batterer treatment program. Journal of Family Violence, 19, 37-47.
Seager, J. A., Jellicoe, D., & Dhaliwal, G. K. (2004). Refusers, dropouts, and completers: Measuring sex offender treatment efficacy. International Journal of Offender Therapy and Comparative Criminology, 48, 600-612.
Serin, R., & Kennedy, S. (1997). Treatment readiness and responsivity: Contributing to effective correctional programming (Research Report No. R-54). Ottawa, Canada: Correctional Service of Canada.
Stewart, L., & Millson, W. A. (1995). Offender motivation for treatment as a responsivity factor. Forum on Corrections Research, 7 (3), 5-7.
Van Voorhis, P., Spruance, L. M., Ritchey, P. N., Listwan, S. J., & Seabrook, R. (2004). The Georgia cognitive skills experiment: A replication of reasoning and rehabilitation. Criminal Justice and Behavior, 31, 282-305.
Wilson, D. B., Bouffard, L. A., & MacKenzie, D. L. (2005). A quantitative review of structured, group-oriented, cognitive-behavioral programs for offenders. Criminal Justice and Behavior, 32, 172-204.
Wormith, J. S., & Olver, M. E. (2002). Offender treatment attrition and its relationship with risk, responsivity, and recidivism. Criminal Justice and Behavior, 29, 447-471.
