FORUM on Corrections Research

Yes, SIR! A stable risk prediction tool

In the 1960s and early 1970s, parole was under attack in the United States. But some supporters of parole argued that offenders could be differentiated in their risk of reoffending, and that corrections officials were taking these variables into account when making parole decisions. The first step was to establish empirical predictors of recidivism that could assist decision makers. This research led to the development of various actuarial risk assessment instruments, notably the Base Expectancy Scale² and the Salient Factor Score.³ Similar efforts in the United Kingdom produced the Parole Prediction Scoring System.⁴

In Canada, research to develop a risk prediction scale for the federal system was undertaken about 20 years ago by Joan Nuffield,⁵ working in the Ministry Secretariat of the Solicitor General of Canada. Nuffield selected a random sample of 2,745 offenders released in the years 1970, 1971 and 1972, and who had entered penitentiary as a result of new offences (excluding revocations and provincial transfers). Information on the current offence, previous criminal history and social/demographic characteristics was collected from the Inmate Records System. In addition, records of arrest and reconviction were obtained from the Canadian Police Information Centre. Reconviction for an indictable offence within three years of parole was used as the measure of general recidivism.

Using the data from half of the cases (the construction sample), Nuffield examined the statistical relationship between the criminal and social/demographic variables and recidivism. She constructed a scale using a simple summation technique where the weighting for a given item is based on the difference between the recidivism rate for offenders with that characteristic and the overall recidivism rate. The scores on the scale ranged from -27 to +30. She then grouped these scores to form five clusters of roughly equal size, with the probability of a successful outcome ranging from 84% (approximately four out of five) for the lowest risk group to 32% (one out of three) for the highest risk, and intermediate probabilities of success for the three groups in between. Nuffield then applied this scale to the second half of the cases (the validation sample), and the results showed that the predicted outcomes for each group held up very well.

Although the scale has been referred to by some authors as the Nuffield Scale or the Risk Prediction Scale, it was officially named the General Statistical Information on Recidivism Scale, or more commonly, the SIR Scale. It was formally introduced in 1988 by the National Parole Board and the Correctional Service of Canada as a component of prerelease decision policies for the assessment of risk. The scoring system of the original SIR Scale instrument was changed by multiplying all the scores by -1 so that positive scores were associated with "good risk" cases and negative scores with "poor risk" cases. This was thought to render the scale more intuitive but, of course, did not change its fundamental properties.

The scale (see next page) combines 15 static factors related to criminal activity and social functioning.

Revalidation and uses in research

There is now a substantial body of research literature confirming the ability of the SIR Scale to differentiate between high risk and low risk cases among federal offenders. In the mid-1980s, the Ministry Secretariat undertook further research on parole decision making and release risk assessment. Although the original focus of the research was primarily on the decision-making process, it was also aimed at improving our ability to predict risk of reoffending. A major part of the research involved a revalidation of the SIR Scale by Hann and Harman.⁶ Data were collected through a manual review of inmate files on a sample of 534 male penitentiary inmates released in 1983-1984, who had been admitted on simple warrants of committal. Recidivism was defined as a conviction that resulted in a custodial sentence, and the follow-up period was 2.5 years, compared with 3 years in the original research. Hann and Harman calculated SIR scores for each subject in their sample, and placed each offender in the designated groups. They then calculated the success rate for each group and compared the rates to those obtained by Nuffield.⁷ The success rates showed very similar patterns, indicating that the scale was still able to discriminate high risk cases from low risk cases

Table 1

General Statistical Information on Recidivism (SIR Scale)

1. Current Offence

2. Age at Admission

3. Previous Incarceration

4. Revocation of Forfeiture

5. Act of Escape

6. Security Classification

7. Age at First Adult Conviction

8. Previous Convictions of Assault

9. Marital Status at Most Recent Admission

10. Interval at Risk since Last Offence

11. Number of Dependents at Most Recent Admission

12. Current Total Aggregate Sentence

13. Previous Convictions for Sex Offences

14. Previous Convictions for Break and Enter

15. Employment Status at Arrest

Wormith and Goldstone⁸ studied a random sample of 203 male inmates released from penitentiaries in the Prairie region. These researchers computed a SIR score for each offender, and the success rates for each group matched closely the results for the national sample. Porporino, Zamble and Higgonbottom⁹ tested the SIR scale on a sample of 77 male offenders released in the Ontario region and confirmed the predictive power of the instrument. Serin¹⁰ reported a significant correlation between SIR scores and general recidivism for a sample of 81 offenders released from minimum and medium security penitentiaries in the Ontario region.

Most recently, Bonta et al.¹¹ reported the results of a revalidation of the SIR Scale based on the full sample of inmates (3,267) released in 1983-1984 who had been admitted on warrants of committal. Recidivism in this study was defined as a custodial admission within three years of release including revocations. Again, the pattern of results across risk categories was remarkably similar to the original findings, confirming the validity of the SIR Scale for the prediction of general recidivism. This study also confirmed the present use of cutoff scores that define the five risk categories.

The results of several other studies lend additional support to the validity of the SIR Scale. For example, Johnson and Motiuk,¹² in their study of factors related to unlawful walkaways from minimum security institutions, calculated SIR scores and found that 80% were in the "poor risk" category and that there were no offenders in the "good" and "very good" categories. In a further comparison of walkaways with a matched sample of similarly situated offenders who had remained in custody,¹³ the walkaways had higher risk scores on the SIR Scale than the non-walkaways.

Because of its established predictive validity, the SIR Scale has been useful in research to control for risk. For example, Motiuk and Belcourt¹⁴ examined the relationship between prison work programs and postrelease outcome, and their findings confirmed that the SIR Scale was significantly related to readmission and conviction for any new offence. They also used the SIR score as a control for risk by comparing the outcomes of participants in the CORCAN prison industry program with the expected outcomes based on their SIR scores. The results indicated that the outcomes of these participants were as good as or better than expected on the basis of the SIR Scale. This suggests that participation in the work programs may have contributed to improved postrelease outcomes.

The SIR Scale has been used to assess the differential impact of treatment on offenders with different base risk levels. Robinson¹⁵ devised a proxy for the SIR Scale to examine the impact of cognitive skills training on postrelease recidivism on higher risk and lower risk cases. He found a treatment effect (a lower recidivism rate for the treatment group as compared with the control group) for the lower risk cases but not for the higher risk cases.

Limitations of the SIR Scale

As with any instrument, the SIR Scale has limitations. In particular, concerns about its limitations relate to the prediction of violence, its use with female, Aboriginal and sex offenders, and its "static" nature. It is important to remember that the SIR Scale was developed to predict general recidivism in a population of male penitentiary inmates. Nuffield¹⁶ was unable to develop a separate scale to predict violent reoffending for several reasons, including the low base rate of violent reoffending in her sample.

Other studies have also found that the SIR Scale fails to predict violent recidivism. For example, Serin examined a sample of 81 male offenders released from penitentiaries in the Ontario region, with an average follow-up period of 30 months. The overall violent recidivism rate was 10%, and there was no correlation between SIR scores and violent recidivism. However, Bonta and Hanson,¹⁷ with a large sample (2,377), a long follow-up period (11 years) and a violent recidivism rate (including robbery) of almost 50%, found a correlation between the SIR scores and violent recidivism. Still, the discrimination across categories was not as great as that found in studies focusing on general recidivism.

Since the SIR Scale was developed on male offenders, the question has been raised regarding its validity as a predictor of risk for federal female offenders. Hann and Harman¹⁸ conducted some preliminary analyses on a small sample of female offenders released from penitentiary in 1983-1984, but the evidence was inconclusive. A further study by Bonta, Pang and Wallace-Capretta¹⁹ also found limited evidence to support the use of the SIR Scale with female offenders. There was a correlation between the total SIR scores and recidivism, but the recidivism rates did not match the risk categories as they consistently do for male offenders. Blanchette²⁰ also found a correlation between total score and general recidivism, but did not report the results by category. Given the sample size (ranging from 59 to 81) in all three studies, the conclusion, at this time, is that the jury is still out on whether the SIR Scale will prove useful in the prediction of recidivism for female offenders.

The issue has also arisen as to whether the scale is valid for Aboriginal offenders. A preliminary test of the SIR Scale on a small sample of Aboriginal male offenders showed a general correspondence between risk category and recidivism, but the relationship was not as strong as that found for a non-Aboriginal male sample. In particular, Hann and Harman²¹ reported large deviations at the high risk end for Aboriginal offenders. A subsequent analysis involving a larger sample (269) of Aboriginal male offenders showed a closer correspondence between SIR score and recidivism outcome.²² However, given the small amount of research examining the SIR Scale with Aboriginal offenders, and considering the gaps in our knowledge of cross-cultural assessment, there is need for caution in this area.

Another issue surrounding the SIR Scale concerns its application to sexual offenders. Although Bonta and Hanson²³ showed that the SIR Scale predicted general recidivism and non-sexual violent recidivism among sexual offenders with reasonable accuracy, it was unable to predict sexual recidivism. Clearly, there are compelling arguments to support the view that sexual offending is different from other crimes²⁴ and, fortunately, there are other instruments being developed to help predict sexual reoffending. Hanson and Bussière²⁵ recently reported the results of a meta-analysis which showed that the most powerful predictors of sexual recidivism were measures of sexual deviance, including phallometric assessments of sexual preferences for children and previous sexual offences. These variables provide the basis for developing appropriate scales for the prediction of sexual offending. The conclusion here is that the SIR Scale can be used with sexual offenders as a measure of risk of general recidivism but not to predict sexual recidivism.

The observation is often made that the SIR Scale is composed of "static" risk factors. This is a serious limitation since it means that the SIR Scale cannot provide targets for treatment interventions or for the possibility of measuring changes in risk over time. However, other instruments such as the Level of Service Inventory²⁶ and the Risk/Needs Scale can be used to tap "dynamic" risk/needs factors (that is, associates, alcohol/drug abuse, employment). This, of course, does not diminish the fact that the SIR Scale provides a basic, general measure of risk²⁷ and that it would be imprudent to ignore a finding of high risk on the SIR Scale irrespective of the results of other assessments.

The SIR Scale could benefit from a little fine tuning. For example, analysis of individual items by Bonta et al.²⁸ found one item -- previous convictions for sexual offence -- that did not correlate with general recidivism. Consideration could be given to removing this item and recalibrating the instrument. However, one could argue that resources might be better spent on developing other specialized tools and other lines of research on prediction and control of criminal behaviour, rather than refining an established one. Furthermore, a recalibration of this sort would have very little impact on the overall performance of the instrument.

Conclusion

The SIR Scale pioneered the use of risk assessment instruments in corrections in Canada. Today, there is overwhelming consensus that empirically based risk assessment is central to sound correctional practice -- it is hard to believe that only 20 years ago, when the research was launched to develop a risk assessment tool, this was not the case. As we move to new generations of dynamic risk assessment tools, the SIR Scale remains a solid instrument for predicting the risk of general recidivism in the federal offender population. In this sense, it is an important tool and, given its ease of administration, still warrants a place in the practitioner's tool kit.

---

1. Department of the Solicitor General, 340 Laurier Avenue West, Ottawa, Ontario, K1A 0P8.

2. D.M. Gottfredson and J.A. Bonds, A Manual for Intake Base Expectancy Scoring (San Francisco: California Department of Corrections, Research Division, 1961).

3. P. Hoffman and J.L. Beck, "Parole Decision-making: A Salient Factor Score," Journal of Criminal Justice, 2 (1974): 195-206.

4. C.P. Nuttall, Parole in England and Wales (London: H.M.S.O., 1977).

5. J. Nuffield, Parole Decision-making in Canada: Research towards Decision Guidelines (Ottawa: Ministry of Supply and Services Canada, 1982). See also J. Nuffield, "The 'SIR Scale': Some Reflections on its Applications," Forum on Corrections Research, 1, 2 (1989), 19-22.

6. R.G. Hann and W.G. Harman, Release Risk Prediction: A Test of the Nuffield Scoring System, User Report No. 5 (Ottawa: Solicitor General Canada, 1989).

7. Nuffield, Parole Decision-making in Canada.

8. J.S. Wormith and C.S. Goldstone, "The Clinical and Statistical Prediction of Recidivism," Criminal Justice and Behavior, 11 (1984): 3-34.

9. F. Porporino, E. Zamble and Higgonbottom, unpublished study, cited in F.J. Porporino, The Statistical Information for Recidivism Scale, Research Brief No. B-01 (Ottawa: Correctional Service Canada, 1989).

10. R.C. Serin, "Violent Recidivism in Criminal Psychopaths," Law and Human Behaviour, 20 (1996): 207-217.

11. J. Bonta, W.G. Harman, R.G. Hann and R.B. Cormier, "The Prediction of Recidivism among Federally Sentenced Offenders: A Re-validation of the SIR Scale," Canadian Journal of Criminology, 38 (January 1996): 61-79.

12. J.C. Johnston and L.L. Motiuk, Factors Related to Unlawful Walkaways from Minimum Security Institutions, Research Report No. R-23 (Ottawa: Correctional Service of Canada, 1992).

13. J.C. Johnston and L.L. Motiuk, Unlawful Departures from Minimum Security Institutions: A Comparative Investigation, Research Report No. R-27 (Ottawa: Correctional Service of Canada, 1992).

14. L.L. Motiuk and R.L. Belcourt, Prison Work Programs and Post-release Outcome: A Preliminary Investigation, Research Report No. R-43 (Ottawa: Correctional Service of Canada, 1996).

15. D. Robinson, The Impact of Cognitive Skills Training on Post-release Recidivism among Canadian Federal Offenders, Research Report No. R-41 (Ottawa: Correctional Service of Canada, 1995).

16. Nuffield, Parole Decision-making in Canada.

17. J. Bonta and R.K. Hanson, "Violent Recidivism of Men Released from Prison," Paper presented at the American Psychological Association Convention (New York, August, 1995).

18. R.G. Hann and W.G. Harman, Release Risk Prediction: Testing the Nuffield Scoring System for Native and Female Inmates, User Report No. 4 (Ottawa: Solicitor General Canada, 1989).

19. J. Bonta, B. Pang and S. Wallace-Capretta, "Predictors of Recidivism among Incarcerated Female Offenders," The Prison Journal, 75 (1995): 277-294.

20. K. Blanchette, "The Relationships between Criminal History, Mental Disorder, and Recidivism among Federal Sentenced Female Offenders," M.A. thesis, Department of Psychology, Carleton University (Ottawa, January, 1996).

21. Hann and Harman, Release Risk Prediction: Testing the Nuffield Scoring System for Native and Female Inmates.

22. R.G. Hann and W.G. Harman, Predicting Release Risk for Penitentiary Inmates, User Report No. 12 (Ottawa: Solicitor General Canada. 1993).

23. J. Bonta and R.K. Hanson, "10-year Recidivism of Canadian Federal Offenders," Unpublished, reported in Hanson and Bussière (see note 25).

24. R.K. Hanson, H. Scott and R.A. Steffy, "A Comparison of Child Molesters and Non-sexual Criminals: Risk Predictors and Long-term Recidivism," Journal of Research in Crime and Delinquency, 32 (1995): 325-337.

25. R.K. Hanson and M.T. Bussière, Predictors of Sexual Offender Recidivism: A Meta-analysis, User Report No. 04 (Ottawa: Solicitor General Canada, 1996).

26. D.A. Andrews and J. Bonta, The Level of Service Inventory - Revised (Toronto: Multi-Health Systems, 1995).

27. L.L. Motiuk and F.J. Porporino, Field Test of the Community Risk/Needs Management Scale: A Study of Offenders on Caseload, Research Report No. R-06 (Ottawa: Correctional Service Canada, 1989).

28. Bonta, Harman, Hann and Cormier, The Prediction of Recidivism among Federally Sentenced Offenders.