Exploring Inconsistencies between Actuarial Measures and Final Decisions

What we looked at

The Correctional Service of Canada (CSC) uses actuarial measures and structured assessments for case management, program and other treatment referrals, and pre-release decision making (Motiuk, 1997). In some cases, final case management decisions may not correspond to the results from these measures due to the consideration of individual factors (this is sometimes called an “override”).

Given the implications of accurately evaluating risk, the literature on assessments resulting in final decisions inconsistent with actuarial assessments was examined.

What we found

While it is well-accepted that there are occasions when staff must make case management decisions based on an offender’s individual qualities rather than solely on assessment results (Andrews & Bonta, 2010), this approach should be applied only where appropriate. Unstructured professional judgment tends to lead to more conservative, less transparent, and less replicable decisions than those based on actuarial measures (Bonta & Motiuk, 1990; Grove et al., 2000). However, when professional judgment is structured, research shows that accuracy is similar to that of actuarial measures and supports its incorporation in decision-making (e.g., Campbell, French, & Gendreau, 2009).

It is important that staff recognize the circumstances that are appropriate for the incorporation of individual factors. For instance, Austin, Johnson, and Weitzer (2005) encourage jurisdictions to abide by a general standard where only 5% to 15% of final assessments differ from initial actuarial scores. Furthermore, these authors suggest that the direction of inconsistencies should be balanced, where half are higher and half are lower than the original actuarial score.

What it means

Risk assessment literature shows that final decisions that are inconsistent with actuarial scores tend to be less predictive and more conservative than the actuarial scores themselves; therefore, decisions that are not in agreement with these scores should be applied judiciously. Moreover, in order to maximize the accuracy of decisions rooted in professional judgment, this judgment should be guided by policy.

Recognizing the importance of accurate assessment, particularly with respect to offender risk management, a thorough understanding of these inconsistencies is necessary. Research is currently underway to further examine their nature and use at CSC.

References

Andrews, D. A., & Bonta, J. (2010).  The psychology of criminal conduct (5^e éd.).   Newark, NJ: LexisNexis.

Austin, J., Johnson, K. D., & Weitzer, R. (2005). Alternatives to Secure Detention and Confinement of Juvenile Offenders. Office of Justice Programs.

Bonta, J., & Motiuk, L. L. (1990). Classification to halfway houses: A quasi-experimental evaluation. Criminology,vol. 23, n^o 3, p. 497 à 506.

Campbell, M. A., French, S., & Gendreau, P. (2009).  The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment.  Criminal Justice and Behavior, 36, 567-590.

Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12, 19-30.

Motiuk, L. L. (1997). Classification of correctional programming: The Offender Intake Assessment (OIA) process. Forum on Corrections Research, 9, n^o 1, p. 19 à 22.

For more information

Please e-mail the Research Branch or contact us by phone at (613) 995-3975.

You can also visit the website for a full list of research publications.

Prepared by: Mary B. Ritchie, Shanna Farrell MacDonald & Renée Gobeil