FORUM on Corrections Research

Using proxy measures for correctional research

Mark Nafekh¹

Research Branch, Correctional Service of Canada

The Correctional Service of Canada (CSC) exercises reasonable, safe, secure and humane control in its endeavours to protect society and assist offenders to reintegrate into the community. In employing the least restrictive measures consistent with this commitment, CSC assesses offenders’ risk to themselves, other offenders, staff members, and society. This is established via professional experiences and judgments, and the use of validated actuarial tools throughout the course of an offender’s sentence. This article reviews research techniques employed for the empirically based contribution to CSC decision making, focusing on procedures of estimation and approximation.

This article outlines techniques of estimation and approximation in;

1. forecasting the federal offender population, and
2. the psychometric examination of actuarial tools that predict static risk. For both cases, empirical techniques are reviewed with respect to their contributions to the Correctional Service of Canada’s (CSC’s) operational-related endeavours.

Estimation and offender population forecasting

For purposes of the National Capital Accommodation and Operational Plan (NCAOP), CSC’s Research Branch uses standard time-series modelling techniques to develop medium-term (five-year) offender population projections. Generally, the NCAOP identifies money and accommodation needs for the foreseeable future. Further, to assist CSC in efficiently accommodating the operational and programming needs specific to non-Aboriginal men, women and Aboriginal offenders, the Research Branch conducted population predictions for these specific groups in the most recent medium-term forecasts.² Over time, changes in technology, legislation and operations affected the way in which data were represented, aggregated, and stored. Following is a brief discussion outlining the specific data issues, and techniques used that reduce the impacts of these changes.

Weekly inmate counts dating back to 1979 are recorded at federal institutions, and housed in a data base referred to as the Inmate Movement System (IMS). These counts represent the total number of offenders physically in, plus outside court, hospital and temporary absences, as well as those under exchange of service agreements (ESAs) with the provinces and territories. Data in the IMS are aggregated at the institution level. As such, estimation techniques are required to break out the aggregated counts to those that fall into the non-Aboriginal men, Aboriginal men, and women groupings. Two sources of information can be used to accomplish this; the Offender Management System (OMS) and the Offender Population Profile System (OPPS).

Since 1994, the Research branch has retrieved offender data for all those incarcerated on the first day of each month — also referred to as “snapshots”. These data are retrieved from CSC’s Offender Management System (OMS). Since snapshot data are offender specific, demographic information such as gender and ethnicity can be tagged to individual offenders. This is done by matching data on unique offender identifiers, such as fingerprint serial numbers (FPS) or offender identification (OID) numbers. The data can then be aggregated at the institution level, and a monthly time series reflecting the proportion of offenders in the above noted groupings can be created. Next, as the process of creating the monthly research snapshots is not automated and is also dependent on the time at which OMS information is ‘refreshed’, there exist gaps in the time series. There are a number of ways to fill the gaps in the series. One method is to fit a statistical model to the time series so that any trends affecting the data are represented in the time series. For instance, legislative changes that increase sentences for particular crimes would result in an accumulation of those offenders with that sentence, and thus over time, an increase in their proportion of the federal incarcerated offender population.

From 1982 to 1994, inmate counts were recorded as part of the OPPS. These data are in the form of hard-copy reports and, for the forecasting purposes, can be recorded into an electronic data base. Similar to the IMS data series, the OPPS data are aggregated over total number of offenders. However, the OPPS data are aggregated within the three groupings of interest. Thus, this information can be married to the snapshot-based OMS series, then applied to the IMS data series such that the total number of offenders is split into total male non-Aboriginal, total male Aboriginal and the total number of women offenders. The SAS statistical software program, commonly utilized by the Research Branch for analytical purposes, can then be used to fit a variety of models to the time series data, by region, for women and non-Aboriginal and Aboriginal males. Finally, the model of best fit can be determined to establish five-year projections for each of the three offender groups.

Approximation of the SIR-R1 scale

The Statistical Information on Recidivism — Revised 1 (SIR-R1) scale combines 15 items in a scoring system that yields probability estimates of re-offending within three years of release. In 2002, CSC’s Research branch examined the reliability, predictive validity and practical utility of the SIR-R1 scale.³ One component of the analyses involved examining the scale for use on women and Aboriginal offenders. Currently, the SIR-R1 is not administered to federally sentenced women and Aboriginal offenders.⁴ As such, a proximal measure of the SIR-R1 scale, the SIR-Proxy, was developed for the investigation.

The SIR-Proxy was developed from offender-specific information routinely collected upon admission to a federal institution. The primary source of information used to develop the SIR-Proxy was data derived from the Criminal Risk Assessment (CRA) and Dynamic Factor Identification and Analysis (DFIA) components of the Offender Intake Assessment process. The OIA is a comprehensive and integrated evaluation of the offender at the time of admission to the federal system.⁵ It involves the collection and analysis of information on each offender’s criminal and mental health history, social situation, education, and other factors relevant to determining criminal risk and identifying offender needs. The CRA component of the OIA provides specific information pertaining to past and current offences. The CRA is based primarily on the criminal history record but may also include case-specific information regarding any other pertinent details pertaining to individual risk factors. The DFIA involves the identification of the offender’s criminogenic needs. More specifically, it considers a wide assortment of case-specific aspects of the offender’s personality and life circumstances, and data are clustered into seven target domains with multiple indicators for each: employment (35 indicators), marital/family (31 indicators), associates/social interaction (11 indicators), substance abuse (29 indicators), community functioning (21 indicators), personal/emotional orientation (46 indicators), and attitude (24 indicators).⁶

To develop the SIR-Proxy, the 15 items of the SIR-R1 were matched to specific dichotomous OIA indicators. Endorsed OIA items were given the equivalent SIR-R1 score. For example, SIR-R1 item 15 — (employment status at arrest) was scored accordingly on the SIR-Proxy with a +1 if item 16 in the employment domain of OIA (was employed at time of arrest) was endorsed. Assessing the proximity of the scale to SIR-R1 ratings can be accomplished by comparing proxy scores to the actual scores of male non-Aboriginal offenders with a completed SIR-R1 scale. In this particular example, the SIR-Proxy was found to be highly correlated to the SIR-R1 (r = 0.90).

Subsequently, the Proxy scale’s performance can be assessed through tests of reliability, predictive validity and practical utility, and compared to those same measures for the SIR-R1. Internal consistency of the SIR-Proxy can be tested using Cronbach’s alpha reliability coefficient. Results showed the scale to be reliable (alpha = 0.78). Next, predictive validity of the SIR-Proxy can be examined using Receiver Operating Characteristic (ROC) analysis. This type of analysis is used to calculate true positive and false positive rates for the SIR Proxy. Plotting the associated rates along an XY axis produced an ROC curve. The “area under the curve” or AUC (between 0 and 1) measures the probability that non-recidivists would score higher on the SIR-R1 scale than recidivists. An AUC of 1 indicates perfect discrimination between recidivists and non-recidivists, while an AUC of 0.5 or less indicates the scale has no power to discriminate. AUC results for the SIR-Proxy on federally sentenced male non-Aboriginal offenders were found to be good at 0.752.

Finally, Prevalence-Value Accuracy (PVA) analyses test the practical utility of a measure. Practical utility is evaluated by incorporating outcome rates and the cost of misclassifications into a quantifiable formula. In the study, this formula was a function of general recidivism rates and associated costs of false-positive and false-negative predictions. By plotting minimum misclassification over a range of success rate and false-positive/false-negative ratio combinations, PVA analysis derives a cost-surface. Analogous to the area under the curve (AUC) for ROC analysis, the volume beneath this cost surface (cost-volume index) is an index of test performance.⁷ A perfect test would have no misclassification costs and would therefore have a volume of 0. Results of the study showed no significant differences between the cost-volume indexes of the SIR-R1 and SIR-Proxy. Given the SIR-Proxy’s high correlation to the SIR-R1, it was not surprising that the SIR-Proxy fared the same or better on all performance tests.

Discussion

The use of estimation and approximation measures throughout the course of research facilitates the Correctional Service of Canada in meeting its operational related endeavours. Specifically, the appropriate application of statistical techniques to fill data gaps enhances the accuracy of population forecasts, aiding the decision processes concerned with capital and accommodation planning. Also, the implementation of a proxy measure for an actuarial tool such as the SIR-Proxy could increase efficiency and save operating costs for the Service. As the SIR-Proxy was derived primarily from the Offender Intake Assessment data base in CSC’s Offender Management System, it is conceivable that the SIR-R1 be replaced by the SIR-Proxy via an automated process. This would reduce the workload for Case Management Teams, and improve predictive accuracy with respect to post release outcome.