Research Reports

Pilot Implementation Of A Custody Rating Scale: Interim Report

The points of view expressed in this research report are those of the authors and do not necessarily reflect the views of the Correctional Service of Canada.
R-02

Prepared by:

Frank Porporino,
Fred Luciani,
Laurence Motiuk,
Michael Johnston,
Brian Mainwaring

Research Branch
Correctional Service Canada

JANUARY 1989

TABLE OF CONTENTS

I. INTRODUCTION

II. DESCRIPTION OF PILOT IMPLEMENTATION

III. PRELIMINARY FINDINGS

IV. DISCUSSION

The development and pilot implementation of the Custody Rating Scale was initiated in collaboration with the Offender Management Division of Correctional Programs and Operations. Terry Sawatsly deserves acknowledgement for anticipating the need for a new offender classification tool, and for encouraging and facilitating the research. Data collection for the pilot testing of the Scale was coordinated by Michael Johnston, and the review and coding of information was completed by Fred Luciani and Brian Mainwaring. Laurence Motiuk assisted in conducting all the statistical analyses. Case management Staff who participated in the Pilot Implementation in the Pacific and Quebec Regions deserve a special thanks for their cooperation.

I. INTRODUCTION

In June 1988, a pilot implementation of a new Custody Rating Scale was approved by the Senior Management Committee of the Correctional Service of Canada. Approval was obtained to integrate the Scale as a component of the Penitentiary Placement Process in the Quebec and Pacific regions (centralized and decentralized systems respectively). All initial placement decisions in these two regions were to be made using the Custody Rating Scale as a replacement for the Benchmark Codes found in the Penitentiary Placement Report. An assessment of the effects of adopting this new classification tool was to be conducted, and this was to include in particular:

• analysis of the implementation process ... to determine how the scale would be accepted by case management staff; and
• an analysis of the impact on classification decisions...to determine the usefulness and appropriateness of the Scale in classifying inmates to the lowest suitable level of security.

A full report outlining what was learned from the Pilot implementation of the Custody Rating Scale will be prepared for review at the March meeting of the Senior Management Committee. This will include a three-month follow-up, to examine institutional behaviour and adjustment, of all cases assessed with the Custody Rating Scale during the first three months of pilot implementation. Although the validity of the scale has been established in an initial research phase, with a retrospective analysis of a randomly selected sample of 575 cases followed-up over a period of two years, an actual pilot application of the instrument was considered essential before considering national implementation. The full report on the pilot will assess the need for any adjustments or refinements of the Scale.

The present interim report describes the staff training and quality-control monitoring that has occurred during the pilot, provides an overview of the degree of staff acceptance of the instrument, and examines some preliminary data on the degree of concordance between actual placement decisions and security level designations derived from the Custody Rating Scale.

II. DESCRIPTION OF PILOT IMPLEMENTATION

Application of the Custody Rating Scale on a pilot basis was initiated in the Pacific Region on July 15, 1988 and in the Quebec Region on August 1, 1988.

In consultation with the Regions, it was decided to start-up the pilot with one-day training workshops for all field staff who would be applying the Scale. A User's Guide was developed to provide detailed instructions on how to complete the Scale and to give definitions and explanations for each item to ensure consistency and uniformity of application.

The training workshops were conducted by the project coordinators, F. Porporino and M. Johnston, on June 24 and July 26 in the Pacific and Quebec Regions respectively. In the Pacific Region, all Community Case Management Officers responsible for the penitentiary placement of offenders received the training. In the Quebec Region, all Case Management Officers at the Regional Reception Centre were trained.

The training workshops consisted of two segments. First, the participants were given an overview of the research underlying the development of the Custody Rating Scale. This included a discussion of the advantages of adopting an objective research-based instrument as a classification tool. With the User's Guide as a framework, a step-by-step instruction session was then provided to ensure an understanding of the actual application of the instrument. Participants also worked through several sample cases, reviewing copies of documentation from actual offender files and discussing any disagreement in the scoring of individual items on the Scale.

Although considerable discussion ensued on the merits of adopting an actuarial instrument for classification, the workshops were viewed as generally useful. The major concerns that were raised related to workload, the unavailability of essential information on offenders, and the perceived encroachment on the professional judgement of the case management officer. These concerns are perhaps more "reflexive" than real, and though addressed in the workshops, they do remain somewhat outstanding.

Data Gathering:

Following the workshops, the Case Management Officers participating in the Pilot were instructed to forward copies of all completed Custody Rating Scales (see Appendix A) and supporting documentation (i.e., Penitentiary Placement Reports and F.P.S. records) to the Offender Management division at National Headquarters. They were also requested to complete a brief form to assess their level of agreement with the Scale and to determine the extent of unavailable information on each case that was assessed (see Appendix B).

Through correspondence, telephone conversations with individual Case Management Officers, as well as several conference calls, ongoing feedback was provided to clarify concerns relating to use of the Scale. In addition, quality control procedures were instituted at National Headquarters, and all cases were reviewed by Offender Management and Research staff to ensure the accurate completion of the documentation and the proper scoring of the Custody Rating Scale.

III. PRELIMINARY FINDINGS

Sample of Cases Reviewed:

For this interim report, all penitentiary placements completed in the Quebec and Pacific regions between August 15, 1988 and October 15, 1988 were included. Those cases completed in the first several weeks of the pilot were excluded since this was considered a trial period for Case Management Officers to familiarize themselves with the application of the Scale.

In the Pacific region, a sample of 60 cases were completed by 11 Community Case Management Officers in five different Parole Offices: Vancouver, Victoria, New Westminster, Kamloops, and Prince George. In Quebec, a sample of 92 cases were completed by 18 Case Management Officers at the Regional Reception Centre.

Sample

Regions	Cases
Pacific	60
Quebec	92
Total Sample	152

Quality Control:

In 36 of the 152 cases (23.7%), the Custody Rating Scale was not properly scored. The errors were fairly evenly distributed between Quebec and Pacific, and they were typically clerical in nature, with mistakes either in the assignment or subtraction and addition of scores on individual items. In five of the 36 cases with errors, the total score on the Scale was affected to the degree that the wrong security level was calculated. The most frequently mis-scored item was an age item that involved some mathematical calculation (35, minus the age in years, times 3).

Nonetheless, the extent and nature of error in completing a relatively simple statistical instrument is somewhat disconcerting. With the inclusion of the Custody Rating Scale in OMS, the scoring will be simplified and automated. Yet accuracy will still depend on the care taken in the gathering and coding of information. Quality control will need to be addressed on an ongoing basis.

Documentation Availability:

In order to accurately complete the scoring for the individual items of the Custody Rating Scale, basic corroborating information and documentation must be available to the Case Management Officer. To assess the availability of this information at the stage of penitentiary reception, the case

FIGURE 1
Frequency of Missing Information

Table 1 - Percentage of Cases for Which Documentation was not Available

&nbps;	Pacific	Quebec
&nbps;	% (#)	% (#)
F.P.S	6.8% (4)	25.6% (23)
Police Report	10.2% (6)	75.8% (69)
Psychological/ Psychiatric Reports	3.4% (2)	4.5% (4)
Institutional History (Federal)	8.5% (5)	11.1% (10)
Institutional History (Provincial)	1.7% (1)	58.9% (54)
Other	12.1% (7)	62.2% (56)

Across all categories, the Pacific region indicated that some piece of information was unavailable in only a few cases. A quite different situation was observed for Quebec. The most notable inter-regional discrepancies were observed with the availability of the F.P.S. record, the Police Report, Institutional History (Provincial), and "Other" reports. For all of these reports, the Quebec region indicated that they were unavailable on a substantially greater proportion of cases.

Figure 1 illustrates with what frequency the regions reported none, one, or up to six missing documents on each case. We see that the Pacific region reported no missing documentation for over 70% of cases, and only one or two missing documents for the remainder. By contrast, Quebec reported three or more missing documents for 55% of the cases. This analysis includes the "other" category which Quebec used quite frequently to indicate that Crown Prosecutor reports or Judges reasons for sentencing were missing. These may not be necessary documents for the initial classification decision. Nevertheless, the analysis highlights the fact that for a substantial number of cases in the Quebec region, classification decisions must be made without complete information.

No Scale or classification procedure can compensate for the lack of information on individual offenders. Classification without some "information base" for the decision is not classification. The situation which is reported in Quebec would seem to be quite unacceptable from the point of view of making quality decisions about offenders. It is a situation which will clearly need to be corrected regardless of what procedure is used to place inmates.

Concordance with Judgements of Case Management Staff:

The Case Management Officers were asked to complete an "Evaluation and Information Data" form for each case. One question requested the case managers to provide a "personal assessment of whether or not you agree with the initial offender security classification as determined by the Custody Rating Scale". A five-point Likert scale was used, with "strongly agree" and "strongly disagree" at opposing poles. Table 2 gives a summary of the degree of concordance with the security designations derived from the Custody Rating Scale.

Note that case managers in Quebec were instructed not to use the middle part of the Likert scale (3). In effect, they provided a "yes" or "no" response.

In the Pacific Region, there was strong or moderate disagreement in only 19.7% of the cases. The overall level of concordance was lower in Quebec where there was strong or moderate disagreement in 31.9% of the cases. Interestingly, although there was less overall concordance, there was strong agreement with the Custody Rating Scale in a larger proportion of cases in Quebec (37.4%). In Pacific, it was more likely for case managers to express uncertainty (relying on the mid-point of the rating scale to express their agreement).

Combining the responses from the two regions, and considering that the first three levels of response indicate general agreement with the security classifications provided by the Scale, we note an overall concordance of 74.3%.

We can conclude, therefore, that the Scale was generally accepted as an accurate tool for classification. This conclusion is buttressed by the fact that only a few case managers accounted for a majority of the disagreements. When we looked at the extent of disagreement by individual case manager, it was noted that in the Pacific, two of the eleven case managers accounted for 70% of the cases where there was disagreement. Similarly in the Quebec Region, five of the eighteen case managers accounted for 70% of the disagreements.

Table 2 - Concordance Between Custody Rating Scale and Judgements of Case Management Staff

Level of Agreement	Pacific	Quebec
Strongly Agree	10.7% (6)	37.4% (34)
Agree	39.3% (22)	25.3% (23)
Uncertain	30.4% (17)	5.5% (5)
Disagree	17.9% (10)	19.8% (18)
Strongly Disagree	1.8% (1)	12.1% (11)

Reasons for Disagreement:

In cases where there was disagreement, case managers were further requested to provide a rationale.
A breakdown of these responses is given in Table 3.

In both regions, no reason was provided in approximately 20% of the cases. For the remaining cases, the reasons for disagreeing with the Scale were not very clear. Program related reasons were given infrequently (in only five out of thirty-nine cases), and other explanations for an over-ride of the scale were not very concrete. A common reason for disagreeing with the security designations derived from the Custody Rating Scale was that the scale, in some vague way, "was incorrect .... e.g., "the scale gave lower than expected security level"; "Custody Rating Scale rated higher than required"; "subject scored lower than expected"; "subject lower than expected given extensive record"; "Custody Rating Scale rates maximum but do not agree". In other cases, particularly in the Quebec region, more basic reasons were articulated. However, these may have more to do with the personal views or theories of case managers than with what should be validly taken into account in making classification distinctions ... e.g., "feels escape risk"; "security factors other than Scale items"; "will not accept treatment for sex offence"; "suicide risk"; "good inmate/poor citizen"; "self-mutilation risk"; "poor attitude"; and "waste of time/nuisance".

In general, there seemed to be no evidence that the Scale was systematically omitting consideration of important factors that relate to custody classification. At the very least, the data suggest that case managers did not point predominantly to any single factor that was being ignored. No tool can cover all possible individual circumstances. The intent of applying an instrument such as the Custody Rating Scale is to cover the most relevant circumstances in most cases, and then to accommodate the exceptions with individual overrides that can be explained.

Table 3 - Reasons for Disagreement

Reason	Pacific	Quebec
	% (#)	% (#)
No Reason Provided	20.0% (2)	17.2% (5)
Protective Custody Override	--	3.4% (1)
Program Override	10.0% (1)	13.8% (4)
Outstanding Charges	10.0% (1)	10.3% (3)
Custody Rating Scale Factor(s)	50.0% (5)	17.2% (5)
Other	10.0% (1)	37.9% (11)
Total	10	29

Concordance with Actual Placement Decisions:

A basic question to ask is to what extent the "actual" placement decisions that were made reflected the security designations derived from the Custody Rating Scale. Table 4 summarizes the distribution of security level classifications derived for the sample. It contrasts the Custody Rating Scale with: a) the levels recommended in the Penitentiary Placement Report, and b) the actual security level where the inmate was assigned.

Considering that the Pilot was conducted independently in the two regions, under conditions that were quite distinct (i.e., centralized versus decentralized placement process), it is striking to note the consistency in the proportions of security level assignments with the Custody Rating Scale. As shown in the first row of Table 4, in both Regions, the Scale assigned approximately 15% of cases to maximum security, 55% to medium security, and 30% to minimum security.

A different picture emerges when we look at the "recommended" and "actual" placement decisions across regions. In general, we see that fewer cases are assigned to the minimum and maximum security extremes relative to what would have occurred with adherence to the Scale. For minimum security, substantially fewer cases are recommended, and then even fewer cases are placed at that level. Less deviation occurs at the maximum security level, but once again we see fewer cases being found appropriate for this higher level of control. The result, of course, is that the bulk of cases are recommended and assigned to medium security, an outcome perhaps more due to accommodation constraints than classification needs.

Table 4 - Percentage Distribution by Security Level

	Pacific			Quebec
	Min	Med	Max	Min	Med	Max
Custody Rating Scale	28.3% (7)	56.7% (34)	15.0% (9)	29.3% (17)	54.3% (50)	16.3% (15)
Pen. Pl. Recommend.	5.1% (3)	84.7% (50)	10.2% (6)	17.4% (16)	71.7% (66)	10.9% (10)
Actual Placement	1.7% (1)	86.4% (51)	11.9% (7)	14.3% (13)	74.7% (68)	11.0% (10)

The hesitancy to recommend inmates for minimum security is particularly noteworthy. A larger proportion of minimum security recommendations was observed for Quebec as compared to Pacific (17.4% and 5.1% respectively). However, for both regions, the numbers were considerably below the proportions suggested by the Custody Rating Scale. When we look at the "actual" placement distributions, the proportion sent to minimum is even lower, and in Pacific it reduces to next to nothing (1 inmate or 1.7%). Clearly, we see that medium security is being used as the "safe" classification alternative, thereby explaining why a much larger percentage of cases is "placed" at medium as compared to "assessed" as medium by the Scale.

Tables 5 and 6 show a more detailed analysis of the degree of correspondence between the Custody Rating Scale and the Recommended and Actual Placements of the regions. Assuming that the Custody Rating Scale provides the "correct" classification level, the tables illustrate the numbers and percentage of cases who were overclassified, underclassified or correctly classified. For example, an inmate who is classified by the Scale as maximum security is considered underclassified if he is recommended for placement in a medium or minimum security facility.

Table 5 - Custody Rating Scale Classification by Penitentiary Placement Recommendation

Custody Rating Scale	Pacific			Quebec
	Min	Med	Max	Min	Med	Max
Min	5.08% (3)	22.03% (13)	1.69% (1)	11.96% (11)	17.39% (16)	0.00% (0)
Med	0.00% (0)	54.24% (32)	1.69% (1)	5.43% (5)	43.48% (40)	5.43% (5)
Max	0.00% (0)	8.47% (5)	6.78% (4)	0.00% (0)	10.87% (10)	5.43% (5)

Table 6 - Custody Rating Scale by Actual Penitentiary Placement

Custody Rating Scale	Pacific			Quebec
	Min	Med	Max	Min	Med	Max
Min	1.69% (1)	22.73% (14)	1.69% (1)	10.99% (10)	18.68% (17)	0.00% (0)
Med	0.00% (0)	55.93% (33)	1.69% (1)	8.80% (3)	45.05% (41)	5.49% (5)
Max	0.00% (0)	6.78% (4)	8.47% (5)	0.00% (0)	10.99% (10)	5.49% (5)

The percentage of correct classifications at the various levels of security are shown in the diagonals,
while the figures to the right and left are over and underclassifications respectively.

If we look at the cross-classifications with the actual placement decisions (Table 6), we see that 66.1% of the cases were classified according to the Custody Rating Scale designation in the Pacific, and 61.5% in Quebec. Very few cases were underclassified in the Pacific (6.8%), although substantially more were underclassified in Quebec (14.3%). A significant number of cases designated as maximum by the Scale were classified to medium in Quebec (11%).

Most noteworthy, however, is the proportion of cases overclassified in both regions - 27% in Pacific and 24% in Quebec. For the most part, we see inmates classified as minimum by the Scale but being placed in medium security institutions.

IV. DISCUSSION

The findings discussed in this interim report provide a sense of what issues must be addressed prior to further implementation of the Custody Rating Scale. A summary discussion of some of these issues follows.

Custody Rating Scale Versus Actual Penitentiary Placements:

The Custody Rating Scale was designed not only as an objective, research-based instrument to assist case managers, but as a classification instrument that would reflect our Corporate beliefs. As a means of bringing consistency and fairness to the classification process, it was designed to more closely align our classification practices with our Mission ..... to rely on "the least restrictive course of action" in dealing with offenders and to place a greater percentage of suitable offenders directly into minimum security institutions.

We have been traditionally cautious and conservative in our approach to classification by security level. Very few inmates have been placed directly to minimum security facilities. By using an instrument which has been empirically tested and considers both institutional adjustment and security risk, we could achieve greater confidence in our decisions to place offenders in institutions that represent the least restrictive alternative.

With the high level of integrity attached to the static security systems of our medium security institutions (e.g., most now are equipped with PIDS), maximum security placements should be reserved for inmates who are assessed as likely to demonstrate behavioral problems in the institutional setting. The Custody Rating Scale changes the traditional focus from a generalized assessment of "escape risk", to giving a greater importance to actual behaviour in determining that initial placement in a maximum security institution is appropriate.

The preliminary data that have been reviewed show that the Custody Rating Scale assessed only a relatively small number of offenders as requiring maximum security. On the other hand, the Scale assessed 29% of the total sample as minimum security offenders, while a significantly smaller number of those individuals were actually placed in minimum security institutions. In view of our desire to avoid overclassification, it is this group that presents the greatest concern.

In both regional samples, the number of inmates who were actually placed in minimum security facilities was much lower than the number who were assessed at this level by the Custody Rating Scale. The difference between the two was greater in the Pacific region where only one of the 17 offenders who were assessed as suitable for minimum security by the Scale was actually placed in a minimum security institution.

What distinguishes those cases who were overclassified in both the Pacific and Quebec regions? We looked at this issue from several perspectives.

First, we looked at whether these offenders fell into particular offence categories. In the Pacific region, 16 inmates were overclassified. Of these, 10 were serving sentences for sex offences, an offence category that we understand is automatically excluded from direct placement in minimum security as regional policy.

Although an offence pattern in Quebec was not as evident it was nonetheless there. Ten of the 22 cases who were overclassified in Quebec were serving sentences for either armed robbery or break and enter.

We also looked at other characteristics of those cases who were overclassified. Some of these are profiled below.

% of Overclassified Cases

Management Scenario
1	2	3	4
48.7%	8.1%	35.1%	8.1%

The picture that emerges is one of an older, generally well adjusted offender, without a substantial criminal career and with no prior history of institutional disruption. One might question why these kinds of cases are being overclassified.

A final analysis we conducted was to look at how the overclassified cases fell into one or other Management Scenarios .... i.e., according to the distinctions made in the recent report of the Task Force on Institutional and Community Programs. This is shown below:

&nbps;	Pacific	Quebec
No Prior Institutional Incidents	87.5%	73.0%
No Prior Escapes or Attemps	100.0%	86.4%
Average or above average Street Stability	87.5%	86.4%
Four or fewer Prior Convictions	74.9%	50.0%
No Outstanding Charges	93.7%	90.9%
Over Age Thirty	75.0%	75.0%

Considering that Management Scenario 1 inmates (short-term, non-violent) were identified by the Task Force as a group that could be released at earliest eligibility, it is interesting to note that close to 50% of the inmates who were overclassified fell into this category.

There may be quite rational and substantial reasons for overriding any actuarial assessment such as the Custody Rating Scale. Such factors as outstanding charges, protective custody considerations and program availability could justify placements in higher security facilities than the level indicated by the Scale. On the other hand, the extent and nature overclassification that was observed in this pilot is more difficult to justify. An instrument on its own cannot change classification practices. What may be required is both an examination of regional policies regarding the initial placement of offenders, as well as a concerted effort to assist case managers in rethinking their traditional conservative and over- cautious views regarding placement.

Several factors may have contributed to the variations found in the percentage distributions between the original offender sample and our present sample. First, it is possible that the profile of our admission population is changing (i.e., the original sample represented admissions into the Federal system from approximately two years ago). More likely, however, is the fact the lack of necessary documentation on offenders resulted in some degree of "underestimation" of scores on the Custody Rating Scale. A retrospective analysis of inmate files was conducted for the sample in the original study, whereas the Custody Rating Scale was actually applied during the pilot at the time of initial placement to federal custody (i.e., when less information is available). This difference in the method of gathering information for the two samples may be partially responsible for the differing outcomes.

While the cut-off scores used to calculate security classifications can be readily adjusted to closely match a desired distribution, it is a more complex matter to assure that this same distribution is actually placed at appropriate levels across regions. It is at this level that the influences of various overriding factors and regional policies must be taken into consideration. In order to effectively control these influences there will need to be ongoing monitoring and analysis of placement decisions.

Offender Management System (OMS):

The Penitentiary Placement Report is currently a component of the Offender Management System (OMS) which is being used on a Pilot Project basis in the Vancouver District Parole Office. Recently a prototype of the Custody Rating Scale was developed by the Systems Division in conjunction with Offender Management which will be incorporated as part of the Penitentiary Placement Report. This development has in effect transferred the Custody Rating Scale instrument to a computer program which will allow the Case Management Officer to complete the Scale directly on a personal computer. In addition, the program provides separate screens for instructing the user how to complete each Scale item, and also automatically calculates any mathematical equations which are required. Finally, the computer program calculates and provides the security assessment of each individual offender.

At this point, it is intended that the prototype be incoroporated into the Offender Management System.