FORUM on Corrections Research

Evaluating the Computerized Item Management System (CIMS) with violent offenders

When offenders enter the federal correctional system, the Psychology Department of the Millhaven Assessment Unit (MAU) provides psychological assessments on offenders who meet certain referral criteria. Traditionally, offenders serving sentences for assault and other violent offences receive a psychological assessment to address criminogenic risk factors and to recommend appropriate programming and treatment options.

An important part of the assessment process is the collection of information, conducted primarily through self-report instruments. These instruments cover a broad range of psychological domains such as anger, psychopathology, attribution, aggression and offender response style, to name only a few.

This article reviews the preliminary results of conducting psychological assessments using the Computerized Item Management System (CIMS). The CIMS presents offenders with psychological test items, and scores the results.

Offender assessment issues are crucial to the way in which an offender is managed. It follows, then, that greater confidence in the validity of our assessment procedures translates into better decisions regarding offender incarceration, treatment and release. These procedures, however, must still remain cost effective. Computerizing psychological testing is one method that may help meet both these goals.

Early use of the CIMS

As part of a larger validation and reliability study, responses to the Novaco Anger Scale² (NAS) were recorded using both the CIMS and paper and pencil methods, and then compared. The NAS, a self-report anger measure, was developed in the 1990s to overcome some of the poor theoretical and scale qualities of other anger inventories. One component of this measure (Part B), which assesses situations that can result in anger, has been fruitful in the long-term prediction of violence.³ This component has 25 items which present offenders with situations where, for example, they are being criticized in front of others or overcharged for a repair.

Table 1

CIMS (n=51) versus Paper and Pencil (n=51) Administration of the NAS (Part B)
Administration	Mean	Standard Deviation	One-month Test retest	Coefficient Alpha
CIMS	48.3	15.1	0.86	0.95
Paper and Pencil	50.3	15.0	0.84	0.88

A sample of 102 consecutive male federal admissions completed a paper and pencil presentation of this self-report anger measure within two weeks of arrival at the Millhaven Assessment Unit. One month later, offenders were retested, half by paper and pencil and half by CIMS presentation. As seen in Table 1, the means, standard deviations and one-month test-retest stability were similar between the two types of administrations. The coefficient alphas⁴ were slightly stronger for the CIMS administration. As a result of these and other findings comparing computer and paper and pencil techniques,⁵ a more in-depth evaluation was conducted with violent offenders.

Evaluation study with violent offenders

The evaluation study compared the time it took to complete the same test battery using the paper and pencil method and the CIMS method. In addition, the validity of the computer-administered Stimulus Seeking and Callousness scales from the Dimensional Assessment of Personality Problems⁶ (DAPP) was compared to the paper and pencil versions of the scale.

This study involved 28 consecutive violent offender admissions alternately assigned to one of two groups. The MAU psychological test battery was divided in half to be presented in two sessions. There were approximately 550 items administered in each of the two sessions. Group 1 was administered the first half of the test battery by CIMS presentation and the second half by traditional paper and pencil method. Group 2 received the first half of the test battery by paper and pencil method and the second half by CIMS. As shown in the figure, less time is required to complete the test battery using the CIMS method. On average, the time saved was approximately half an hour per offender tested. This represents a 30% time saving in test administration. In addition, the automated scoring programs of the CIMS dramatically reduce the time required for scoring each test and eliminate the potential for clerical error.

Figure 1

The assessment process included an interview with each of the 28 offenders. Clinical ratings were gathered on stimulus seeking and callousness that targetted the Stimulus Seeking and Callousness scales of the DAPP. The correlations between the self-report scales and the clinical ratings are presented in Table 2. The scales administered with the CIMS (n = 14) had a stronger relationship with their respective clinical ratings than did the paper and pencil group.

Table 2

Validity Comparisons between CIMS and Paper and pencil Presentations
Method of Self-report
Clinical Rating	CIMS (n=14)	Paper and Pencil (n=14)
Stimulus seeking	0.53	0.11
Callousness	0.59	0.06

Conclusion

Using the CIMS to assess violent offenders has several benefits. For the more situationally based anger items, the results of this study suggest the potential for stronger reliability with computerized administration. With the CIMS, items are presented consecutively and require a response before one can proceed to the next item. This prevents both revisiting items and looking ahead, which in turn results in a more standardized presentation. This gain is not only important for reliability, but for practicality, as it allows for the increased prediction of targetted behaviours.

In terms of time efficiency, computer administered items are completed more quickly and without a loss of scale properties. Furthermore, the automatic scoring programs of the CIMS reduce scale scoring time and eliminate the potential for clerical error.

This study suggests that there is greater validity for some violence-related content areas with CIMS compared to paper and pencil procedures.

Computerized psychological testing with a program like the CIMS may provide more confidence in our assessment procedures, resulting in better decisions in a time efficient and cost effective manner.

---

1. P.O. Box 280, Bath, Ontario, K0H 1G0. We thank Al Stevenson and Donna Morrin for their support in the development of the CIMS. CIMS is flexible and can accommodate a wide variety of self-report presentations.

2. R.W. Novaco, "Anger as a Risk Factor for Violence Among the Mentally Disordered," in Violence and Mental Disorder: Developments in Risk Assessment, J. Monahan and H. Steadman (eds.) (Chicago: University of Chicago Press, 1994): 21-59.

3. R.W. Novaco, personal communication, July, 1996.

4. Coefficient alpha is a measure of reliability and refers to how well a test holds together, with 0.15 being very weak and 0.95 being very strong.

5. R.R. Holden, G.C. Fekken and D.H.G. Cotton, "Clinical Reliabilities and Validities of the Microcromputerized Basic Personality Inventory," Journal of Clinical Psychology, 46 (1990): 815-849.

6. W.J. Livesley and D.N. Jackson, Manual for the Dimensional Assessment of Personality Problems - Basic Questionnaire
(Port Huron, Michigan: Sigma, in press).