Generally used methods for risk assessment

Unaided Clinical Judgment (UCJ)  
Continued Reliance on UCJ in Clinical Practice 
Guided Clinical Risk Assessments (GCRAs)  
Sexual Violence Risk-20 (SVR-20)   
Overview of GCRA

[Page 118]

Unaided Clinical Judgment (UCJ)

When relying on unaided clinical judgment (UCJ) -- making predictions of future violence based on idiosyncratic impressions obtained from interview and test data -- the performance of psychologists and psychiatrists is abysmal.

Monahan's (1978) review of the accumulated literature reported the rate of error associated with attempts at predicting future violence ranged from 54 to 94%. In this same review, Monahan indicated that the majority of studies reported an error rate of 80% or more when attempting to predict future violence.

A subsequent review by Monahan (1981 b) concluded that mental health professionals are:

 "accurate in no more than one out of three predictions of violent behavior over a several year period among institutionalized populations that have both committed violence in the past (and thus had high base rates for it) and who were diagnosed as mentally ill" (p. 14). 

In 1984, Schopp and Quattrochi further reviewed the accuracy of UCJ in predicting violence. They emphasized:

A rather large and consistent body of empirical evidence indicates that the standards of the profession include no ability to accurately predict dangerous behavior. Not only have psychologists and psychiatrists been unable to predict dangerousness to a degree of accuracy which would justify infringing on a client's rights, they have been unable to predict any more accurately than nonprofessionals (p. 23).

More recent reviews have also confirmed unacceptable levels of accuracy associated with UCJ related to risk assessment

(Bonta, Law, & Hanson, 1998; Grove & Meehl, 1996).

In examining the predictive accuracy of UCJ regarding sexual offender recidivism risk, Hanson and Bussière (1998) found an average correlation of .10. Quite obviously, then, relying on UCJ for assessing the recidivism risk of a previously convicted sexual offender is inconsistent with the  ethical obligations of a psychologist. In particular, the levels of sensitivity and specificity, and the frequencies of false positive and false negative classifications, associated with UCJ remain unknown.

Ethical Standard 2.04(a), and Testing Standard 6.1, also prohibit psychologists from resorting to UCJ when undertaking risk assessments. Any reasonably well informed psychologist must acknowledge that the available validity data related to assessments premised on UCJ contraindicate such a practice. Moreover, there are no cutoff scores, or well defined decision-making criteria, available for UCJ when used to assess recidivism risk. UCJ therefore is also inconsistent with Testing Standard 1.1, and Ethical Standard 2.02(a). Without a generally available manual for UCJ, relying on it for predator evaluations also disregards Standard 5.1 of the testing standards.

Continued Reliance on UCJ in Clinical Practice

This overview of UCJ poses a further question. Do practicing psychologists recognize the many shortcomings of UCJ when assessing recidivism risk in cases of 

[Page 119]

previously convicted offenders? Related data, for example, indicate that practicing psychologists remain unaware of the many problems undermining clinical judgment in general (Rock, 1994). Absent sufficient familiarity with the many pitfalls associated with clinical judgment, might practicing psychologists inappropriately rely on it when undertaking predator evaluations?

Consider, for example, how a Florida psychologist responded to questions, regarding clinical judgment and actuarial instruments, during a pre-trial deposition prior to a sexual predator hearing.

The fact is, what I do is to go in and I interview each person and then I make a decision based on my opinion before I score the [actuarial] instruments. I guess I want to say that -- I might as well go on record now as saying this. I hate these instruments [actuarial methods] myself. Emotionally these instruments annoy me, and before I ever used them I could just go in and impress the court with my credentials and then give my opinion and then you had to shoot me down to shoot mine down, my opinion.

Now I have to use these instruments developed by geeks on computers who never saw a sex offender, and I'll confess that it triggers me (Florida v. Wydell, pp.43-44).

Above and beyond demonstrating how woefully misinformed he is, this psychologist's persisting affinity for clinical judgment increases the likelihood of his own biases influencing his opinions. 

The elasticity of clinical judgment allows stretching it to conform with the a priori expectations of an evaluator. If predisposed to "rule in," or "rule out," future sexual dangerousness, evaluators will seek information consistent with their expectations. When people respond to strong expectations, they selectively pay attention to the information available to them. They remember expectancy-consistent information better because it confirms what they expected to find. They also discount inconsistencies as random variation, further regarding them as insignificant exceptions to what they expected (Rothbart, Evans, & Fulero, 1979). Relying on clinical judgment, therefore, easily allows psychologists to indulge whatever biases they bring to an evaluation. 

It can also be argued that the biases of practicing psychologists predispose them to "rule in" future dangerousness. For a practicing psychologist undertaking a predator evaluation, the costs of false positive and false negative errors are far from equal. In the instance of a false positive error, that outcome rarely if ever receives public attention. An offender mistakenly classified as a predator may protest his fate. His protests, however, typically fall upon deaf ears as he lingers in the obscurity of indefinite confinement. In cases of false negative errors, however, the psychologist involved can endure the harsh spotlight of public criticism.

Assume, for example, that Dr. Jones evaluates Mr. Smith pursuant to a sexual predator statute. Dr. Jones concludes that Mr. Smith does not warrant civil commitment as a sexual predator. Subsequent to his release into the community, Mr. Smith commits a particularly brutal sexual assault. Scenarios such as these attract considerable media attention with the potential of enormous costs related to Dr. Jones' professional reputation. These considerations indicate that when relying on clinical judgment in sexual predator evaluations, psychologists may be more  inclined to tolerate false positive than false negative errors.

 [Page 120]

Guided Clinical Risk Assessments (GCRAs)

In contrast to UCJ, guided clinical risk assessments (GCRAs) address various empirically validated risk factors, arriving at recidivism estimates based on an offender's status on these factors. Previous work has identified the following factors as increasing the risk of recidivism for sexual offenders: 

Hanson and Bussière (1998), however, reported data inconsistent with the factors cited above. In their meta-analysis of 61 follow-up studies examining sexual offender recidivism (n = 23 393), they found no significant correlations between recidivism and any 

Hanson and Bussière (1998) additionally found no significant correlations between recidivism and the following factors: 

Quite clearly, then, attempts at identifying risk factors for sexual offender recidivism have led to some inconsistent and equivocal results. In this writer's experience, reviewing sexual predator evaluations made in California, Florida, Ohio, and Wisconsin, he has encountered numerous instances of psychologists invoking factors for which Hanson and Bussière found no empirical support. 

It should also be pointed out that Hanson and Bussière (1998) did identify other risk factors that correlated significantly with sexual offender recidivism: 

Sexual Violence Risk-20 (SVR-20)

The SVR-20 (Boer, Hart, Kropp, & Webster, 1997) is a commercially available instrument (Psychological Assessment Resources, Inc.) for GCRAs. It directs users to assess recidivism risk for previously convicted sexual offenders by addressing 20 risk factors. Each factor is coded according to whether it is definitely present, possibly present, or not present. The SVR-20 manual, however, does not report data regarding inter-rater reliabilities for these factors. The extent to which two or more evaluators reach the same or similar conclusions when independently  evaluating the same offender therefore remains unknown. 

The manual advises evaluators to summarize their ratings into an overall risk rating of low, moderate, or high. Nevertheless, the manual does not specify what values of the SVR-20 (ranging from 0 to 20) correspond to different risk levels. In particular, the manual contends: 

"For clinical purposes, it makes little sense to 

[Page 121]

sum the number of risk factors present in a given case and then use fixed, arbitrary cutoffs to classify the individual as low, moderate, or high risk" (Boer et al., 1997, p.36).

Additionally, available data (Hanson & Bussière, 1998) do not support at least five of the SVR-20 factors: 

The SVR-20 manual acknowledges "... there is little evidence supporting a specific link between [being a victim of] child sexual abuse and later sexual violence" (p. 44). The manual nonetheless invokes considerations of "professional reviews" to support its position. 
For the factor "extreme minimization/denial of sex offenses," the manual again acknowledges: 

"There is no clear evidence supporting this factor's ability to predict future sexual violence, although it predicts general criminality in sexual offenders" 
(Hanson & Bussière, 1996) (p. 74).

The manual then proceeds to defend relying on this factor indicating:

"According to professional reviews, it is an important factor to consider in clinical evaluations of risk"
(Greer, 1991; McGovern & Peters, 1988; Ross & Loss, 1991) (p. 74).

Elevating "professional reviews" to a level of equal or greater importance than relevant data is obviously ill advised. Doing so ultimately embraces theory in preference to empirically supported fact; and, in turn, encourages those who use the SVR-20 to rely on what is -- at best -- a modest improvement over UCJ.

The SVR-20 can claim a generally available manual; but, beyond that consideration, there is little else to recommend it for assessing the recidivism risk of previously convicted sexual offenders. Unfortunately, the commercial availability of the SVR-20 suggests that practitioners may be more inclined to rely on it.

Overview of GCRA

As Hanson (1998) has acknowledged, GCRAs inevitably contend with the problem of how to best weight and combine the different factors. Despite Testing Standard 5.1, there is no manual available for GCRAs to address this issue. Without an explicitly defined method for converting various risk factors into recidivism probabilities, different evaluators can reach very different conclusions when assessing the same offender. When they occur, these inconsistent conclusions correspond to variations in clinical judgment between two or more evaluators assessing the same offender. Psychologist A attributes considerable significance to the offender's age, but psychologist B views the offender's failure to complete treatment as more compelling. Ultimately, then, the value of GCRAs is undermined by their unavoidable reliance on clinical judgment.

Moreover, there are no data available for GCRAs to answer the four critical questions for evaluating an assessment method for sexual predators: 

As a result, GCRAs fail to comply with Ethical Standard 2.02(a). Without a generally available manual for GCRAs (except for the SVR-20), relying on them for predator evaluations also disregards Testing Standard 5.1. Relatedly, then, opinions premised on GCRAs regarding recidivism risk also neglect Testing Standard 6.1 and Ethical Standard 2.04(a).

[Page 122]

The considerations discussed above lead to a sobering conclusion -- for purposes of assessing the recidivism risk for previously convicted sexual offenders, GCRAs are -- at best -- experimental procedures. As experimental procedures, they do not possess sufficient evidentiary reliability to support expert testimony in a legal proceeding. In particular, GCRAs present an alarming potential for misleading and misinforming a trier of fact.