DECISION PROCESSES IN PERCEPTIONpsych.colorado.edu/~lharvey/P5665 Prosem/P5665_2016/Class... ·...

Psychological Review1961, Vol. 68, No. 5, 301-340

DECISION PROCESSES IN PERCEPTION1

JOHN A. SWETSMassactmsetts Institute of Technology

WILSON P. TANNER, JR., AND THEODORE G. BIRDSALLUniversity of Michigan

About 5 years ago, the theory ofstatistical decision was translated intoa theory of signal detection.2 Althoughthe translation was motivated by prob-lems in radar, the detection theory thatresulted is a general theory for, likethe decision theory, it specifies an idealprocess. The generality of the theorysuggested to us that it might also berelevant to the detection of signals byhuman observers. Beyond this, wewere struck by several analogies be-tween this description of ideal behaviorand various aspects of the perceptualprocess. The detection theory seemedto provide a framework for a realisticdescription of the behavior of thehuman observer in a variety of per-ceptual tasks.

1 This paper is based upon Technical Re-port No. 40, issued by the Electronic De-fense Group of the University of Michiganin 1955. The research was conducted in theVision Research Laboratory of the Univer-sity of Michigan with support from theUnited States Army Signal Corps and theNaval Bureau of Ships. Our thanks are dueH. R. Blackwell and W. M. Kincaid fortheir assistance in the research, and D. H.Howes for suggestions concerning the pres-entation of this material. This paper wasprepared in the Research Laboratory ofElectronics, Massachusetts Institute of Tech-nology, with support from the Signal Corps,Air Force (Operational Applications Lab-oratory and Office of Scientific Research),and Office of Naval Research. This is Tech-nical Report No. ESD-TR-61-20.

2 For a formal treatment of statisticaldecision theory, see Wald (1950) ; for a briefand highly readable survey of the essentials,see Bross (1953). Parallel accounts of thedetection theory may be found in Peterson,Birdsall, and Fox (1954) and in Van Meterand Middleton (1954).

The particular feature of the theorythat was of greatest interest to us wasthe promise that it held of solving anold problem in the field of psycho-physics. This is the problem of con-trolling or specifying the criterion thatthe observer uses in making a percep-tual judgment. The classical methodsof psychophysics make effective provi-sion for only a single free parameter,one that is associated with the sensi-tivity of the observer. They containno analytical procedure for specifyingindependently the observer's criterion.These two aspects of performance areconfounded, for example, in an experi-ment in which the dependent variableis the intensity of the stimulus that isrequired for a threshold response. Thepresent theory provides a quantitativemeasure of the criterion. There is left,as a result, a relatively pure measureof sensitivity. The theory, therefore,promised to be of value to the studentof personal and social processes in per-ception as well as to the student ofsensory functions. A second featureof the theory that attracted us is thatit is a normative theory. We believedthat having a standard with which tocompare the behavior of the humanobserver would aid in the descriptionand in the interpretation of experi-mental results, and would be fruitfulin suggesting new experiments.

This paper begins with a brief re-view of the theory of statistical decisionand then presents a description of theelements of the theory of signal detec-tion appropriate to human observers.

301

302 J. A. SWETS, W. P. TANNER, JR., AND T. G. BIRDSALL

.£U

.16

.12

» .08

I .04a.

0

1 1

!

_ _rii — '_ iii i

5^r jr

-i | . _

1' .,.,„. -T

1. 1

1

1

7 8 9 10 I ITo ta l o f Th ree Dice

12 I 3 14 15

FIG. 1. The probability distributions for the dice game.

Following this, the results of some ex-perimental tests of the applicability ofthe theory to the detection of visualsignals are described.

The theory and some illustrative re-sults of one experimental test of it werebriefly described in an earlier paper(Tanner & Swets, 1954). The presentpaper contains a more nearly adequatedescription of the theory, a more com-plete account of the first experiment,and the results of four other experi-ments. It brings together all of thedata collected to date in vision experi-ments that bear directly on the valueof the theory.3

THE THEORYStatistical Decision Theory

Consider the following game ofchance. Three dice are thrown. Twoof the dice are ordinary dice. Thethird die is unusual in that on each ofthree of its sides it has three spots,whereas on its remaining three sidesit has no spots at all. You, as the

3 Reports of several applications of thetheory in audition experiments are availablein the literature; for a list of references, seeTanner and Birdsall (19S8).

player of the game, do not observe thethrows of the dice. You are simplyinformed, after each throw, of the totalnumber of spots showing on the threedice. You are then asked to statewhether the third die, the unusual one,showed a 3 or a 0. If you are correct—that is, if you assert a 3 showedwhen it did in fact, or if you assert a0 showed when it did in fact—you wina dollar. If you are incorrect—that is,if you make either of the two possibletypes of errors—-you lose a dollar.

How do you play the game? Cer-tainly you will want a few minutes tomake some computations before youbegin. You will want to know theprobability of occurrence of each of thepossible totals 2 through 12 in theevent that the third die shows a 0, andyou will want to know the probabilityof occurrence of each of the possibletotals 5 through 15 in the event thatthe third die shows a 3. Let us ignorethe exact values of these probabilities,and grant that the two probability dis-tributions in question will look muchlike those sketched in Figure 1.

Realizing that you will play the gamemany times, you will want to establish

DECISION PROCESSES IN PERCEPTION 303

a policy which defines the circum-stances under which you will makeeach of the two decisions. We canthink of this as a criterion or a cutoffpoint along the axis representing thetotal number of spots snowing on thethree dice. That is, you will want tochoose a number on this axis such thatwhenever it is equaled or exceeded youwill state that a 3 showed on the thirddie, and such that whenever the totalnumber of spots showing is less thanthis number, you will state that a 0showed on the third die. For the gameas described, with the a priori proba-bilities of a 3 and a 0 equal, and withequal values and costs associated withthe four possible decision outcomes, itis intuitively clear that the optimal cut-off point is that point where the twocurves cross. You will maximize yourwinnings if you choose this point asthe cutoff point and adhere to it.

Now, what if the game is changed?What, for example, if the third die hasthree spots on five of its sides, and a 0on only one? Certainly you will nowbe more willing to state, following eachthrow, that the third die showed a 3.You will not, however, simply statemore often that a 3 occurred withoutregard to the total showing on the threedice. Rather, you will lower your cut-off point: you will accept a smallertotal than before as representing athrow in which the third die showeda 3. Conversely, if the third die hasthree spots on only one of its sides andO's on five sides, you will do well toraise your cutoff point—to require ahigher total than before for stating thata 3 occurred.

Similarly, your behavior will changeif the values and costs associated withthe various decision outcomes arechanged. If it costs you 5 dollarsevery time you state that a 3 showedwhen in fact it did not, and if you win5 dollars every time you state that a 0

showed when in fact it did (the othervalue and the other cost in the gameremaining at one dollar), you will raiseyour cutoff to a point somewhere abovethe point where the two distributionscross. Or if, instead, the premium isplaced on being correct when a 3 oc-curred, rather than when a 0 occurredas in the immediately preceding exam-ple, you will assume a cutoff some-where below the point where the twodistributions cross.

Again, your behavior will change ifthe amount of overlap of the two dis-tributions is changed. You will assumea different cutoff than you did in thegame as first described if the threesides of the third die showing spotsnow show four spots rather than three.

This game is simply an example ofthe type of situation for which thetheory of statistical decision was devel-oped. It is intended only to recall theframe of reference of this theory. Sta-tistical decision theory—or the specialcase of it which is relevant here, thetheory of testing statistical hypotheses—specifies the optimal behavior in asituation where one must choose be-tween two alternative statistical hy-potheses on the basis of an observedevent. In particular, it specifies theoptimal cutoff, along the continuum onwhich the observed events are ar-ranged, as a function of (a) the apriori probabilities of the two hypothe-ses, (&) the values and costs associatedwith the various decision outcomes,and (c) the amount of overlap ofthe distributions that constitute thehypotheses.

According to the mathematical the-ory of signal detectability, the problemof detecting signals that are weak rela-tive to the background of interferenceis like the one faced by the player ofour dice game. In short, the detectionproblem is a problem in statistical deci-sion ; it requires testing statistical hy-


O b s e r v a l i o n ( x )

FIG. 2. The probability density functions ofnoise and signal plus noise.

potheses. In the theory of signal de-tectability, this analogy is developed interms of an idealized observer. It isour thesis that this conception of thedetection process may apply to thehuman observer as well. The nextseveral pages present an analysis ofthe detection process that will makethe bases for this reasoning apparent.4

Fundamental Detection ProblemIn the fundamental detection prob-

lem, an observation is made of eventsoccurring in a fixed interval of time,and a decision is made, based on thisobservation, whether the interval con-tained only the background interferenceor a signal as well. The interference,which is random, we shall refer to asnoise and denote as N; the other alter-native we shall term signal plus noise,

4 It is to be expected that a theory recog-nized as having a potential application inpsychophysics, although developed in anothercontext, will be similar in many respects toprevious conceptions in psychophysics. Al-though we shall not, in general, discussexplicitly these similarities, the strong re-lationship between many of the ideas pre-sented in the following and Thurstone'searlier work on the scaling of judgmentsshould be noted (see Thurstone, 1927a,1927b). The present theory also has muchin common with the recent work of Smithand Wilson (1953) and of Munson andKarlin (1956). Of course, for a new theoryto arouse interest, it must also differ in somesignificant aspects from previous theories—•these differences will become apparent as weproceed.

SN. In the fundamental problem, onlythese two alternatives exist—noise isalways present, whereas the signal mayor may not be present during a speci-fied observation interval. Actually, theobserver, who has advance knowledgeof the ensemble of signals to be pre-sented, says either "yes, a signal waspresent" or "no, no signal was present"following each observation. In the ex-periments reported below, the signalconsisted of a small spot of light flashedbriefly in a known location on a uni-formly illuminated background. It isimportant to note that the signal is al-ways observed in a background ofnoise; some, as in the present case,may be introduced by the experimenteror by the external situation, but someis inherent in the sensory processes.

Representation of Sensory InformationWe shall, in the following, use the

term observation to refer to the sensorydatum on which the decision is based.We assume that this observation maybe represented as varying continuouslyalong a single dimension. Althoughthere is no need to be concrete, it maybe helpful to think of the observationas some measure of neural activity,perhaps as the number of impulses ar-riving at a given point in the cortexwithin a given time. We assume fur-ther that any observation may arise,with specific probabilities, either fromnoise alone or from signal plus noise.We may portray these assumptionsgraphically, for a signal of a given am-plitude, as in Figure 2. The observa-tion is labeled x and plotted on theabscissa. The left-hand distribution,labeled f y ( x ) , represents the proba-bility density that x will result giventhe occurrence of noise alone. Theright-hand distribution, fsy(^), is theprobability density function of x giventhe occurrence of signal plus noise.(Probability density functions are used,


rather than probability functions, sincex is assumed to be continuous.) Sincethe observations will tend to be ofgreater magnitude when a signal is pre-sented, the mean of the SN distributionwill be greater than the mean of theN distribution. In general, the greaterthe amplitude of the signal, the greaterwill be the separation of these means.

Observation as a Value of LikelihoodRatio

It will be well to question at thispoint our assumption that the observa-tion may be represented along a singleaxis. Can we, without serious viola-tion, regard the observation as uni-dimensional, in spite of the fact thatthe response of the visual system prob-ably has many dimensions? The an-swer to this question will involve someconcepts that are basic to the theory.

One reasonable answer is that whenthe signal and interference are alike incharacter, only the magnitude of thetotal response of the receiving systemis available as an indicator of signalexistence. Consequently, no matterhow complex the sensory informationis in fact, the observations may be rep-resented in theory as having a singledimension. Although this answer isquite acceptable when concerned onlywith the visual case, we prefer to ad-vance a different answer, one that isapplicable also to audition experiments,where, for example, the signal may bea segment of a sinusoid presented in abackground of white noise.

So let us assume that the responseof the sensory system does have severaldimensions, and proceed to representit as a point in an w-dimensional space.Call this point 31. For every such pointin this space there is some probabilitydensity that it resulted from noisealone, f#(;y), and, similarly, someprobability density that it was due tosignal plus noise, isy(y). Therefore,

there exists a likelihood ratio for eachpoint in the space, X(y) =fsjr(y)/fy(y), expressing the likelihood thatthe point y arose from SN relative tothe likelihood that it arose from N.Since any point in the space, i.e., anysensory datum, may be thus repre-sented as a real, nonzero number, thesepoints may be considered to lie alonga single axis. We may then, if wechoose, identify the observation x withA(y) ; the decision axis becomes like-lihood ratio.5

Having established that we mayidentify the observation x with \(y),let us note that we may equally wellidentify x with any monotonic trans-formation of A(y) . It can be shownthat we lose nothing by distorting thelinear continuum as long as order ismaintained. As a matter of fact wemay gain if, in particular, we identifyx with some transformation of A.(y)that results in Gaussian density func-tions on x. We have assumed theexistence of such a transformation inthe representation of the density func-tions, isx(x) and f»(^), in Figure 2.We shall see shortly that the assump-tion of normality simplifies the problemgreatly. We shall also see that thisassumption is subject to experimentaltest. A further assumption incorpo-rated into the picture of Figure 2, onemade quite tentatively, is that the twodensity functions are of equal variance.This is equivalent to the assumptionthat the SN function is a simple trans-lation of the N function, or that addinga signal to the noise merely adds aconstant to the N function. The re-

5 Thus the assumption of a unidimensionaldecision axis is independent of the characterof the signal and noise. Rather, it dependsupon the fact that just two decision alterna-tives are considered. More generally, it canbe shown that the number of dimensionsrequired to represent the observation isM — 1, where M is the number of decisionalternatives considered by the observer.


suits of a test of this assumption arealso described below.

To summarize the last few para-graphs, we have assumed that an ob-servation may be characterized by avalue of likelihood ratio, A.(y), i.e., thelikelihood that the response of the sen-sory system y arose from SN relativeto the likelihood that it arose from N.This permits us to view the observa-tions as lying along a single axis. Wethen assumed the existence of a par-ticular transformation of A(y) such thaton the resulting variable, x, the densityfunctions are normal. We regard theobserver as basing his decisions on thevariable x.

Definition of the CriterionIf the representation depicted in Fig-

ure 2 is realistic, then the problemposed for an observer attempting todetect signals in noise is indeed similarto the one faced by the player of ourdice game. On the basis of an ob-servation, one that varies only in mag-nitude, he must decide between twoalternative hypotheses. He must de-cide from which hypothesis the ob-servation resulted; he must state thatthe observation is a member of the onedistribution or the other. As did theplayer of the dice game, the observermust establish a policy which definesthe circumstances under which the ob-servation will be regarded as resultingfrom each of the two possible events.He establishes a criterion, a cutoff xcon the continuum of observations, towhich he can relate any given observa-tion Xi. If he finds for the «th ob-servation, xt, that Xi > xc, he says"yes"; if xt < xc, he says "no." Sincethe observer is assumed to be capableof locating a criterion at any pointalong the continuum of observations, itis of interest to examine the variousfactors that, according to the theory,will influence his choice of a particular

criterion. To do so requires some ad-ditional notation.

In the language of statistical decisiontheory the observer chooses a subsetof all of the observations, namely theCritical Region A, such that an ob-servation in this subset leads him toaccept the Hypothesis SN, to say thata signal was present. All other ob-servations are in the complementarySubset B; these lead to rejection ofthe Hypothesis SN, or, equivalently,since the two hypotheses, are mutuallyexclusive and exhaustive, to the ac-ceptance of the Hypothesis N. TheCritical Region A, with reference toFigure 2, consists of the values of xto the right of some criterion value xc.

As in the case of the dice game, adecision will have one of four out-comes: the observer may say "yes" or"no" and may in either case be corrector incorrect. The decision outcome, inother words, may be a hit (SN-A, thejoint occurrence of the Hypothesis SNand an observation in the Region A~),a miss (SN-B), a correct rejection(N-B), or a false alarm (N-A). Ifthe a priori probability of signal occur-rence and the parameters of the distri-butions of Figure 2 are fixed, thechoice of a criterion value xc completelydetermines the probability of each ofthese outcomes.

Clearly, the four probabilities areinterdependent. For example, an in-crease in the probability of a hit,p(SN-A), can be achieved only byaccepting an increase in the probabilityof a false alarm, p(N-A), and de-creases in the other probabilities,p (SN • B) and p (TV • B). Thus a givencriterion yields a particular balanceamong the probabilities of the four pos-sible outcomes; conversely, the balancedesired by an observer in any instancewill determine the optimal location ofhis criterion. Now the observer maydesire the balance that maximizes the


expected value of a decision in a situa-tion where the four possible outcomesof a decision have individual values,as did the player of the dice game. Inthis case, the location of the best cri-terion is determined by the same pa-rameters that determined it in the dicegame. The observer, however, may de-sire a balance that maximizes someother quantity—i.e., a balance that isoptimum according to some other defi-nition of optimum—in which case adifferent criterion will be appropriate.He may, for example, want to maxi-mize p(SN-A) while satisfying a re-striction on p(N-A), as we typicallydo when as experimenters we assumean .05 or .01 level of confidence. Al-ternatively, he may want to maximizethe number of correct decisions. Again,he may prefer a criterion that willmaximize the reduction in uncertaintyin the Shannon (1948) sense.

In statistical decision theory, and inthe theory of signal detectability, theoptimal criterion under each of thesedefinitions of optimum is specified interms of the likelihood ratio. That isto say, it can be shown that, if we de-fine the observation in terms of thelikelihood ratio, A.(#) = fstf(#)/fff(.*0,then the optimal criterion can alwaysbe specified by some value JB of A(JF).In other words, the Critical Region Athat corresponds to the criterion con-tains all observations with likelihoodratio greater than or equal to /?, andnone of those with likelihood ratio lessthan /3.

We shall illustrate this manner ofspecifying the optimal criterion for justone of the definitions of optimum pro-posed above, namely, the maximizationof the total expected value of a decisionin a situation where the four possibleoutcomes of a decision have individualvalues associated with them. This isthe definition of optimum that we as-sumed in the dice game. For this pur-

pose we shall need the concept of con-ditional probability as opposed to theprobability of joint occurrence intro-duced above. It should be stated thatconditional probabilities will have aplace in our discussion beyond their usein this illustration; the ones we shallintroduce are, as a matter of fact, thefundamental quantities in evaluatingthe observer's performance.

There are two conditional probabili-ties of principal interest. These arethe conditional probabilities of the ob-server saying "yes" : psx(A), the prob-ability of a Yes decision conditionalupon, or given, the occurrence of asignal, and pN(A), the probability ofa Yes decision given the occurrenceof noise alone. These two are suffi-cient, for the other two are simply theircomplements: PHN(B) — \ — pSN(A)and ps(B) - 1 - pN(A). The condi-tional and joint probabilities are relatedas follows:

PN(A) =

p(SN-A)p(SN)

p ( N - A )p(N)

[1]

where: p (SN) is the a priori probabilityof signal occurrence and p(N)=\ —p(SN) is the a priori probability of occur-rence of noise alone.

Equation 1 makes apparent the con-venience of using conditional ratherthan joint probabilities—conditionalprobabilities are independent of the apriori probability of occurrence of thesignal and of noise alone. With refer-ence to Figure 2, we may definePSN(A), or the conditional probabilityof a hit, as the integral of iaN(x) overthe Critical Region A, and pN(A), theconditional probability of a false alarm,as the integral of f#(#) over A. Thatis, pN(A) and pan (A) represent, re-spectively, the areas under the two


curves of Figure 2 to the right of somecriterion value of x.

To pursue our illustration of how anoptimal criterion may be specified by acritical value of likelihood ratio ft, letus note that the expected value of adecision (denoted EV) is defined instatistical decision theory as the sum,over the potential outcomes of a deci-sion, of the products of probability ofoutcome and the desirability of out-come. Thus, using the notation V forpositive individual values and K forcosts or negative individual values, wehave the following equation:

EV = VsN.Ap(SN-A)+ VN.Bp(N-B)- K3N.Bp(SN-B)

-KN.Ap(N-A) [2]

Now if a priori and conditionalprobabilities are substituted for thejoint probabilities in Equation 2 fol-lowing Equation 1, for example,p(SN)psN(A) for p(SN-A), thencollecting terms yields the result thatmaximizing EV is equivalent to maxi-mizing :

PSN(A) — 0pff(A) [3]where

0 P(N) (VN.B + KN.A) ,_,_,

It can be shown that this value of ftis equal to the value of likelihood ratio,X(x], that corresponds to the optimalcriterion. From Equation 3 it may beseen that the value ft simply weightsthe hits and false alarms, and fromEquation 4 we see that ft is determinedby the a priori probabilities of occur-rence of signal and of noise alone andby the values associated with the indi-vidual decision outcomes. It shouldbe noted that Equation 3 applies to alldefinitions of optimum. Equation 4

shows the determinants of ft in onlythe special case of the expected-valuedefinition of optimum.

Return for a moment to Figure 2,keeping in mind the result that ft is a.critical value of \(x) = ias(x)/is(x).It should be clear that the optimal cut-off xc along the x axis is at the pointon this axis where the ratio of the ordi-nate value of ias(x) to the ordinatevalue of ifi(x) is a certain number,namely ft. In the symmetrical case,where the two a priori probabilities areequal and the four individual values areequal, ft = 1 and the optimal value ofxc is the point where iss(x) — fw(^),where the two curves cross. If thefour values are equal but p(SN) =5/6 and p (N) = 1/6, another case de-scribed in connection with the dicegame, then ft = 1/5 and the optimalvalue of xc is shifted a certain distanceto the left. This shift may be seenintuitively to be in the proper direction—a higher value of p ( S N ) should leadto a greater willingness to accept theHypothesis SN, i.e., a more lenient cut-off. To consider one more examplefrom the dice game, if p(SN) = p ( N )= 0.5, if VN.S and KN.A. are set at 5dollars and VBN-A and Kas.B are equalto 1 dollar, then ft = 5 and the optimalvalue of xc shifts a certain distance tothe right. Again intuitively, if it ismore important to be correct when theHypothesis N is true, a high, or strict,criterion should be adopted.

In any case, ft specifies the optimalweighting of hits relative to falsealarms: xe should always be located atthe point on the x axis correspondingto ft. As we pointed out in discussingthe dice game, just where this value ofx0 will be with reference to the x axisdepends not only upon the a prioriprobabilities and the values but alsoupon the overlap of the two densityfunctions, in short, upon the signalstrength. We shall define a measure


of signal strength within the next fewpages. For now, it is important to notethat for any detection goal to which theobserver may subscribe, and for anyset of parameters that may characterizea detection situation (such as a prioriprobabilities and values associated withdecision outcomes), the optimal crite-rion may be specified in terms of asingle number, /?, a critical value oflikelihood ratio."

Receiver-Operating-CharacteristicWhatever criterion the observer ac-

tually uses, even if it is not one of theoptimal criteria, can also be describedby a single number, by some value oflikelihood ratio. Let us proceed to aconsideration of how the observer'sperformance may be evaluated with re-spect to the location of his criterion,and, at the same time we shall see howhis performance may be evaluated withrespect to his sensory capabilities.

As we have noted, the fundamentalquantities in the evaluation of per-formance are p N ( A } and p8N(A}, thesequantities representing, respectively,the areas under the two curves of Fig-ure 2 to the right of some criterionvalue of x. If we set up a graph ofpsx(A) versus pn(A) and trace on itthe curve resulting as we move thedecision criterion along the decision

6 We have reached a point in the discus-sion where we can justify the statementmade earlier that the decision axis may beequally well regarded as likelihood ratio oras any monotonic transformation of likeli-hood ratio. Any distortion of the linearcontinuum of likelihood ratio, that maintainsorder, is equivalent to likelihood ratio interms of determining a criterion. The de-cions made are the same whether thecriterion is set at likelihood ratio equal to§ or at the value that corresponds to /3 ofsome new variable. To illustrate, if acriterion leads to a Yes response when-ever \(y)>2, if x= [\(y)Y the decisionswill be the same if the observer says "yes"whenever x > 4.

FIG. 3. The receiver-operating-character-istic curves. (These curves show pan (A)vs. ps(A) with d' as the parameter. Theyare based on the assumptions that the prob-ability density functions, f»O) and f «»(#),are normal and of equal variance.)

axis of Figure 2, we sketch one of thearcs shown in Figure 3. Ignore, fora moment, all but one of these arcs.If the decision criterion is set way atthe left in Figure 2, we obtain a pointin the upper right-hand corner of Fig-ure 3: both psn(A) and p y ( A ) areunity. If the criterion is set at theright end of the decision axis in Figure2, the point at the other extreme ofFigure 3, pan (A} — pN(A) = 0, is ob-tained. In between these extremes liethe criterion values of more practicalinterest. It should be noted that theexact form of the curve shown in Fig-ure 3 is not the only form which mightresult, but it is the form which willresult if the observer chooses a crite-rion in terms of likelihood ratio, andthe probability density functions arenormal and of equal variance.

This curve is a form of the operatingcharacteristic as it is known in statis-tics; in the context of the detectionproblem it is usually referred to asthe receiver-operating-characteristic, orROC, curve. The optimal "operatinglevel" may be seen from Equation 3 to

310 J. A. SWETS, W. P. TANNER, JR., AND T, G. BIRDSALL

be at the point of the ROC curve whereits slope is /?. That is, the expressionPair (A) — /3pN (A) defines a utility lineof slope /?, and the point of tangencyof this line to the ROC curve is theoptimal operating level. Thus thetheory specifies the appropriate hitprobability and false alarm probabilityfor any definition of optimum and anyset of parameters characterizing thedetection situation.

It is now apparent how the observ-er's choice of a criterion in a givenexperiment may be indexed. The pro-portions obtained in an experiment areused as estimates of the probabilities,px(A) and pan (A} ; thus, the observ-er's behavior yields a point on an ROCcurve. The slope of the curve at thispoint corresponds to the value of like-lihood ratio at which he has locatedhis criterion. Thus we work backwardfrom the ROC curve to infer the cri-terion that is employed by the observer.

There is, of course, a family of ROCcurves, as shown in Figure 3, a givencurve corresponding to a given separa-tion between the means of the densityfunctions iu(x~) and fay(.*•). The pa-rameter of these curves has been calledd'', where d' is defined as the differencebetween the means of the two densityfunctions expressed in terms of theirstandard deviation, i.e.:

d' = — MfN (*> [5]

Since the separation between the meansof the two density functions is a func-tion of signal amplitude, d' is an indexof the detectability of a given signalfor a given observer.

Recalling our assumptions that thedensity functions iy(x) and !«»(#) arenormal and of equal variance, we maysee from Equation 5 that the quantitydenoted d' is simply the familiar nor-mal deviate, or x/a measure. From the

pair of values px(A) and ps^(A) thatare obtained experimentally, one mayproceed to a published table of areasunder the normal curve to determinea value of d'. A simpler computationalprocedure is achieved by plotting thepoints [pir(A'), PSN(A)} on graphpaper having a probability scale and anormal deviate scale on both axes.

We see now that the four-fold tableof the responses that are made to aparticular stimulus may be treated ashaving two independent parameters—the experiment yields measures of twoindependent aspects of the observer'sperformance. The variable d' is ameasure of the observer's sensory capa-bilities, or of the effective signalstrength. This may be thought of asthe object of interest in classical psy-chophysics. The criterion /3 that isemployed by the observer, which deter-mines the ps(A) and pSN(A) for somefixed d', reflects the effect of variableswhich have been variously called theset, attitude, or motives of the observer.It is the ability to distinguish betweenthese two aspects of detection perform-ance that comprises one of the mainadvantages of the theory proposed here.We have noted that these two aspectsof behavior are confounded in an ex-periment in which the dependent varia-ble is the intensity of the signal that isrequired for a threshold response.

Relationship oj d' to Signal EnergyWe have seen that the optimal value

of the criterion, /3, can be computed.In certain instances, an optimal valueof d', i.e., the sensitivity of the mathe-matically ideal device, can also be com-puted. If, for example, the exact waveform and starting time of the signalare determinable, as in the case of anauditory signal, then the optimal valueof d' is equal to ^2E/N0, where E isthe signal energy and N0 is the noise


power in a one-cycle band (Peterson,Birdsall, & Fox, 1954). A specifica-tion of the optimal value of d' forvisual signals has been developed veryrecently.7 Although we shall not elabo-rate the point in this paper, it is worthnoting that an empirical index of de-tectability may be compared with idealdetectability, just as observed and opti-mal indices of decision criteria may becompared. The ratio of the squares ofthe two detectability indices has beentaken as a measure of the observer'ssensory efficiency. This measure hasdemonstrated its usefulness in thestudy of several problems in audition(Tanner & Birdsall, 1958).

Use of Ideal Descriptions as ModelsIt might be worthwhile to describe

at this point some of the reasons forthe emphasis placed here on optimalmeasures, and, indeed, the reasons forthe general enterprise of considering atheory of ideal behavior as a model forstudies of real behavior.8 In view ofthe deviations from any ideal which arebound to characterize real organisms,it might appear at first glance that anydeductions based on ideal premisescould have no more than academic in-terest. We do not think this is thecase. In any study, it is desirable tospecify rigorously the factors pertinentto the study. Ideal conditions generallyinvolve few variables and permit theseto be described in simple terms. Hav-ing identified the performance to beexpected under ideal conditions, it ispossible to extend the model to includethe additional variables associated withreal organisms. The ideal perform-ance, in other words, constitutes a con-venient base from which to explore the

7 W. P. Tanner, Jr. & R. C. Jones, per-sonal communication, November 1959.

8 The discussion immediately following is,in part, a paraphrase of one in Horton(1957).

complex operation of a real organism.In certain cases, as in the problem

at hand, values characteristic of idealconditions may actually approximatevery closely those characteristics of theorganism under study. The problemthen becomes one of changing the idealmodel in some particular so that it isslightly less than ideal. This is usuallyaccomplished by depriving the idealdevice of some particular function.This method of attack has been foundto generate useful hypotheses for fur-ther studies. Thus, whereas it is notexpected that the human observer andthe ideal detection device will behaveidentically, the emphasis in early stud-ies is on similarities. If the differencesare small, one may rule out entireclasses of alternative models, and re-gard the model in question as a usefultool in further studies. Proceeding onthis assumption, one may then in laterstudies emphasize the differences, theform and extent of the differences sug-gesting how the ideal model may bemodified in the direction of reality.

Alternative Conceptions of theDetection Process

The earliest studies that were under-taken to test the applicability of thedecision model to human observerswere quite naturally oriented towarddetermining its value relative to exist-ing psychophysical theory. As a re-sult, some of the data presented beloware meaningful only with respect todifferences in the predictions basedupon different theories. We shall,therefore, briefly consider alternativetheories of the detection process.

Although it is difficult to specify withprecision the alternative theories of de-tection, it is clear that they generallyinvolve the concept of the threshold inan important way. The developmentof the threshold concept is fairly ob-scure. It is differently conceived by


different people, and few popularusages of the concept benefit from ex-plicit statement. One respect, how-ever, in which the meaning of thethreshold concept is entirely clear is itsassertion of a lower limit on sensitivity.As we have just seen, the decisionmodel does not include such a boun-dary. The decision model specifies nolower bound on the location of thecriterion along the continuous axis ofsensory inputs. Further, it implies thatany displacement of the mean of lay(.v)from the mean of iy(x), no matter howsmall, will result in a greater value ofPSN(A) than pa (A), irrespective of thelocation of the criterion.

To permit experimental comparisonof decision theory and threshold theory,we shall consider a special version ofthreshold theory (Blackwell, 1953).Although it is a special version, webelieve it retains the essence of thethreshold concept. In this version, thethreshold is described in the sameterms that are used in the descriptionof decision theory. It is regarded asa cutoff on the continuum of observa-tions (see Figure 2) with a fixed loca-tion, with values of x above the cutoffalways evoking a positive response, andwith discrimination impossible amongvalues of x below the cutoff. This de-scription of a threshold in terms of a

\

\\

.0001 .001, 01

FIG. 4. The relationship between d' andps(A) at threshold.

fixed cutoff and a stimulus effect thatvaries randomly, it will be noted, isentirely equivalent to the more commondescription in terms of a randomlyvarying cutoff and a fixed stimulus ef-fect. There are several reasons forassuming that the hypothetical thresh-old cutoff is located quite high relativeto the density function £#(#), say atapproximately +3o- from the mean oftv (.*•). We shall compare our datawith the predictions of such a "highthreshold" theory, and shall indicatetheir relationship to predictions from atheory assuming a lower threshold.We shall, in particular, ask how low athreshold cutoff would have to be tobe consistent with the reported data.It may be noted that if a high thresholdexists, the observer will be incapableof ordering values of x likely to resultfrom noise alone, and hence will be in-capable of varying his criterion overa significant range.

If a threshold exists that is rarelyexceeded by noise alone, this fact willbe immediately apparent from the ROCcurves (see Figure 3) that are obtainedexperimentally. It can be shown thatthe ROC curves in this case arestraight lines from points on the left-hand vertical axis—pay (A)—to theupper right-hand corner of the plot.These straight line curves represent theimplication of a high threshold theorythat an increase in pir(A') must be ef-fected by responding "yes" to a randomselection of observations that fail toreach the threshold, rather than by ajudicious selection of observations, i.e.,a lower criterion level. If we followthe usual procedure of regarding thestimulus threshold as the signal inten-sity yielding a value of psy(A) —0,5for py(A) = 0.0, then an appreciationof the relationship between d' andp l f ( A ) at threshold may be gained byvisualizing a straight line in Figure 3from this point to the upper right-hand


corner. If we note which of the ROCcurves drawn in Figure 3 are inter-sected by the visualized line, we seethat the threshold decreases with in-creasing ps(A). For example, a re-sponse procedure resulting in a px(A)— 0.02 requires a signal of d' = 2.0 toreach the threshold, whereas a responseprocedure yielding a. pN(A) — 0.98 re-quires a signal of d' < 0.5 to reach thethreshold. A graph showing whatthreshold would be calculated as a func-tion of ps(A) is plotted in Figure 4.The calculated threshold is a strictlymonotonic function of p$(A) rangingfrom infinity to zero.

The fundamental difference betweenthe threshold theory we are consideringand decision theory lies in their treat-ment of false alarm responses. Accord-ing to the threshold theory, these re-sponses represent guesses determinedby nonsensory factors; i.e, pn(A} isindependent of the cutoff which is as-sumed to have a fixed location. Deci-sion theory assumes, on the other hand,that ps (A) varies with the temporaryposition of a cutoff under the observer'scontrol; that false alarm responsesarise for valid sensory reasons, andthat therefore a simple correction willnot eliminate their effect on PSN(A).A similar implication of Figure 4 thatshould be noted is that reliable esti-mates of PSN(A) or of the stimulusthreshold are not guaranteed by simplytraining the observer to maintain a low,constant value of pN(A}. Since ex-treme probabilities cannot be estimatedwith reliability, the criterion may varyfrom session to session with the varia-tion having no direct reflection in thedata. Certainly, false alarm rates of0.01, 0.001, and 0.0001, are not dis-criminable in an experimentally feasi-ble number of observations; the differ-ences in the calculated values of thethreshold associated with these differentvalues of px(A) may be seen from

Figure 4 to be sizeable. The experi-ments reported in the following weredesigned, in large measure, to clarifythe relationship that exists betweenPN(A) and pSN(A), to show whetheror not the observer is capable of con-trolling the location of his criterion fora Yes response.

SOME EXPERIMENTSFive experiments are reported in the

following. They are the first experi-ments that were undertaken to test theapplicability of decision theory to psy-chophysical tasks, and it must be em-phasized that they were intended toexplore only the general relationshipsspecified in the theory. We shall referalso to more recent experiments con-ducted within the framework of deci-sion theory. The later experiments,although not focused as directly ontesting the validity of the theory, sup-port the principal thesis of this paper.

The experiments reported here aredevoted to answering the two principalquestions suggested by a considerationof decision theory. The first of thesemay be stated in this way: is sensoryinformation (or the decision axis) con-tinuous, i.e., is the observer capableof discriminating among observationslikely to result from noise alone ? Thealternative we consider is that thereexists a threshold cut, on the decisionaxis, that is unlikely to be exceeded byobservations resulting from noise, andbelow which discrimination among ob-servations is impossible. The secondquestion has two parts: is the observercapable of using different criteria, and,if so, does he change his criterion ap-propriately when the variables that weexpect will determine his criterion(probabilities, values, and costs) arechanged ?

Three of the five experiments to bedescribed pose for the observer whatwe have called the fundamental detec-


tion problem, the problem that occupiedour attention throughout the theoreticaldiscussion. Of these, two test the ob-server's ability to use the criterion thatmaximizes the expected value of a de-cision. The a priori probability of asignal occurrence and the individualvalues associated with the four possibledecision outcomes are varied systemati-cally, in order to determine the rangeover which the observer can vary hiscriterion and the form of the resultantROC curve. A third experiment teststhe observer's ability to maximize theproportion of hits while satisfying arestriction on the proportion of falsealarms. This experiment is largely con-cerned with the degree of precisionwith which the observer can locate acriterion.

The remaining two experiments dif-fer in that the tasks they present to theobserver do not require him to estab-lish a criterion, that is, they do not re-quire a Yes or No response. Theytest certain implications of decisiontheory that we have not yet treatedexplicitly, but they will be seen to fol-low very directly from the theory andto contribute significantly to an evalua-tion of it. In one of these the observeris asked to report after each observa-tion interval his subjective probabilitythat the signal existed during the in-terval. This response is a familiar one;it is essentially a rating or a judgmentof confidence. The report of "a poste-riori probability of signal existence,"as it is termed in detection theory, maybe regarded as reflecting the likelihoodratio of the observation, This case isof interest since an estimate of likeli-hood ratio preserves more of the in-formation contained in the observationthan does a report merely that the like-lihood ratio fell above or below a criti-cal value. We shall see that it is alsopossible to construct the ROC curvefrom this type of response.

The other experiment not requiringa criterion employs what has beentermed the temporal forced-choicemethod of response. On each trial asignal is presented in exactly one ofM temporal intervals, and the observerstates in which interval he believes thesignal occurred. The optimal proce-dure for the observer to follow in thiscase, if he is to maximize the proba-bility of a correct response, is to makean observation x in each interval andto choose the interval having the great-est value of x associated with it. Sincedecision theory specifies how the pro-portion of correct responses obtainedwith the forced-choice method is re-lated to the detectability index d', theinternal consistency of the theory maybe evaluated. That is to say, if theobserver follows the optimal procedure,then the estimate of the detectability ofa signal of a given strength that isbased on forced-choice data will becomparable to that based on yes-nodata. The forced-choice method mayalso be used to make a strong test ofa fundamental assumption of decisiontheory, namely, that sensory informa-tion is continuous, or that sensory in-formation does not exhibit a thresholdcutoff. For an experiment requiringthe observer to rank the n intervalsaccording to their likelihood of contain-ing the signal, the continuity andthreshold assumptions lead to very dif-ferent predictions concerning the prob-ability that an interval ranked otherthan first will be the correct interval.

All of the experiments reported inthe following employed a circular sig-nal with a diameter of 30 minutes ofvisual angle and a duration of %,>oof a second. The signal was presentedon a large uniformly illuminated back-ground having a luminance of 10 foot-lamberts. Details of the apparatus havebeen presented elsewhere (Blackwell,Pritchard, & Ohmart, 1954).


Maximising the Expected Value of aDecision-—An Experimental Analysis

A direct test of the decision modelis achieved in an experiment in whichthe a priori probability of signal occur-rence or the values of the decision out-comes, or both, are varied from onegroup of observations to another-—inshort, in which ft (Equations 3 and 4)assumes different values. The ob-server, in order to maximize his ex-pected value, or his payoff, must varyhis willingness to make a Yes re-sponse, in accordance with the changein /?. Variations in this respect willbe indicated by the proportion of falsealarms, py(A). The point of interestis how PSN(A), the proportion of hits,varies with changes in pN(A), i.e., inthe form of the observer's ROC curve.If the experimental values of py(A)reflect the location of the observer'scriterion, if the observer responds onthe basis of the likelihood ratio of theobservation, and if the density func-tions (Figure 2) are normal and ofequal variance, the ROC curve of Fig-ure 3 will result. If, on the other hand,the location of the criterion is fixed insuch a position that it is rarely ex-ceeded by noise alone, then the result-ing ROC curve will be a straight line,as we have indicated above. We shallexamine some empirical ROC curveswith this distinction in mind.

This experiment can be made toyield another and, in one sense, astronger test of these two hypotheses,by employing several values of signalstrength within a single group of ob-servations, i.e., while a given set ofprobabilities and values are in effect.For in this case stimulus thresholdscan be calculated, and correlationaltechniques can be used to determinewhether the calculated threshold is de-pendent upon pir(A) as predicted bydecision theory, or independent of

as predicted by what we havetermed the high threshold theory. Wewill grant that presenting more thanone value of signal strength, within asingle group of observations to whichfixed probabilities and values apply, isnot, conceptually, the simplest experi-ment that could have been performedto test our hypotheses. Nevertheless,a little reflection will show that thisexperimental procedure is entirely le-gitimate from any of our present pointsof view. We simply associate severalvalues of psy(A) with a given valueof pN(A), and thereby obtain at oncea point on each of several ROC curvesand an estimate of the stimulus thresh-old that is associated with that valueo f p N ( A ) .

First Expected-Value Experiment.The first of the two expected-valueexperiments that were performed em-ployed four values of signal strength.

Three observers, after considerable prac-tice, served in 16 2-hour sessions. In eachsession, signals at four levels of intensity(0.44, 0.69, 0.92, and 1.20 foot-lamberts)were presented along with a "blank" or "no-signal" presentation. The order of presen-tation was random within a restriction placedupon the total number of occurrences of eachsignal intensity and the blank in a givensession. Each of the signal intensities oc-curred equally often within a session. Theproportion of trials on which a signal (ofany intensity) was presented, p(SN), waseither 0.80 or 0.40 in the various sessions.In all, there were 300 presentations in eachsession—six blocks of SO presentations, sepa-rated by rest periods. Thus each estimateof ps(A) is based on either 60 or 180 ob-servations, and each estimate of pan (A) isbased on 30 or 60 observations, dependingupon p(SN).

In the first four sessions, no values wereassociated with the various decision out-comes. For the first and fourth sessions theobservers were informed that p(SN) = Q.&Qand, for the second and third sessions, thatp(SN)=OAO. The average value of ps(A)obtained in the sessions with p(SN) = Q.&0was 0.43, and, in the sessions with p(SN) =0.40, it was 0.15—indicating that the ob-server's willingness to make a Yes response


O.I 0.2 0.3 04 0.5 06 0.7 08 0.9 10PM ( A )

O.I 0.2 03 0.4 0.5 0.6 0.7PN Wl

08 0.9 1.0

FIG. 5. Empirical receiver-operating-characteristic curves obtained from threeobservers in the first expected-value experi-ment.

is significantly affected by changes in j>(SN)alone. In the remaining 12 sessions, thesetwo values of p(SN) were used in conjunc-tion with a variety of values placed on thedecision outcomes. In the fifth session, forexample, the observers were told thatp(SN)= 0.80 and were, in addition, giventhe following payoff matrix:

No Yes

Signal

No Signal

- 1KSN-B

+ 2VN.B

+ 1VsN-A

- 2K f f . A

A variety of simple matrices was used.These included, reading from left to rightacross the top and then the bottom row:(-1, +1, +3, -3) and (-1, +1, +4,-4) with p(SN)= 0.80, and (-1, + 1, + 2,-2), (-1, +1, +1, -1), (-2, +2, +1,- 1), and (-3, +3, +1, -1) withp(SN)=QAQ. By reference to Equation 4,it may be seen that these matrices andvalues of p(SN) define values of /3 rangingfrom 0.25 to 3.00. The observers wereactually paid in accordance with these payoffmatrices, in addition to their regular wage.The values were equated with fractions ofcents, these fractions being adjusted so thatthe expected earnings per session remainedrelatively constant, at approximately onedollar.

The obtained values of p$(A) variedin accordance with changes in the val-ues of the decision outcomes as well aswith changes in the a priori probabilityof signal occurrence. Just how closelythe obtained values of pN(A~) ap-proached those specified as optimal bythe theory, we shall discuss shortly.For now, we may note that the rangeof values of PN(^) obtained from thethree observers is shown in Figure 5.The parts of this figure also show fourvalues of PSN(A) corresponding toeach value of PN(A) ; the four valuesof pas (A), one for each signalstrength, are indicated by differentsymbols. We have, then, in the partsof Figure 5, four ROC curves.


Although entire ROC curves are notprecisely defined by the data of thefirst experiment, these data will con-tribute to our purpose of distinguishingbetween the predictions of decisiontheory and the predictions of a highthreshold theory. It is clear, for ex-ample, that the straight lines fitted tothe data do not intersect the upperright-hand corner of the graph, as re-quired by the concept of a highthreshold.

We have mentioned that anotheranalysis of the data is of interest indistinguishing the two theories we areconsidering. As we have indicatedearlier in this paper, and developed inmore detail elsewhere (Tanner &Swets, 1954), the concept of a highthreshold leads to the prediction thatthe stimulus threshold is independentof pN(A), whereas decision theory pre-dicts a negative correlation betweenthe stimulus threshold and pN(A).Within the framework of the highthreshold model that we have de-scribed, the stimulus threshold is de-fined as the stimulus intensity thatyields a pSN(A) - 0.50 for pv(A)— 0.0. This stimulus intensity may bedetermined by interpolation from psy-chometric functions—PSN(A) vs. sig-nal intensity—that are normalized sothat pif(A) — 0.0. The normalizationis effected by the equation:

* (A\ ps»(A)-pK(A)PSN (A ,) corrected = , _ , , , -. L°J

commonly known as the "correctionfor chance success." The intent of thecorrection is to remove what has beenregarded as the spurious element ofPSN (A) that is contributed by an ob-server's tendency to make a Yes re-sponse in the absence of any sensoryindication of a signal, i.e., to make aYes response following an observa-tion that fails to reach the threshold

level. It can be shown that the validityof this correction procedure is impliedby the assumption of what we havetermed a high threshold. The decisionmodel, as we have indicated, differs inthat it regards sensory information asthoroughly probabilistic, without afixed cutoff—it asserts that the pres-ence and absence of some sensory indi-cation of a signal are not separablecategories. According to the decisionmodel, the observer does not achievemore Yes responses by respondingpositively to a random selection of ob-servations that fall short of the fixedcriterion level, but by lowering his cri-terion. In this case, the chance cor-rection is inappropriate; the stimulusthreshold will not remain invariantwith changes in pN(A).

The relationship of the stimulusthreshold to ps(A) in this first experi-ment is illustrated by Figures 6 and 7.The portion of data comprising each ofthe curves in these figures was selectedto be relatively homogeneous with re-spect to pN(A). The curves are aver-age curves for the three observers.

FIG. 6. The relationship between thestimulus threshold and ps(A) with the pro-portion of positive responses to four positivevalues of signal intensity, pas (A), and tothe blank or zero-intensity presentation,ps(A), at three values of ps(A).


FIG. 7. The relationship between thestimulus threshold and pa (A) with the threecurves corrected for chance success, byEquation 6.

Figure 6 shows ps(A~) and pan (as a function of the signal intensity,Al. The intercepts of the three curvesmay be seen to indicate values ofps(A) of 0.35, 0.25, and 0.04, respec-tively. Figure 7 shows the correctedvalue of PSN(A) plotted against signalintensity. It may be seen in Figure 7that the stimulus threshold — the valueof A I corresponding to a correctedpsn(A) of 0.50 — is dependent uponPN(A) in the direction predicted bydecision theory.9

Figures 6 and 7 portray the relation-ship in question in a form to whichmany of us are accustomed ; they arepresented here only for illustrative pur-poses. We can, of course, achieve astronger test by computing the coeffi-cients of correlation between px(A)and the calculated threshold. We have

9 AT is plotted in Figures 6 and 7 in termsof the transmission values of the filters thatwere placed selectively in the signal beamto yield different signal intensities. Thesevalues (0.365, O.S7S, 0.76S, 1.000) are con-verted to the signal values in terms of foot-lamberts that we have presented above, bymultiplying them by 1.20, the value of thesignal in foot-lamberts without selectivefiltering.

made this computation, and have in theprocess avoided the averaging of dataobtained from different observers anddifferent experimental sessions. Theproduct-moment coefficients for thethree observers are —.37 (/> =0.245),-.60 0 = 0.039), and -.81 (p =0.001), respectively. For the threeobservers combined, p = 0.0008. Theimplication of these correlations is thesame as that of the straight lines fittedto the data of Figure 5, namely, thata dependence exists between the con-ditional probability that an observa-tion arising from SN will exceed thecriterion and the conditional probabil-ity that an observation arising from Nwill exceed the criterion. Stated other-wise, the correlations indicate that theobserver's decision function is likeli-hood ratio or some monotonic functionof it and that he is capable of adoptingdifferent criteria.

Second Expected-Value Experiment.A second expected-value experimentwas conducted to obtain a more precisedefinition of the ROC curve than thatprovided by the experiment just de-scribed. In the second experimentgreater definition was achieved by in-creasing the number of observationson which the estimates of pan (A) andpx(A) were based, and by increasingthe range of values of ps(A).

In this experiment only one signal inten-sity (0.78 foot-lamberts) was employed.Each of 13 experimental sessions included200 presentations of the signal, and 200presentations of noise alone. Thus, p(SN)remained constant at 0.50 throughout thisexperiment. Changes in the optimal criterion/3, and thus in the obtained values of ps(A),were effected entirely by changes in thevalues associated with the decision outcomes.These values were manipulated to yield (3's(Equation 4) varying from 0.16 to 8.00. Adifferent set of observers served in thisexperiment.

The results are portrayed in Figure8, It may be seen that the experimen-


tally determined points are fitted quitewell by the type of ROC curve that ispredicted by decision theory. It isequally apparent, excepting Observer 1,that the points do not lie along astraight line intersecting the pointpir(A) — PSN(A) = 1.00, as predictedby the high threshold model.

One other feature of these figures isworthy of note. It will be recalled thatin our presentation of decision theorywe tentatively assumed that the densityfunctions of noise and of signal plusnoise, iy(x) and f «#(.*•)> are of equalvariance. Although we did not, inorder to preserve the continuity of the

discussion, we might have acknowl-edged at that point that the assumptionof equal variance is not necessarily thebest one. In particular, one mightrather expect the variance of fsw(#)to be proportional to its mean. At anyrate, the assumption made about vari-ances represents a degree of freedomof the theory that we have not empha-sized previously. We have, however,used this degree of freedom in the con-struction of the theoretical ROC curvesof Figure 8. Notice that these curvesare not symmetrical about the diagonal,as are the curves of Figure 3 that arepredicated on equal variance. The

L

70 Q 1 02 05 0 4 O S

V*1

0.7 06 09 1.0

0 O.I 0.:

FIG. 8. Empirical receiver-operating-characteristic curves for four observers inthe second expected-value experiment.


FIG. 9. The probability of a correctchoice in a four-alternative forced-choiceexperiment as a function of d'.

curves of Figure 8 are based on theassumption that the ratio of the incre-ment of the mean of fs.v(^) to the in-crement of its standard deviation isequal to 4, AJkf/Ao- = 4. A close lookat these figures suggests that ROCcurves calculated from a still greaterratio would provide a still better fit.Since other data presented in the fol-lowing bear directly on this questionof a dependence between variance andsignal strength, we shall postpone fur-ther discussion of it. We shall alsoconsider later whether the exact formof the empirical ROC curves supportsthe assumption of normality of thedensity functions is(x) and f8Af(^).For now, the main point is that deci-sion theory predicts the curvilinearform of the ROC curves that areyielded by the observers.

Forced-Choice ExperimentsWe have indicated above that an ex-

tension of the decision model may bemade to predict performance in aforced-choice test. On each trial of atypical forced-choice test, the signal ispresented in one of n temporal inter-vals, and the observer selects the inter-val he believes to have contained thesignal. It will be intuitively clear that,

to behave optimally, in the sense ofmaximizing the probability of a correctresponse, the observer must make anobservation x in each interval, andchoose the interval having the greatestvalue of x associated with it. Equiva-lently, he may rank the intervals ac-cording to their values of likelihoodratio and choose that interval yieldingthe greatest value of likelihood ratio.

If the observer behaves optimally,then the probability that a correct an-swer will result, p ( c ) , for a given valueof d', is expressed by:

/.+O

=J—00

[7]

where: i(x) is the area of the noise func-tion to the left of x, g(x) is the ordinateof the signal-plus-noise function, and n isthe number of intervals used in the test.

This is simply the probability that onedrawing from the distribution due tosignal plus noise is greater than thegreatest of w-1 drawings from the dis-tribution due to noise alone.

It is intuitively clear that if the sig-nal produces a large shift in the noisefunction, i.e., if d' is large, then theprobability that the greatest value of xwill be obtained in the interval thatcontains the signal is also large, andconversely—indeed (for a fixed num-ber of intervals) p ( c ) is a monotonicfunction of d'. Equation 7 can be seento be a function of d' by noting that,under the assumption of equal vari-ance, the signal-plus-noise function issimply the noise function shifted byd', i.e., g(x~) = i(x — d'). Thus d' maybe defined in a forced-choice experi-ment by determining a value of p ( c }for some signal intensity and thenusing Equation 7 to determine d'. Aplot of p ( c ) versus d', for the case offour intervals, and under the assump-tion of equal variance, is shown inFigure 9.


Estimates of Signal DetectabilityObtained from Different Procedures.According to detection theory, the esti-mates of d' for a signal and backgroundof given intensities should be the sameirrespective of the psychophysical pro-cedure used to collect the data. Thuswe may check the internal consistencyof the theory by comparing estimatesof d' based on yes-no and on forced-choice data. The results of such acomparison have been reported in an-other paper (Tanner & Swets, 1954).It was shown there that estimates ofd' based on the data of the first ex-pected-value experiment that we havepresented above, and on forced-choicetests conducted in cqnjunction with it,are highly consistent with each other.Comparable estimates of d' have alsobeen obtained in auditory experiments—from yes-no and forced-choice pro-cedures, and from forced-choice proce-dures with from two to eight alterna-tives (Swets, 1959). Hence, decisiontheory provides a unification of thedata obtained with different proce-dures; it enables one to predict theperformance in one situation from datacollected in another.

It is a commonplace that calculatedvalues of the stimulus threshold are notindependent of the psychophysical pro-cedure that is employed (Osgood,1953). Of particular relevance to ourpresent concern is the finding thatthresholds obtained with the forced-choice procedure are lower than thoseobtained with the yes-no procedure(Blackwell, 1953). This finding is ac-counted for, in terms of decision the-ory, by the fact that the calculatedthreshold varies monotonically with thefalse alarm rate (see Figure 4)—withhigh thresholds corresponding to lowfalse alarm rates such as were obtainedin these experiments. The dependenceof the stimulus threshold upon the falsealarm rate, however the threshold is

calculated, precludes the existence ofa simple relationship between thresh-olds obtained with the yes-no proce-dure and those obtained with otherresponse procedures. It is also thecase that the normalization of the psy-chometric function provided by thecorrection for chance, or the normali-zation achieved by defining the thresh-old as the stimulus intensity yieldinga proportion of correct responses half-way between chance performance andperfect performance, does not serve torelate forced-choice thresholds obtainedwith different numbers of alternatives.

Theoretical and Experimental Analy-sis of Second Choices. As we haveindicated, a variation of the forced-choice procedure—in which the ob-server indicates his second choice aswell as his first—provides a powerfultest of a basic difference between thedecision model and the high thresholdmodel. If the observer is capable ofdiscriminating among values of the ob-servations x that fail to reach what wehave termed the threshold, i.e., a crite-rion fixed at approximately +3<r fromthe mean of the noise function, thenthe proportion of second choices thatare correct will be considerably higherthan if he is not.10

According to the high thresholdmodel, only very infrequently will morethan one of the n observations of aforced-choice trial exceed the thresh-old. Since the observations which donot exceed the threshold are assumedby the model to be indiscriminable, thesecond choice will be made among then — 1 alternatives on a chance basis.

10 This experiment was suggested to usby R. Z. Norman, formerly a member of theElectronic Defense Group, now at Prince-ton University. The general rationale ofthis experiment, and the results of its appli-cation to the perception of words exposedfor short durations, have been presented byBricker and Chapanis (19S3) and by Howes(19S4).


FIG. 10. The results of the second-choiceexperiment. (The proportions of correctsecond choices are plotted against d'. Thecurve labeled "2nd Choice" represents theprediction of decision theory, assuming thedensity functions to be normal and of equalvariance. The prediction of the high thresh-old theory is shown by the dotted line.)

Thus, for a four-alternative experimentas described in the following, the highthreshold model predicts that, whenthe first choice is incorrect, the proba-bility that the second choice will becorrect is 0.33. This predicted value,it may be noted, is independent of thesignal strength.

Decision theory, on the other hand,implies that the observer is capable ofordering the four alternatives accord-ing to their likelihood of containing thesignal. If this is the case, the propor-tion of correct second choices will begreater than .33. Should one of thesamples of the noise function be thegreatest of the four, leading to an in-correct first choice, the probability thatthe observation from the signal-plus-noise distribution will be the secondgreatest is larger than the probabilitiesthat either of the observations of thenoise distribution will be the secondgreatest. Again, it is intuitively clearthat this probability is a function ofd', or of signal strength—=i.e., the prob-ability that the observation of the sig-nal-plus-noise value will be greaterthan two of the observations of noise

increases with increases in d'. Specifi-cally, the probability of a correct sec-ond choice in a four-alternative, forced-choice test, for a given value of d', isgiven by the expression:

[8]

where the symbols have the samemeaning as in Equation 7. This rela-tionship is plotted in Figure 10 underthe assumptions that the density func-tions of noise and signal plus noise areGaussian and of equal variance. (Thefunction predicted by decision theoryfor the proportion of correct firstchoices in a three-alternative situationis included in Figure 10 to show thatthis function is not the same as thepredicted function of the probability ofof a correct second choice, given anincorrect first choice, for the four-alternative situation).

To distinguish between the two predictions,data were collected from four observers;two of them had served previously in thesecond expected-value experiment, whereasthe other two had received only routineforce-choice training. Each of the observersserved in three experimental sessions. Eachsession included ISO trials in which botha first and second choice were required.

The resulting 12 proportions of cor-rect second choices are plotted againstd' in Figure 10. The values of d' weredetermined by using the proportions ofcorrect first choices as estimates of theprobability of a correct choice, p ( c ) ,and reading the corresponding valuesof d' from the middle curve of Figure10, which is the same curve shown inFigure 9. Although just one value ofsignal intensity was used (0.78 foot-lamberts as in the second expected-value experiment), the values of d'differed sufficiently from one observer


to another to provide an indication ofthe agreement of the data with the twopredicted functions. Additional varia-tion in the estimates of d' resulted fromthe fact that, for two observers, a con-stant distance from the signal was notmaintained in all three of the experi-mental sessions.

A systematic deviation of the datafrom a proportion of 0.33 clearly ex-ists. Considering the data of the fourobservers combined, the proportion ofcorrect second choices is 0.46. Fur-ther, a correlation between the propor-tion of correct second choices and d'is evident.

Two control conditions aid in inter-preting these data. The first of theseallowed for the possibility that requir-ing the observer to make a secondchoice might depress his first-choiceperformance. During the experiment,blocks of 50 trials in which only a firstchoice was required were alternatedwith blocks of 50 trials in which botha first and a second choice were re-quired. Pooling the data from the fourobservers, the proportions of correctfirst choices for the two conditions are0.650 and 0.651, a difference that isobviously not significant. A prelimi-nary experiment in which data wereobtained from a single observer for fivevalues of signal intensity also servesas a control. In that experiment, 150observations were made at each valueof signal intensity. The relative fre-quencies of correct second choices forthe lowest four values of signal inten-sity were, in increasing order of sig-nal intensity: 26/117 (0.22), 33/95(0.35), 30/75 (0.40), and 20/30(0.67). For the highest value of sig-nal intensity, none of five secondchoices was correct. In this experi-ment, then, the proportion of correctsecond choices is seen to be correlatedwith a physical measure of signal in-tensity as well as with the theoretical

measure d'—this eliminates the possi-bility that the correlation found with aconstant value of signal intensity, in-volving d' as one of the variables (Fig-ure 10), is an artifact of theoreticalmanipulation.

It may be seen from Figure 10 thatthe second-choice data also deviate sys-tematically from the predicted functionderived from decision theory. Thisdiscrepancy, as will be seen, resultsfrom the inadequacy of the assumption—of equal variance of the noise andsignal-plus-noise density functions—upon which the predicted functions inFigure 10 are based. It was pointedout above that the data obtained in thesecond expected-value experiment (seeFigure 8 and accompanying text) indi-cate that a better assumption would bethat the ratio of the increment in themean of the signal-plus-noise functionto the increment in its standard devia-tion is equal to 4. Figure 11 showsthe second-choice data and the pre-dicted four-alternative and second-choice curves derived from the theoryunder this assumption that AM/A<r= 4. In view of the variance associ-ated with each of the points (each first-choice d' was estimated on the basis

FIG. 11. The results of the second-choiceexperiment calculated under another as-sumption. (The predictions from deci-sion theory for first and second choicesare plotted under the assumption thatAAf/A<r = 4.


of 300 observations and each second-choice proportion on less than 100observations), the agreement of thedata and the predicted function shownin Figure 11 is quite good.

The conclusion to be drawn fromthese results of the second-choice ex-periment, though perhaps more obvioushere, is the same as that drawn fromthe yes-no, or expected-value, experi-ments : the sensory information, or thedecision axis, is continuous over agreater range than allowed for by thehigh threshold model. If a thresholdcutoff, below which there is no dis-crimination among observations, existsat all, it is located in such a positionthat it is exceeded by much of the noisedistribution.

Note on the Variance AssumptionBefore considering the two remain-

ing experiments, we should pausebriefly to take up the problem of therelative sizes of the variances of thenoise and signal-plus-noise distribu-tions. We have seen, as indicated inthe theoretical discussion, that an as-sumption concerning these variancesmay be tested by experiment. We havefound that two sets of data, fromyes-no and forced-choice experiments,support the assumption that the vari-ance of the signal-plus-noise distribu-tion increases with its mean. In par-ticular, the assumption that AM/Ao-= 4 is seen to fit those data reasonablywell, and noticeably better than theassumption of equal variance. Weshould like to point out three aspectsof this topic in the following para-graphs : first, the assumption of AM/Ao- = 4 is probably not generally appli-cable ; second, that we have good rea-son to suspect in advance of experi-mentation, in the visual case, that thevariance of the signal-plus-noise distri-bution is greater than that of the noisedistribution; and, third, that the very

assumption of unequal variances re-quires that we qualify a statement madeearlier in this paper.

It will be apparent that if the vari-ance of these sampling distributions isa function of sample size, then theirvariances will differ as a function ofthe duration and the area of the signal.The assumption of AM/Ao- = 4 willprobably not fit the results of experi-ments with different physical parame-ters. Further, as we have indicated,we have not explored the extent ofagreement between other specific as-sumptions and our present data. Itappears likely that more precise datawill be required to determine the rela-tive adequacy of different assumptionsabout the increase in variance withsignal strength.

Peterson, Birdsall, and Fox (1954),after developing the general theory ofsignal detectability, spelled out the spe-cific forms it takes in a variety of dif-ferent detection problems. By way ofillustration, we may mention the prob-lems in which the signal is knownexactly, the signal is known exactlyexcept for phase, and the signal is asample of white Gaussian noise. Aprincipal difference among these prob-lems lies in the shape of the expectedROC curve. For our present purposes,we may regard these problems as dif-fering in the degree of variance con-tributed by the signal itself. For thefirst case mentioned, the signal con-tributes no variance—the signal-plus-noise distribution is simply a transla-tion of the noise distribution, the twohave equal variances. In the othertwo cases, the signal itself has a varia-bility which increases with its strength.

Clearly, if we are to select one ofthe specific models incorporated withinthe theory of signal detectability toapply to a visual detection problem, wewould not select the one that assumesthat the signal is known exactly, for


the visual signal does not contain phaseinformation. Thus, the second modelis more likely to be applicable than thefirst. Actually, the third model, whichassumes that the signal is a sample ofnoise, is the best representation of avisual signal. The fundamental pointhere is that either of the last twomodels leads to predicted results quitesimilar to those that are predictedunder the assumption that AM/Atr= 4. Further discussion of this pointwould lead us too far off the path; wewould like simply to note here that aspecific form of the theory of signaldetectability, which on a priori groundsis most likely to be applicable to visionexperiments, predicts results very simi-lar to those obtained. It is interestingto note in this connection that the re-sults of auditory experiments usingpure tones as signals are in close agree-ment with the signal-known-exactlymodel, with the assumption of equalvariance.

The discerning reader will havenoted that the assumption of a varianceof the signal-plus-noise distributionthat increases with its mean is incon-sistent with a statement made in thetheoretical discussion. In particular,the assumption of a greater varianceof fsff(#) than of iy(x) conflicts withthe statement that the decision axis xmay be regarded as a likelihood-ratioaxis. It was stated above (see thediscussion following Figure 2) that amultidimensional response of the sen-sory system, i.e., one that might berepresented by a point y in a multi-dimensional space, could be mappedinto a line by considering the likeli-hood that y arose from SN relative tothe likelihood that y arose from N, orA (30 = WiO/iJvCy)- We then statedthat we could identify the observationvariable x with some monotonic trans-formation of A(;y). If, now, the vari-ance of fsy(^) is greater than the

variance of f^(^), then as x decreasesfrom a high value, A(^r) will decrease—but, at some point below the meanof the function iN(x}, A(>) will beginto increase again, and will, as a matterof fact, become greater than unity.Thus, if we choose to maintain the as-sumption of a greater variance ofisN(x), then the variable x cannot beregarded, throughout its range, as alikelihood ratio. Given that we dowant to maintain the assumption ofincreasing variance of W(#), for thetime being at least, we may take anyof several possible steps to correct thedifficulty. We can, for example, as-sume that there exists a low threshold,near the mean of ix(x), such that val-ues of x less than this threshold are notordered by the observer, and hence thefact that x cannot be considered as alikelihood ratio below this point is ofno consequence. Another alternativeis to assume outright that the variablex is unidimensional, without recourseto the likelihood-ratio argument tomake the assumption reasonable.Which particular solution we shalladopt will depend upon further experi-mentation.

Analysis 0} the Rating ScaleWe have concluded from the experi-

ments described above that the observ-er's decision axis is continuous overa large range, i.e., that he can orderobservations likely to result from noisealone. We might expect then, in thelanguage of decision theory, that hewill be able to report the a posterioriprobability of signal existence, i.e., thathe will be able to state, following anobservation interval, the probabilitythat a signal existed during the inter-val. In more familiar terms, we areexpecting that the observer will becapable of reporting a subjective prob-ability, or of employing a rating scale.Experimental verification of this hy-

326 J. A. SWETS, W. P. TANNER, JR., AND T. G. BJRDSALL

pothesis is required, of course, for areasonable doubt remains whether theobserver will be able to maintain themultiple criteria essential to the use ofa rating scale. If, for example, sixcategories of a posteriori probabilityare used, or a six-point rating scale,the observer must establish five criteriainstead of just one as in the yes-noprocedure—this may be considerablymore difficult.

The ability to make a probability orrating response is of interest, in part,because such a response is highly effi-cient—in principle, a probability re-sponse retains all of the informationcontained in the observation. In con-trast, breaking up the observation con-tinuum into Yes and No sections is aprocess that loses information. Froma procedure forcing a binary response,one learns from the observer only thatthe observation fell above or below acritical value, and not how far aboveor below. In some practical detectionproblems, the finer-grain informationgained from a probability response canbe utilized to advantage: the observermay record a posteriori probability sothat Yes and No decisions concerningthe action to be taken can be made at alater time, or by someone else who maybe more responsible or who may pos-sess more information about the valuesand costs of the decision outcomes.

More to the point in terms of ourpresent interests, an experimental testof the ability to make a rating responsecontributes to the evaluation of deci-sion theory, and also to distinguishingbetween the adequacy of decision the-ory and the high threshold theory.Since the data obtained with a ratingprocedure may be used to constructROC curves, this experiment attacksthe same problem as those describedabove, i.e., whether the observer candiscriminate among observations likelyto result from noise alone. It is also

the case, as pointed out by Egan,Schulman, and Greenberg (1959), thatthe rating procedure generates ROCcurves, of a given reliability, with aconsiderable economy of time com-pared to the yes-no procedure. There-fore it is of interest, with respect tofuture applications of decision theory,to determine whether the observer canperform as well, as indexed by d', withthe rating procedure as with the yes-no procedure.

The observer's task in this experiment wasto place each observation in one of six cate-gories of a posteriori probability. Fourcategories of equal size (0.2) were used inthe range between 0.2 and 1.0; the other twocategories were 0.0-0.04 and 0.05-0.19. Theboundaries of the categories were chosen inconference with the observers; they believedthat they would be able to operate reasonablywithin this particular scheme. Actually, thespecific sizes of the categories used are notimportant for most purposes; we can as wellthink of a six-point rating scale and assumeonly the property of order.

The four observers in this experimentwere those who served in the second ex-pected-value experiment. Further, the samesignal intensity (0.78 foot-lamberts) andthe same a priori probabilities—p(SN) —p(N)=0.50—that were employed in thatexperiment were employed in this one. Theobservers made a total of 1,200 observationsin three experimental sessions.

Results. The raw data for each ob-server consist of the number of ob-servations of signal plus noise and thenumber of observations of noise alonethat were placed in each of the sixcategories of a posteriori probability.Before proceeding with more complexanalyses, we shall first make a roughdetermination of the validity of theobservers' use of the categories, i.e.,of whether we are, in fact, dealing witha scale. This may be achieved bycomputing the proportion of the totalnumber of observations placed in eachcategory that were actually observa-tions of a signal. If the categorieswere used properly, this proportion


will increase with increases in theprobabilities that define the categories.

The results of this analysis areshown in Figure 12. Five curves areplotted there, one for each of the fourobservers and one showing the averageresult. We may note, as an aside, thatObserver 4 is considerably more cau-tious than the others. A look at theraw data reveals that he used the low-est category twice to four times asoften as the other observers; as a mat-ter of fact, he placed 60% of his ob-servations in that category. We maylook for this difference to reappear inother analyses of the data of this ex-periment. The major point here, how-ever, is that three of the four indi-vidual curves are monotonic increasing,whereas the fourth shows only onereversal. This result indicates thefeasibility of using a scaling procedure—it indicates that requiring an ob-server to maintain five criteria simul-taneously in a detection problem is notunreasonable. The result is consistentwith an ability to order completely theobservations, those arising from noisealone as well as those arising fromsignal plus noise.

ROC Curves Obtained from theRating Data. ROC curves can begenerated from data obtained with therating procedure since these data canbe compressed to those of the binary-decision procedure with any of severalcriterion levels. That is to say, we cancalculate the pair of values, ps(A) andPSN(A), ignoring all but one of the(five) criteria, or category boundaries,employed by the observer. We suc-cessively calculate five pairs of thesevalues, each time singling out a differ-ent criterion, and thus trace out anROC curve. In particular, we firstcompute the conditional probabilitiesthat observations arising from noisealone and from signal plus noise will beplaced in the top category; then these

2 3 1 50.0-0.04 .05-.19 .20-.39 .40-.59 .60-79 .80-I0C

Categories of A Posterior i Probability

FIG. 12. The results of the ratingexperiment.

probabilities are computed with respectto the top two categories, and so forth.We assume, in these calculations, thatobservations placed in a particularcategory would fall above the criteriathat define a lower category.

The ROC curves so obtained areshown in the upper left-hand portionsof each part of Figure 13. (Ignore,for now, the other curves in Figure13.) We may note that the data arewell described by the type of ROCcurve predicted from decision theory.As is the case with the empirical ROCdata from yes-no experiments, theycannot be fitted well by a straightline intersecting the point pss(A) =pu(A) = 1.0, the prediction made fromthe high threshold theory. This resultindicates that the observers can dis-criminate among observations likely toresult from noise alone, and are capableof maintaining the multiple criteriarequired for the rating response.

Comparison of ROC Curves Ob-tained from Ratings and Binary De-cisions. It is intuitively clear that anestimate of d' of given reliability canbe achieved with fewer observations bythe rating procedure than by the yes-noprocedure. This proposition is sup-


FIG. 13. Empirical receiver-operating-characteristic curves for four observers inthe rating experiment—two alternative presentations.

ported by a comparison of the yes-nodata shown in Figure 8 with the ratingdata shown in Figure 13—the ratingdata, which show considerably lessvariation, are based on 1,200 observa-tions whereas the yes-no data are basedon approximately 5,000 observations.

The economy provided by the ratingprocedure makes it desirable to deter-mine whether the two procedures areequivalent means of generating theROC curve. Unfortunately, to answerthis question immediately, there aresome clear differences between theROC curves we have obtained with thetwo procedures. These differences are

best illustrated by plotting the data onnormal coordinates, i.e., on probabilityscales transformed so that the normaldeviates are linearly spaced. Thesescales are convenient since on themthe ROC curve specified by decisiontheory becomes a straight line. Fur-ther, the slope of this line representsthe relative variances of the densityfunctions, iy(x) and f«^(^), that un-derlie the ROC curve. In particular,it can be shown that the reciprocal ofthe slope (with respect to the normaldeviate scales) is equal to the ratio<rSN/vN.

The empirical ROC curves obtained


with the rating and yes-no proceduresare shown on normal coordinates inFigure 14. It is immediately evidentfrom this figure that a lower detecta-bility resulted from the rating proce-dure for all four observers. We maysee from the alternative presentationsof these data in Figures 8 and 13 thatthe values of d' range from 2.0 to 3.0for the yes-no data and from 1.5 to2.0 for the rating data.11 It is furtherapparent in Figure 14 that, consistentwith the difference in d', the ratingcurve has a greater slope than theyes-no curve. This difference is small—the greater variance of fsw(^) underthe yes-no procedure did not showclearly in the plots on linear proba-bility axes—but it is regular. We mayalso note again, as this way of plottingthe data makes very clear, that the rat-ing data show considerably less scatterthan the yes-no data.

The values and costs associated withthe decision outcomes in this situationmake us hesitant, on the basis of thedata we obtained, to reject the hypothe-sis that the rating and yes-no proce-dures are equivalent means of generat-ing ROC curves. It is possible, ofcourse, that some undetected differenceexisted between the experimental con-ditions in the two experiments; onewas conducted after the other was com-pleted. Such a difference might easilyaccount for the relatively small dis-crepancies observed. Again, it has re-cently been shown in an auditory ex-periment that the two procedures resultin essentially the same ROC curve,both with respect to d' and to slope(Egan, Schulman, & Greenberg, 1959).Still, we cannot discount the present

11 Values of d' can, of course, be computedfrom the normal deviate scales of the plots inFigure 14. A problem arises, however, ifthe slope of the line fitted to the data is notunity. A solution to this problem is proposedin Clarke, Birdsall, and Tanner (1959).

results on the basis of the auditory ex-periment, for we have noted severaldifferences between visual and auditorydata that are likely to be real—one per-haps relevant to this issue is that theROC curves obtained with pure toneshave slopes that are uniformly near one.We should perhaps be content, at thispoint, with the admittedly weak con-clusion that no data exist to support thehypothesis that the two procedures areequivalent in the case of visualstimuli.12

Test of the Normality of the DensityFunctions. At this juncture, it is con-venient to turn briefly, but explicitly,to a topic first considered in the theo-retical discussion. It was stated therethat we would assume the density func-tions on the observer's decision axis tobe Gaussian in form, but that the as-sumption was subject to experimentaltest. A test of this assumption is pro-vided by plotting the empirical ROCcurves on normal coordinates. Havingnow introduced plots of the data in thisform in Figure 14, we may use themfor this purpose. If the observer's den-sity functions are normal, then the em-pirical points of an ROC curve plottedon normal coordinates will be fittedbest by a straight line. Clearly, astraight line provides an adequate de-scription of the data in these figures.Thus the assumption of normality, animportant one for the sake of simplicityof analysis, is supported by the data.

Approach to Optimal BehaviorIn the presentation of experimental

results thus far, we have concentratedon the continuity of the observer's deci-sion axis, and on his ability to adopt

12 As this article goes to press we can re-port that in a repetition of this experimentwith visual stimuli (unpublished) no reliableor regular differences were found betweenROC curves obtained from ratings andbinary decisions.


0.99 -2

Normal Deviate

0

PcN(A) 0.60

Observer 1© Rat ing Datax Yes-No Data

0.01 _L

0 foe

-2

0.01 0.05 0.10 0.20 0.40 0.60 0.80PN(A)

0.95 0.99

FIG. 14. Comparison of the receiver-operating-characteristic curves obtained fromratings and binary decisions.

various criteria along this axis. A re-maining question is how closely thecriteria he adopts correspond to thosespecified by decision theory as the opti-mal criteria, To answer this questionwe shall consider some further analy-ses of experimental results already de-scribed, and the results of an additionalexperiment.

It should be recalled that decisiontheory specifies as the optimal decisionfunction either likelihood ratio, X ( x ) ,or some monotonic function of likeli-hood ratio, call it \(x)'. That is tosay, any transformation of the decisionaxis is acceptable as long as order ismaintained. If the decision functionis A(.*•), then the optimal criterion is

the value of X(x) equal to /? (Equa-tion 3). If the decision function is\(x)', then the optimal criterion is thevalue of this function that correspondsto /?, call it /?'. The monotonic re-lationship means that A(#)' > /?' <•»\(x) > /3. Thus to establish the ap-plicability of decision theory, it is suffi-cient to demonstrate that the observer'scriteria are monotonically related to /?.If sampling error is taken into account,it is sufficient to demonstrate a signifi-cant correlation between the observer'scriteria and p. It is of interest, how-ever, to determine just how closely theobserver's criteria do approach the op-timal criteria as specified by /9. Inexamining this question we shall make


0.99 -ZNormal Dev ia te

0

0.010.01 0.05 0.10 0.20 0.40 0.60 0.80

P N (A )

FIG. 14—Continued

0.95 0.99

use of the fact that, in order to indexthe observer's criterion, it is notstrictly necessary to compute a valueof likelihood ratio from the proportionsof hits and false alarms; it is moreconvenient, and for purposes of inter-pretation, more direct, to take simplythe proportion of false alarms as theindex.

Criteria Employed in the Expected-Value Experiments. In the first ex-pected-value experiment, the observerswere told only the a priori probabilitiesof signal and noise and the values ofthe various decision outcomes thatwere in effect during each experimentalsession. They were not told that anycombination of these factors can be ex-pressed by a single number (ft) which,

in conjunction with a value of d', speci-fies the optimal criterion or the optimalfalse alarm rate. The rank-order cor-relations between /3 and the obtainedproportions of false alarms that werecomputed from the data of this firststudy were .70, .46, and .71 for thethree observers, respectively. A corre-lation of .68 is significant at the .01level of confidence. This result indi-cates that the observer did not merelyvary his criterion from one session toanother, but that his criterion variedappropriately with changes in /?.

In the second expected-value experi-ment, the observers were told the opti-mal proportion of false alarms foreach session as well as the a prioriprobabilities and decision values. This


0.99 -2Normol Dev ia te

0

PcN(A) 0.60

Observer 3® Rat ing Datax Yes -No Data

<ua

-I

-2

0.05 0.10 0.20 0.40 0.60 0.80P N ( A )

FIG. 14—Continued

0.95 0.99

information was available to the ex-perimenter since values of d' had pre-viously been determined by the forced-choice procedure during a trainingperiod. Thus, in the second study,we were asking how closely the ob-server would approach the optimal falsealarm rate given knowledge of it. Therank-order correlations between thefalse alarm rates announced as optimaland the false alarm rates yielded by thefour observers were .94, .97, .86,and .98. Again, a coefficient of .68is significant at the .01 level of con-fidence. Data obtained later in an audi-tory experiment showed coefficients ofthis magnitude—as a matter of fact,the rank-order cofficient based on fivepairs of measures for each of two ob-

servers in the auditory experiment was1.0—when the observers were not in-formed of the optimal false alarm rate(Tanner, Swets, & Green, 1956).

Satisfying a Restriction on the Pro-portion of False Alarms. A more di-rect attack on the question of theobserver's ability to reproduce a givenfalse alarm rate is provided by an ex-perimental procedure not previouslydescribed in detail, one involving adifferent definition of optimal behavior.Under this definition of optimal behav-ior, no values and costs are assignedthe various decision outcomes; instead,a restriction is placed on the propor-tion of false alarms permitted. Theoptimal behavior is to maximize theproportion of hits while satisfying the


0.99

Normal Deviate

0

Observer 4© Ra t i ng Datax Yes-No Data

a

-2

0.010.01 0.05 0.10 0.20 0.40 0.60

P N ( A )

FIG. 14—Continued

0.80 0.95 0.99

restriction on false alarms. This, itwill be recognized, is the proceduremost popular among experimenters fortesting statistical hypotheses.

An experiment using this procedurewas conducted with a different set offour observers. The a priori probabil-ity of signal occurrence was 0.72throughout the experiment. Therewere, then, 14 presentations of noisealone in a block of 50 presentations.There were four different experimentalconditions, each extending over 18blocks of 50 presentations. In each ofthese conditions, the observers wereinstructed to adopt a criterion thatwould result in Yes responses to ap-proximately n or n + 1 of the 14 pres-entations of noise alone in a block of

50 presentations. For the four condi-tions of the experiment, n was equalto 0, 3, 6, and 9, respectively. Thusthe acceptable range for the proportionof false alarms was .0-.07, .21—.28,.43-.SO, or .64-.71. The primarydata consist of four values of falsealarm rate for each observer; eachvalue is based on 252 presentations ofnoise alone.

The data are shown in Figure 15.The false alarm rates obtained areplotted against the restricted ranges offalse alarm rate. The four observersare represented by different symbols;the vertical bars designate the accept-able range. It may be seen that thelargest deviation from the range stipu-lated is .04. This result suggests that


s1 30

X

X

0.0-.07 ,21— ,28 43 — 50 ,64 — 71Restrict ion on F^fA)

FIG. 15. The reproduction of a givenfalse alarm rate.

the observer is able to adjust his crite-rion with considerable precision.

Two other pieces of information areneeded, however, to interpret the datashown in Figure 15. For, of course,if the observer were given informationabout the correctness of his responseafter each response, these data couldbe obtained even if the observer wereunable to vary his criterion. The ob-server could then approximate anyfalse alarm rate by saying "yes" untilthe desired number of false alarms wasachieved, and then by saying "no" onthe remaining presentations. That pro-cedure would entail a severe depressionof d'. Actually, the observers weregiven information about correctnessonly after each block of 50 presenta-tions, and the values of d' were notdepressed. Thus the false alarm ratesthat were obtained may legitimately beregarded as reflecting the observer'scriteria.

Criteria Employed in the RatingScale Experiment. We may also in-vestigate how closely the multiple cri-teria adopted by the observers in therating scale experiment approach theoptimal criteria. Stated otherwise, we

may examine the relationship that ex-isted between the subjective and objec-tive probabilities of signal occurrencein that experiment. It may be notedin advance that an alternative presenta-tion of the results, in Figure 12, givesan indication of the extent of agree-ment we may expect.

As stated earlier, the a posterioriprobability of signal existence is amonotonic function of likelihood ratio.In particular, the optimal relationshipbetween the two is:

P.(SN) =\(x)p(SN)

\(x)p(SN) + p(N) [9]

where: p,(SN*) denotes the probabilitythat the signal existed given the observa-tion x (i.e., the a posteriori probability),

\(#) is the likelihood ratio, and p(Sff) andP(N) are the a priori probabilities (Peter-son et al., 19S4).

For our experiment, with p(SN) =p (N) = 0.50, this equation reduces to:

P,(SN) = \(x) [10]

As described above, a point on theROC curve can be obtained for eachof the boundaries of the six categoriesemployed by the observer, i.e., for thefive criteria he employed. Since, aswe have also pointed out, the criterionvalue of \(x) corresponds to the slopeof the ROC curve at the point in ques-tion, this criterion value of A(^) canbe determined. Thus px(SN} =X(x)/\(x} + 1 can be computed foreach of the criteria employed by theobserver. Assuming now that the ob-server's decision function is likelihoodratio, then if he is behaving accordingto the optimal relationship betweenpx(SN) and A(», the values of \(x)/A.(#) + 1 computed from his data willcorrespond directly to probability val-ues that were announced as denningthe categories. In short, we know thevalues of px(SN') that were announced


as marking off the categories; by pur-suing a route through the empiricalROC curve and \(x) we can calculatethe values of px(SN) that bound thecategories the observer actually used—therefore we can assess how well thetwo sets of criterion values of px(SN),the objective and subjective probabili-ties, agree.

The lower right-hand portions ofFigure 13 show the probability valuesthat were announced as denning thecategories, plotted against the proba-bility values that characterize the cri-teria actually employed by the observ-ers, i.e., against px(SN) = X(x)/\(x)+ 1 as determined from the data.(Some points are missing since \(x)is indeterminate at very low values ofpN(A}.~) It is apparent from theseplots that Observers 1, 2, and 3 areoperating with a decision function simi-lar to likelihood ratio and approxi-mately according to the optimal rela-tionship between />3,(SW) and \(x).The pattern exhibited by Observers 1and 3, that of overestimating smalldeviations from a probability of 0.50,will be familiar to those acquaintedwith the literature on subjective prob-ability. Observer 4, as we notedearlier, is quite different from theothers. His tendency, also evidencedbut to a far lesser extent by Observer2, is to consistently underestimate thea posteriori probability, i.e., to set allof his criteria too high.

To summarize our discussion of hownearly the criteria adopted by the ob-servers in these several experimentscorrespond to the optimal criteria, wemay say that the observer, for want ofa better term, behaves in an "optimalfashion." He is responsive to changesin both the a priori probability of sig-nal occurrence and the values of thedecision outcomes; the criteria headopts are highly correlated with theoptimal criteria. Subjective trans-

formations of the real probability scaleand of the "real" value scale do, ofcourse, exist, and differ somewhat fromone observer to another. Undoubtedly,values also play a role in those experi-ments in which no values are explicitlyassigned by the experimenter. Never-theless, we have seen that the observercan adopt successively as many as 10different criteria, on the basis of differ-ent combinations of probabilities andvalues presented to him, that are al-most perfectly ordered. He can main-tain simultaneously at least five criteriathat are a reasonable facsimile of theoptimal criteria. If he is told the opti-mal false alarm rate, he can, providedit is not very large or very small, ap-proximate it with a small error.

SUMMARY, CONCLUSIONS, ANDREVIEW OF IMPLICATIONS

We imagine the process of signaldetection to be a choice between twoGaussian variables. One, having amean equal to zero, is associated withnoise alone; the other, having a meanequal to d', is associated with signalplus noise. In the most common de-tection problem the observer decides,on the basis of an observation that isa sample of one of these populations,which of the two alternatives existedduring the observation interval. Theparticular decision that is made de-pends upon whether or not the obser-vation exceeds a criterion value; thecriterion, in turn, depends upon theobserver's detection goal and upon theinformation he has about relevant pa-rameters of the detection situation.The accuracy of the decision that ismade is a function of the variable d'which is monotonically related to thesignal strength.

This description of the detectionprocess is an almost direct translationof the theory of statistical decision.The main thrust of this conception, and


the experiments that support it, is thatmore than sensory information is in-volved in detection. Conveniently, alarge share of the nonsensory factorsare integrated into a single variable,the criterion. There remains a meas-ure of sensitivity (d1) that is purerthan any previously available, a meas-ure largely unaffected by other thanphysical variables. This separation ofthe factors that influence the observer'sattitudes from those that influence hissensitivity is the major contribution ofthe psychophysical application of sta-tistical decision theory.13

We have indicated several times inthe preceding that another conceptionof the detection process, one involvingwhat we termed a "high threshold," isinconsistent with the data reported. Itshould be noted, however, that thesedata, to the extent analyzed in thispaper, do not preclude the existence ofa lower threshold. The analyses pre-sented do not indicate explicitly howfar down into the noise the observa-tions are being ordered, i.e., how lowa threshold must be relative to the

18 It is interesting to note that the presentaccount is not the first to model psycho-physical theory after developments in thetheory of statistical decision—as a matterof fact, Fechner was influenced by Bernoulli'ssuggestion that expectations might be ex-pressed in terms of satisfaction units. AsBoring (1950, p. 285) relates the story,Bernoulli's interest in games of chance ledhim to formulate the concept of "mentalfortune"; he believed changes in mentalfortune to vary with the ratio of the changein physical fortune to the total fortune. Thismathematical relationship between mentaland physical terms was the sort of relation-ship that Fechner sought to establish withhis psychophysics. It should also be ob-served that Fechner anticipated the decisionmodel under discussion in a much moredirect way. His concept of "negative sen-sations," largely dismissed by subsequentworkers in- the field, denies the existence ofsuch a cut in the continuum of observationsthat the magnitudes of observations belowthe cut are indiscriminable.

noise distribution in order to be com-patible with the data. As it happens,further analyses of the yes-no andforced-choice results show them to beconsistent with a threshold slightlyabove the mean of the noise distribu-tion. If, for example, we examine theempirical ROC curves of Figures 8and 13, we see that at values of pN(A)greater than 0.16, the curves are ade-quately fit by a straight line throughthe upper right-hand corner. Thusthese data are consistent with a thresh-old cutoff that is located one sigmaabove the mean of the noise distribu-tion.

Of course, a determination of thelevel at which a threshold may possiblyexist is neither critical nor useful. Athreshold well within the noise distri-bution is not a workable concept. Sucha concept, since it is inconsistent withthe correction for chance, complicatesrather than facilitates the mathematicaltreatment of the data. Moreover, athreshold that is low is, for practicalpurposes, not measurable. The forced-choice experiment is a case in point;the observer conveys less informationthan he is capable of conveying if onlya first choice is required. That thesecond choice contains a significantamount of information has been dem-onstrated; auditory experiments haveshown that the fourth choice conveysinformation (Tanner et al., 1956).Thus it is difficult to determine whenenough information has been extractedto yield a valid estimate of a lowthreshold. In addition, the existenceof such a threshold is of little conse-quence for the application of the deci-sion model—for example, yes-no dataresulting from a suprathreshold crite-rion depend upon the criterion but arecompletely independent of the thresh-old value.

One of the major reasons for ourconcern with the threshold concept is


that this concept supports several com-mon psychophysical procedures thatare invalidated by the results we havedescribed. The correction for chancesuccess has already been mentioned asa technique that stems from a highthreshold theory, and one that is in-consistent with the data. This correc-tion is frequently applied to data col-lected with the method of constantstimuli. It is used implicitly wheneverthe threshold is denned as the stimu-lus intensity that yields a probabilityof correct response halfway betweenchance and perfect performance. Themethod of adjustment and the standardmethod of serial exploration are alsoinappropriate, given the mechanism ofdetection described above. When themethod of serial exploration is usedwith the signal always present, or withinsufficient "catch trials" to estimatethe probability of a false alarm, the rawdata will not permit separating thevariation in the observer's criterionfrom variation in his sensitivity.Changes in an observer's criterion fromone session to another can be estimatedonly if it is assumed that his sensi-tivity has not changed, and conversely.The same applies to data collected withthe method of adjustment.

To be sure, unrecognized variationsin the criterion are not important inmany psychophysical measurements forthey may be expected to contributerelatively little variation to the com-puted value of the threshold. Fairlylarge changes in the criterion will affectthe threshold value by less than 3 db.in the case of vision, and by no morethan 6 db. in the case of audition. Thisdegree of reliability is acceptable inclinical audiometry, for example, inwhich the method of limits is usuallyemployed. Neither would it distortappreciably curves of the course ofdark adaptation. In many experiments,however—in experiments concerned

with substantive as well as with theo-retical problems—a reliability of lessthan 1 db. is required, and in thesecases a knowledge of the criterion usedby the observer is essential.

To illustrate the problems in whichthe threshold concept and its associatedprocedures may have led to improperconclusions, we may single out one ofcurrent interest, that of "subliminalperception." In most of the studiesof this phenomenon, the evidence forit consists of the finding that subjectswho first report seeing no stimulus canthen identify the stimulus with greater-than-chance accuracy when forced tomake a choice.14 We have mentionedabove as a typical result in psycho-physical work that the forced-choiceprocedure yields lower threshold val-ues than does the yes-no procedure.We have also suggested that this resultmay be accounted for by the fact thatwith the yes-no procedure the calcu-lated value of the threshold varies di-rectly with the observer's criterion, andthat a strict criterion is usually em-ployed by the observers under this pro-cedure. That a strict criterion isusually used with the yes-no procedureis not surprising in view of the factthat observers are often instructed toavoid making false alarm responses. Itis also likely that the stigma associatedwith "hallucinating" promotes the useof a strict criterion in the absence ofan explicit caution against false alarms.Thus it may be expected that on manyoccasions when an observer does notchoose to report the existence of thestimulus, he nevertheless possessessome information about it. It may be,

14 This procedure was used explicitly inthe earlier studies of subliminal perception;several of these studies are reviewed byMiller (1942). With minor variations, thisprocedure also underlies many of the morerecent studies—see, for example, Bricker andChapanis (1953).


therefore, that subliminal perceptionexists only when a high criterion isincorrectly identified as a limen.15

Having presented a theory of detec-tion behavior and some detection ex-periments, and having just discussedthe relationship of this work to "psy-chophysics," it remains to articulatewith the title and the introductoryparagraph of this paper, to considerthe relationship of the work to thestudy of "perception."

In principle, the general scheme wehave outlined may apply to perceptionas well as to detection. It seems rea-sonable to suppose that perception isalso a choice among Gaussian variables.Consistent with the existence of manyalternatives in the case of perception,we may imagine many critical regionsto exist in the observation space. Thisspace will have more dimensions thanare involved in detection—as we havepreviously indicated, one less dimen-sion than the number of alternativesconsidered. We may presume, in per-ception as in detection, that the boun-daries of the critical regions are de-fined in terms of likelihood ratio, andare determined by the a priori proba-bilities of the alternatives and the rela-tive values of the decision outcomes.

It may also be contended that whatwe have been referring to as a detec-tion process is itself a perceptualprocess. Certainly, if perceptual proc-esses are to be distinguished from sen-sory processes on the grounds that theformer must be accounted for in termsof events presumed to occur at highercenters whereas the latter can be ac-counted for in terms of events occur-ring within the receptor systems, thenthe processes with which we have beenconcerned qualify as perceptual proc-esses. Since, in detecting signals, the

15 This analysis of the problem of sub-liminal perception has been elaborated byGoldiamond (1958).

observer's detection goal and the in-formation he possesses about probabili-ties and values play a major role, wemust assume either that signal detec-tion is a perceptual process, or thatthe foregoing distinction between sen-sory and perceptual processes is oflittle value.

Thus the thesis of the present paperis, in one of its aspects, another stagein the history of the notion that theprocess of perceiving is not merely oneof passively reflecting events in theenvironment, but one to which theperceiver himself makes a substantialcontribution. Various writers havesuggested that our perceptions arebased upon unconscious inferences,that sensory events are interpreted interms of unconscious assumptionsabout their probable significance, thatour responses to stimuli reflect the in-fluence of our needs and expectancies,that we utilize cues in selectively plac-ing sensory events in categories ofidentity, and so forth. The presentview differs from these in regardingthe observer as relating his sense datato information he has previously ac-quired, and to his goals, in a mannerspecified by statistical decision theory.The approach from decision theory hasthe advantage that it specifies the per-ceiver's contribution to perception atother than the conversational level; itprovides quantitative relationships be-tween the nonsensory factors and boththe independent and dependent vari-ables.

We submit then that the presentpaper, although confined to detectionexperiments, is aptly named. We mayview detection and perception as madeof the same cloth. Of course, signaldetection is a relatively simple per-ceptual process, but it is exactly itssimplicity that makes the detection set-ting most appropriate to a preliminaryexamination of the value of statistical


decision theory for the study of per-ception. Because detection experi-ments permit precise control over thevariables specified by the theory aspertinent to the perceptual process,they provide the rigor desirable in theinitial tests of a theory. Once thesetests are passed, the theory may beextended and applied to more complexproblems. Recent studies within theframework of decision theory includethe recognition of one of two signals(Tanner, 1956), combined detectionand recognition (Swets & Birdsall,1956), problems in which a single de-cision is based on a series of observa-tions (Swets, Shipley, McKey, &Green, 1959), problems in which theobserver decides sequentially whetherto make another observation beforemaking a final decision (Swets &Green, in press), and the recognitionof speech (Decker & Pollack, 1958;Egan, 1957; Egan & Clarke, 1956;Egan, Clarke, & Carterette, 1956; Pol-lack & Decker, 1958).

REFERENCESBLACKWELL, H. R. Psychophysical thresh-

olds : Experimental studies of methods ofmeasurement. Bull. Eng. Res. Inst. U.Mich., 1953, No. 36.

RLACKWELL, H. R., PRITCHARD, B. S., &OHMART, T. G. Automatic apparatus forstimulus presentation and recording invisual threshold experiments. /. Opt. Soc.Amer., 1954, 44, 322-326.

BORING, E. G. A history of experimentalpsychology. (2nd. ed.) New York:Appleton-Century-Crofts, 1950.

BRICKER, P. D., & CHAPANIS, A. Do in-correctly perceived tachistoscopic stimuliconvey some information? Psychol. Rev.,1953, 60, 181-188.

BROSS, I. D. J. Design for decision. NewYork: Macmillan, 1953.

CLARKE, F. R., BIRDSALL, T. G., & TANNER,W. P., JR. Two types of ROC curves anddefinitions of parameters. /. Acoust. Soc.Amer., 1959, 31, 629-630.

DECKER, L. R., & POLLACK, I. Confidenceratings and message reception for filteredspeech. /. Acoust. Soc. Amer., 1958, 30,432-434.

EGAN, J. P. Monitoring task in speechcommunication. /. Acoust, Soc. Amer.,1957, 29, 482-489.

EGAN, J. P., & CLARKE, F. R. Source andreceiver behavior in the use of a criterion./. Acoust. Soc. Amer., 1956, 28, 1267-1269.

EGAN, J. P., CLARKE, F. R., & CARTERETTE,E. C. On the transmission and confirma-tion of messages in noise. /. Acoust. Soc.Amer., 1956, 28, 536-550.

EGAN, J. P., SCHULMAN, A. I, & GREEN-BERG, G. Z. Operating characteristics de-termined by binary decisions and by rat-ings. /. Acoust. Soc. Amer., 1959, 31,768-773.

GOLDIAMOND, I. Indicators of perception:I. Subliminal perception, subception, un-conscious perception: An analysis in termsof psychophysical indicator methodology.Psychol. Bull., 1958, 55, 373-411.

HORTON, J. W. Fundamentals of sonar.Annapolis: United States Naval Institute,1957.

HOWES, D. H. A statistical theory of thephenomenon of subception. Psychol. Rev.,1954, 61, 98-110.

MILLER, J. G. Unconsciousness. NewYork: Wiley, 1942.

MUNSON, W. A., & KARLIN, J. E. Themeasurement of the human channel trans-mission characteristics. /. Acoust. Soc.Amer., 1956, 26, 542-553.

OSGOOD, C. E. Method and theory in experi-mental psychology. New York: OxfordUniver. Press, 1953.

PETERSON, W. W., BIRDSALL, T. G., & Fox,W. C. The theory of signal detectability.IRE Trans., 1954, PGIT-4, 171-212.

POLLACK, L, & DECKER, L. R. Confidenceratings, message reception, and the re-ceiver operating characteristic. /. Acoust.Soc. Amer., 1958, 30, 286-292.

SHANNON, C. E. The mathematical theoryof communication. Bell Sys. tech. J., 1948,27, 379-423.

SMITH, M., & WILSON, EDNA A. A modelof the auditory threshold and its applica-tion to the problem of the multiple ob-server. Psychol. Monogr., 1953, 67(9,Whole No. 359).

SWETS, J. A. Indices of signal detectabilityobtained with various psychophysical pro-cedures. /. Acoust. Soc. Amer., 1959, 31,511-513.

SWETS, J. A., & BIRDSALL, T. G. The hu-man use of information: III. Decision-making in signal detection and recognitionsituations involving multiple alternatives.IRE Trans., 1956, IT-2, 138-165.


SWETS, J. A., & GREEN, D. M. Sequentialobservations by human observers of sig-nals in noise. In C. Cherry (Ed.), Fourthsymposium on information theory. Lon-don: Butterworth, in press.

SWETS, J. A., SHIPLEY, ELIZABETH P.,McKEY, MARY J., & GREEN, D. M. Mul-tiple observations of signals in noise. /.Acoust. Soc. Amer., 1959, 31, 514-521.

TANNER, W. P., JR. A theory of recogni-tion. J. Acoust. Soc. Amer., 1956, 28,882-888.

TANNER, W. P., JR., & BIRDSALL, T. G.Definitions of d' and 1 as psychophysicalmeasures. /. Acoust. Soc. Amer., 1958,30, 922-928.

TANNER, W. P., JR., & SWETS, J. A. Adecision-making theory of visual detection.Psychol. Rev., 1954, 61, 401-409.

TANNER, W. P., JR., SWETS, J. A., & GREEN,D. M. Some general properties of thehearing mechanism. Technical Report No.30, 1956, University of Michigan, Elec-tronic Defense Group.

THURSTONE, L. L. A law of comparativejudgment. Psychol. Rev., 1927, 34, 273-286. (a)

THURSTONE, L. L. Psychophysical analysis.Amer. J. Psychol., 1927, 38, 368-389. (b)

VAN METER, D., & MIDDLETON, D. Modernstatistical approaches to reception in com-munication theory. IRE Trans., 1954,PGIT-4, 119-145.

WALD, A. Statistical decision functions.New York: Wiley, 1950.

(Received December 16, 1959)

DECISION PROCESSES IN PERCEPTIONpsych.colorado.edu/~lharvey/P5665 Prosem/P5665_2016/Class... ·...

Documents

Transcript of DECISION PROCESSES IN PERCEPTIONpsych.colorado.edu/~lharvey/P5665 Prosem/P5665_2016/Class... ·...