Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.
-
Upload
silvester-hill -
Category
Documents
-
view
217 -
download
4
Transcript of Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.
![Page 1: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/1.jpg)
Psychophysics 3
Research Methods
Fall 2010
Tamás Bőhm
![Page 2: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/2.jpg)
Signal detection theory
• Aka. sensory decision theory (SDT)• A model & a data analysis method for decision
problems with uncertainty (noise)• Originates from World
War II: aircraft detection on radar signals
• Today: widely used in psychophysics, medicine, radiology and machine learning
![Page 3: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/3.jpg)
Signal detection theory
• Experiment setup:– In some trials a stimulus (signal) is presented, in
others there is no stimulus;– Observer reports if she/he saw a signal or not– Calculate how many times the observer detected a
signal when she/he was presented one (hit rate)• Is the hit rate all we want to know?
Two observers achieved the same hit rate. Are they certainly behaving the same way?
• NO, we also need to know how many times the observer said “I see” when there was no signal (false alarm rate)
![Page 4: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/4.jpg)
Signal detection theory
• Confusion matrix: contains all the information about the observer’s performance
![Page 5: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/5.jpg)
Signal detection theory
• Confusion matrix: contains all the information about the observer’s performance
• As columns add up to 100%, it is enough to know one item from each column
40 trials
20 20
18
2
6
14
= 100% = 100%
= 90%
= 10% = 70%
= 30%
![Page 6: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/6.jpg)
Signal detection theory
• Perfect detection:
100%
100%0%
0%
![Page 7: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/7.jpg)
Signal detection theory
• No detection at all (1st example): always reporting “Seen”
100%
0%0%
100%
![Page 8: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/8.jpg)
Signal detection theory
• No detection (2nd example): always reporting “Not seen”
0%
100%100%
0%
![Page 9: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/9.jpg)
Signal detection theory
• No detection (3rd example): flipping a coin
50%
50%50%
50%
![Page 10: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/10.jpg)
Signal detection theory
• No detection (4th example): reporting “Seen” in 30% of the trials (no matter what is presented)
30%
70%70%
30%=
=
Rows equal no detection
![Page 11: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/11.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
![Page 12: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/12.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
90% 30%
10% 70%
![Page 13: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/13.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
100% 0%
0% 100%
Perfect detection
![Page 14: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/14.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
100% 100%
0% 0%
No detection: always “yes”
![Page 15: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/15.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
0% 0%
100% 100%
No detection: always “no”
![Page 16: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/16.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
50% 50%
50% 50%
No detection: reporting “yes” in 50% of the trials (flipping a coin)
![Page 17: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/17.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
40% 40%
60% 60%
No detection: reporting “yes” in 40% of the trials
![Page 18: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/18.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
30% 30%
70% 70%
No detection: reporting “yes” in 30% of the trials
![Page 19: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/19.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%
60% 60%
40% 40%
No detection: reporting “yes” in 60% of the trials
![Page 20: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/20.jpg)
Signal detection theory
• Receiver operating characteristic (ROC):
false alarm rate
hit
rat
e
100%
100%Diagonal: no detection
![Page 21: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/21.jpg)
Signal detection theory
• SDT model:
• No way to remove the noise• But sensation can be separated from decision by
using ROCs
Sensation
Noise
DecisionSignal
present/absent
Sensation level (SL)
SL ≥ β
Criterion (β)
SL < β
YES
NO
![Page 22: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/22.jpg)
Signal detection theory
Sensation
(Noise)
DecisionSignal
present/absent
Sensation level (SL)
SL ≥ β
Criterion (β)
SL < β
YES
NO
sensation level
pro
bab
ility
Without noise: perfect detection is possible
criterion signal present
signal absent
![Page 23: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/23.jpg)
Signal detection theory
Sensation
(Noise)
DecisionSignal
present/absent
Sensation level (SL)
SL ≥ β
Criterion (β)
SL < β
YES
NO
sensation level
pro
bab
ility
criterion signal present
signal absent
100% 0%
0% 100%
![Page 24: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/24.jpg)
Signal detection theory
Sensation
Noise
DecisionSignal
present/absent
Sensation level (SL)
SL ≥ β
Criterion (β)
SL < β
YES
NO
sensation level
pro
bab
ility
Noise: smears the distributions perfect detection is impossible (if the two distributions overlap)
signal absent(noise only)
signal present(signal+noise)criterion
online demo
![Page 25: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/25.jpg)
Signal detection theory
Sensation level
Sensation level
http://www-psych.stanford.edu/~lera/psych115s/notes/signal/
![Page 26: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/26.jpg)
Signal detection theory
Sensation level
Sensation level
false alarm rateh
it r
ate
![Page 27: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/27.jpg)
Signal detection theory
false alarm rateh
it r
ate
ROC curve
β = 8
β = 6
β = 10
β = 6
β = 8
β = 10
![Page 28: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/28.jpg)
Signal detection theory
false alarm rateh
it r
ate
β
sensation level
pro
bab
ility
• Criterion (β): specifies where we are on the ROC curve
• The ROC curve is specified by sensory capacities only(discriminability)
![Page 29: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/29.jpg)
Signal detection theory
• Discriminability: how well the observer can separate the presence of signal from its absence~ overlap between the two distributions~ bowing out of the ROC curve
• Measured by d’ (discriminability index,also called sensitivity)
http://www-psych.stanford.edu/~lera/psych115s/notes/signal/
![Page 30: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/30.jpg)
Signal detection theory
d’: selects the ROC curve
β: specifies a point on the selected ROC curve
same information as hit rate & false alarm rate, but:
hit rate, false alarm rate:both reflect sensation & decision characteristics;cannot separate the two
d’: depends only on sensation
β: depends only on decision
β
The two processes are separated
http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.ROC.html
![Page 31: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/31.jpg)
Signal detection theory
Fechner’s methods:Is a stimulus detectable? Yes or no?
• Clear-cut threshold value (with some variability) that can be measured– Stimulus intensity >
threshold detectable– Stimulus intensity <
threshold not detectable
• Dichotic outcome, categorical model
Signal detection theory:How well is it detectable? How sensitive the observer is to the stimulus?
• Measured by d’– The higher d’ is, the more
the stimulus is detectable– d’ = 0
not detectable at all
• Scalar outcome, dimensional model
![Page 32: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/32.jpg)
Signal detection theory
Sensation
(Noise)
Stimulus
Sensation level (SL)
Different task
Correct Incorrect
Forced-choice: eliminates the criterion
SDT: separates the criterion
Decision
SL ≥ β
Criterion (β)
SL < β
YES
NO
• Problem with Fechner’s methods: criterion
![Page 33: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/33.jpg)
Signal detection theory
Psychophysical measurements with SDT:1. Create a stimulus set with a range of intensities (like
in the method of constant stimuli)2. Test each stimulus many times with each observer3. On each trial, either present a randomly selected
stimulus or do not present anything4. Ask the observer if he/she detected the stimulus5. Calculate the hit rate and false alarm rate for each
observer, for each stimulus intensity6. Use the formula/table to calculate d’ for each case7. Examine how d’ changes with intensity: the higher
d’ is for a stimulus intensity, the greater the observer’s ability to detect this intensity
http://psych.hanover.edu/JavaTest/Media/Chapter2/MedFig.SignalDetection.html
![Page 34: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/34.jpg)
Signal detection theory
• Main results: changes in d’ values
Caudek–Rubin Vision Res. 2001
![Page 35: Psychophysics 3 Research Methods Fall 2010 Tamás Bőhm.](https://reader030.fdocuments.us/reader030/viewer/2022032601/56649dd15503460f94ac70ef/html5/thumbnails/35.jpg)
Signal detection theory
• There is also a β value for each d’ value• It can be informative about the decision
behavior:– Balanced: false alarm and
miss rates are equal– Liberal: the observer
says “yes” whenever there may be a signal
– Conservative: decision is yes only when it is almost certain that there is a signal
sensation level
pro
bab
ility
balanced
conservativeliberal