By Mari Riess Jones The Dynamics of Attending and Scene Analysis.
-
Upload
rodney-holt -
Category
Documents
-
view
217 -
download
0
Transcript of By Mari Riess Jones The Dynamics of Attending and Scene Analysis.
By
Mari Riess Jones
The Dynamics of The Dynamics of Attending Attending andand Scene AnalysisScene Analysis
Thanks to Thanks to
Ralph BarnesRalph Barnes
Devin McAuleyDevin McAuley
Heather MoynihanHeather Moynihan
Noah MacKenzieNoah MacKenzie
Amandine PenelAmandine Penel
Jennifer PuenteJennifer Puente
Nate VaughanNate Vaughan
Special Thanks Special Thanks to to
Edward Edward LargeLarge
I. Scene Analysis: A Neo Gestalt view
II. Scene Analysis: A dynamic perspective
III. The Changing Scene: Tracking in real time
OverviewOverview
I. Scene Analysis:I. Scene Analysis: A Neo Gestalt view A Neo Gestalt view
What’s a Scene?What’s a Scene?
The term scenescene is taken from vision. Two cases are most common:
1. A stationary viewer
2. A moving viewer
The scene is the static visual layout.The scene is the static visual layout.
Sound ScenesSound Scenes
A common assumption:
We use the proximal sound stimulus to recover the location and/or identity of a distal, vibrating, object within a scene.
This is the source identity principle.
Bregman’s TheoryBregman’s Theory
of of Scene AnalysisScene AnalysisBregman builds on this idea:Bregman builds on this idea:
Goal of auditory perception… Goal of auditory perception… is identification of one is identification of one or more sources of a or more sources of a proximal mixture of sounds. proximal mixture of sounds.
If an unfolding sound pattern is perceived as a structurally intact , i.e. as one streamone stream,then it must come from ONE ONE source.source.
Other patterns? Other patterns?
If a pattern is perceived to break up, i.e. form N different subjective units, streamsstreams, ,
then, it must …
come from N separate sources.
Patterns NOT perceived as intact….
Time
high
low
Another source?
Example: A sound Example: A sound patternpattern
frequencyA stream ?One source?
The Big QuestionThe Big Question
What makes a pattern seem seem structurally intact?
Or… What determines a stream?stream?
Bregman’s AnswerBregman’s Answer
Involves a two stage theory
in which
Neo Gestalt principles are critical.
The Neo Gestalt ViewThe Neo Gestalt View
I. Primitive processes Innate, automatic, effortless, Determine coherent perceptual groups at fast rates.
II. Schema driven processes Learned, voluntary, effortful, Determine attention to groupings at slow rates.
Two Stages
Auditory Scene PerceptionAuditory Scene Perception Stage I: Primitive Processes Operate on Gestalt grouping principles:
proximity in frequency and time; similarity (e.g., timbre); continuity; exclusive allocation… and so forth
These are domain general (I.e. speech, music, natural environs,etc.)
Proximity vs ContinuityProximity vs Continuity
Tougas and Bregman (1985)
In a series of famous studies pitted proximityproximity against continuity. continuity.
Goal: To assess which principle dominated, and to dispute a trajectory explanation.
Bouncing versus Crossing trajectories
Time in seconds
Possible Percepts
ConclusionConclusion
Proximity in frequency dominates
People hear bouncing patterns
Rhythm PerceptionRhythm Perception
Speeding up a sequence can break it , perceptually, into interleaved interleaved streams streams
This changes its perceived This changes its perceived rhythm! rhythm!
A stream usually conveys a rhythmrhythm
RhythmRhythm
is perceived to have a ‘is perceived to have a ‘galloping galloping ‘‘rhythmrhythm
This slow melody…This slow melody…
Time
Rhythm perception… depends on the relationshiprelationship of frequency change (f ) to time change (t).
Hi Hi frequencyfrequency
Lo Lo frequencyfrequency
f
t 2 t
But… That Rhythm But… That Rhythm DisappearsDisappears
In a In a fasterfaster melody with the same tones. melody with the same tones.
Now we hear two melodies (streams) each with an isochronousisochronous rhythmrhythm..
A Neo Gestalt A Neo Gestalt ExplanationExplanation
Builds on Gestalt proximity:
1.1. Temporal proximityTemporal proximity is heightened at fast rates.
2. But frequency proximity frequency proximity dominates.
3.3. Two streamsTwo streams form based on pitch proximity: A high tone stream and a low tone stream.
II.II. Scene Analysis:Scene Analysis:A Dynamic PerspectiveA Dynamic Perspective
Auditory PatternsAuditory Patterns
Most of our environment comprises
sources that turn on and off and
modulate over time to project pattern
structure that is….
motion-like and rhythmicmotion-like and rhythmic
Important Pattern Properties Important Pattern Properties
1. Motionlike properties: Rates of change.
[ e.g., f/ t ]
2. Rhythmic properties: Relative timing.
[e.g., ti/tk ]
Motionlike TrajectoriesMotionlike Trajectories
Pitch VelocityPitch Velocity: f / t
In 1976, I proposed that the rate-of-change of frequency (log scale) may be perceived as a sort of velocitylocity. .
Time
high
low
Determinants of Pitch Determinants of Pitch Velocity Velocity
frequency
A sudden increase in rate-of-change in an isochronous rhythm is perturbing.
f
t
f/ t
Motionlike Trajectory Motionlike Trajectory limits on pitch velocity?limits on pitch velocity?
f log
FAST t Slow
Intact Serial Percepts
Vel
ocity
Lim
it
(Jones, 1976)
RATE
SMALL
LARGE
Stimulus patterns
streamed
Rhythmic Properties: Rhythmic Properties: Relative TimingRelative Timing
flog
FAST t =IOI SLOW
Lim
it
RATE
Slower patterns
Alternative View: Alternative View: A focus on timeA focus on time
How do rhythmic properties of a pattern affect real time responding to it?
In the remainder of this talk, I address this question.
General AssumptionsGeneral Assumptions
Connecting through time Connecting through time
Pattern timing determines how living Pattern timing determines how living things things connect connect with a changing scene. with a changing scene.
Living thingsLiving things
Animate creatures possess Animate creatures possess internal internal periodicitiesperiodicities that can that can
synchronize to patterns. synchronize to patterns.
EEnvironment nvironment
Scenes often project dynamic patterns Scenes often project dynamic patterns with with relative timing that is relative timing that is quasi- quasi- periodicperiodic. .
Great Gray OwlGreat Gray Owl
Horned Horned LarkLark
CardinalCardinal
t = IOI = 700 t = IOI = 700 msms
Some quasi-periodic patternsSome quasi-periodic patterns
Internal Periodicities: Internal Periodicities: Biological Oscillations Biological Oscillations
Most studied are circadian Most studied are circadian rhythms; oscillatory periods of ca. rhythms; oscillatory periods of ca. 24 hours. 24 hours.
Many internal periodicities exist; Many internal periodicities exist; periods of 1 periods of 1 ms, several minutes, years !. ms, several minutes, years !.
ConnectingConnecting
“How do internal oscillations of an organism connect with pattern timing? “
One answer: By One answer: By synchronizing. synchronizing.
More precisely: By entrainmentMore precisely: By entrainment
Biological EntrainmentBiological Entrainment
Pittendrigh & DaanPittendrigh & Daan (1976) . (1976) . Circadian rhythms of 24 Circadian rhythms of 24 hours hours synchronize to synchronize to light/dark cycles. light/dark cycles.
Daan et al. Daan et al. (2001) recently (2001) recently updated updated this two oscillator this two oscillator model. model.
SummeSummerr
WinterWinter
From Daan et al. J.Biological Rhythms (2001)
Example: Two Entraining OscillationsExample: Two Entraining Oscillations
M Phase locked
E Phase locked
Shorter Periods?Shorter Periods?
In speciesIn species
with refined use of fast sound with refined use of fast sound patterns,patterns,
Can we postulate,Can we postulate,
internal oscillations with periods internal oscillations with periods lessless than the circadian than the circadian period ?period ?
Generalizing…..Generalizing…..
Long Long natural natural periodsperiods
Respond to Respond to light light changes changes
Most Biological Most Biological
OscillatorsOscillators
Alternatively Alternatively
Other Other OscillatorsOscillators
ShortShort natural periods natural periods
Respond to Respond to soundsound changeschanges
Background:Background: Rhythmic (periodic) ExpectanciesRhythmic (periodic) Expectancies
Quasi-periodicities inQuasi-periodicities in
Rhythmical sound patternsRhythmical sound patterns
Induce Induce Periodic expectanciesPeriodic expectancies
Quasi-periodicities inQuasi-periodicities in
Rhythmical sound patternsRhythmical sound patterns
Induce Induce Periodic expectanciesPeriodic expectancies
Perhaps such expectancies reflect Perhaps such expectancies reflect entrainments entrainments to to patterns of stimulus IOIs?patterns of stimulus IOIs?
Back to Synchrony: Back to Synchrony: AA simple premise simple premise
When we attend , When we attend , an internal activity is an internal activity is elicited … elicited … that co-occurs that co-occurs with the attended object. with the attended object.
This activity involves This activity involves Internal oscillations Internal oscillations
(Jones, 1976). (Jones, 1976).
Where elements appear… then disappear
Attending to Dynamic Attending to Dynamic Patterns Patterns
Time Time
Attention must occur in synchrony with elements.
Dynamic AttendingDynamic Attending
Selective attending in time is Selective attending in time is pacedpaced by stimulus timing. by stimulus timing.
Postulates that :Postulates that :
Attentional focus in time becomes Attentional focus in time becomes phase-locked phase-locked to elements. to elements.
III. The Changing Scene:III. The Changing Scene:Tracking sounds in real Tracking sounds in real
timetime
Attending oscillates in Attending oscillates in timetime
Resources are periodically distributed as internal oscillationsinternal oscillations. .
Time Pulses of Pulses of attention attention energyenergy
Inter-onset-interval, IOI
Expected point in time
PhasePeriod
Formalized as an Oscillator Formalized as an Oscillator ModelModel :
by by Ed Large Ed Large (Large & Jones, 1999)(Large & Jones, 1999)
Stimulus timing paces the oscillator. Oscillator period and phase determine this entrainment .
Periodic Periodic ExpectanciesExpectancies
Stimulus IOI
Phase, iPeriod, Pi
Tones, i
Period and Phase Period and Phase change: change: They They draw draw attending rhythms attending rhythms to “fit” a time to “fit” a time pattern. pattern.
Period changes
Phase changes
The adaptiveadaptive oscillator
The entrained oscillator gravitates to an attractor state of:
1. Phase Synchrony:1. Phase Synchrony: A zero time difference between an sound onset and a pulse peak. 2. Period matching:2. Period matching: A zero time difference between an IOI and oscillator period (limit cycle).
A simple dynamical system.
Relative Phase, Relative Phase, i i
Gives observed - expected times
ii is the order variable in this system. It summarizes the state of the system at any instant i.
i+1 i+1 = = ii + IOI + IOIi+1i+1/P/Pii - - F(F(ii))
The coupling term, F, here is sinusoidal: F(F()= {1/2)= {1/2}sin 2}sin 2 => =>
-
Phase Attractor: Synchrony
-.25 .25
Relative Phase,
i
Phase Response Curve, PRC
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
0.2
-0.5 -0.25 0 0.25 0.5
Ph
ase C
orr
ecti
on
,
Entrainment Region
{ { = = 1.0 }1.0 }
Theoretical
Phase Attractor: synchrony
-.25 .25
What does this approach What does this approach purchase purchase
??
A few experimentsA few experiments….….
Preview: Preview: Our Experimental Our Experimental
DesignsDesigns Will involve a context rhythm, given by tones:
Followed by a comparison, C, (of some sort)
Time Judgment
Comparison IOI Pitch Judgment
Comparison tone
C
Time Judgment TasksTime Judgment Tasks RationaleRationale
If relative timing in sound arrays entrains attending,
then the context rhythm should
Shape temporal expectancies
Affect time judgments.
Example Study:Example Study:
Judgment TaskJudgment Task: Is a comparison time interval “shorter”, “same” or “ longer” ?
““Ignore context IOIs“Ignore context IOIs“
Context IOI
Standard IOI
Comparison IOI
Context IOIs are 600 ms. This rhythm should synchronize attending.
+/-T
Standard Ending Varies:Standard Ending Varies:
Expected Standard Ending: = 600 ms Unexpected Standard Endings:
Early = 579 ms
Late = 621 ms
Very Early = 524 ms
Very Late = 676 ms
23A comparison IOI is always yoked to standard IOI
STANDARD IOI
ExpectedEnding
ExpectedEnding:
“Same”
Phase
COMPARISON IOI
Period
Modeling Oscillatory Attending
Context IOIs
600 600 msms
600600
When a standard ends as expected, When a standard ends as expected, the oscillator’s period is the oscillator’s period is accurately preserved accurately preserved
….as a memory of the standard ….as a memory of the standard IOI.IOI.
The Reason: The Reason: The Attentional The Attentional PulsePulse
Pulse location depends on rhythmic context
Expected time
15
Pulse Pulse locationlocation
Very Early Expected Very Late
Standard Ending Time
PC
One Prediction:
A quadratic accuracy Expectancy ProfileExpectancy Profile:
Attentional Attentional PulsePulse
Predicted and ObservedPredicted and ObservedExpectancy Profiles Expectancy Profiles (Large & Jones 1999)(Large & Jones 1999)
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Very Early Early Expected Late Very Late
Standard Ending Time
PC
PredictedObserved
P < .01
Subsequent Subsequent ExperimentsExperiments
Over a dozen experiments later (e.g., Barnes & Jones, 2000; McAuley & Jones, 2002) We Find:
• The profile vanishes with no context; sharpens with longer sequences .
• Holds for different rates.
• Reveals harmonic scaling of timing.
Different Induction Different Induction RatesRates
Three conditions: different induction rates.
Condition Condition Induction IOIs Standard Induction IOIs Standard durationsdurations
1. 300 ms 300 ms 524, 600, 676 ms
2. 500 ms 500 ms 524, 600, 676 ms
3. 600 ms 600 ms 524, 600, 676 ms
Isochronous sequencesIsochronous sequences
This task required same vs different duration judgments
Observed Expectancy Profiles Observed Expectancy Profiles Three Three
Time Judgment Conditions Time Judgment Conditions
0.4
0.5
0.6
0.7
0.8
0.9
1
524 600 676
Duration of Standard
Ag
300
500600
Conclude Conclude Back to Relative timing
1. Ratios Ratios of induction IOIsinduction IOIs to expected expected standardsstandards seem influential.
2. 2. Harmonic ratiosHarmonic ratios of 1:1 and 1:2 yield different profiles than the 500 ms with a more complex ratio.
Pitch JudgmentsPitch Judgments
We generalized this approach to attending to tone frequency.
Does rhythmic context affect pitch judgments?
StandarStandard d pitchpitch
ComparisoComparison n pitch pitch
TaskTask: Is a comparison pitch: “same”, “higher”, ”lower” than a
standard pitch?
““Ignore distracting Ignore distracting pitches!”pitches!” “higher”
same”
“lower”
A Pitch Judgment Task A Pitch Judgment Task
Critical IOICritical IOI
21
Independent Variable: Critical IOI
RationaleRationale
In a rhythmic context, In a rhythmic context, attending is paced in time. attending is paced in time.
Pitch judgments will be best Pitch judgments will be best for for temporally expectedtemporally expected tones: tones: When critical IOI = context IOI.When critical IOI = context IOI.
Very Early Expected Very Late
Onset Time of Comparison
PC
Similar Predictions
AA quadratic expectancyquadratic expectancy profile
Attentional Pulse
Experiment 1: Expectancy Profile
0.5
0.55
0.6
0.65
0.7
0.75
0.8
Very Early Early On Time Late Very Late
Onset Time of Comparison
Prop
ortio
n Cor
rect
P<.005
Related ExperimentsRelated Experiments
Found a:
flattened expectancy profile
with irregular context timing
Recent findingsRecent findings
Suggest the importance of Suggest the importance of higher-order time higher-order time structurestructure … …
In focal attending over time In focal attending over time levels levels
An Accent Rhythm An Accent Rhythm
21
Accent Timing
Accented Tones
Higher time level : 4 IOIs
Lower time level: IOI
Accent Rhythm is VariedAccent Rhythm is Varied
21
RegulaRegularr
IrregularIrregular
Accented Tones
Standard Standard pitch pitch
Comparison Comparison pitchpitch
Same Pitch Judgment Same Pitch Judgment TaskTask
Critical IOI is Varied Critical IOI is Varied
21
Two Oscillators?Two Oscillators?
Remember those two circadian Remember those two circadian oscillators (Morning, Evening)?oscillators (Morning, Evening)?
Well, Well, these are these are different!different!
Shorter periods Shorter periods Sensitive to Sensitive to
soundsound
Two Different OscillatorsTwo Different Oscillators
Two oscillators: Different periods
1. Lower-order oscillator, period ~ period ~ IOIIOI..
2. Accent oscillator, period ~ 4 IOIsperiod ~ 4 IOIs
If the Accent oscillator dominates, then only only the Regular Rhythm will give an Expectancy Profile.
Which dominates?
50
60
70
80
90
Very Early Early On Time Late Very Late
Regular Rhythm
Irregular RhythmPCPC
P<.001
Two Oscillators Two Oscillators In listening to sound patternsIn listening to sound patterns
Differential responses to Differential responses to sounds: sounds: AcAccented versus cented versus Unaccented Unaccented
Flexible attendingFlexible attending:: DDominance of one oscillator in ominance of one oscillator in focal attending. focal attending.
Final CommentsFinal Comments
The dynamics of sound patterns together the dynamics of attending may explain real time tracking of events.
Assumptions about rhythm and attentional synchrony can enhance theoretical power.
Entrainment to dynamic contexts may play a role in judgments of both time and pitch.
Thank you !
Thank you !
Two routes to SynchronyTwo routes to Synchrony
1. Anticipatory attending: 1. Anticipatory attending:
Attending heightens Attending heightens priorprior to an element. to an element.
Oscillator Oscillator period period is important. is important.
2.2. Reactive attending:Reactive attending:
If anticipation fails, If anticipation fails,
Attending shifts Attending shifts afterafter an element occurs. an element occurs.
Oscillator Oscillator pphasehase is important. is important.