Cognitive neuroscience approach to telemedicine - … · Cognitive neuroscience approach to...
Transcript of Cognitive neuroscience approach to telemedicine - … · Cognitive neuroscience approach to...
Tokyo, Japan, 4-5 February 2013
Cognitive neuroscience approach to telemedicine
Makio KASHINO, Ph.D. Senior Distinguished Scientist, Executive Manager
NTT Communication Science Laboratories [email protected]
ITU Workshop on “E-health services in low-resource settings:
Requirements and ITU role”
(Tokyo, Japan, 4-5 February 2013)
Tokyo, Japan, 4-5 February 2013 2
Human Information Lab, NTT CSL
���������� �
����������
������������������
�01���
23453#678479:�
25:;73453#678479:�
<=>7:�
!7?#7@A7347BC7�
������
������
���������� �
����������
������������������
�01���
23453#678479:�
25:;73453#678479:�
<=>7:�
!7?#7@A7347BC7�
������
������
Informatics
Neuroscience
Psychophysics
Man-machine interface
Perception, Emotion, Motor control
Tokyo, Japan, 4-5 February 2013 3
Challenges in telemedicine
! Transmission delay and asynchrony
! Guiding a remote collaborator
! Face-to-face-like communication
Compensation for transmission delay
Tokyo, Japan, 4-5 February 2013 4
����� �������
�+�����
�� "����
����
����� ��
��������
Takamuku, Gomi (2011) ECVP2011
No delay With delay Delay+flow Delay Delay
Heavy
Smooth Smooth Awkward
Compensation for transmission delay
Tokyo, Japan, 4-5 February 2013 5 Takamuku, Gomi (2011) ECVP2011
Compensation for transmission delay
Tokyo, Japan, 4-5 February 2013 6 Takamuku, Gomi (2011) ECVP2011
Involuntary arm movement induced by vision
Tokyo, Japan, 4-5 February 2013 7
1717
�� ������������ ��������������������������
Movement duration: 1.0 sassisted by beeping soundsFixation marker
0.05 [m]
XY 0.
05
[m]
RLC
110 ms
85 ms
91 ms
norm
aliz
ed
Saijo et al. 2005 J.Nneurosci.
MFR (Manual Following Response)
����������
1717
�� ������������ ��������������������������
Movement duration: 1.0 sassisted by beeping soundsFixation marker
0.05 [m]
XY 0.
05
[m]
RLC
110 ms
85 ms
91 msno
rmal
ized
Saijo et al. 2005 J.Nneurosci.
MFR (Manual Following Response)
����������
1717
�� ������������ ��������������������������
Movement duration: 1.0 sassisted by beeping soundsFixation marker
0.05 [m]
XY 0.
05
[m]
RLC
110 ms
85 ms
91 ms
norm
aliz
ed
Saijo et al. 2005 J.Nneurosci.
MFR (Manual Following Response)
����������
Saijo, Murakami, Nishida, Gomi (2005) The Journal of Neuroscience
Manual Following Response (MFR)
・Involuntary ・Fast
Different visual processing systems for perception and motor control
Tokyo, Japan, 4-5 February 2013 8 Gomi, Abekawa, Nishida (2006) The Journal of Neuroscience
Fast motion of a coarse pattern is not salient perceptually, but effective to induce MFR
Audiovisual simultaneity
Tokyo, Japan, 4-5 February 2013 9
How to judge A-V timing in the presence of significant external and internal asynchrony?
Adaptation to A-V asynchrony induces…
Tokyo, Japan, 4-5 February 2013 10 Fujisaki, Shimojo, Kashino, Nishida (2004) Nature Neuroscience
Recalibration of A-V simultaneity
Tokyo, Japan, 4-5 February 2013 11
“Sim
ult
aneo
us”
res
ponse
A first V first
N=7
Test AV lag (ms)
adaptation (0ms)
adaptation (-235ms)
adaptation (+235ms)
Fujisaki, Shimojo, Kashino, Nishida (2004) Nature Neuroscience
Tokyo, Japan, 4-5 February 2013 12
Challenges in telemedicine
! Transmission delay and asynchrony
! Guiding a remote collaborator
! Face-to-face-like communication
Perceptual attraction force: Buru-Navi
Tokyo, Japan, 4-5 February 2013 13
Amemiya, Gomi (2012) IEEE Transactions on Haptics
Amemiya, Ando, Maeda (2007) Laval Virtual 2007
How to generate “pull” or “push” sensation in the mobile devices having no external fulcrum?
Navigation by Buru-Navi
Tokyo, Japan, 4-5 February 2013 14 Amemiya, Sugiyama (2009) ASSETS
Successfully navigated ~90% (21/23) of visually-impaired people in a maze
Tokyo, Japan, 4-5 February 2013 15
Challenges in telemedicine
! Transmission delay and asynchrony
! Guiding a remote collaborator
! Face-to-face-like communication
Implicit InterPersonal Information
Tokyo, Japan, 4-5 February 2013 16
Supported by JST CREST (PI: Makio Kashino [NTT], Collaborators: Sinsuke Shimojo [Caltech], Katsumi Watanabe [U Tokyo])
������������������������� �������
���� ���������������������� �������
!"#
!$#
!%#
!&#
!'#
!(#
)#
(#
'#
&#
%#
$#
"#
*#
+#
,#
()#
)# (# '# &# %# $# "# *# +#
-./#
01#
)#
()#
')#
&)#
%)#
$)#
")#
)# (# '# &# %# $# "# *# +#
-./#
01#
!"#
!$#
!%#
!&#
!'#
!(#
)#
(#
'#
&#
%#
$#
"#
)# (# '# &# %# $# "# *# +#
!"#$%&'()*+),#!(-'.)
01#
!"#
!$#
!%#
!&#
!'#
!(#
)#
(#
'#
&#
%#
$#
"#
)# (# '# &# %# $# "# *# +#
,#!(-'.)*+),#!(-'.)
01#
Gamma, 40-60 Hz (650ms) Theta, 5-8 Hz (500ms)
ERPs after Face1 presented (200-400ms)
Wavelet-based time-freq. plot, after Face2 presented
All before decision! (Lindsen, et al., NeuroImage, '10)
Two face presented sequentially.
22
ⓒ Caltechⓒ Caltech
IIPI (interpersonal interaction of involuntary and subconscious body movements and physiological responses) may play critical roles in communication → Decoding and controlling IIPI for better communication systems
Interpersonal interaction of brain and ANS activities
Interpersonal interaction of body movements
Acoustic IIPI for talker “presence”
Tokyo, Japan, 4-5 February 2013 17
Kobayashi, Ooishi, Enomoto, Kitagawa, Ueno, Ise, Kashino (in preparation)
Spontaneous utterances recorded and reproduced by the sound field sharing system “BoSC” (CREST Ise team)
→ Subjective rating, ANS activity, hormone concentration
Dynamic Static
Acoustic IIPI for talker “presence”
Tokyo, Japan, 4-5 February 2013 18
Kobayashi, Ooishi, Enomoto, Kitagawa, Ueno, Ise, Kashino (in preparation)
n Significant differences also found for sympathetic nerve activity and salivary hormone concentration
n Subjective rating 1. Artificial sound? 2. Feel talker presence? 3. Like the talker? 4. Interested in the content?
Concluding remark
Tokyo, Japan, 4-5 February 2013 19
! Challenges in telemedicine may be overcome by taking advantage of brain mechanisms underlying human perception, emotion, motor control, and communication.