Sensor-based Situated, Individualized, and Personalized Interaction in Smart Environments Simone...
-
Upload
bruce-simpson -
Category
Documents
-
view
216 -
download
0
Transcript of Sensor-based Situated, Individualized, and Personalized Interaction in Smart Environments Simone...
Sensor-based Situated, Individualized, and Personalized Interaction
in Smart Environments
Simone Hämmerle, Simone Hämmerle, Matthias WimmerMatthias Wimmer, Bernd Radig, Michael Beetz, Bernd Radig, Michael Beetz
Technische Universität München – Informatik IXTechnische Universität München – Informatik IX
Sensor-based Situated, Individualized, and Personalized Interaction
in Smart Environments
SIP via sensorsSIP via sensors
Situation detection:Situation detection: information about persons: name, location, focus information about persons: name, location, focus
of attention, posture, motion,…of attention, posture, motion,…
Individualized settings:Individualized settings: desktop, avatar, desktop, avatar,
input settings (gestures, voice commands,…)input settings (gestures, voice commands,…)
Personalized settings: Personalized settings: user’s role, right management,…user’s role, right management,…
SIP detection using sensorsSIP detection using sensors more comprehensive SIP informationmore comprehensive SIP information more intuitive HCImore intuitive HCI
Our Test BedOur Test Bed
Sensors: cameras, microphones, laser-range-sensorsSensors: cameras, microphones, laser-range-sensorsActuators: monitor, speaker, video-wallActuators: monitor, speaker, video-wall
Scenarios:Scenarios: person localizationperson localization automatic loginautomatic login meeting remindermeeting reminder individualized individualized
gesture interactiongesture interaction
laser range sensor
camera
video wall
loud speaker
automatic
login
seamless track
identification
control video wall
using gestures
natural language
interaction
Techniques (Computer Vision)Techniques (Computer Vision)
person detectionperson detectionOpenCV (Haar-Face-Detector)OpenCV (Haar-Face-Detector)
person recognitionperson recognitionOpenCV (Hidden Markov Models)OpenCV (Hidden Markov Models)
person trackingperson trackingdeveloped at TUMdeveloped at TUMlaser-scanner based multiple hypothesis tracking,…laser-scanner based multiple hypothesis tracking,…
gesture recognitiongesture recognitiondeveloped at TUMdeveloped at TUMmotion templates, multiple classifiers,…motion templates, multiple classifiers,…
mimic recognitionmimic recognitiondeveloped at TUMdeveloped at TUMpoint distribution model, optical flow,…point distribution model, optical flow,…
Techniques (others)Techniques (others)
natural language input natural language input Java Sphinx 4 (origin CMU, now open source)Java Sphinx 4 (origin CMU, now open source)
phonemes are already trained phonemes are already trained
we defined the words ( = concatenation of phonemes)we defined the words ( = concatenation of phonemes)
we defined the grammar ( = allowed sentences)we defined the grammar ( = allowed sentences)
natural language outputnatural language output provides the user with audio informationprovides the user with audio information
user can be mobileuser can be mobile
FreeTTS 1.2 (sourceforge)FreeTTS 1.2 (sourceforge)
Software architectureSoftware architecture
Multi Agent Framework
Loudspeaker Agent
Loudspeaker
Microphone Agent
Microphone Module
Microphone
D
Camera
Person Recognition
Agent
Person Recognition
Module
Robot
Robot AgentRobot Module
Illumination Agent
Illimination Module
LampBrain
Dis
patc
her
Knowledge Base Accessor
Knowledge Base
Command-Generator
Command-Generator
Command-Generator
Loudspeaker Module
FilterFilterFilter
D
Camera
Gesture Detection
Agent
Gesture Detection Module
semantic communication
proprietary communication
Dis
pa
tch
er
multi agent framework
ConclusionConclusion
Advantages using sensorsAdvantages using sensors additional and more exact context knowledgeadditional and more exact context knowledge unobtrusive systemunobtrusive system
Multi agent frameworkMulti agent framework distributed and scalable systemdistributed and scalable system simply extensible to further scenariossimply extensible to further scenarios
Overall semanticOverall semantic semantic agent communicationsemantic agent communication central aggregation of semantic context knowledgecentral aggregation of semantic context knowledge
Leads toLeads to more comprehensive SIP informationmore comprehensive SIP information seamless integration of SIP informationseamless integration of SIP information intuitive HCIintuitive HCI
Setup & BenefitSetup & Benefit
sensors for detection of SIP context:sensors for detection of SIP context: camerascameras microphonesmicrophones laser-range-sensorslaser-range-sensors pressure-sensors, …pressure-sensors, …
sensors provide knowledge about the SIP contextsensors provide knowledge about the SIP context situation dependant servicessituation dependant services intuitive HCI (human computer interface)intuitive HCI (human computer interface)
application scenarios:application scenarios: support in meetings and presentationssupport in meetings and presentations intelligent House intelligent House external robot controlexternal robot control
Our Test BedOur Test Bed
Sensors: Cameras, Microphones, Laser-Range-SensorsSensors: Cameras, Microphones, Laser-Range-SensorsActuators: Monitor, Speaker, Video-WallActuators: Monitor, Speaker, Video-Wall
Scenarios: Scenarios: automatic loginautomatic login meeting remindermeeting reminder individualized gesture interactionindividualized gesture interaction
intuitive robot controlintuitive robot control person localizationperson localization
SensorsSensors
person recognitionperson recognition
(Bild)(Bild)
gesture recognitiongesture recognition
(Bild)(Bild)