ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)
description
Transcript of ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)
![Page 1: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/1.jpg)
ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)
The AVRL of the Informatics and Telematics Institute exhibits substantial research activity, both basic and industry-oriented, as
well as technology transfer actions, in the area of virtual, augmented and mixed reality. This area includes modeling and
simulation of complex virtual environments, haptic force feedback applied research, 3D modeling and compression, deformable objects and cloth modeling, collaborative virtual environments, new sensor
development for head/hands/body tracking, multimedia analysis/synthesis, gesture recognition, head motion modelling and tracking, advanced applications of virtual/augmented/mixed reality in education and training, e-health, assistive technologies, culture,
industry and e-business.
![Page 2: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/2.jpg)
The ITI-CERTH AVRLab operates a fully operational and well-
equipped Virtual Reality Laboratory that serves as a
reference for augmented and virtual reality applications and
demonstrations in Greece. It employs a high quality scientific
group of personnel for the design, development and demonstration
of advanced augmented and virtual reality applications and
services. It participates in research networks with assorted institutes and industrial partners
in Greece and Europe and has established a spin-off company, to commercially exploit the research
results of the Laboratory.
Some significant applied research fields where ITI is activated are:
Haptics Computer Graphics 3D Semantics Mixed and Augmented
Reality Virtual, Augmented and
Mixed Reality Applications
![Page 3: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/3.jpg)
HapticsExtensive study and development of very
efficient approaches for haptic rendering, real time force feedback and real time collision detection for
complex virtual environments. Applications include virtual
environments for the visually impaired, geometry education in 3D
environments, virtual assembly planning, real time interaction with
complex cultural objects, etc
Computer GraphicsResearch and development of efficient
algorithms for modeling, representation animation and
monitoring of deformable and complex virtual objects. It includes new approaches for cloth modeling,
collision and self-collision detection, new interaction paradigms, real time
avatar simulations, smart objects and complex behavioral modeling
![Page 4: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/4.jpg)
3D Semantics Research and development of advanced
algorithms for search and retrieval of 3D content from large databases. Research includes new modeling approaches, descriptor extraction using rotation/translation/scaling
invariant features and identification of appropriate matching and data fusion
strategies for optimal descriptor selection
Mixed and Augmented RealityExtensive study and development of new
approaches in real time head/hands/body tracking for mixed
reality applications. Research directions include real time 3D
acquisition, stereo and multiview image sequence processing, feature detection and tracking, coding and
compression, advanced prediction and detection approaches, etc
![Page 5: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/5.jpg)
Virtual, Augmented and Mixed Reality Applications
Application oriented research in areas such as education and training, e-
health, assistive technologies, culture, industry, e-business, phobia
treatment, etc. Includes various approaches utilizing and integrating
appropriate virtual/augmented/mixed reality strategies for enhanced
application realism
![Page 6: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/6.jpg)
![Page 7: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/7.jpg)
The goals of the SIMILAR European network
• Integrating a task force on multimodal interfaces that respond efficiently to speech, gestures, vision, haptics and direct brain connections
• Merging 32 excellent European laboratories in Human-Computer Interaction (HCI) and in Signal Processing in a single research network.
• Address challenges in medical, edutainment and disabled areas
![Page 8: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/8.jpg)
The SIMILAR partners
SIMILAR institutions are spread all over Europe (and one in Africa)
Red : Research Blue : Industry
![Page 9: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/9.jpg)
Secondary Objectives
– SIMILAR will develop a common theoretical framework for fusion and fission of multimodal information using Signal Processing tools constrained by Human Computer Interaction rules.
– SIMILAR will develop a network of usability test facilities.
– SIMILAR will manage an International Journal, Special Sessions in conferences, summer schools, interact with industrial partners and promote new research activities.
– SIMILAR will address a series of grand challenges in edutainment, disabled people and medical applications and will develop new interfaces for environments where the user is unable to use his hands, like in surgical operation rooms, or in cars.
![Page 10: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/10.jpg)
The 7 SIGs (Special Interest
Groups )
– Information Fusion and Fission– Multimodal Analysis and Synthesis– Context Aware Adaptation– Usability– Medical Applications – Edutainment Applications– Disabled People Applications
![Page 11: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/11.jpg)
car
Multimodal Signal Proc.
EdutainmentDisabledAnd Cars
Medical
Platforms(OpenInterface) Usability
Fusion &Fission (IT)
Contextaware.
SIMILAR Synopsis
![Page 12: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/12.jpg)
A general specification for multimodal interfaces geared to solve interconnection problems between hardware and software from heterogeneous origins.
OpenInterface is mainly driven by : FPMs with Multitel (Belgium), ITI/CERTH (Greece), UCL-Tele (Belgium), CLIPs-IMAG(France).
OpenInterface will be accessible on the web : http://www.openinterface.org .
What is OpenInterface ?
![Page 13: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/13.jpg)
Collaborative research between a large amount of partners leads to interconnection
problems…
Interconnection problems (1)
![Page 14: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/14.jpg)
• John is working on a voice recognition algorithm and would like to test it into a real application…
• Brenda has a 3D visualisation system that would be interesting to drive with voice…
Interconnection problems (2)
… but programming interfaces are not the same !
?
![Page 15: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/15.jpg)
Interconnection problems (3)
• Brenda’s 3D visualisation system looks very promising and Wiliam would like to use it to inspect his 3D segmented medical structures…
… but programming interfaces are not the same !
?
![Page 16: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/16.jpg)
•Solution :
SIMILAR Platforms (OpenInterface)
Creation of a common language, using XML, that will be understood by anyone and used to describe module’s interfaces.
Interconnection problems (4)
SIMILARPlatforms
![Page 17: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/17.jpg)
Interconnection problems (5)
• Once a common language as been created, everybody will be able to drive partner’s plugin easily and in a way that could be re-used to drive other partner’s module of the same type.
Zoom 1,5Zoom 1,5
![Page 18: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/18.jpg)
• Easy development of multimodal applications • Avoid redundances of common tasks
– data formats support, usual audio/image/video processing,…
• Easy integration between partners’ modules using SIMILARPlatforms specifications
• Once a module is finished, he could then be integrated in the OpenInterface library to facilitate his deployment.
OpenInterface Main Goals
![Page 19: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/19.jpg)
OpenInterface Architecture
![Page 20: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/20.jpg)
OpenInterface : Component based architecture (1)
• Each functionality is a component
• Components are grouped into plugins (shared libraries)
• The content of each plugin is described using the SimilarPlatforms language
![Page 21: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/21.jpg)
KERNEL
speech
visualization
Video plugins
Image plugins
Fusion/fission
gesturerecognition
Conception: Component based architecture (2)
![Page 22: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/22.jpg)
OpenInterface Editor
•The OpenInterface Editor will– Facilitate the generation of new multimodal applications by
simple drag and drop of partner’s modules in a intuitive graphical user interface,
– Use the OpenInterface library and the SIMILARPlatforms language to automaticaly add new modules,
– Allow rapid creation of protopytes to test new multimodal interfaces.
![Page 23: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/23.jpg)
Similar eNTERFACE workshops
• Similar organizes annual workshops on multimodal signal processing and interaction– Graduate, master, PhD, postdoctoral students as well as
Seniors meet for four weeks and work together in common or complementary research areas and develop novel applications!
• eNTERFACE 2005 in Mons, Belgium was a complete success
• eNTERFACE 2006 in Dubrovnik was also successful• Hopefully eNTERFACE projects will continue far
beyond SIMILAR
![Page 24: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/24.jpg)
eNTERFACE 2005 projects
• Project 1: Combined Gesture-Speech Analysis and Synthesis
• Project 2: Multimodal Caricatural Mirror • Project 3: Biologically-driven musical instrument • Project 4: Multimodal Focus Attention Detection in an
Augmented Driver Simulator • Project 5: Multilingual Multimodal Biometric
Identification/Verification • Project 6: Speech Conductor • Project 7: A Multimodal (Gesture+Speech) Interface
for 3D Model Search and Retrieval Integrated in a Virtual Assembly Application
![Page 25: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/25.jpg)
eNTERFACE 2006 projects
• Project 1: An Agent Based Multicultural User Interface in a Customer Service Application
• Project 2: Multimodal tools and interfaces for the intercommunication between visually impaired and “deaf and mute” people
• Project 3: Sign Language Tutoring Tool • Project 4: Multimodal Character Morphing • Project 5: Introducing Network-Awareness for Networked
Multimedia and Multi-modal Applications • Project 6: An instrument of sound and visual creation driven by
biological signals • Project 7: Emotion Detection in the Loop from Brain Signals and
Facial Images • Project 8: Realtime and Accurate Control of Expression in
Singing Synthesis • Project 9: Multimodal Driving Simulator
![Page 26: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/26.jpg)
ITI-CERTH in eNTERFACE 2005
• ITI-CERTH was the leader of Project 7.
• The work has been published in IEEE Multimedia. “Masterpiece: Physical Interaction and 3D content-based search in VR Applications”, vol.13, no.3, pp. 92-100, July, 2006.
![Page 27: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/27.jpg)
ITI-CERTH in eNTERFACE 2006• ITI-CERTH was the
leader of Project 2.– Intercommunication
between blind and deaf-mute users
– Utilizing cross-modal transformation
– Tools are integrated in treasure hunting game that serves for both entertainment and education to the new technologies for the disabled
– Modalities: Haptics, Speech, Sign language, Gesture
CSCW Tool(Game)
I
nt
er
ne
t
SpeechRecognition
text Text to SignLanguage
Sign LanguageSynthesis
AvatarAnimation
Haptic Device
SpeechSynthesis
Sign LanguageAnalysis
text
SynchronisedEnvironmentParameters
SynchronisedEnvironmentParameters
Haptic ForceFeedbackInterface
CSCW Tool(Game)
Voice Data
text
text
Net
wo
rk
![Page 28: ITI-CERTH Augmented and Virtual Reality Laboratory (AVRLab)](https://reader035.fdocuments.us/reader035/viewer/2022062309/56814d53550346895dba8fc8/html5/thumbnails/28.jpg)
THANK YOU!
INFORMATICS & TELEMATICS INSTITUTE
1st km. Thermi-Panorama Road
PO BOX 361, 57001 THERMI THESSALONIKI, GREECE
TEL: +30 2310 464160 FAX: +30 2310 464164
http://www.iti.gr
Konstantinos Moustakas
Email: [email protected]