S S1
Intelligent MultiModal Interfaces
Manuel J. FonsecaJoaquim A. Jorge
http://sketch.inesc-id.pt
S S2
Presentation Outline
• Problem
• Tools/Solution
• Objectives
• Project Overview
• Task Analysis
• 2D Sketch Editor
• Sketch-Based Retrieval
• Current Status
S S3
Problem
• Longer product cycles• Inflexible separation - design and engineering• Raise manufacturing costs and Time-to-market
• CAD systems allow very complicated models• “Bad” Usability at early stages of product design
(pencil/paper reign)• Early design phase dissociated from product
definition• Large conceptual gaps
Mental model vs Geometric models
S S4
S S5
S S6
S S7
• 9 Weeks to get the Virtual model
S S8
Tools/Solution
• Alternative input modalities and new interface technologies
• New sketching applications supporting what we call Calligraphic Interfaces. – Remove artificial dialogue constraints (WIMP Uis)
– Allow designers to work as they would with more traditional media (pencil/paper)
S S9
Objectives
• Rethinking Usability of design
• Research into better input modalities
• RISC Interfaces, Drawing as metaphor
• User-centered approach
• Computer-based tools in initial stages of
product design
S S10
Constraint BasedConstraint BasedModel AssemblyModel Assembly
Personal Design AssistantsPersonal Design Assistants
Immersive Surface ModelingImmersive Surface ModelingRetrieval by ExampleRetrieval by Example
S S11
Task Analysis
• We visited Industrial partners– CENTIMFE– ItalDesign– ELASIS
• Observed/Videotaped users in action• Interviewed them and presented
questionnaires• They answered to 11 questions• Reports for each site
How ?
S S12
Task Analysis
1. Who is going to use the system?2. What tasks do they perform?3. What tasks are desired?4. How are the tasks learned?5. Where are the tasks performed?6. How often are the tasks performed?7. What are the time constraints?8. etc.
What did we ask?
S S13
Task Analysis
• Identify critical tasks
• Define user requirements
• Recommendations and guidelines for the project
• Assure that our work is based on Real Data not on our imagination!
Why ?
S S14
Task Analysis• User Requirements
– Larger drawing area– Bigger and easier to identify icons– New tools for the left hand – No mouse and no keyboard– Access to past projects– Possibility to use the touch sense for modelling– Software integration (2D+3D+presentation+...)– ...
• Final TA meeting (Sept 9-10, Lisbon)– Final report almost done
Results/Conclusions
S S15
On Going WPs
• API Specification
• 2D Sketch Editor for Modelling
• 3D Surface sketching in Immersive
Environments
• Constraint-Based Scene Modelling
• Sketch-Based Retrieval
S S16
2D Sketch Editor
• Recognition-based Interfaces
• Reduced Instruction Set Interfaces
• Handle Imprecision and Ambiguity
in uniform manner
• Less (More expressive) Commands
S S17
Current Status
• Simple prototype using IRIT (geometric kernel)
• Make it more robust using better
geometric kernel such as ACIS or ICEM
• Add more functionality
2D Editor
S S18
Sketch-Based Retrieval
• Designer re-use drawings
• CAD Systems do not support retrieval
• Text indexing not sufficient for CAD
• Need Innovative methods for CAD databases
• Simple geometric features + Spatial relations
• Indexing + Matching algorithms
S S19
Query by Sketch
• How do designers retrieve?(Library Component, Past project) – Videos from TA
• Our proposalSketched Query Suggested Results
Retrieval
S S20
Matching
System ArchitectureClassificationQuery
Retrieval
S S21
Current Status
• Description using Topology Graphs
• Graph Matching using Eigenvalues
• New Indexing Structure, NB-Tree– High-Dimensional Data points
– Outperforms recent structures• SR-Tree [Katayama 97]
• Pyramid Technique [Berchtold 98]
• A-Tree [Sakurai 00]
Retrieval
S S22
Performance Evaluation
• 100,000 points
• PII@233MHz
• 384MB RAM
• Knn = 10
• Uniform Data
Retrieval
S S23
Project Outline and Status
Integration and Final Prototype
Integration and Final Prototype
Final User Evaluation
Final User Evaluation
User Requeriments
& Task Analysis
User Requeriments
& Task AnalysisAPI
Specification
APISpecification
Field and User Tests
Constraint-Based Scene Modelling
Constraint-Based Scene Modelling
Sketch-Based Retrieval
Sketch-Based Retrieval
2D Sketch Editor for Modelling
2D Sketch Editor for Modelling
3D Surface Sketching
3D Surface Sketching
Prototypes
Done Almost Done
Working On It