Post on 19-Jul-2015
Constructing Conceptual Knowledge Artefacts: Activity Patterns in the Ontology Authoring Process
Markel Vigo, Caroline Jay, Robert Stevens
University of Manchester (UK)
CHI 2015, Seoul (Korea)
@markelvigo
markel.vigo@manchester.ac.uk
What are ontologies?
Mad cow
Lion
Giraffe Cow
Animal
Classes
eats
SubClassOf
Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
What are ontologies?
Mad cow
SubClassOf
Lion Giraffe Cow
Animal
Classes
Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal eats
What are ontologies?
Mad cow
SubClassOf
Lion Giraffe Cow
Animal
Classes
Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
Vegetarian à Animal and (eats only (not (Animal)))
eats
What are ontologies?
Mad cow
SubClassOf
Lion
Giraffe Cow
Animal Classes
Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
Vegetarian à Animal and (eats only (not (Animal)))
eats
Vegetarian
What are ontologies?
Mad cow
SubClassOf
Lion
Giraffe Cow
Animal Classes
Properties Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
Vegetarian à Animal and (eats only (not (Animal)))
eats
Vegetarian
Complex artefacts
• Large size
• Domain expertise required
• Critical
• Highly expressive
Problem
We don’t know...
§ if existing ontology authoring tools are effective
§ how ontologist go about authoring
§ the authoring workflows
Authoring environment: Protégé
Authoring environment: Protégé
File menu
Class hierarchy
Property hierarchy
Annotations-Usage
Description
Mad cow
SubClassOf
Lion Giraffe Cow
Animal
Classes
Properties
Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
Vegetarian à Animal and (eats only (not (Animal)))
eats
File menu
Class hierarchy
Property hierarchy
Annotations-Usage
Description
Pop up
Explanation
Edit entity
Authoring environment: Protégé
Mad cow
SubClassOf
Lion Giraffe Cow
Animal
Classes
Properties
Axioms Cow, Giraffe, Lion à SubClassOf Animal
Mad cow à SubClassOf Cow
Mad cow, lion à Animal eats some Animal
Vegetarian à Animal and (eats only (not (Animal)))
eats
Study • 16 ontology authors, 3 typical tasks • Protégé à Protégé4US • Eye-tracking • Sync and merge interaction + fixation data • N-gram analysis
1389973572771,eye,Class hierarchy!1389973577038,eye,Description!1389973584775,log,EntitySelected!1389973586349,log,DescriptionSelected!1389973598978,log,EntityModified!1389973603166,log,EntitySelected!1389973605053,log,DescriptionSelected!1389973607847,log,EntityModified!1389973616404,eye,Description!1389973616754,eye,Class hierarchy!1389973617221,eye,Description!
Findings: interaction log data
• Interaction events account for 65% of events while authoring events are 30%
• The top 3 events (entity selection, description selection and invocation of editing menu) account for 56% of events
61219232839476182
113139142
182199
267314332
426617
9601004
14052793
BackUndo
Get explanationEntity renamed
Set propertyEntity dragged
Property additionEntity deletedLoad ontology
Hierarchy collapsed(i)Save
Description selected(i)Run reasoner
Hierarchy collapsedConvert into defined
Hierarchy expanded(i)Class addition
Entity selected(i)Hierarchy expanded
Entity edited:finishEntity edited:start
Description selectedEntity selected
0 1000 2000
Findings: eye-tracking data
The class hierarchy is the pivotal area • Index of the ontology • External memory
Transitions between AOIs
from
to
Ann−Usage
Class hierarchy
Description
Explanation
File menu
Pop up
Edit Entity
Prop. hierarchy
Ann−U
sage
Class h
ierarc
hy
Descri
ption
Explan
ation
File men
u
Pop up
Edit Enti
ty
Prop. h
ierarc
hy0
1000
2000
3000
4000
5000
6000
Findings: eye-tracking data
The class hierarchy receives users’ attention 45% of the time
0
100
200
300
400
File men
u
Ann.−U
sage
Class h
ierarc
hy
Descri
ption
Popup
Edit en
tity
Prop. h
ierarc
hy
Explan
ation
AOI
time
(sec
)
Findings: workflows
Select description
Select entity
0.29 Modify entity
0.37
0.63
0.59
Editing activity
Run reasoner
Convert into defined class
Save
Select description
0.16
0.15
0.17
0.40
Expand inferred
hierarchy
0.30
Select entity
0.41
0.37
0.43
Select inferred entity
0.54
0.25 0.12
Reasoning activity
Select entity
Expand hierarchy
0.48
0.31
Select inferred entity
Expand inferred
hierarchy
0.25
0.43
0.12
0.54
Load ontology
0.52
0.31
Expand hierarchy
Select description
0.29
0.37
Exploration activity Exploration workflow Editing workflow
Reasoning workflow
Implications: from raw data to workflows Workflows can be automatically identified
raw data
cleaning
merging
filtering
workflow detection
~7K rows
~200 rows
• Different authoring styles • Time distribution per workflow • Identification of confounding variables
Implications for design
• Support for bulk editing
• Anticipation of reasoner invocation
• Automatic detection of authoring problems
• Make changes to the inferred hierarchy explicit
tl;dr
• Identification of activity patterns when dealing with complex interactive artefacts
• Interaction log data + eye gaze data
• Data-driven
• Application on knowledge artefacts
Markel Vigo, Caroline Jay, Robert Stevens
University of Manchester (UK)
@markelvigo
markel.vigo@manchester.ac.uk