A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI...
Transcript of A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI...
![Page 1: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/1.jpg)
A system for interactive learning in dialogue with a tutor
Pedagoška konferenca FRI Ljubljana, 6 June 2013
Danijel Skočaj
University of Ljubljana
![Page 2: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/2.jpg)
Motivation
industrial robots
SF
human
cognitive robots
communication
perception action
attention goals
planning reasoning
learning
![Page 3: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/3.jpg)
Learning
Why learning?
It is impossible to encode all the knowledge by hand!
It is impossible to envision all the data in advance!
Gather information from the environment (and a human companion)
![Page 4: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/4.jpg)
Learning efficiency
Learn
as good as you can
as fast as you can
as autonomously as you can
number of training samples/ tutoring cost
recognition success
![Page 5: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/5.jpg)
Interactive learning
intr
ospective learn
ing
exstr
ospective learn
ing
![Page 6: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/6.jpg)
Robot George
George, the curious robot
George is curious.
He wants to know!
He wants to learn!
Curiosity driven extension of knowledge.
Use the information given by the tutor!
Ask the tutor for additional information!
Learn as much as you can!
Learning of categorical knowledge
![Page 7: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/7.jpg)
Scenario setup
Table top scenario
Extendable to more general scenarios
Learning about objects
Colour
Shape
Type
Baseline knowledge/abilities
Perceptual input
Low, mid-level vision
Situated dialogue
Ontology mediated reference resolution
System architecture: CAS (CAST)
Mixed initiative learning:
tutor driven
robot driven
![Page 8: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/8.jpg)
System overview
![Page 9: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/9.jpg)
Self-understanding for self-extension
![Page 10: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/10.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
![Page 11: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/11.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
H: Look left.
H: What colour is this object?
R: It is red./Do you mean this one?
![Page 12: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/12.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
H: The elongated object is red.
R: OK, red.
H: It is not blue.
R: OK.
![Page 13: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/13.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
Detect and attend the objects.
![Page 14: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/14.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
R: I see that this object is green.
I will update my models.
![Page 15: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/15.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
R: Is this object green?
H: Yes it is.
R: OK, green.
![Page 16: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/16.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
Looking around to see if there are any
new objects.
![Page 17: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/17.jpg)
Behaviours
Prioritization of motive drives
Interaction drive
Answering tutor's requests Situated tutor-driven learning
Extrospection drive
Attention mechanism Situated autonomous learning Situated tutor-assisted learning Exploring the scene
Introspection drive
Non-situated tutor-assisted learning
Priority
R: Can you show me a white object?
H: This object is white.
R: OK.
![Page 18: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/18.jpg)
Video
http://cogx.eu/results/george
![Page 19: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/19.jpg)
System diagram
![Page 20: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/20.jpg)
Individual competencies
Plane-pop-out
object detection
and tracking
odKDE-based
learning of
object properties
Incremental
view-based learning
of object models
MLN-based
reference
resolution Motive
management
Continual
planning
Robot arm
manipulation
Belief-based
information
fusion
Abduction-based
situated language
processing
CAST-based
integration
![Page 21: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/21.jpg)
Conclusion
T-60 T-30 T T+30
![Page 22: A system for interactive learning in dialogue with a tutor€¦ · Pedagoška konferenca FRI Ljubljana, 6 June 2013 Danijel Skočaj University of Ljubljana . Motivation industrial](https://reader034.fdocuments.us/reader034/viewer/2022051915/6006ea91b39b064472089538/html5/thumbnails/22.jpg)