Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong...
Transcript of Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong...
![Page 1: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/1.jpg)
Neural Architectures for Lifelong Learning on Humanoid Robots
Vadym Gryshchuk
19.11.2018
![Page 2: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/2.jpg)
Outline
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
2Neural Architectures for Lifelong Learning on Humanoid Robots
![Page 3: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/3.jpg)
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
3Neural Architectures for Lifelong Learning on Humanoid Robots
![Page 4: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/4.jpg)
What is Lifelong Learning?
• Continual acquisition of knowledge
• Fine-tuning of knowledge
• Learning from experiences
• Retaining of previously learnt experiences
Neural Architectures for Lifelong Learning on Humanoid Robots 4
Figure 1.1: NICO – Neuro-Inspired COmpanion (Source: Kerzel et al. [2]).
![Page 5: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/5.jpg)
Catastrophic Forgetting
• Interference of learnt representations with new information
Neural Architectures for Lifelong Learning on Humanoid Robots 5
Representation 1 Representation 2
![Page 6: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/6.jpg)
Inspiration from Biological Systems
• Neurosynaptic plasticity
• Hippocampus and cerebral cortex
• Transfer learning
• Intrinsic motivation
• Crossmodal learning
• Incremental learning
Neural Architectures for Lifelong Learning on Humanoid Robots 6
![Page 7: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/7.jpg)
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
7Neural Architectures for Lifelong Learning on Humanoid Robots
![Page 8: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/8.jpg)
Neural Networks
Neural Architectures for Lifelong Learning on Humanoid Robots 8
Figure 2.1: Neural network representation (Source: McDonald [3]).
![Page 9: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/9.jpg)
Convolutional Neural Networks (CNNs)
Neural Architectures for Lifelong Learning on Humanoid Robots 9
Figure 2.2: Convolutional neural network (Source: Cavaioni [1])
![Page 10: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/10.jpg)
Self-Organizing Networks
• Self-Organizing Map (SOM)
• Grow When Required Network (GWR Network)
• Recurrent GWR
Neural Architectures for Lifelong Learning on Humanoid Robots 10
![Page 11: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/11.jpg)
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
11Neural Architectures for Lifelong Learning on Humanoid Robots
![Page 12: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/12.jpg)
Object Recognition: CNN + Classifier
• Learning from video sequences
• Visual transformations of objects
• Changing environment
Neural Architectures for Lifelong Learning on Humanoid Robots 12
Figure 3.1: iCub (Source: Pasquale et al. [6]).
![Page 13: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/13.jpg)
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 13
CNN
Classifier
app
le
cup
bal
l
tom
ato
![Page 14: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/14.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 14
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 15: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/15.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 15
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 16: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/16.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 16
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 17: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/17.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 17
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 18: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/18.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 18
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 19: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/19.jpg)
iCub: Object Learning
Neural Architectures for Lifelong Learning on Humanoid Robots 19
Source: https://www.youtube.com/watch?v=ghUFweqm7W8
![Page 20: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/20.jpg)
Sensorimotor Learning: Self-Organization
• Latency in sensorimotor systems
• Predictive mechanisms for future motor states
• Online learning
Neural Architectures for Lifelong Learning on Humanoid Robots 20
Source: https://upload.wikimedia.org/wikipedia/commons/4/47/Nao_Robot_%28Robocup_2016%29.jpg
![Page 21: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/21.jpg)
Sensorimotor Learning: Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 21
Figure 3.2: The imitation scenario (Source: Mici et al. [4]).
![Page 22: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/22.jpg)
Sensorimotor Learning: Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 22
Figure 3.3: Visuomotor learning (Source: Mici et al. [4]).
![Page 23: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/23.jpg)
Object Recognition: CNN + Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 23
RGB sequence
Pre-trained CNN
Pre-trained CNN
Depth sequence
Feat
ure
s
Self-organizing network
Label
Figure 3.4: Recognition pipeline (Adapted from Part et al. [5]).
![Page 24: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/24.jpg)
• Motivation
• Background
• Approaches
• Results
• Discussion
• Conclusion
24Neural Architectures for Lifelong Learning on Humanoid Robots
![Page 25: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/25.jpg)
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 25
Figure 4.1: Classification accuracy of the model, which was trained on an incremental number of objects (Source: Pasquale et al. [6]).
![Page 26: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/26.jpg)
Object Recognition: CNN + Classifier
Neural Architectures for Lifelong Learning on Humanoid Robots 26
Figure 4.2: Classification accuracy of the model trained incrementally on different days (Source: Pasquale et al. [6]).
![Page 27: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/27.jpg)
Sensorimotor Learning: Self-Organizing Architecture
Neural Architectures for Lifelong Learning on Humanoid Robots 27
Figure 4.3: Behaviour of the architecture (Source: Mici et al. [4]).
![Page 28: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/28.jpg)
Object Recognition: CNN + Self-Organization
Neural Architectures for Lifelong Learning on Humanoid Robots 28
Figure 4.4: Recognition pipeline (Source: Part et al. [5]).
![Page 29: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/29.jpg)
Discussion
• CNN + Classifier architecture for object recognition:• Features extracted from a CNN are dependent on a dataset the model was
trained on
• Old representations are overwritten by the new information
• Self-organizing architecture for sensorimotor learning:• Incremental online learning and prediction
• Unreliability of visual body tracking framework in complex body positions
• CNN + self-organization for object recognition:• Self-organizing network grows when required
• Temporal context is not considered
Neural Architectures for Lifelong Learning on Humanoid Robots 29
![Page 30: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/30.jpg)
Conclusion
• Lifelong learning is crucial for intelligent robots
• Biological systems provide a basis for the incremental learning
• Self-organizing networks preserve the topology
• CNNs learn efficient feature descriptors
• Catastrophic forgetting increases during incremental tasks
Neural Architectures for Lifelong Learning on Humanoid Robots 30
![Page 31: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/31.jpg)
Thank You!Questions?
Neural Architectures for Lifelong Learning on Humanoid Robots 31
![Page 32: Neural Architectures for Lifelong Learning on Humanoid Robots · Neural Architectures for Lifelong Learning on Humanoid Robots 23 RGB sequence Pre-trained CNN Pre-trained CNN Depth](https://reader034.fdocuments.us/reader034/viewer/2022051812/602b7cc1d7ec913a046cd15c/html5/thumbnails/32.jpg)
References• [1] Cavaioni, M. Deep Learning series: Convolutional Neural Networks. https://medium.com/machine- learning-
bites/deeplearning-series-convolutional-neural-networks-a9c2f2ee1524 . [Online; accessed 13-November-2018].
• [2] Kerzel, M., Strahl, E., Magg, S., Navarro-Guerrero, N., Heinrich, S., Wermter, S.NICO - neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction. In26th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN 2017, Lisbon, Portugal, August 28 - Sept. 1, 2017, pages 113–120, 2017.
• [3] McDonald, C. Machine learning fundamentals (II): Neural networks. https://towardsdatascience.com/machine-learning-fundamentals-ii-neural-networks-f1e7b2cb3eef. [Online; accessed 13-November-2018].
• [4] Mici, L., Parisi, I. G., Wermter, S. An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction. CoRR, abs/1712.08521, 2017.
• [5] Part l. J., Lemon, O. Incremental online learning of objects for robots operating in real environments. In Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2017, Lisbon, Portugal, September 18-21, 2017, pages 304–310, 2017.
• [6] Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., Natale, L. Real-world object recognition with off-the-shelf deep conv nets: How many objects can iCub learn? CoRR, abs/1504.03154, 2015.
Neural Architectures for Lifelong Learning on Humanoid Robots 32