Cognitive Robotics - Object Grasping · NIRYO ONE ROS STACK overview. ROBOT MANAGER NODE...
Transcript of Cognitive Robotics - Object Grasping · NIRYO ONE ROS STACK overview. ROBOT MANAGER NODE...
COGNITIVE ROBOTICS PROJECT
OBJECT GRASPING
Project requirements
2
Intel® RealSense™ Depth Camera D435 NVIDIA® Jetson Nano™
Niryo One Robot Operating System
CameraThe only camera used is fixed on the end effector of the arm, making the environment more compact.
BoxesOnce the program has started, the boxes cannot be moved anymore.
BallsYou can add, remove and move the balls between each operating cycle.
3
Constraints
4
Software design
TRAJECTORY LISTENER NODEMoving Niryo One.
5NIRYO ONE ROS STACK overview
ROBOT MANAGER NODEChoose how to move.
6ROBOT MANAGER command list
VISION NODELook around.
7RGB VIDEO and POINT CLOUD acquisition
Intel® RealSense™ Stereo Depth TechnologyIn a similar way to how human vision works, stereo depth is used to provide the distance of a precise point in the visual field.
VISION NODELook around.
8DEEP NEURAL NETWORK recognition
MobileNet SSD v2 (COCO)Pre-trained to detect the location of 90 types objects from COCO dataset.Input size: 300x300Fine tuning for 8-classes target.
VISION NODELook around.
9
0,98
0,63 0,78 0,92
0,7 0,81
0,28 0,
650,82
0,71
0,41 0,
76
RED BLUE GREEN WHITE
BALL RECOGNITIONPrecision Recall F1 Score
1 0,96
0,97
0,92
0,63 0,82
0,84
0,410,
77 0,88
0,9
0,57
RED BLUE GREEN WHITE
BOXES RECOGNITIONPrecision Recall F1 Score
0,89
5
0,64
25 0,72
75
MEAN VALUESPrecision Recall F1 Score
VISION NODELook around.
10DEEP NEURAL NETWORK confusion matrix
ENGINE NODEAchieve your target.
11ENGINE FLOWCHART
12
VISUAL VERBOSE NODELet me show youwhat I’m doing.
PLANNED IMPROVEMENTS
› Managing an even more dynamic workspace› Improve the recognition of targets› Speed up the use of Niryo One Python API
… and surely contemplate a new and nicer hardware structure.
13
THANKS!OBJECT GRASPING for Cognitive Robotics
Allegretti GiovanniDi Prisco GiovanniScaldaferri AntonelloSchettini Marco
14