Features-Based Affordance Detection of Tool Parts

Introduction Method Results

References

Student: Raghad Al-abboodi Supervisor: Walterio Mayol-Cuevas Department of Computer Science, University of Bristol

Outlook!

Conclusions In this work, we present an affordance detection framework for a 3D point cloud, using 3D robust histogram features which characterize the point local geometry. Model-scene matching is computed using Euclidean distance and the shortest returned distance between the feature vectors which represent the most relevant features and accordingly the enquire affordance. The features-based classification gave promising results during testing.

Robots are increasingly being used to perform daily tasks usually performed by humans, as well as task requiring human-robot collaboration. As such it is important for them to be able to detect and interact with various human tools and objects. Gibson [1] refers to affordance as “properties of an object. that determine what actions a human can perform on them.” In this sense, man-made tools usually consist of many affordances (multi-affordance) such as contain, grasp or cut. If the robot has the ability to detect these affordances, then it would be possible to interact with objects even if it was seeing them for the first time. Moreover, learning the functional part of the tool (ex: mug-handle, knife-blade) rather than the tool itself helps generalize to define novel set of tools tht have the same functional part. For example, when the robot learns the functional part of the knife(the blade) use for cut, then variety of tools which have sharp edge can be used. In this work, we address the problem of learning affordances for part of the object based of the local features.

[1] J. J. Gibson. The theory of affordance. In Perceiving, Acting, and Knowing. Lawrence Erlbaum Associates, Hillsdale, NJ, 1977. [2] R. B. Rusu, N. Blodow, and M. Beetz. Fast Point Feature Histograms (FPFH) for 3D Registration. In In Proceedings of the International Conference on Robotics and Automation (ICRA), 2009.

The proposed approach show promising result after determining the optimal parameters to use during calculation. The figures bellow shows comparisons of histogram features for different object, illustrating similarities and differences.

Training Point cloud extraction: a relative 3d point cloud acquired from RGB-D kinect. Normal estimation: Use approximation to infer the surface normals from the point cloud dataset directly. Choosing the right scale: Since calculating the normal value using kd-tree needs to use a right scale for the size of the neighbor points, therefore; a series of cross validation are performed as shown an example in Table 1. Table 1: Cross validation result for the jar with different normal radius. The left column means the training data, and the top row means the testing radius. The values represent the number of testing data that were wrongly classified. The lowest error average are the best to use.

Keypoints computing: Dawn sample the point cloud regulate the point density of the resulting file. Histogram Feature computing: set of 3D local point features (Fast Point Feature Histograms) [2] are used to represent the local descriptor. For the training, the result features are manually labeled with the ground truth model.

Testing

Model/Radius

0.005 cm

0.004 cm

0.006 cm

0.05 cm 0.2 cm

Jar1 0 4 0 0 0

Jar2 0 0 1 2 5

Jar3 0 3 0 0 2

Jar4 3 0 4 5 5

Jar5 0 0 0 0 1

Jar6 0 0 0 0 1

Avg. 0.5 1.16667 0.8333 1.1667 2.3333

Mean histograms for two different object(Jar, Can) share the same affordance (grasp)

Two different objects(Jar, Can) with different affordances (grasp, place).

The mean histograms for various objects. Objects with the same affordance have similar histograms, while those of

different affordance do not.

New set of point cloud

Match found, return

affordance type

FPFH

Keypoint Extraction

Match not found

Euclidean distance

Grasp Detection

Place Detection

Multi-affordance tool (cut, grasp)

Multi-affordance tool (Pound, grasp)