PCL :: Features: Hands-onSuat Gedikli

November 6, 2011

Hands-on

In this part of the session, we’ll have a look to an exampleapplication that uses most of the methods described so far.

I Reading PCD files and displaying point cloudsI Plane detection and segmenationI Euclidean clusteringI Keypoint detection using various methodsI Feature extraction using various methodsI Feature matchingI Feature rejectionI Initial Alignment (manual)I Registration using ICPI Surface Reconstruction (Triangulation)

Point Cloud Library (PCL)

Accessing the Code

Code is be available online here:http://pointclouds.org/media/iccv2011.htmlchange to the folder where you extracted the source code andrun:> make release

the binaries should be in the folder ./build/Release We willconcentrate on the sourcode tutorial.cpp. Other useful tutorialcode is also available in the src folder and also will be installedinto the ./build/Release folder.

Point Cloud Library (PCL)

Running the Example

To run the application just type in> ./tutorial <source-pcd-file> <target-pcd-file> <

keypoint-detection-method> <feature-descriptor> <surface-

reconstruction-method>

the available keypoint detectors are listed if you run theexample without arguments> ./tutorial

To use the SHOT descriptor with the SIFT keypoint detectorand Marching Cubes as the desired surface reconstructionmethod, type:> ./tutorial 1 2 2

Point Cloud Library (PCL)

The Example Framework

tutorial.cpp:

The example framework is implemented in the template-class:template<typename FeatureType> class ICCVTutorial

which is templated on the feature-descriptor type used forcorrespondence finding.

Point Cloud Library (PCL)

The Example Framework

tutorial.cpp:

The constructorICCVTutorial (pcl::Keypoint<pcl::PointXYZRGB, pcl::PointXYZI

>::Ptr, typename pcl::Feature<pcl::PointXYZRGB, FeatureType

>::Ptr, typename pcl::PointCloud<pcl::PointXYZRGB>::ConstPtr

, typename pcl::PointCloud<pcl::PointXYZRGB>::ConstPtr);

Point Cloud Library (PCL)

The Example Framework

tutorial.cpp:

The actual work is done in following methods:void segmentation (input, segmented)const;

void detectKeypoints (input, keypoints)const;

void extractDescriptors (input, keypoints, features);void findCorrespondences (source, target, correspondences)

const;void filterCorrespondences ();

void determineInitialTransformation ();

void determineFinalTransformation ();

void reconstructSurface ();

Point Cloud Library (PCL)

Keypoint Detectors

tutorial.cpp:

Using different keypoint detectors: main ()Sift:pcl::SIFTKeypoint<...>* sift3D = new pcl::SIFTKeypoint<...>;

sift3D->setScales(0.01, 3, 2);

sift3D->setMinimumContrast(0.0);

Point Cloud Library (PCL)

Keypoint Detectors

tutorial.cpp:

Using different keypoint detectors: main ()Harris-like:pcl::HarrisKeypoint3D<...>* harris3D = new pcl::

HarrisKeypoint3D<...>;harris3D->setMethod(...);

Point Cloud Library (PCL)

Feature Descriptors

tutorial.cpp:

Using different feature descriptors: main ()Since CorrespondenceViewer is templated on featuredescriptor type:

Point Cloud Library (PCL)

Feature Descriptors

tutorial.cpp:

Using different feature descriptors: main ()Since CorrespondenceViewer is templated on featuredescriptor type:CorrespondenceViewer<pcl::FPFHSignature33> ... ;

CorrespondenceViewer<pcl::SHOT> ... ;

CorrespondenceViewer<pcl::PFHSignature125> ... ;

CorrespondenceViewer<pcl::PFHRGBSignature250> ... ;

Point Cloud Library (PCL)

Plane Detection

tutorial.cpp: segmentation ()

pcl::ModelCoefficients::Ptr coefficients (new pcl::

ModelCoefficients);pcl::PointIndices::Ptr inliers (new pcl::PointIndices);

pcl::SACSegmentation<pcl::PointXYZRGB> seg;

seg.setOptimizeCoefficients (true);

seg.setModelType (pcl::SACMODEL_PLANE);

seg.setMethodType (pcl::SAC_RANSAC);

seg.setDistanceThreshold (0.02);

seg.setInputCloud (source);

seg.segment (*inliers, *coefficients);

Point Cloud Library (PCL)

Segmentation

tutorial.cpp: segmentation ()

pcl::ExtractIndices<pcl::PointXYZRGB> extract;

extract.setInputCloud (source);

extract.setIndices (inliers);

extract.setNegative (true);

extract.filter (*segmented);

Point Cloud Library (PCL)

Segmentation

tutorial.cpp: segmentation ()

pcl::ExtractIndices<pcl::PointXYZRGB> extract;

extract.setInputCloud (source);

extract.setIndices (inliers);

extract.setNegative (true);

extract.filter (*segmented);

Point Cloud Library (PCL)

Segmentation Result

Point Cloud Library (PCL)

Segmentation Result

Point Cloud Library (PCL)

Clustering

tutorial.cpp: segmentation ()

std::vector<pcl::PointIndices> cluster_indices;

pcl::EuclideanClusterExtraction<pcl::PointXYZRGB> clustering

clustering.setClusterTolerance (0.02);

clustering.setMinClusterSize (1000);

clustering.setMaxClusterSize (250000);

clustering.setSearchMethod (tree);

clustering.setInputCloud(segmented);

clustering.extract (cluster_indices);

...

Point Cloud Library (PCL)

Clustering Result

Point Cloud Library (PCL)

Clustering Result

Point Cloud Library (PCL)

Keypoint Detection

tutorial.cpp: detectKeypoints ()

keypoint_detector_->setInputCloud(input);

keypoint_detector_->compute(*keypoints);

Point Cloud Library (PCL)

Keypoint Result

Point Cloud Library (PCL)

Keypoint Result

Point Cloud Library (PCL)

Keypoint Result

Point Cloud Library (PCL)

Feature Extraction

tutorial.cpp: extractDescriptors ()

feature_extractor_->setSearchSurface(input);

feature_extractor_->setInputCloud(kpts);

feature_extractor_->compute (*features);

calculate and set normals is feature_extractor is derived fromFeatureFromNormalsfeature_from_normals->setInputNormals(normals);

Point Cloud Library (PCL)

Feature Matching

tutorial.cpp: findCorrespondences ()

pcl::KdTreeFLANN<FeatureType> matcher;

matcher.setInputCloud (target);

std::vector<int> indices (1);

std::vector<float> distances (1);

for (size_t i = 0; i < source->size (); ++i){

matcher.nearestKSearch (*source, i, 1, indices, distances);

correspondences[i] = indices[0];

}

Point Cloud Library (PCL)

Feature Matching Result

source to target correspondences:

Point Cloud Library (PCL)

Feature Matching Result

target to source correspondences:

Point Cloud Library (PCL)

Correspondence Filtering

tutorial.cpp: filterCorrespondences ()

for (cIdx = 0; cIdx < source2target.size (); ++cIdx){

if (target2source[source2target[cIdx]] == cIdx)

correspondences.push_back( <cIdx, source2target[cIdx]>);

}

CorrespondenceRejectorSampleConsensus rejector

rejector.setInputCloud(source_keypoints_);

rejector.setTargetCloud(target_keypoints_);

rejector.setInputCorrespondences(correspondences_);

rejector.getCorrespondences(*correspondences_);

Point Cloud Library (PCL)

Correspondence Filtering Result

Point Cloud Library (PCL)

Correspondence Filtering Result

Point Cloud Library (PCL)

Correspondence Filtering Result

Point Cloud Library (PCL)

Initial Alignment

tutorial.cpp: determineInitialTransformation ()

TransformationEstimation<...>::Ptr te (new

TransformationEstimationSVD<...>);te->estimateRigidTransformation (source, target, corrs,

trans);transformPointCloud(source, transformed_, trans);

Point Cloud Library (PCL)

Registration via ICP

tutorial.cpp: determineFinalTransformation ()

Registration<...>::Ptr registration (new

IterativeClosestPoint<...>);registration->setInputCloud(source);

registration->setInputTarget (target);

registration->setMaxCorrespondenceDistance(0.05);

registration->setRANSACOutlierRejectionThreshold (0.05);

registration->setTransformationEpsilon (0.000001);

registration->setMaximumIterations (1000);

registration->align(*registered);

transformation = registration->getFinalTransformation();

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

merging registered point clouds to one cloudPointCloud<...>::Ptr merged (new PointCloud<...>);

*merged = *source_transformed_;

*merged += *target_segmented_;

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

subsampling point cloud, so we have a equal densityVoxelGrid<PointXYZRGB> voxel_grid;

voxel_grid.setInputCloud(merged);

set voxel size to 2 mmvoxel_grid.setLeafSize(0.002, 0.002, 0.002);

voxel_grid.setDownsampleAllData(true);

voxel_grid.filter(*merged);

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

assemble pointcloud with normalsPointCloud<PointXYZRGBNormal>::Ptr vertices (new PointCloud

<...>);pcl::copyPointCloud(*merged, *vertices);

estimate normals using 1 cm radiusNormalEstimation<PointXYZRGB, PointXYZRGBNormal> ne;

ne.setRadiusSearch (0.01);

ne.setInputCloud (merged);

ne.compute (*vertices);

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

initialize search method for surface reconstructionKdTree<PointXYZRGBNormal>::Ptr tree (new KdTree<...>);

tree->setInputCloud (vertices);

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

reconstruct surfacesr->setSearchMethod(tree);

sr->setInputCloud(vertices);

sr->reconstruct(surface_);

Point Cloud Library (PCL)

Surface Reconstruction

tutorial.cpp: reconstructSurface ()

result for the GreedyProjectionTriangulation method

Point Cloud Library (PCL)