3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha...
-
Upload
benjamin-swindells -
Category
Documents
-
view
218 -
download
2
Transcript of 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha...
![Page 1: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/1.jpg)
3DVCR Group, Department of Machine Intelligence
Computing Object-based Saliency
in Urban Scenes Using Laser Sensing*Yipu Zhao, M. He, H. Zhao, F. Davoine, and
H. Zha
Department of EECS, Peking UniversitySino-French Lab, CNRS & LIAMA
![Page 2: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/2.jpg)
3DVCR Group, Department of Machine Intelligence
Motivation Object discovery from mobile laser scanning.
![Page 3: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/3.jpg)
3DVCR Group, Department of Machine Intelligence
Background
Different applications may concern different objects. Put more focus on the objects of interest.
![Page 4: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/4.jpg)
3DVCR Group, Department of Machine Intelligence
Objective: Compute the object-based saliency of laser points
Computing Object-based
Saliency
This Research
Laser Points Object Detection
Geometric Feature
Extraction
Geometric Feature
Extraction
Object Candidate Generation
Object Candidate Generation
Object-based Saliency
Computing
Object-based Saliency
Computing
Step1 Step2 Step3
![Page 5: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/5.jpg)
3DVCR Group, Department of Machine Intelligence
Experimental Platform
LMS
GPS IMU
LMS
![Page 6: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/6.jpg)
3DVCR Group, Department of Machine Intelligence
Four types of geometric featuresVertical line Horizontal lineVertical plane Horizontal
plane
Seed Selection
Region Growing
Range Image
Geometric
Features
Step 1. Geometric Feature Extraction
![Page 7: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/7.jpg)
3DVCR Group, Department of Machine Intelligence
Extraction results
Step 1. Geometric Feature Extraction
Vertical Line
Vertical Plane Horizontal Plane
Horizontal Line
![Page 8: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/8.jpg)
3DVCR Group, Department of Machine Intelligence
Objects Combination of geometric features Car several surface planes Road lamp a long pole Traffic sign a board with a supporting stick
Finding combination of geometric features
Step 2. Object Candidate Generation
Voting Candidate Centers
Clustering Centers
Object Candidate
s
Geometric Features
![Page 9: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/9.jpg)
3DVCR Group, Department of Machine Intelligence
Step 2. Object Candidate Generation
Voting car candidate
![Page 10: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/10.jpg)
3DVCR Group, Department of Machine Intelligence
The object-based saliency depends on Type & size of the related geometric features Spatial relationship between geometric features
To contain these information A graphical object representation is
introduced
Step 3. Object-based Saliency Computing
Graph Generati
on
Graph Matching
Salient Object
s
Object Candidat
es
![Page 11: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/11.jpg)
3DVCR Group, Department of Machine Intelligence
Graph definition: Node: Type & size of geometric features Edge: Spatial relationship of different
geometric features
3.1 Graph Generation
i
j
x
z
y
Object coordinate
�⃗� ′ 𝑗 ′�⃗� ′ 𝑖 ′ & i
j
![Page 12: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/12.jpg)
3DVCR Group, Department of Machine Intelligence
Some model graphs of objects of interest
3.1 Graph Generation
Car Bus Road lamp Traffic light Traffic sign
![Page 13: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/13.jpg)
3DVCR Group, Department of Machine Intelligence
Evaluate matching score between a previously trained model graph and a data graph
Step 1. Inexact graph matching Only concern edge attributes Generate 2 sub-graphs &
Step 2. Score evaluating
3.2 Graph Matching
)
where denotes for the th node in node set , and is the area of node 's corresponding geometric feature
![Page 14: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/14.jpg)
3DVCR Group, Department of Machine Intelligence
1. Highway scene (the 4th ring road, Beijing) Collecting time cost: 35 minutes Data volume: about 14,300,000 laser points Sample: 26 model graphs for 8 object classes Processing time: 18 minutes (on a 2.8GHz & 8G PC)
Experiment
![Page 15: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/15.jpg)
3DVCR Group, Department of Machine Intelligence
Result in Highway Scene
Road lamp
Traffic light
Traffic sign
Road belt
Car
Signpost
![Page 16: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/16.jpg)
3DVCR Group, Department of Machine Intelligence
Result in Highway Scene
Bus
Road lamp
Traffic light
Traffic sign
Road belt
Building
![Page 17: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/17.jpg)
3DVCR Group, Department of Machine Intelligence
2. Street scene (Street ShangDi, Beijing) Collecting time cost: 30 minutes Data volume: about 13,210,000 laser points Sample: 38 model graphs for 11 object classes Processing time: 20 minutes (on a 2.8GHz & 8G PC)
Experiment
![Page 18: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/18.jpg)
3DVCR Group, Department of Machine Intelligence
Result in Street Scene
Truck
SignpostCar
Building
Road lamp Ad board
Result in street scene
Data Graph Model Graph
![Page 19: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/19.jpg)
3DVCR Group, Department of Machine Intelligence
Result in Street Scene
Traffic signBus
Building
Data Graph Model Graph
![Page 20: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/20.jpg)
3DVCR Group, Department of Machine Intelligence
Some Errors
Road lamp
Traffic sign
Road belt
Bus Signpost
Car
Trash box Building
![Page 21: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/21.jpg)
3DVCR Group, Department of Machine Intelligence
Statistical Result
*Highway scene only
![Page 22: 3DVCR Group, Department of Machine Intelligence *Yipu Zhao, M. He, H. Zhao, F. Davoine, and H. Zha Department of EECS, Peking University Sino-French Lab,](https://reader031.fdocuments.us/reader031/viewer/2022020306/5517fddf550346c6568b50a9/html5/thumbnails/22.jpg)
3DVCR Group, Department of Machine Intelligence
Compute the object-based saliency of urban laser sensing data Highlight the data of objects of interest Help object detection in the subsequent
procedures
In the future On-line application More comprehensive approach (include context
information)
Summary