Using multiple optical images 3D surface reconstrunction
Transcript of Using multiple optical images 3D surface reconstrunction
![Page 1: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/1.jpg)
Using multiple optical images3D surface reconstrunction
20205309 Chungsu Jang
2020 12.10
![Page 2: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/2.jpg)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
![Page 3: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/3.jpg)
DSM(Digital Surface Model)
![Page 4: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/4.jpg)
Korea Daejeon, WorldView3 Satellite
Test area 1
Test area 2
Analysis Data characteristics3D dense points obtained using satellite imagery and SGM(Semi Global Mapping) matching technique
Test area 1
Test area 2
![Page 5: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/5.jpg)
Area 1(Daejeon) : Disparity Maps
Area 2(Daejeon) : Disparity Maps
Refinementblock_size = 15
➢ All Census & MI data : noise vertical range ~ 2 m, Many miss-matching points
Refinement
Problem Derivation through Analysis Data characteristics
![Page 6: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/6.jpg)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
![Page 7: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/7.jpg)
Related Works
●Urban 3D reconstruction– Urban Semantic 3D reconstruction from Multiveiw Satellite Imagery, CVP
R 2019
![Page 8: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/8.jpg)
Related Works
• Geo referencing –mapping image pixels to global coordinates, photometry
GtoI()
ItoG()
Ref: Introduction to Photogrammetry and Remote sensing, 1st Edition
R : RPC(The rational polynomial coefficient)
![Page 9: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/9.jpg)
Related Works
• Semi Global Matching(SGM)– Algorithm and architecture of disparity estimation with mini-Census
adaptive support weight, IEEE Transactions on Circuits, 2006
![Page 10: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/10.jpg)
Related Works
• Point cloud set matching– ICP(Iterative Closest Point)
• A Method for Registration of 3-D Shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1992
–Least-squares Height Difference(LHD) Matching• Three-dimensional absolute orientation of stereo models using digital elevation models,
PHOTOGRAMMETRIC ENGINEERING AND REMOTE SENSING, 1988
![Page 11: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/11.jpg)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
![Page 12: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/12.jpg)
For 3D building shape restoration , approach
![Page 13: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/13.jpg)
High-rise Building Reconstructionusing Warped Images
No way to recover these missing buildings?
![Page 14: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/14.jpg)
Projected points from image # 1
Alignment and color information in the building
Left P.C.
Left image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
Right P.C.
Right image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
Projected points from image # 2
Warped Image
When knowing information about the floor plan of the building
Alignment and color information in the building
Projected points from image # 1
Left P.C.
Left image
1 4
Building
Object space
Plane
segmented
from LiDAR
2
3
![Page 15: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/15.jpg)
Two warped image comparison /
Inside and outside the building Similarity
Warped Image
fl frelbl dl erbr dr
LiDAR plane
PCL PCR
A
B
C
D E
Ground
Rooftop
Wall
Hig
h s
imila
rity
Lo
w s
imila
rity
Hig
h s
imila
rity
F
Hig
h s
imila
rity
Left warped image
Right warped image
g(bl) g(dl) g(el) g(fl)
g(br) g(dr) g(er) g(fr)
![Page 16: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/16.jpg)
Left satellite image
Right satellite image
Left warped image
Building rooftop plane
Right warped image
➢ Similarity measure between two warped images and building height estimation
High-rise Building Reconstructionusing Warped Images
![Page 17: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/17.jpg)
Similarity Check: Hamming distance
![Page 18: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/18.jpg)
19
Proposed Workflow – 1st track approach
Two warped image generation
Panchromatic Image (Right)
Min < Height< Max
Similarity Calculation (Hamming Dist.)
Best Height Estimation
Panchromatic Image (Left)
DBM generation
Yes
No
Check the min. costRecord costs
for each height
![Page 19: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/19.jpg)
Best Views
Three-dimensional geometric condition
Convergence Angle : 25~45
Bisector Elevation Angle : > 60
Satellite Azimuth Angle : <70
Satellite Azimuth Angle : <15
![Page 20: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/20.jpg)
Similarity Measure Map
Left warped image
Right warped image
High similarity
Low similarity
Similarity Check
➢ High similarity measure at the correct height
➢ Low similarity measure at the incorrect height
![Page 21: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/21.jpg)
H : 81m H : 107mH : 99m
Test results
Best estimated Building Height : 99m at the min. cost
![Page 22: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/22.jpg)
※ Reference height is extracted by intersection after manually measuring the same feature from left and right satellite images
Test results
Reference Height (m)
Estimated Height(m)
diff (m)
‘ㄷ’type building 72.61 72.50 0.11
‘ㅁ’type building 99.29 99.00 0.29
Apartment 100.75 101.00 0.25
High building 233.45 233.50 0.05
Low builidng– 1 70.47 70.00 0.47
Low building– 2 70.71 71.00 0.29
![Page 23: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/23.jpg)
24
Using Warped image Outline and plane extraction method
Warped image (Right)
Line matching
Warped image (Left)
Best height
Line extraction (Left) Line extraction (Right)
Line merge (Left) Line merge (Right)
Line intersection
Space partitioning
Cumulative min cost image generation
Space Selection
Space Union
DBM
Cost check for each partition
![Page 24: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/24.jpg)
Linear Information Extraction and Merge
Line detection Line merge
Line detection Line merge
Left warped image at the Best height
Right warped imageat the Best height
Best height estimation using similarity
measure
![Page 25: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/25.jpg)
Matching, Intersection, Space partitioning and Building area selection
Line matching Line intersection Space Partitioning
Space SelectionSpace Union DBM
![Page 26: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/26.jpg)
Test results (two satellite images, Redundancy)
ㄷtype
ㅁtype
Hig
h
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
![Page 27: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/27.jpg)
Apartm
ent
Low
-
1
Low
-
2
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Space partitioning Space selection Space unionImage
Test results (two satellite images, Redundancy)
![Page 28: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/28.jpg)
DBM
ㄷtype
ㅁtype
Hig
h
DBM
DBM
Apartm
ent
Low
-1Low
-
2
DBM
DBM
DBM
Test results (two satellite images, Redundancy)
![Page 29: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/29.jpg)
Test results (Improved DSM Generation)
DSM from point cloud
Build
ing 1
Build
ing 2
Build
ing 3
DSM from point cloud
DSM from point cloud
Improved DSM
Improved DSM
Improved DSM
![Page 30: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/30.jpg)
Test results (Improved DSM Generation)
Build
ing 4
Build
ing 5
DSM from point cloud Improved DSM
DSM from point cloud Improved DSM
![Page 31: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/31.jpg)
Sensor Fusion
IKONOS-2
Test area
7340 7340
7341 7341
Epipolar Image SGM Disparity Map
WV-3
![Page 32: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/32.jpg)
Sensor Fusion
7340 7341
DEM
IKONOS-2 WV-3
![Page 33: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/33.jpg)
Sensor Fusion
Point Cloud Matching : Use 12,322 points
RMSE = 23.4m; Max = 89.1m
LHD Matching (reference)
![Page 34: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/34.jpg)
Sensor Fusion
3D matching
(7340+7341)
Point Cloud
Matching
Mosaic
(7340+7341)
![Page 35: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/35.jpg)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
![Page 36: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/36.jpg)
Conclusion
➢ Suggest to generate 3d surface model from several sat
ellite image
➢ Show you the generated 3d model surface, visually
➢ Limit : Completeness? / Accuracy?
![Page 37: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/37.jpg)
●Introduction
●Related Works
•Approach & Experiment
●Conclusion
●Q&A
Contents
![Page 38: Using multiple optical images 3D surface reconstrunction](https://reader033.fdocuments.us/reader033/viewer/2022052013/62860dbee5789e26260aefeb/html5/thumbnails/38.jpg)
Thank youQ & A