Simulation Symmetric N-dimensional Cube Network-on-Chip Architecture by Using Ns-2
Symmetric Architecture Modeling with a Single Image
description
Transcript of Symmetric Architecture Modeling with a Single Image
Symmetric Architecture Modeling with a Single Image
Author: Nianjuan Jiang, Ping Tan, Loong-Fah CheongDepartment of Electrical & Computer Engineering, National
University of Singapore
Presenter: Feilong Yan
MotivationModel architecture from single image is common task in 3D creation due to the lack of the more images.
Historic Photo:
Internet Photo:
MotivationSingle image based modeling is very difficult!Due to the trouble on camera calibration and texture loss
The recent methods only can handle simple and planar façade
Pascal Mulle et al. Image-based procedural modeling of facades
Changchang Wu et al. Repetition-based Dense Single-View Reconstruction
Motivation
But what about this one?And If we only have this single photo.
Complex and not planar
Motivation
Fortunately , the symmetry is very prevalent in the architecture
Complex and not planar
Symmetry is a breakthrough which magically can generate more images from the input
This is reasonable, but exciting to me
Main IdeaBilateral Symmetry
Main IdeaRotational Symmetry
Main Idea2 even more views
Reconstruction
Modeling Pipeline
Input Image and Frustum Vertices
Calibration and 3D Reconstruction
Model Initialization
Texture Enhancement
Model Refinement
3D Reconstruction Surface Modeling
3D ReconstructionCamera Calibration 3D points
Reconstruction
Camera CalibrationPrevious Methods
Calibrate the camera from the vanishing points of 3 mutually orthogonal directions in a single image.
But many photos do not have 3 vanishing points, and this method is often numerical unstable
HARTLEY, R., AND ZISSERMAN Multiple View Geometry in Computer Vision
Camera CalibrationPrevious Methods
Parallelipiped is used to represent a building block. Under the constraint of parallelipiped, the visible 6 spatial vertices may be estimated.
WILCZKOWIAK, M. et. al Using geometric constraints through parallelepipeds for calibration and 3d modeling
This method is stable but not very suitable for some architecture
If enough(>=6) correspondences between spatial vertices and the image pixels are known, the camera calibration may be immediately computed.
Camera CalibrationNew Method:
Inspired by parallelipiped method, the author found the frustum more general to represent the architecture
Camera CalibrationDemo:
Frustum is symmetric
Camera Calibration
𝛬=(𝑙1 𝑙2 cos𝜃 0 00 𝑙2 sin 𝜃 0 0
0 0𝑙3𝛼 0
0 0 1𝛼 1 )𝑃 𝑖=𝛬⋅ �̂� 𝑖 ¿
1 2
4
3
56
Coordinate represented in world:
𝑥 ,𝑖❑ 𝑦 𝑖∈ {1 ,−1} 𝑧𝑖∈ {0,1 }
𝛬=(𝑙1 0 0 00 𝑙2 0 0
0 0𝑙3𝛼 0
0 0 1𝛼 1 )Of this example
Camera Calibration
¿1 2
4
3
56
𝑝𝑖≃𝑀𝑃 𝑖=𝑀 𝛬 �̂�𝑖=�̂� �̂� 𝑖
M=K ⋅ [R|t ] ¿
=
K=
= Quaternion( unit vector(x,y,z),)
t =t(x, y, z)
15 parameters to estmate =1 𝑙1 𝑙2 𝜃𝛼❑
Camera Calibration
1 2
4
3
56
K=
Simplification:
11 parameters to estimate, now the calibration is formulated as a non-linear optimization
Camera CalibrationOptimization Initialization:
= =
The Quadratic :
¿Extend the right multiplication, since the R is unit orthogonal matrix, then we obtain:
1 22 2 2ˆ ˆ( )T Tm K K m l
�̂�1𝑇 (𝐾 −𝑇 ⋅ 𝐾− 1)�̂�1=𝑙12
�̂�1𝑇 (𝐾 −𝑇 ⋅ 𝐾− 1)�̂�2=𝑙1𝑙2cos𝜃
�̂�2𝑇 (𝐾 −𝑇 ⋅ 𝐾− 1)�̂�2=𝑙22
User gives the
3D Points ReconstructionSymmetry-Based Triangulation:
3D Points ReconstructionSymmetry-Based Triangulation:
Surface Modeling
Geometry Modeling Model Refinement Texture Mapping
User-Interaction Assisted Modeling
Geometry ModelingPlanar Structure
Roof
Model Refinement
Texture MappingSingle image inevitably lack texture samples due to the foreshortening and occlusion.
But to achieve a good texture effect, there are 2 requirements:1. the final texture should be consistent with the foreshortened image ; 2, the final texture should have consistent weathering pattern.
Detect Texture Quality
Refine low quality region
Texture the occluded region
We need to know where is well textured and where not
Texture Quality Detection
Back Project
Ratio = Triangle.size / imageProjection.size
Ratio > Threshold and Ratio is finite: large texture distortionRatio<Threshold: distortion freeRatio is infinite: occluded
Texture in distortion free region will be used as the texture sample
Refinement for Low-Quality
Super- Resolution Problem
Occluded Region TexturingThe simplest way is to repeat the same texture as those of their symmetric counterparts, but this makes the model look artificial.
It is better to synthesize the texture in these region with common method
Another feature of the texture is weathering pattern, a constraint texture map is used according to the height of the architecture.
Input sample Synthesized
Result
Result
Conclusion
• Contribution:– Novel Calibration Method– Texture Enhance Method
• Limitations:– Strong assumption for simplification of
camera calibration
Message from this Paper