Procedural Modeling of Architectures towards 3D Reconstruction Nikos Paragios Ecole Centrale Paris /...
-
Upload
marjory-robbins -
Category
Documents
-
view
217 -
download
2
Transcript of Procedural Modeling of Architectures towards 3D Reconstruction Nikos Paragios Ecole Centrale Paris /...
Procedural Modeling of Architectures towards 3D Reconstruction
Nikos ParagiosEcole Centrale Paris / INRIA Saclay Ile-de-France
Joint Work: P. Koutsourakis, L. Simon, O. Teboul & N. Komodakis
Outline• The language of Architecture
• Automatic generation of buildings
• Toward single image-based reconstruction
• Discrete MRFs, LP and Duality
04/11/0810/04/08
Part I:Shape Grammars
3D reconstruction• Emerging technology in information society– Post-production, games, navigation
• Current Methods: image based– Not scalable, computational expensive, low
level,
04/11/0810/04/08
The Geometry of Architecture• Very complex geometry…• …but deeply structured due to:
– Laws– Practical constraints– Economical constraints– Aesthetic motivation
• Each architectural style is constrained differently. But buildings of the same style obey the same rules.
04/11/0810/04/08
Simple example of structure
04/11/0810/04/08
Architecture as a large scale Lego• Architecture is made of repetitions of basic
elements, place holders or real elements.– Facades, floors, windows, balconies, doors…
• A specific style can be describe by :– The specification of the repetitions (geometry)– The nature of the elements (semantics)
04/11/0810/04/08
Shape Grammars• Procedural modeling is a set of techniques for
creating 3D models from a set of rules.• Traditional 3D modeling can be very time
consuming for large models. • Large scale architectural models are especially
suited for this kind of techniques.• There is a strong analogy with classical string
grammars.
04/11/0810/04/08
Basic Shapes and Scopes• The Basic Shapes are for Shape Grammars what
Symbols are for String Grammars• A Basic Shape is composed of
– A Semantic– A fixed geometry– Appearance attributes
• A Scope provides the orientation, scaling, position of a basic shape
04/11/0810/04/08
Shape
04/11/0810/04/08
• The shape is a hierarchical representation (a tree) of the building.
• Each node holds a basic shape with an associated scope.
Rules : Interacting with shapes• A rule can be described as:
precondition : LHS RHS
• Given a precondition on the context, a LHS shape is replaced by the RHS shape. From the hierarchical point of view some children are added to the node LHS, increasing the depth of the tree.
• Rules are built from operators
04/11/0810/04/08
The operators of the grammar
• Except the roof operators (which rely on computational geometry) the operators are fairly simple.– Transformation : rotation,translation, scaling– Split– Repeat– Mirror– Component Split– Roof operators : hipped, mansard
04/11/0810/04/08
Operators continued• Splits subdivide a scope along an axis• Component split decomposes a mesh into faces• Roof operators are based on Weighted Straight
Skeleton algorithm
04/11/0810/04/08
Step by step generation
04/11/0810/04/08
Generating : from details to large scale
04/11/0810/04/08
04/11/0810/04/08
04/11/0810/04/08
Demos• Video\ComplexBuilding.ogg• video\roofsAndTexture.ogg
04/11/0810/04/08
Grammar-based reconstruction
• A building is not described by a geometrical attributes anymore
• Building = sequence of rules
• Goal : optimize the sequence of rules that best explains a given facade image
04/11/0810/04/08
Model Quality Evaluation• Assuming that the camera has been weakly
calibrated, the elements of the resulting model can be reprojected on the image.
• The score of the model can be defined in terms of a distance between the observed regions and the known semantics.
04/11/0810/04/08
Part II:MRF optimization viathe primal-dual schema
The MRF optimization problem vertices G = set of objects
edges E = object relationshipsset L = discrete set of labels
Vp(xp) = cost of assigning label xp to vertex p(also called single node potential)
Vpq(xp,xq) = cost of assigning labels (xp,xq) to neighboring vertices (p,q) (also called pairwise potential)
Find labels that minimize the MRF energy (i.e.,
the sum of all potentials):
MRF hardness
MRF pairwise potential
MRF hardness
linear
exact global optimum
arbitrary
local optimum
metric
global optimum approximation
Move right in the horizontal axis,
But we want to be able to do that efficiently, i.e. fast
and remain low in the vertical axis (i.e., still be able to provide approximately optimal solutions)
The primal-dual schema Say we seek an optimal solution x* to the following
integer program (this is our primal problem):
(NP-hard problem)
To find an approximate solution, we first relax the integrality constraints to get a primal & a dual linear program:
primal LP: dual LP:
The primal-dual schema Goal: find integral-primal solution x, feasible dual solution y
such that their primal-dual costs are “close enough”, e.g.,
Tb y Tc x
primal cost of solution x
primal cost of solution x
dual cost of solution y
dual cost of solution y
*Tc x
cost of optimal integral solution x*
cost of optimal integral solution x*
*fT
T
c x
b y
**f
T
T
c x
c x
Then x is an f*-approximation to optimal solution x*
Then x is an f*-approximation to optimal solution x*
(only one label assigned per vertex)
enforce consistency between variables xp,a, xq,b and variable xpq,ab
The primal-dual schema for MRFs
Binaryvariables
xp,a=1 label a is assigned to node p
xpq,ab=1 labels a, b are assigned to nodes p, q
xp,a=1 label a is assigned to node p
xpq,ab=1 labels a, b are assigned to nodes p, q
The primal-dual schema for MRFs During the PD schema for MRFs, it turns out that:
each update of primal and dual
variables
each update of primal and dual
variables
solving max-flow in appropriately
constructed graph
solving max-flow in appropriately
constructed graph
Max-flow graph defined from current primal-dual pair (xk,yk) (xk,yk) defines connectivity of max-flow graph (xk,yk) defines capacities of max-flow graph
Max-flow graph is thus continuously updated
Resulting flows tell us how to update both: the dual variables, as well as the primal variables
for each iteration of primal-dual schema
primal-dual framework
primal-dual framework
Handles wide class of MRFsHandles wide class of MRFs
Approximatelyoptimal solutions
Approximatelyoptimal solutions
Theoretical guarantees AND tight certificates
per instance
Theoretical guarantees AND tight certificates
per instance
Significant speed-up
for static MRFs
Significant speed-up
for static MRFs
Significant speed-up
for dynamic MRFs
Significant speed-up
for dynamic MRFs
- New theorems- New insights into
existing techniques- New view on MRFs
- New theorems- New insights into
existing techniques- New view on MRFs
04/11/0810/04/08
Future Work• Efficient optimization framework, through neuro-
dynamic programming like methods• Image-based grammar learning• Augmentation of the expressiveness of the
grammar• Large scale representation of forests of buildings
04/11/0810/04/08
Questions ?
04/11/0810/04/08