Topics In-memory Spatial Database based on key-value pair Building Information Model based on...
Transcript of Topics In-memory Spatial Database based on key-value pair Building Information Model based on...
IN-MEMORY SPATIAL DATABASE BASED ON KEY-VALUE PAIR
Topics
In-memory Spatial Database based on key-value pair
Building Information Model based on relationship inference
In-memory Spatial Database based on key-value pair
Background
Architectural Overview
Features
Background
This research is supported to solve
kernel issues during the State Grid information construction.
The achievements have been applied to the power information systems.
Deploy view
State Grid
Shanghai Grid
…
User1
User2
…
Hubei Province
Grid
Beijing Grid
User1
User2
…
User1
User2
…
User1
User2
…
Network
Storage
Software …
Physical
Resource
Graphics
…Vector
Database
ManageAnalysi
s…Spatial
Power Grid Services
ERPMarkin
gGIS …
Systems
PlanningSchedul
eSale …
Businesses
SecurityData
Network
System
…
Application IntegrateProduct
Monitor
Emergency
…
Technical Architecture
Background
Store Data
Organize Data
Access Data
Others
Large scale dataKey-valueTransaction Locks Lazy write to DBMS
High concurrency High throughout Batch to submit data long period, offline data edit Increment data query Can’t be accessed directly by user hybrid index based on spatial and topology
Change notify Reuse existing power database, such as Oracle Spatial, SDE, DB2 Spatial Extender Data synchronize …
Key problems
Partition based on spatial and topology logical data model Version History Database
Background
Why not traditional spatial database?
Why not DBMS, such as Oracle?
Why not SDE or ArcServer?
Why not object-oriented, such as Oracle spatial?
Why not traditional spatial index, such as Quad-tree, R*-tree?
Bad I/O throughout Bad performance when high concurrency and batch editCan’t provide necessary features Bad performance There are some bugs in LinuxCan’t provide necessary features
More slow than raw data type, such as BLOB Spatial index has bugs and has to be rebuilt at fixed periodSpatial index performance is badPrecise filter is even slow, such as sdo_relateSpatial function is not necessary
No. We need a hybrid index, including spatial, topology and attributes information
Background
Why in-memory spatial database on key-pair
Why store data in memory? How is the data persistence?
Why use key-pair? What is your spatial model?
Do you use spatial index?
Do you support condition query like SQL?
in-memory database is much fast than disk database The changed data will be written into DBMS
Simple means high performance! In-memory database doesn’t need complex search
We use the hybrid index, including spatial, topology and attribute information
We support simple compare function ,such as greater, equal, less…
httpd
Server computer
http
d
Serv
er
comp
uter
httpd
Server
computer
httpd
Server computer
http
d
Serv
er
comp
uter
httpd
Server
computer
Data synchronize
Data synchronize Data synchronize
cluster cluster
Lazy wri te Lazy
wri
te
DBMS
Architectural-Logical View
Apache httpd
Server computer
In-memory Database
Architectural-Framework
Memory management
Cache
Memory schedule
Page management Index library
Access API
Thread management
Persistence Pool refresh
Connect pool
Index
Hash Index
Spatial Index
In-memory database
Data edit
Pool FillQuery
HTTP protocolData
synchronize
Features-Data Model
Indeed, every record is a memory block!
Typical data struct
Index for attributesB-tree index, Hash index
Spatial index Spatial index is a hybrid index: B-tree + grid index + topology based index
protocol head description field value 1 field value 2 … field value n
Features-Cluster In my research, cluster is a group of virtual, loosely
coupled in-memory database that work together closely.
In a cluster, all in-memory databases have the same data.
For different group, maybe they have intersected data.
How to partition the power data for cluster is based on the spatial , topology and attribute information
When some data are updated in one in-memory database, other in-memory database which has these data will be synchronized almost real-time.
Features-Change notify
When data is update, clients will receive the notify.
The notification granularity is based on object, not record, which help to
The notify is based on subscribe & notify model.
User can only download changed records.message server
2012年2月21日 2012年8月21日
2012年4月1日 2012年7月1日
Snapshot 1 Snapshot 2
Ti me axi s
Features-Version & History DB Provide version mechanism and history
database based on time axis .
Sometimes, work maybe needs several days, even a month to finish. These data will be managed as a version and other client can’t see them before finishing the work.
At any time, in-memory can create database snapshot for archive or future comparative analysis.
confl ict resolve
confl ict resolve
commit commit
version 3version 2
version 1
BASE
BIM based on relationship inference
Background
Roadmap
Background
Design
Management
Construction
In common, the life cycle of a building includes 3 phases:
IFC is a vendor-neutral BIM data repository for the semantic information of building objects, including geometry, associated properties, and relationships, to facilities.
IfcPropertyDefinition
1
*IfcTypeObject
HasProperty Sets *L[1:?]
(INV) Def inesTy pe S[0:1]
(ABS)
IfcPropertySetDefinition
(INV) PropertyDef initionOf
S[0:1]
IfcRelDefines (ABS) (ABS)
IfcRelDefines ByType
IfcRelDefines ByProperties
1
RelatingTy pe
RelatingProperty Def inition
(INV) Ty peObjectOf
S[0:1]
IfcPropertySet IfcXxxProperties
1
IfcObject (ABS)
RelatedObjects S[1:?]
come from T.E. El-Diraby
(ABS)
IfcRelationship
1
IfcRelAssigns
IfcRelConnects
IfcRelDecomposes
IfcRelAssociates
IfcRelDefines
(ABS)
(ABS)
(ABS)
(ABS) (ABS)
come from T.E. El-Diraby
I f cRoot
I f cObj ect
I f cRel ati onshi p
I f cProperty
0 200 400 600 800
IFC1.0
IFC1.5
IFC2.0
IFC2x
IFC2…
number of classes in IFC
BackgroundImpression
IFC is a complex, relatively complete and huge specification.
From the OO point of view, IFC is a well designed and loose coupled class set.
Background
Problems and Limitations for IFC
For OO design, message or event driven is an import design pattern, but it seems there is no definition about it.
IFC doesn’t define something about analysis, which is important for management phase during the life cycle of a building.
The ability of relationship description is much weak and can not describe the implicit relationship automatically, which is import for analysis and modeling.
GIST
belong to
OTB
Liubelong
toGIST
Liubelong
toOTB
Roadmap
Goals
Provide the ability of relationship inference.
Improve the degree of automation during the process of building information modeling.
Provide a mechanism for building information analysis
Roadmap-relationship inference
I FC(I f cRel ati onshi p)
Formal Descri pti on
OWL
Rel ati onshi p Operator(aggregati on, group, order, add, subtract, . . )
I f cXXXRel ati ons (deri ved f rom I f cRel ati onshi p)
Bui l di ng Model i ng Bui l di ng Anal ysi s
rel ati onships
Cl assi fi cati on
Property
Constrai n
Classification based on the number of relation
unaryBinary triple
Classification based on type
subsumption(subClass-of, superClass-of, …)mereology(is-part-of,) topology(next-to, touch-with, …)….
transitive
symmetric
inverse
equality
hasValue
exists at
occur at
maximum minimum
Roadmap-analysis based on event driven
bui l di ng i nformati on pi pel i ne
I fcObject 1 IfcObject 2 IfcObject . . .
analysi s analysi s analysi s
Initial thoughts
Information transmission is modeled a virtual pipeline.
Every object will get information from pipeline and analyze them. The result is also pushed into the pipeline.
The purpose: 1) object is self-contain; 2) object is independent of other object; 3) combine different analysis
Thank you