Algebraic Manipulation of Scientific Datasets
Bill Howe and David MaierOGI School of Science and
Engineering at Oregon Health and Science University
Portland State University
Environmental Observation and Forecastingon the Columbia River
Sensors
Simulation
Data Products
Gridded Scientific Datasets
3.2 3.1
3.8
12.1C
12.6C
13.1C
13.2C
12.8C
12.5C
3.63.23.33.4
3.63.6
4.0
4.1
4.0
4.0 4.04.0
4.0
4.0
Some CORIE Grids
H = 2d Horizontal Grid
T = 1d Time Grid
V = 1d Vertical Grid
Mean Sea Level
Underground
(not shown)
Thesis
• Grid topology requires explicit data model support.
• Transformations can be expressed via composition of a few logical operators.
• Performance can be preserved via algebraic optimization and specialized operator implementations.
Roadmap
• Domain Introduction
• Model Introduction
• Conventional Approaches
• Examples of Optimization
• Conclusion
Grid Topology
• Grid Topology
– A collection of cells of various dimensions,
– implicit or explicit incidence relationships
1
A B0
23
mn
op
q
2-Cells 0-CellsA 0A 1A 3B 1B 2B 3
1-Cells 0-Cellsm 0m 1n 1n 2
: :
2-Cells = {A,B}1-Cells = {m,n,o,p}0-Cells = {0,1,2,3}
Grid Properties
• Topology <> geometry
• A grid may contain cells of – multiple dimensions – multiple “shapes”
• Dimension of a grid is the maximum dimension of its cells
1
AB
02
3
mn
o
pq
4
r
GridField: Grid with Bound Data
• Tuples of numeric primitives
• Total functions over cells of dimension k
• Two gridfields may share a grid
x y salt temp
x1 y1 29.4 12.1
x2 y2 29.8 12.5
x3 y3 28.0 12.0
x4 y4 30.1 13.2
flux area
11.5 3.3
13.9 5.5
13.1 4.5
Roadmap
• Domain Introduction
• Model Introduction
• Conventional Approaches
• Examples of Optimization
• Conclusion
1) Modeling with Relations
• trivial join dependency embedded in the key– decomposition won’t help– no notion of “grid”
x y t salt temp
x1 y1 1 29.4 12.1
x2 y2 1 29.8 12.5
x3 y3 1 28.0 12.0
x4 y4 1 30.1 13.2
x1 y1 2 30.6 12.1
x2 y2 2 31.5 12.2
x3 y3 2 31.7 11.8
x4 y4 2 32.0 10.1
Node Data cid flux area
a 11.5 3.3
b 13.9 5.5
c 13.1 4.5
Cell Data
G
G
cid x y
a x1 y1
a x2 y2
a x4 y4
b x2 y2
: : :
Incidence
2) Spatial Extensions
• Incidence relationship dependent on geometry rather than topology• Geometry information redundantly defined in nodes and cells• No concept of a “grid”: impedance mismatch with visualization applications
Node::Point t salt temp
Point(x1,y1) 1 29.4 12.1
Point(x2,y2) 1 29.8 12.5
Point(x3,y3) 1 28.0 12.0
Point(x4,y4) 1 30.1 13.2
Point(x1,y1) 2 30.6 12.1
Point(x2,y2) 2 31.5 12.2
Point(x3,y3) 2 31.7 11.8
Point(x4,y4) 2 32.0 10.1
Node Data
Cell::Polygon flux area
Polygon(Point(x1,y1),…) 11.5 3.3
Polygon(Point(x2,y2),…) 13.9 5.5
Polygon(Point(x1,y1),…) 13.1 4.5
Cell Data
3) Visualization Libraries
• Different algorithms, each dependent on data characteristics. • Programmer’s responsibility to match algorithms with data• Logical equivalences are obscured
vtkExtractGeometryvtkThresholdvtkExtractGridvtkExtractVOIvtkThresholdPoints
Grid restriction:
With VTK:
restrict
Roadmap
• Domain Introduction
• Model Introduction
• Conventional Approaches
• Examples of Optimization
• Conclusion
associate grids with data
combine grids topologically
reduce a grid using data values
transform grids or data
bind (b)
union, intersection, cross product ()
restrict (r)
aggregate (a)
Task Operator
Operators
25
26 21
19
Restrict Semantics
24
26
25
27
25
24
26
21
19
26
27
25
24
restrict(<24)
restrict(<24)
Values bound to 0-cells (nodes)
Values bound to 2-cells (triangles)
Working With GridFields
H : (x,y,b)
V : (z)
r(z>b) b(s) r(region) render
H V (H V) r(H V) b(r(H V)) r(b(r(H V)))
“wetgrid”
Optimize: Push Restricts
• salt,temp defined on G
• Materialize pointers to elements of salt, temp
• Bind salt, temp to a subgrid of G, G'
G =
s1 s2 s3 s4 s5
t1 t2 t3 t4 t5
::
G' =
s1 s3 s5
::
t1 t3 t5
salt =
temp =
salt' =
temp' =
r(p(x,y))
r(p(z))
r(z>b) b(s)
H : (x,y,b)
V : (z)
Optimization Results
0
5
10
15
20
25
30
35
40
45
0 0.2 0.4 0.6 0.8 1
selectivity
tim
e (s
ec)
unopt
opt
vtk
rdbms
Horizontal Slice
H(x,y,b)
V(z)
r(z>b) b(s) slice
H(x,y,b)
<depth>
r(z>b) b(s)apply
Transect (Vertical Slice)
H(x,y,b)
V(z)
r(z>b) b(s) “join”
PP V
CA B
Transect (Vertical Slice)
V(z)P
H(x,y,b)
“join” b(s) “join”
A B
CP
A
B
C
Transect Optimizations
05
1015202530354045
Top Related