Babu Netezza
-
Upload
manoj-kumar -
Category
Documents
-
view
108 -
download
5
Transcript of Babu Netezza
Data-centric computing with Netezza Architecture
DISC reading groupSeptember 24, 2007
High Level Points
• Supercomputer use model today:– Compile, submit, wait– Does a poor job of taking advantage of
human insight available in interactive models
• Large datasets can be interactively processed using Netezza
What is Netezza?
• Essentially: A big, fast SQL database
What is Netezza?
• Frontend provides SQL interface• Backend is a large rack of specialized blades
Custom Backend Blades
• Commodity CPU, NIC, disk• Custom FPGA replaces disk interface
– Can do basic filtering in hardware, i.e., stream processing before data hits main memory
Division of Data
• Database distributed across multiple (100+) SPUs
• Each SPU controls, manages its slice of DB
• No info on data management, replciation, etc.
Division of Labor
• SPU FPGA handles basic filtering tasks• SPU CPU handles record level processing:
filtering, parsing, projecting, logging, etc.• SPU CPU handles most operations on
intermediate results: sorts, joins, aggregates• Frontend CPU handles remaining operation
>>> Processing close to disk
What can this be used for?
• Paper gives 3 examples:– Citation graph processing– Search for particular structure in electrical
netlist– Word meaning disambiguation through search
of ontology
Citation graph example
• Look through large, sparse graph (16 million nodes, 388 million edges)
• Find both strong (direct edge) and weak couplings (e.g., two papers cite the same work)
• Essentially same code for workstation and Netezza – no need to expose parallel architecture
• Workstation DNF; 80-100x speedup on smaller tests
IC netlist example• Flattened netlist of 3.5 million transistors, 10
million wires• Search for AND structure
IC example results
• Combinatorial explosion makes directly joining all possibilities for each element impossible
• Can constrain better using fanouts of signals internal to the circuit
• Individual SQL queries for finding possible matches for the individual transistors took under 10 seconds
• Found all uses of the AND macro, as well as many other (1300+) identical structures generated through other means
Ontology example
• Expand out all possible interpretations of a phrase
• Ontology specifies lexical elements, IS-A relations, concepts, and constraints on concepts
• Goal is to search the space, expand concepts to find all matches to given phrase
Ontology results
• Partially unfolded ontology– Greatly expands database size, but reduces
iterations / recursions
• Recoded ontology triples as integers
• 5.58 sec. vs. 262 sec.
• can pipeline multiple queries
Issues
• Works if you can reduce your problem to SQL queries• All of the problems were based on graph expansion /
exploration – how about other domains?• Issues of database partitioning? How does arbitrary
slicing across 108 blades affect performance / scalability, esp. for non-sparse problems?
• Strawman comparison to workstation class machine: how does a traditional DB server / storage cluster compare?