1. BoostingperformancewithColumnstoreIndexes MichaelZilbersteinDBArt LtdMichael@dbart.co.il

2. DEMO 3. History Column-oriented databases: Sybase IQ Vertica Aster Data Greenplum Excel PowerPivot. VertiPaq. xVelocity Columnstore index. 4. C1 C2 C3 C4 C5 C6Uses VertiPaqcompression 5. Reduced IO Fetches only needed columns from diskSELECT C2, SUM (C3) C2 C3Columns are compressed C1 C4 C5 C6 Less IO Better buffer hit rates 6. New query execution technology Batch mode execution of some operations processes rows in batches groups of batch operations in query plan Better parallelism, better algorithms 7. Dictionary-based compression Year of Code Birth 19961Internal Dictionary 197515Year of 194850Birth 193258On-the-fly build dictionary1996 60 with all distinct value.1975 Substitute non-selective values with ID.1975 Index in our example 6 bits per row.1948Year ofBirth1932Code1 Compressed Fact1515505860 8. SegmentsC1 C2 C3 C4 C5 C6 Column segment Set of about contains values from 1M rowsone column for a setof about 1M rowsColumn segmentsare compressedEach columnsegment stored inseparate LOBColumn segment isColumnunit of transfer fromSegment disk 9. Data Dictionary Views 10. New execution plan elements 11. Best practices / worst practices Best practices: Put columnstore indexes on large tables only. Include every column of the table in the columnstore index. Structure your queries as star joins with grouping and aggregation as much as possible. Worst practices: Avoid JOIN and/or filter on string columns in the tablewith columnstore index. Avoid OUTER JOIN, UNION ALL, IN/NOT IN. Avoid JOIN between 2 Fact tables. 12. Datatype Limitations Decimal > 18 digits (Var)Binary BLOB (N)Varchar(max) Uniqueidentifier Date/Time types > 8 bytes (Datetime2 etc) CLR 13. Issues and Workarounds: good, bad and ugly Outer Joins NOT IN Scalar Aggregates Multiple DISTINCT UNION ALL IN and EXISTS 14. Data loading and columnstore index Columnstore index makes table read-only. DEMO 15. Data loading and columnstore index 2 ways to load data: Partition switching Disable/drop index -> load data -> create/rebuild index. 16. References http://channel9.msdn.com/Events/TechEd/NorthAmerica/2011/DBI312 http://social.technet.microsoft.com/wiki/contents/articles/4995.sql-server-columnstore-performance-tuning.aspx 17. Q&AMichael ZilbersteinTel: 052-4767219E-Mail: Michael@dbart.co.il