Columnstore Customer Stories 2016 by Sunil Agarwal

36

Transcript of Columnstore Customer Stories 2016 by Sunil Agarwal

Page 1: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 2: Columnstore Customer Stories 2016 by Sunil Agarwal

Agenda

Page 3: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 4: Columnstore Customer Stories 2016 by Sunil Agarwal

World Record Breaking Performance (TPC-H)

SQL Server Leading the TPC-H benchmark

Page 5: Columnstore Customer Stories 2016 by Sunil Agarwal

SQL Server 2016: Columnstore ImprovementsPerformance

Functionality: Data Warehouse (DW)

Functionality: Real-Time Operational Analytics

Page 6: Columnstore Customer Stories 2016 by Sunil Agarwal

World Record Breaking Performance (TPC-H)

SQL Server 2016 gives 40% improved

performance over SQL Server 2014

Page 7: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 8: Columnstore Customer Stories 2016 by Sunil Agarwal

Customer – Daman (Health Insurance, UAE)Scenario• Traditional DW with ~100 target tables, ~200 source

tables several billion rows

• Multi-layered, holistic, dimensional model, used by all

users

• Queries involve multi-table joins

Page 9: Columnstore Customer Stories 2016 by Sunil Agarwal

Customer – Daman Requirements/Challenges

Requirements

Challenges

Page 10: Columnstore Customer Stories 2016 by Sunil Agarwal

Approach

JulJunMayAprMarFebJan

2015 2016

DecNov

Go: POC SQL Server 2016

POC SQL Server 2016 Go

Production

Go

In the cloud

6 contenders

Synthetic data

Same data model

Similar characteristics

On the premises

SQL Server 2016 vs.

Sybase IQ

Full productive data set Daily load on SQL Server 2016

Automated test framework

Full test end user tools

Page 11: Columnstore Customer Stories 2016 by Sunil Agarwal

Customer – Daman: Benchmark Results

Solution - CCI

Page 12: Columnstore Customer Stories 2016 by Sunil Agarwal

Results – testing & production

50 reports weekly

Scheduled Reports

60% improvement

SQL 2016: 10 h

Sybase IQ: 28 h

Weekly

Runs at EOM

Business Tool

5 x improvement

SQL 2016: 10h

Sybase IQ: 2 d

Monthly

100s with 30 concurrent users

AdHoc Analytics

Significant Improvement

Daily

Many queries < 20

seconds

Page 13: Columnstore Customer Stories 2016 by Sunil Agarwal

Demo – Screen Shots

Page 14: Columnstore Customer Stories 2016 by Sunil Agarwal

Customer Daman: Learnings

ETL

Optimizer

Case Studyhttps://customers.microsoft.com/en-us/story/daman

Page 15: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 16: Columnstore Customer Stories 2016 by Sunil Agarwal

First American - Business Problem

• Scope of Data:150 Million Properties in USA

• Querying

• Not a traditional Columnstore Index Scenario

• Failed Solution

Page 17: Columnstore Customer Stories 2016 by Sunil Agarwal

First American - Requirements

Success Criteria

Examples of Slow SQLs SELECT

FROM

WHERE

SELECT

FROM

WHERE

Page 18: Columnstore Customer Stories 2016 by Sunil Agarwal

19

Relational Table (disk-based)(Clustered Index)

Page 19: Columnstore Customer Stories 2016 by Sunil Agarwal

20

Btree index

Rowstore Table (disk-based)

Hybrid queries are directed appropriately

Challenge

Solution

Page 20: Columnstore Customer Stories 2016 by Sunil Agarwal

First American - Hybrid Query

SELECT <Select List>FROM NormandySearch.Search.PropertyWHERE propertyid in(SELECT propertyid

FROM NormandySearch.Search.PropertyCSWHERE [NumericStreetNumber] = 416

and OwnerName like 'BOW%‘ And ( [City] = 'Bay Village'Or [PlaceName] = 'Bay Village')

Page 21: Columnstore Customer Stories 2016 by Sunil Agarwal

First American - Results with Columnstore Index

Data Compression

CPU Consumption

Query Performance with 100 concurrent users

With Rowstore (PAGE Compressed) With CCI

Database Size (including Indexes) 560 GB 44 GB (13x)

Page 22: Columnstore Customer Stories 2016 by Sunil Agarwal

First American - Learnings

Page 23: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 24: Columnstore Customer Stories 2016 by Sunil Agarwal

FIS – Business Overview and Challenges

Application Overview

Current Solution

Page 25: Columnstore Customer Stories 2016 by Sunil Agarwal

FIS – Application Challenges

Challenge

No Columnstore Index Benefits

What to do?

Page 26: Columnstore Customer Stories 2016 by Sunil Agarwal

FIS – Solution

Considered two configurations

Baseline with all 1.2 million rows compressed

Delivered solution with NCCI with force compression

Page 28: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 29: Columnstore Customer Stories 2016 by Sunil Agarwal

what something is instead of where it's stored.

Organize information based on

Proposal

ESTT Corporation

9/30/2016

Website Renewal

Y:

Proposals 2016

Projects

ESTT

Official docs

Website renewal

M-FILES – Document Management

Page 30: Columnstore Customer Stories 2016 by Sunil Agarwal

Each document inserted results in approximately 60 SQL inserts into Metadata

Metadata change in a

document can result

updating metadata of

large number of other

documents

Document loading and Viewing

Page 31: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 32: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 33: Columnstore Customer Stories 2016 by Sunil Agarwal

Metadata Table

Btree Index

Page 34: Columnstore Customer Stories 2016 by Sunil Agarwal

Query Before (sec) After (sec)

Contracts by Pricing Type and Agency 6.5 0.5

Contracts by year and month 2.6 0.05

Page 35: Columnstore Customer Stories 2016 by Sunil Agarwal
Page 36: Columnstore Customer Stories 2016 by Sunil Agarwal

Thank you

for your time!