1c

Jimmy May, MCMSQL Server on Flash: Re-Thinking Best Practices

SQL Server Solutions Architect | Fusion ioMemory™ Solutions

1

DBA Fundamentals :: Down Under

2

Preamble

2

3

LegaleseDuring our meeting today, we may make forward-looking statements.

Any statement that refers to expectations, projections or other characterizations of future events or circumstances is a forward-looking statement, including those relating to market position, market growth, product sales, industry trends, supply chain, future memory technology, production capacity, production costs, technology transitions, construction schedules, production starts, and future products. This presentation contains information from third parties, which reflect their projections as of the date of issuance. Actual results may differ materially from those expressed in these forward-looking statements due to factors detailed under the caption “Risk Factors” and elsewhere in the documents we file from time to time with the SEC, including our annual and quarterly reports. We undertake no obligation to update these forward-looking statements, which speak only as of the date hereof.

©2014 SanDisk Corporation. All rights reserved. SanDisk is a trademark of SanDisk Corporation. Lightning, Lightning Eco, Lightning Ascend, Lightning Ultra, Optimus, Optimus Eco, Optimus Ascend, Optimus Ultra, Optimus Extreme, CloudSpeed, CloudSpeed Eco, CloudSpeed Ascend, CloudSpeed Ultra, CloudSpeed Extreme, FlashSoft, ZetaScale, Guardian Technology, FlashGuard, DataGuard, EverGuard, SanDisk ION Accelerator, Fusion ioMemory, Fusion ioSphere, Fusion ioTurbine, Fusion ioVDI, and others are trademarks of SanDisk Enterprise IP LLC. Other brand names mentioned herein are for identification purposes only and may be the trademarks of their respective holder(s).

4

Introduction

Flash storage is a game-changer for virtually every app in which it is used. Whether used locally—SATA, SAS, or PCIe; or for tiered storage or all-flash arrays—flash is changing what we thought we knew about SQL Server performance.

This presentation provides a brief survey of history and a high-level, vendor-agnostic review of options for implementing flash in the enterprise.

4

5

Jimmy May, MCM• SanDisk SQL Server Solutions Architect

Formerly:• Sr. Program Manager, SQL CAT• SQL Server Customer Advisory Team• MSIT Principal Architect: Database

Microsoft Certified Master: SQL Server (2009)MS IT Gold Star Recipient (2008) Microsoft Oracle Center of Excellence (2008)SQL Server MVP Nominee (2006)Indiana Windows User Group www.iwug.net

• Founder & Board of DirectorsIndianapolis Professional Association for SQL Server www.indypass.org

• Founder & Member of Executive CommitteeSQL Server Pros: Founder & Visionary-in-ChiefSQL Innovator Award Recipient (2006)

• Contest sponsored in part by MicrosoftFormerly Chief Database Architect for high-throughput OLTP VLDB at ExactTarget (recent IPO)Senior Database Administrator for OpenGlobe/Escientwww.twitter.com/aspiringgeek jimmy.may@sandisk.com jimmymay@outlook.com http://sqlblog.com/blogs/jimmy_may

Bio

6

Data Propulsion Laboratory — DPL

7

C:\>sqlservr.exe /faster

8

Acknowledgements Kevin Kline, MVP Brent Ozar, MCM, MVP Thomas LaRock, MCM, MVP Denny Cherry, MCM, MVP Gavin Payne, MCA, MCM Chuck Suter, Pure Storage Mike Anderson, Disney Mike Ruthruff, Bungie Studios Thomas Kejser Tim Radney, MVP Mindy Curnutt, MVP

Brian Walters, SanDisk Niall MacLeod, SanDisk John Gatch, SanDisk Peter Plamondon, SanDisk David Reeves, SanDisk Grant Jacobson, SanDisk Sean Hehir, SanDisk

9

Navigating this deck1.On the left, right-click on any section name

2.Click “Collapse All”)

3.Review the section names, then click to expand desired content

10

PASS Summit 2015 Pre-ConThe Complete Primer to SQL Server Virtualization

Learn More: http://www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8149

11

SanDisk Speaking Sessions Title: The Complete Primer to SQL Server Virtualization (Full Day Session)

– Speakers: • Jimmy May, MCM, SQL Server Solutions Architect, SanDisk• David Klee, MVP, Founder & Chief, Heraflux Technologies• Argenis Fernandez, MVP, MCM, SQL Server Solutions Architect, Pure Storage

– Date: Monday, October 26– Time: 9 a.m. – 6:00 p.m. PST– Location: 611 – 614– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8149 [DBA-299-P]

Title: SQL Server 2016 on Flash– Speaker: Brian Walters, Solution Architect, SanDisk– Date: Wednesday, October 28– Time: 10:15 a.m. PDT– Location: TCC101

Title: Columnstore Indexes in SQL Server: Flipping the DW Faster Bit [AD-206]– Speaker: Jimmy May, MCM, SQL Server Solutions Architect, SanDisk– Date: Wednesday, October 28– Time: 4:45 p.m. – 6:00 p.m. PST– Location: 6A– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8623

Title: SQL Server on Flash: Re-Thinking Best Practices [DBA-210]– Speaker: Jimmy May, MCM, SQL Server Solutions Architect, SanDisk and Brian Walters, Solution Architect, SanDisk– Date: Friday, October 30– Time: 8:00 – 9:15 a.m. PST– Location: TCC 304– Learn More: www.sqlpass.org/summit/2015/Sessions/Details.aspx?sid=8601

SanDisk Theater– Visit the SanDisk booth (#22) for brief sessions—throughout the day, throughout the week—for an opportunity to learn & to win!

12

Agenda

12

13

Agenda History and Introduction to Enterprise Flash Flash Best Practices for SQL Server SQL Server Use Cases and Implementation Details Flash Myths, Misinformation,

& Re-visiting Best Practices PCIe Application Accelerators Microsoft SQL Server 2014 Fast Track (DWFTRA) Hardware for Nothing, Get Your Flash for Free

14

Enterprise Flash Unlocks Microsoft SQL Server 2014 Business Value

• Increase business productivity – more transactions, faster

• Improve customer experience – faster response times

• Reduce costs – scale more users and workloads per server

• Increase SLAs – faster SQL database startup, failover, & maintenance

15

Flash: Don’t be this guy. Don’t be in this org.

Hat tip to Bob Pusateri (@SQLBob)

via Jason Horner (@jasonhorner) & Argenis Fernandez (@DBArgenis )

16

Introduction to Enterprise Flash

16

17

Flash is Revolutionizing Enterprise Storage

19

May’s I/O Mantra

X capacity (GB)at Y throughput (MB/sec)at Z IOPs (transfers/sec) at <=10ms latency for OLTP data files

at <=30ms latency for DW data files at 0ms-2ms for log files

20

What is I/O?Performance Monitor: Logical Disk and PhysicalDisk Latency

– Measured in milliseconds (ms) – Measured in microseconds (µs)– Performance Monitor: Logical Disk

• Avg. disk sec/transfer• Avg. disk sec/read • Avg. disk sec/write

IOPS• Disk transfers/sec• Disk read/sec• Disk writes/sec

Throughput• Disk bytes/sec• Disk read bytes/sec• Disk write bytes/sec

Capacity– Measured in GB/TB– The easy one!

21

What is the business impact of flash on I/O?Google found an extra half-second in search-page generation time dropped traffic by 20%.

Amazon found every 100ms of latency cost them 1% in sales.

If a trading platform is 5 milliseconds behind the competition, it could lose millions in revenues per millisecond.

Performance is paramount: Asking users to wait is like asking them to leave.

If it is fast and ugly, they will use it and curse you; if it is slow, they will not use it. —Computer science professor, entrepreneur, & billionaire David CheritonApplication performance is not an event or something you do once—it's a practice. —Mark Kremer, CEO, Precise Software Solutions Inc.

22

Spinning Media vs. PCIe

How many conventional hard drives to match the IOPs of a single PCIe flash device?

23

24

Spinning Media vs. PCIe: Some Math Note: We could do lots of calculations with various types of I/O

– 4KB IOPs are one of the most commonly published– Numbers will vary for I/O characteristic of SQL Server – The ratios are nonetheless equally remarkable

Random per 4K Read IOPs– 600GB 15K HDD: 181– 3.2TB PCIe Flash: 345,000

Devices for same Random Read IOPs – 340,000 IOPs/PCIe device ÷ 181 IOPs/15K HDD = 1,906 HDDs/PCIe device

Height– 1,906 * 1” height/HDD * 1 foot/12” * 1 story/10 foot = 15.8 stories

Weight– 1,906 HDDs/PCIe * 1.51 pound/HDD = 2,878 pounds

Sources– https://www.sandisk.com/business/datacenter/products/flash-devices/pcie-flash/sx350– http://www.seagate.com/files/docs/pdf/datasheet/disc/cheetah-15k.7-ds1677.3-1007us.pdf– http://archive.benchmarkreviews.com/index.php?option=com_content&task=view&id=375&Itemid=60&limitstart=5

25

Relative Performance of Conventional HDD vs. Flash There are many ways to look at disk performance:

– Read/Write– Random/Sequential– IOPs/Throughput– Block size impacts IOPs, throughput, & latency– Let’s look briefly at one perspective: IOPs vs.

latency

Spinning media (15K Enterprise SAS)– Random 4K reads: 210 IOPs @2000 µs– Random 4K writes: 190 IOPs @2000 µs

Flash vendor 1:– Random 4K reads: 75,000 IOPs @50µs– Random 4K writes: 36,000 IOPs @65µs

Flash vendor 2: – Random 4K reads: 252,000 IOPs @29µs– Random 4K writes: 236,000 IOPs @29µs

Spinning Media vs. Flash

180x – 1,000x+ IOPs

30x – 70x faster

38

Flash Best Practices for SQL Server

38

39

Flash Best Practices for SQL Server

Though flash can seem like a magic bullet …– Flash can mask many performance, configuration, and design challenges

Best practices remain fundamental There is no substitute for engineering discipline in terms of

application & database design

If you don't have time to do it right, when will you have time to do it over?

—John Wooden

40

Workload Characteristics and Pain Points

OLTP Small block, random I/O High IOPS Low throughput Low Latency

Data Warehouse Large block, sequential I/O Lower IOPS High throughput High Latency

41

Flash Addresses Pain Points

Pain Points Overprovisioned Shared

Storage – random I/O Introduces I/O Bottleneck:

– High Latency– Low IOPS – Low CPU utilization– High Cost / Depressed Return

Flash Liberation Solid State minimizes seek time

– Ultra Low latency (µs)– Small & Large Block Size– High Throughput (29 GB/s @ 2 ms)– Low cost / High return

42

Flash Best Practices: SQL Server Configuration tempdb Configuration

– -T1117 (to enable consistent AutoGrow)– -T1118 (to eliminate SGAM contention)– Multiple, equisized tempdb data files– Note: These are baked into SQL Server 2016– Consider making these mandatory components for all SQL Server deployments

Local Security Policy – secpol.msc > Local Policies > User Rights Assignment– Add SQL Server service account to the following:

• Perform volume maintenance tasks (to enable Instant File Initialization).

• Lock pages in memory (to prevent paging).

43

Best Practice – SQL Server

Mitigate memory latch contention:• User DBs: Multiple equally-sized data files per data file filegroup• Tempdb: Multiple equally-sized data files • 8 files is the sweet spot for most implementations

Consider segregating log files from user and tempdb data files if demanded by performance

Configure 'optimize for ad hoc workloads' – mitigate single use statements from overly consuming plan cache & from monopolizing excessive CPU

Dispelling some myths...

44

Flash Best Practices: Windows Power Configuration Power options

– Windows power options:Control Panel > Hardware > Power Options >

High Performance

PoSh: Powercfg -SETACTIVE SCHEME_MIN

45

Flash Best Practices: BIOS

Logical processor / hyper-threading: Enable– Contemporary hyper-threading vs. the original incarnation

Fan: high or increased cooling C-States: Disable all “power saving” options

– Configure for maximum clock speed & voltage– Note: Sometimes called P-States

46

Flash Best Practices: Storage Devices Logical volume configuration

– Disk Partition Alignment remains important– Windows-level format (FAUS): 64KB

Drivers: – Update drivers– More important for flash devices than for conventional spinning media

Vendor-specific recommendations:– Default power settings for device may not be optimal

• This is in addition to physical server & Windows power settings

– “Low-level” formatting: 4KB (not 512B) recommended

47

Power Optimization Summary Maximize power settings in three different locations:

– BIOS• Disable C-State settings

• Maximize fan or cooling, if available

– Windows: High Performance– PCIe configuration

48

Flash Best Practices: Bandwidth Maximum performance is limited by system bottlenecks 2.5” flash drives in server front-end

– It doesn’t take much flash to overwhelm last gen buses– Contemporary buses can accommodate 20 – 24 disks

• Up to 100TB!

• Lenovo X3650 w/Optimus Max 100TB storage (100sec)www.twitter.com/aspiringgeek/status/615909659916898304

– Be aware of potential thermal issues

Validate PCIe slots best-suited for cards– PCIe version– Bus speed: 4x, 8x, 16x– Bus power

49

Best Practice – SQL Server Backup

Design a User & System Database Backup strategy compliant with SLOs & SLAs

Backup Types: FULL, DIFFERENTIAL, LOG, COPY_ONLY Test & Validate:

– Full, Diff, Log—including point-in-time– Integrity: DBCC CHECKDB

You don’t have a backup until it’s been restored—& validated. —Kimberly Tripp

50

Best Practice – SQL Server Maintenance Fragmentation

– SQL Server “read ahead” relies on logically contiguous pages

SORT_IN_TEMPDB Index Build Option– Significant reduction in I/O within SQL Server data files– Especially important for AlwaysOn

51

Backup & Restore Performance Options Backup

– Backup File Count (multiple streams)– COMPRESSION– MAXTRANSFERSIZE

• 4,194,304 (4MB) is max permitted block size– BUFFERCOUNT

• 2x to 4x logical core count Restore

– Presumably multiple files & compression were implemented during backup– Leverage MAXTRANSFERSIZE & BUFFERCOUNT

Note: Consider enabling these options via sp_configure– backup checksum default– backup compression default

52

Prototypical T-SQL BACKUP SyntaxDECLARE @bkpname sysname = 'MyFastBackup‘ -- NAME is exposed via RESTORE HEADERONLY FROM DISK = 'path to first bkp file‘ -- & also in SELECT * FROM msdb.dbo.backupsetBACKUP DATABASE MyDB TO DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#1.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#2.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#3.bak' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#4.bkp' WITH NAME = @bkpname , STATS = 1 , COMPRESSION , MAXTRANSFERSIZE = 4194304 -- 4MB is max permitted block size , BUFFERCOUNT = 96 -- 2x to 4x logical core count , STOP_ON_ERROR -- CHECKSUM error

53

Prototypical T-SQL RESTORE SyntaxDECLARE @bkpname sysname = 'MyFastBackup‘ -- NAME is exposed via RESTORE HEADERONLY FROM DISK = 'path to first bkp file‘ -- & also in SELECT * FROM msdb.dbo.backupsetRESTORE DATABASE MyDB FROM DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#1.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#2.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#3.bkp' , DISK = '<local or \\UNC path>\DBbkps\MyDB_20150909_#4.bkp' WITH REPLACE , RECOVERY , STATS = 1 , MAXTRANSFERSIZE = 4194304 --4MB is the max allowed , BUFFERCOUNT = 112 --up to 4xcorecount , STOP_ON_ERROR -- CHECKSUM error , MOVE 'tpch_300_Primary_SystemData01' TO 'C:\mount\data\MyDB_Primary_SystemData01.mdf' , MOVE 'tpch_300_Primary_SystemData02' TO 'C:\mount\data\MyDB_Primary_SystemData02.mdf' , MOVE 'tpch_300_Primary_SystemData03' TO 'C:\mount\data\MyDB_Primary_SystemData03.mdf' , MOVE 'tpch_300_Primary_SystemData04' TO 'C:\mount\data\MyDB_Primary_SystemData04.mdf' , MOVE 'tpch_300_Secondary01' TO 'C:\mount\data\MyDB_Secondary01.ndf‘ , ...

, MOVE 'tpch_300_Log' TO 'C:\mount\data\MyDB_Log.ldf'

54

Automated Backup Tuning What’re the best options for your environment? Sir SQL—our knight in shining armor—with a PoSh-based solution!

Automated Backup Tuninghttps://bit.ly/SirSQLBackupTuning

Nic Cain @SirSQL

55

Data Compression Long a best practice for SQL Server data Yet it’s often not implemented Typically, relatively small CPU cost (3%) in exchange for storage

savings of ~2x – 5x Corollary: If extant data is not compressed, compression while

migrating to flash can effectively discount your new storage costs by half or more.

Note: Some flash vendors provide compression or de-dupe by default.

56

Comparing Space Savings

19.7GB

10.9GB

5.0GB4.0GB

6.9GB

1.8GB

56

57

#2014-003

#2014-003

#2014-002

#2014-009

#2014-023

#2014-001

#2014-028

#2014-032

0

500

1000

1500

2000

2500

3000

3500

4000

0

10

20

30

40

50

60

70

80

90

100

Query Thruput

RS MeasuredThruput(Queries/Hr/TB)

CS MeasuredThruput(Queries/Hr/TB)

Max User Capacity(TB)

(Que

ries/

Hr/T

B

TB

Row Store vs. Columnstore: I/O vs. Query Throughput

Copyright © 2015 SanDisk. All rights reserved.

#2014-003

#2014-003

#2014-002

#2014-009

#2014-023

#2014-001

#2014-028

#2014-032

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000

0

10

20

30

40

50

60

70

80

90

100

I/O Thruput

RS Scan RatePhysical(MB/sec)

CS Scan RatePhysical(MB/sec)

Max User Capacity(TB)

MB/

sec

TB

58

Best Practice – Data Warehouses New deployments should consider Data Warehouse Fast Track

Reference Architectures (DWFTRA) Columnstore Indexes

– Best, underutilized feature in SQL Server 2014– Microsoft recommends that columnstore be the default choice for

warehouse tables– Leverage table partitioning

59

Stay Tuned for Guidance

-E & -T834– These options direct SQL Server to use Windows large-page

memory allocations for the buffer pool & large physical page allocations, respectively.

– We are investigating interesting memory and I/O behaviors when these traces flags are enabled with certain SQL Server features

Indirect Checkpoint Delayed Durability TDE (& Always Encrypted)

60

Flash Myths

60

61

Flash Myths, Misinformation, & Re-visiting Best Practices

Log files & data files Fragmentation OLTP vs. OLAP/DW Cost Durability/Reliability Availability & RAID for PCIe Cards CPU0 bottlenecks

62

Fragmentation Concept of “sequential” access has no physical analog in flash. Defragmentation remains a nominal best practice, especially for

OLTP/DW workloads – SQL Server readahead requires logically contiguous data

Yet even within Microsoft, benchmarks on flash are often run without the benefit of defragging(!)

63

Log Files & Data files

Once upon a time...– Physical segregation of indexes from data files was a best

practice• Storage abstraction rendered this largely unnecessary

– We still often needed to segregate log files from data files– However, this may no longer be necessary—test!

• Bottom line: Look for consistent, low (<1ms) log file latency

64

OLTP vs. OLAP/DW

Once upon a time...– We didn’t intermingle workloads– Today? Flash may render this moot– Depends on server resources & application workload

65

Availability & RAID for SAS, SATA, & PCIe Not all flash is created equal!

– Consumer vs. Enterprise

Contemporary Enterprise Flash is robust– Massively over-provisioned, internal error checking, etc.– Example warranty: 50PBW (petabytes written), i.e., 50 * 1024TB of writes

• 512GB each day for 6 years

• Doable, yet...

Is it necessary to RAID enterprise flash?– It depends– Be clear: I am not advocating an SPOF!– Perhaps a better use of resources to focus on server/system HA

66

CPU0 bottlenecks Not a myth, but antiquated Windows Software RAID leverages CPU0 Some flash drivers used to do so as well

– Result, high I/O could result in one core with little headroom– Even though other cores were relatively unused– Must examine core-by-core % Processor Time

For example: 16 core server, CPU0 at 99%, CPU1-15 at 10%– Average CPU <16%

Contemporary flash drivers render this moot because they either: – On-board device processor, or – Balance CPU across all cores

67

Cost Once upon a time...

– Flash was expensive– If you think flash is expensive, try pricing a SAN

• And supporting it...

– Yet time is money • Remember the latency slide?

– Today: • Glacial SATA is ~50¢/GB

• Depending on format, vendor, re-seller, etc., Enterprise Flash has recently been as economical as $3/GB & quickly approaching $1/GB

• As of March 2015: InfiniFlash is as low as $2/GB!

• SAS & SATA SSD offerings as low as 50¢ - $1/GB

68

Raw Cost/Capacity vs. Access Density

Raw Cost/Capacity – $/TB is as simplistic as classifying storage simply in terms of capacity– Remember the I/O Mantra: latency, IOPs, thruput, capacity

Consider: Access Density

AD = work ÷ capacity ÷ time• e.g., IOs (work) / GB (unit of storage) / second (unit of time)

Related: – Warehouse workloads rated in terms of Queries/Hr/TB

69

TCO & TCA: Flash is Fast—& Less Expensive than HDDs

The Accelerating Economics of Flash and the Retreat of Hard Disk Driveshttp://itblog.sandisk.com/the-accelerating-economics-of-flash-and-the-retreat-of-hard-disk-drives

15K SAS Enterprise HDD vs. 4TB Optimus Max

70

SQL Server Use Cases

70

71

SQL Server Use Cases: Single-Instance Options tempdb as a Cluster Shared Resource AlwaysOn HA using Readable Replicas AlwaysOn Multi-site HA with DR Windows Scale Out File Server (SOFS) Compression Fragmentation Flash for Development Environments

72

Entire Database on In-Server Flash

Parts of DB on In-Server Flash

(e.g., “hot files” or tempdb)

Cache data read from SAN on In-Server Flash

AlwaysOn Availability Groups ION Applianceas Shared Storage for FCI

FlashSoft®

Virtualized Database Servers for Consolidation, Upgrade, Mixed

Workloads

SQL Server 2014 Architectural Configurations

Data Warehouse Fast Track Reference Architectures (FTDWRA)

76

AlwaysOn Multi-site HA with DR

Synchronous

Asynchronous

Primary (server 1)

Local Replica

Remote Replica

Primary DC Disaster Recovery DC

Availability Group

77

Windows Scale Out File Server (SOFS) Software-defined Storage

https://en.wikipedia.org/wiki/Software-defined_storage– Software-defined storage (SDS) is evolving concept for computer data storage software to manage policy-based provisioning and management of

data storage independent of hardware. Software-defined storage definitions typically include a form of storage virtualization to separate the storage hardware from the software that manages the storage infrastructure. The software enabling a software-defined storage environment may also provide policy management for feature options such as deduplication, replication, thin provisioning, snapshots and backup.

– By consensus and early advocacy, SDS software is separate from the hardware it is managing.

Windows implementation of a virtual SAN (VSAN) aka SANless storage JBODs on steroids

– Storage Spaces– Storage Spaces Direct

Features– Redundancy– Performance– Amazingly cheap—er, ah inexpensive

Introduced in Windows Server 2012, significant enhancements in Windows Server 2016 Be aware—pay attention—we’ll be hearing more-&-more about Software-defined Storage & SOFS Windows Server 2012 Scale-Out File Server for SQL Server 2012 - Step-by-Step Installation by Jose Barreto

78

Windows Scale Out File Server (SOFS)

7979

SanDisk: Test ScenarioEnhanced AlwaysOn

Workload scenario High volume transactional workload I/O

Small block reads & writes (8K – 16K) Random reads & writes

Continuous writes to transaction log Synchronous availability mode High volume log transport across the

wire Likewise high volume for redo on

secondary

8080

Systems Under Test (SUT) 2 Lenovo ThinkServer RD650 Servers

each with:

SanDisk Fusion ioMemory

6.4TB

3.2TB

Processors

2 socket, 56 cores ~2.6Ghz Haswell-EP

Network

2x Mellanox ConnectX-3 Ethernet Adapter 56Gbs (7MB/s)

BIOS set to high performance

Fan offset increased

C-States disabled

Virtualization disabled

Load Injector (LI) 256GB DRAM 16 cores/32 LP Windows Server 2012 R2 10Gbe NIC

ID

RD650

ID

RD650

SUTs

LI

Load Drivers

SanDisk: Hardware Configuration

ID

RD650

Workload: OLTP HekatonHA: AlwaysOn

Enhanced AlwaysOn

8181

Windows Server 2012 R2 Datacenter

Windows High Performance(Powercfg -SETACTIVE SCHEME_MIN)

secpol.msc Lock pages in memory Perform volume maintenance tasks

SanDisk: OS & SQL Server ConfigurationSQL Server 2016 CTP2.2+SQL Server 2014 SP1

Server configuration options: max server memory: 128000 max worker threads: 2016 (default:

1344) AlwaysOn configuration

Availability mode: SYNCHRONOUS_COMMIT

Failover mode: AUTOMATIC Workload: TPCC Hekaton, assigned

to a Resource Governor resource pool: CREATE RESOURCE POOL

Pool_Hekaton WITH (MIN_MEMORY_PERCENT = 50, MAX_MEMORY_PERCENT = 90);

82

AlwaysOn Testing: SQL Server 2014 vs. 2016The engine & the hardware are liberated to do what they’re supposed to do Processor: ~4x CPU

More transactions: >5x XTP transactions

More flash: ~3x throughput; >4x IOPs; 0ms – 1ms latency

Source of ImprovementAlwaysOn Transport: >4x Log Bytes Received

83

AlwaysOn Log Synchronization

84

Application Performance

CPU (%) XTP Transactions/sec (x1000)

Log Bytes Received (MB/s)

Redone Bytes/sec (MB/s)

0

50

100

150

200

250

300

350

400

450

500

12 22

110 109

52

120

467

117

SQL2014 SP1

~4x CPU Utilisation

>5x XTP Transactions

>4x Log Bytes Received

85

SanDisk Flash Performance

Disk: IOPs (x1000) Disk: Thruput (MB/s) Disk: Latency (ms)0.000

100.000

200.000

300.000

400.000

500.000

600.000

700.000

800.000

900.000

1,000.000

18

200

053

873

1

SQL2014 SP1 SQL2016 CTP2.2

~3x IOPs

>4x Throughput

0ms – 1ms Latency

86

Performance PatternCorrelation: ProcessorAlwaysOn TransportXTP TransactionsFlash

87

Flash for Development Environments Consider using flash for dev environments I’ve always been a fan of providing developers with a robust

environment Providing a flash platform (& plenty of all the right resources)

encourages experimentation & innovation

88

Microsoft SQL Server 2014 Fast Track (DWFTRA)

88

89

Data Warehouse Fast Track Reference Architectures(DWFTRA)

Perfectly balanced hardware solutions (CPU, memory, storage) Official Microsoft Certification issued for each RA OEMs often provide BoMs for the solutions, significantly

simplifying ordering (e.g., Lenovo’s doc_id) Integral for our upcoming SQL Server 2005 EOS consolidation story Numerous reference architectures published publicly

90

DWFTRA (cont.)

OEM Server RackUnits

Protocol DB Size(TB)

MeasuredThruput

(Queries/Hr/TB)

Microsoft RACertification

Date OEMDocumentation

HP DL380 G8 2 PCIe 28 202 #2014-003* 9/3/2014 linkHP DL380 G8 2 PCIe 45 198 #2014-003* 9/3/2014 linkHP DL580 G8 4 PCIe 90 366 #2014-002 9/3/2014 link

Lenovo x3650 M5 2 PCIe 20 265 #2014-009 10/15/2014 linkLenovo x3850 X6 4 PCIe 55 360 #2014-023 3/10/2015 linkLenovo x3850 X6 4 PCIe 60 371 #2014-001 8/11/2014 linkLenovo x3850 X6 4 PCIe 95 433 #2014-028 5/5/2015 linkLenovo x3650 M5 2 SAS 37 192 #2014-032 7/14/2015 link

*#2014-003 reflect two certifications issued on the same day & available in the same document.

91

DWFTRA (cont.)

Internal resources– DPL: Sprocket >> MSSQL2014FTRA – EPS: Sprocket >> SQL Server > Reference Architectures and Tech Guides

Pros – These RAs are elegant solutions & the DPL strongly endorses them– DWFTRA are inexpensive & compact compared to spinning media – The Fast Track program is mature—Microsoft is currently on v.3, work on v.4 underway– The DPL collaborates closely with the Microsoft Fast Track team

Cons– Customer awareness is low– Microsoft incents sales for APS (formerly PDW) & Azure, so DWFTRA has a low profile

92

HP 28TB Data Warehouse Fast Track CertificationDWFT Certification

#2014-003 HP ProLiant DL380 Gen8 with HP PCIe LE Workload

Accelerator 28TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track

Report Date 9/3/2014

DWFT Rev. 5.4

System Provider System Name Processor Type Memory

HP DL380 G8 Intel Xeon E5-2697v2 2.7 GHz (12/24/48)

768 GB

Operating System SQL Server Edition

Windows Server 2012 R2 SQL Server 2014 Enterprise Edition

Storage Provider Storage Information

4 x 2.6TB HP PCIe LE Workload Accelerator for data and tempdb 2 x 300GB HDD for OS (RAID 1)

8 x 15K 300GB HDD for log (RAID 10)

Primary Metrics

Rated User Data Capacity1

(TB) Row Store Relative

Throughput2 Column Store Relative

Throughput3 Maximum User Data Capacity

(TB)

28 170 218 32

Row Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

170 202 4,508 5,500 5,004 96

Column Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

218 1,414 1,389 N/A N/A 98

Breakthrough Performance - 5.5GB/s throughput - 1414 queries / Hr / TB - 98% average CPU utilization

SanDisk-based 2.6TB HP Workload Accelerators

HP DL380 Server

93

Breakthrough Performance – 11.4GB/s scan rate

Lenovo 60TB SQL 2014 Data Warehouse - Ref Architecture

System x 3850 X6 Server

SanDisk-based 2400GB Lenovo io3 Flash Adapters

94

HP 45TB Data Warehouse Fast Track CertificationDWFT Certification

#2014-003 HP ProLiant DL380 Gen8 with HP PCIe LE Workload

Accelerator 45TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track

Report Date 9/3/2014

DWFT Rev. 5.4

System Provider System Name Processor Type Memory

HP DL380 G8 Intel Xeon E5-2697v2 2.7 GHz (12/24/48)

768 GB

Operating System SQL Server Edition

Windows Server 2012 R2 SQL Server 2014 Enterprise Edition

Storage Provider Storage Information

4 x 5.2TB HP PCIe LE Workload Accelerator for data and tempdb 2 x 300GB HDD for OS (RAID 1)

8 x 15K 300GB HDD for log (RAID 10)

Primary Metrics

Rated User Data Capacity1

(TB) Row Store Relative

Throughput2 Column Store Relative

Throughput3 Maximum User Data Capacity

(TB)

45 161 227 72

Row Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

161 198 4,220 5,240 4,730 97

Column Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

227 1,476 1,443 N/A N/A 99

Breakthrough Performance - 5.2GB/s throughput - 1476 queries / Hr / TB - 45TB compressed data in 2U

HP DL380 Server

SanDisk-based 5.2TB HP Workload Accelerators

95

HP 90TB Data Warehouse Fast Track CertificationDWFT Certification

#2014-002 HP ProLiant DL580 Gen8 with HP PCIe LE Workload

Accelerator 90TB reference architecture for Microsoft SQL Server 2014 Data Warehouse Fast Track

Report Date 9/3/2014

DWFT Rev. 5.4

System Provider System Name Processor Type Memory

HP DL580 G8 Intel Xeon E7-4890 v2

2.8 GHz (4/60/120) 1536 GB

Operating System SQL Server Edition

Windows Server 2012 R2 SQL Server 2014 Enterprise Edition

Storage Provider Storage Information

6 x 5.2TB HP Workload Accelerator PCIe Flash devices for data and tempdb 2 x 500GB 7200RPM HDD for OS (RAID 1)

2 x 1.3TB HP Workload Accelerator PCIe Flash devices for log (RAID 1)

Primary Metrics

Rated User Data Capacity1

(TB) Row Store Relative

Throughput2 Column Store Relative

Throughput3 Maximum User Data Capacity

(TB)

90 314 419 108

Row Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

314 366 8,351 10,139 9,245 92

Column Store

Relative Throughput2

Measured Throughput

(Queries/Hr/TB)

Measured Scan Rate Physical

(MB/Sec)

Measured Scan Rate Logical

(MB/Sec)

Measured I/O Throughput

(MB/Sec)

Measured CPU (Avg.)

(%)

419 2,721 3,392 N/A N/A 96

Breakthrough Performance - 10.1GB/s throughput - 2721 queries / Hr / TB - 90TB compressed data in 4U

HP DL580 Server

SanDisk-based 5.2TB HP Workload Accelerators

96

Hardware for Nothing, Get Your Flash for Free

96

97

SQL Server Licensing Labyrinth

98

Licensing Variables

Cores & Sockets SQL Server Edition Virtualized vs. Bare Metal Host vs. Guest Software Assurance (SA) or not Active or Passive Legacy Licensing Is your sales rep under quota & is it near end-of-quarter or better yet near end-of-fiscal

year? Which highly-trained professional sales rep you talk to

– Ask 2 people, get 3 opinions

99

Weasel Clause Licensing is complex Directions on Microsoft has a multi-day boot camp costing

thousands of dollars dedicated to Microsoft licensing– Windows & Office portion: 4 – 6 hours– SQL licensing: several more hours

Failure to respect the complexities risks leaving out so many important details that we’ll be “wrong” in some aspects

Yet the case studies which follow are eminently defensible

100

Data Center Economics

SanDisk Flash isn’t transformative merely because of performance - the economics are disruptive

5 SQL Server Licensing/Hardware scenarios– #1. Using Flash to eliminate AlwaysOn Read-only Secondaries & their

licenses– #2. The Simple Upgrade: Spare compute & budget– #3. Application-level: Scaling up to big money– #4. Data Center Scale: Literally saving millions– #5. ...

Hardware for Nothing, Get Your Flash for Free

101

#1. AlwaysOn Readable Secondaries

3 Sustained Simultaneous Workstreams Scenario⁻ 2 HP DL580 4-socket servers, 64 logical cores per server⁻ Highly transactional TPCC-like environment – 1,000,000 New Orders per Minute⁻ 4 AlwaysOn Ags in synchronous availability mode, no queuing ⁻ Database backups at 4GB/s⁻ I/O Latency: < 0.0ms, i.e., µs range

Recommended Configuration⁻ Readable Secondary used for backup stream⁻ This requires licensing all cores on both servers

Configuration on SanDisk Fusion ioMemory Flash⁻ Backup offloaded from Secondary, so the Read-Only bit can be turned off⁻ This requires only one server to be licensed

SQL Server License Savings:⁻ 32 EE licenses (64 cores)

• @$13,473.00 USD = $ 431,136 (non-SA)• @$20,209.00 USD = $ 646,688 (SA)

103

Hardware for Nothing, Get Your Flash for Free

I want my...

I want my...

I want my SSDs!

104

#2. The Simple Upgrade Case Study

104

Scenario⁻ Four SQL Server instances⁻ Windows Server 2003 & SQL Server 2005 EOL approaching⁻ Four physical servers⁻ SAN attached

Solution₋ Four SQL Server instances₋ Windows Server 2012 R2 + SQL Server 2014 ₋ Three physical servers ₋ SanDisk PCIe Fusion ioMemory Flash ₋ ...with lots of compute & storage headroom

105

#2. The Simple Upgrade (cont.)

105

Consider:⁻ Dual socket quad-core to E5-2699 v3 octo-core, 128GB RAM @$9,943.50⁻ Each 2-core SQL Server EE license @$13,472.50

• 32 logical cores per server >> 16 SQL licenses per server >> $215,560.00⁻ Fusion ioMemory SX 6.4TB card @$39,779

Solution⁻ Classic hardware upgrade:

• 4 servers, each requiring 16 SQL licenses• 4 * ($9,944 + $215,560) = $ 902,016

⁻ SanDisk PCIe Application Accelerator solution:• 3 servers , again each requiring 16 SQL licenses + 3 Fusion ioMemory cards• 3 x ($9,944 + $215,560 + $39,779) = $ 795,849

⁻ Δ: Savings of $ 106,167⁻ …with lots of compute & storage headroom⁻ …enough to buy two more servers & Fusion ioMemory cards ⁻ And a leftover SAN!

106

Hardware for Nothing, Get Your Flash for Free

I want my...

I want my...

I want my SSDs!

THE CHALLENGE• Reduce power, cooling, and footprint• Near real time reporting for 15,000

users running up to 1.2 million queries per month

• Meet failover requirements

RESULTS• 3x more concurrent users• 3x more queries• 2x greater workload• Eliminated maintenance overhead for 27

disk arrays and over 400 disks• 16:1 footprint consolidation• 1/16th power and cooling

USAF Financial Data WarehouseTEKSOUTH

Source: Case Study - TekSouth Fights US Air Force Datacenter Sprawl with ioMemory; http://www.fusionio.com/case-studies/teksouth

#3. US Air Force & TEKSOUTH

108

#3. US Air Force System Changes (cont.)

109

#3. US Air Force System Changes Summary (cont.)

Before⁻ Database Servers (20RU)

⁻ 5 x 4U servers, four quad-core AMD Opteron™ 822 SE processors @ 3.0GHz, 32GB RAM⁻ 80 cores, 48 active

⁻ Storage⁻ Hard disks: 5 x 15K RPM SAS drives, 146GB Storage (81RU)⁻ 27 x 3U disk arrays⁻ Hard disks: 15 x 15K RPM SAS drives

After⁻ Database Servers (6RU)

⁻ 3 x 2U database servers, four hex-core Intel Xeon E5-4610 processors @ 2.40GHz, 64GB RAM⁻ 72 cores, 36 active

⁻ Storage⁻ Data stored on one ioDrive2 Duo 2.4TB and one ioDrive2 1.2TB in each server

Savings: (72-48) cores ÷ (2 cores/EE license) ÷ $13,472.50/license = $161,670

Case Study: http://www.fusionio.com/case-studies/teksouth

110

Hardware for Nothing, Get Your Flash for Free

I want my...

I want my...

I want my SSDs!

111

#4. Data Center Scale: Literally saving millions

Scenario– Windows Server 2003 & SQL Server 2005 EOL approaching– SAN-attached – Core SQL Server EE licenses @$13,472.50 each– 1,500 servers

Solution– Windows Server 2012 R2 leveraging Hyper-V– SQL Server 2014 leveraging AlwaysOn– Fusion ioMemory cards– 150 servers– Hundreds of fewer servers– Thousands of fewer licenses– Millions of dollars saved– ...with lots of compute & storage headroom– And leftover SANs & shelves & fiber

112

Hardware for Nothing, Get Your Flash for Free

I want my...

I want my...

I want my SSDs!

113

#5. <your company name here>

Scenario– Windows Server 2003 & SQL Server 2005 EOS approaching– New hardware likely to accompany software upgrades – SQL Server upgrades require per core licensing– 2-Core SQL Server EE license MSRP $13,472.50 (~$6,500/core)

Solution– Contemporary multi-core servers– Windows Server 2012 R2 leveraging Hyper-V– SQL Server 2014/2016 leveraging AlwaysOn, In-Memory, Columnstore, etc.– SanDisk Flash options:

• SanDisk Fusion ioMemory cards• SanDisk front-side SSDs• SanDisk FlashSoft caching software, SanDisk ION Accelerator™ shared storage,

etc.

114

Hardware for Nothing, Get Your Flash for Free

I want my...

I want my...

I want my SSDs!

115

Hardware for Nothing, Get Your Flash for Free

Challenges:– How to handle millions of IOPs/device, GBs of I/O throughput, µsec

latency– What to do with the leftover spinning media– What to do with the leftover budget?

Scenarios such as described here aren’t unusualThey’re typical, & they’re becoming more-&-more common

I reiterate: Actual customers today are literally saving hundreds of thousands of dollars per server

Takeaways

116

Upgrade for Nothing & the Flash is Free Customer after customer: Hundreds of thousands, even millions in

savings Licensing exercise / Consolidation story EOL: Windows Server 2003 & SQL Server 2005

– Upgrade h/w– Virtualize & Consolidate

117

SanDisk PCIe Application Accelerators

117

118

Accelerate your time-to-insight

Microsoft certified SQL 2014 solutions 9.2GB/s of I/O Throughput 700% improvement in query execution

times when using Clustered Columnstore Indexes

Database consolidation made easy. 3TB- 90TB size warehouses

Leveraging Fusion ioMemory Flash

119

Fusion ioMemory™ SX350Fusion ioMemory™ PX600

Leading the Industry: New! Improved! Gen 3.5

Capacities up to 6.4TBIndustry leading reliabilityStrongest ECC using LDPC

Unprecedented performance

NEW!

Fusion ioMemoryBlade Mezzanine Cards

Capacities up to 1.6TBFrom Cisco and HP

NEW!

120

Industry Leading Performance

Previous Fusion ioMemory™

Read Performance2X

HIGHEST

350K 4K Random Read IOPS

385K 4K Random Write IOPS

2.7GB/S ReadBandwidth

2.2GB/S WriteBandwidth

15μS WriteLatency

Write Performance in the Industry

121

Customer Testimonial: Plenty of Fish

At any given time, we can have over 700,000 users active, 30,000 messages a minute, 500,000 SQL statements per second across 30 servers.

The ROI is ridiculous. These cards pay for themselves in two weeks.

—Owen Morley Director of Infrastructure Plenty of Fish

https://youtu.be/zIWgmWEsvpM (2:13)

122

Thank You!