Session 7-24HOP DmitriKorotkevitch The Ambiguous Case...
Transcript of Session 7-24HOP DmitriKorotkevitch The Ambiguous Case...
![Page 1: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/1.jpg)
The Ambiguous Case of Off-Row Storage in In-Memory OLTPDmitri Korotkevitch, aboutsqlserver.comModerated By: Sander Stad
![Page 2: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/2.jpg)
Technical Assistance
If you require assistance
during the session, type
your inquiry into the
question pane on the
right side.
Maximize your screen
with the zoom button
on the top of the
presentation window.
Please fill in the short
evaluation following
the session. It will
appear in your web
browser
![Page 3: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/3.jpg)
3
Thank You
Empower users with new
insights through familiar tools while balancing the
need for IT to monitor and manage user created
content. Deliver access to all data types across structured
and unstructured sources.
attunity.com
Attunity, a leader in data
integration and management software, helps move,
transform and analyze data efficiently in SQL
Server/Azure environments.
idera.commicrosoft.com
IDERA’s award-winning
SQL Server database solutions and multi-platform
database, application and cloud monitoring tools
ensure your business never slows down.
![Page 4: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/4.jpg)
Access to online training and content
Enjoy discounted event rates
Join Local Chapters and Virtual Chapters
Get advance notice of member exclusives
PASS is a not-for-profit organization which offers
year-round learning opportunities to data
professionals
Membership is free, join todayat www.sqlpass.org
JOINPASS
![Page 5: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/5.jpg)
Save on PASS Summit 2016 Registration! • The world’s largest gathering of SQL Server & BI professionals• Learn from the world’s top data experts, in over 190 technical sessions• More than 4000 attendees from all over the world• Meet the Microsoft engineering team!
Save $200 right now using discount code 24HOP200!
$2,195until September 18, 2016
www.passsummit.com
![Page 6: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/6.jpg)
BIO
20+ Years in IT
15+ Years working with SQL Server
Microsoft Data Platform MVP
Microsoft Certified Master
Author:� Pro SQL Server Internals
� Expert SQL Server In-Memory OLTP
http://aboutsqlserver.com
@aboutsqlserver
![Page 7: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/7.jpg)
The Ambiguous Case of Off-Row Storage in In-Memory OLTP
Dmitri Korotkevitch, aboutsqlserver.com
![Page 8: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/8.jpg)
In-Memory OLTP…
![Page 9: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/9.jpg)
In-Memory OLTP…
![Page 10: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/10.jpg)
Agenda
Hash and nonclustered (range) indexes overview
LOB and row-overflow storage in In-Memory OLTP
Columnstore indexes on memory-optimized tables
![Page 11: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/11.jpg)
In-Memory Data Storage
create table dbo.Community
(
Name nvarchar(64) not null
constraint IDX_NAME primary key nonclustered hash
with (bucket_count=1024),
City nvarchar(32) not null,
FirstName nvarchar(64) not null,
index IDX_CITY nonclustered hash(City)
with (bucket_count=128),
)
with (memory_optimized=on, durability=schema_and_data);
![Page 12: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/12.jpg)
A
B
C
D
E
V
A
B
C
M
N
O
S
T
10, ∞Andreas Wolter
Cologne
51, ∞VictorIsakov
Sydney
83, ∞Denny Cherry
Oceanside
61, ∞DmitriKorotkevitch
Tampa
94, ∞DmitriPilugin
Moscow
Hash Indexes
41, ∞Andy Warren
Orlando
13, 60DmitriKorotkevitch
Novosibirsk
![Page 13: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/13.jpg)
Range Index
alter table dbo.Community
add index IDX_FIRSTNAME
nonclustered(FirstName);
![Page 14: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/14.jpg)
Range Index0
1
2
3
4
5
N
41, ∞AndyWarren
Orlando
Andreas1
94, ∞ Dmitri Pilugin Moscow
83, ∞DennyCherry
Oceanside
61, ∞Dmitri Korotkevitch
Tampa
13, 60Dmitri Korotkevitch
Novosibirsk
Dmitri2
Zack3
Adam Andy Andreas Boris Denny Dmitri
10, ∞AndreasWolter
Cologne
![Page 15: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/15.jpg)
15
Hash vs. Range Indexes
15
Hash Indexes Range Indexes
� Useful only with point-lookups and equality joins
• SQL Server needs to calculate the hash of the key
• All columns from the composite keys must be present
� Faster than nonclustered (range) indexes when
bucket_count is correct
• Ideal bucket_count is 1.5-2 times of the # of unique index keys.
� Good choice for catalog entities
� Similar to on-disk B-Tree indexes
• Unidirectional
� Good choice for the range scans
� Should be used when correct bucket_count estimation is
impossible
![Page 16: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/16.jpg)
Demo: bucket_count and performance
16
![Page 17: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/17.jpg)
On-Disk Allocation Units
IN-ROW: Main data row structure and all fixed-length data types
LOB: Variable-length data• (n)varchar(max); varbinary(max) when size > 8,000 bytes
ROW-OVERFLOW: Variable-length data that does not fit IN-ROW• (n)varchar(N); varbinary(N)
• (n)varchar(max); varbinary(max) when size <= 8,000 bytes
Decision is made based on the row/column data
![Page 18: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/18.jpg)
In-Memory OLTP Memory Consumers
In-Memory OLTP objects allocate memory from separate VARHEAPs• Data rows
• Hash tables in Hash indexes
• Range indexes
![Page 19: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/19.jpg)
A
B
C
D
E
0
1
2
3
4
583, ∞Denny Cherry
Oceanside
61, ∞DmitriKorotkevitch
Tampa
94, ∞DmitriPilugin
Moscow
Memory Consumers
13, 60DmitriKorotkevitch
Novosibirsk
Andreas1
Dmitri2
Zack3
Boris Denny Dmitri
Adam Andy Andreas
Table Heap
Hash Index
Heap
Range
Index Heap
![Page 20: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/20.jpg)
Demo: Memory Consumers
20
![Page 21: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/21.jpg)
51,∞
Victor Isakov
Sydney
Off-Row Storage
61,∞
DmitriKorotkevitch
Tampa100 500 101 501
101100
51,
∞100 Victor’s BIO
61,
∞101 Dmitri’s BIO
501500
51,
∞500 Ptr
61,
∞101 Ptr
Victor’s Picture
Dmitri’s Picture
Row-Overflow Data
LOB Data
Table Table
Heap
Range
Index Heap
Range Index
Heap
Table
Heap
LOB
Allocator
In-Row Data
![Page 22: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/22.jpg)
51,∞
Victor Isakov
Sydney
Off-Row Storage
61,∞
DmitriKorotkevitch
Tampa100 500 101 501
101100
51,
∞100 Victor’s BIO
61,
∞101 Dmitri’s BIO
501500
51,
∞500 Ptr
61,
∞101 Ptr
Victor’s Picture
Dmitri’s Picture
95,∞
DmitriKorotkevitch
Seattle 101 501
99,
∞100
Even More
Achievements
61,94
51,
98
![Page 23: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/23.jpg)
Demo: Off-Row Columns and Performance
23
![Page 24: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/24.jpg)
Off-Row Columns
Stored in the separate internal tables• Referenced through artificial internal keys
• Have separate lifetime from in-row data
• Decided at CREATE TABLE phase based on data type/column size
Add significant overhead • 50+ extra bytes per each value
• Slower INSERT/DELETE operations
• Slower SELECT: Not covered by In-Row indexes
![Page 25: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/25.jpg)
Clustered Columnstore Indexes
Memory-Optimized Data Rows
Columnstore Data
Delta Store
Range Index Range Index
and Table
Heaps
Table Heap Table Heap
HkCS
Allocator
Delete
Bitmap
![Page 26: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/26.jpg)
Demo: Columnstore Indexes
26
![Page 27: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/27.jpg)
Columnstore Indexes
Delta Store is another Table Heap with memory-optimized rows• Tuple mover compresses the rows and moves them to the main Table Heap
when it estimates ~>1M rows in delta store
• COMPRESSION_DELAY index option delays row compression
Delete Bitmap is the range index
Large Delta Stores and Delete Bitmaps negatively affect performance
Tables with columnstore indexes do not support off-row storage
![Page 28: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/28.jpg)
Takeaways
Do not use HASH indexes unless you can correctly estimate and maintain the bucket_count
Avoid off-row storage in In-Memory OLTP• Defining columns as varchar(max) just because is clearly the bad idea
• Performance critical cases can benefit from multi-column design
Control the size of columnstore index delta store and delete bitmap with COMPRESSION_DELAY option
![Page 29: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/29.jpg)
We covered
Hash and nonclustered (range) indexes
Off-row data storage in In-Memory OLTP
Columnstore indexes in In-Memory OLTP
Email: [email protected]
Blog: http://aboutsqlserver.com
![Page 30: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/30.jpg)
Questions?
![Page 31: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/31.jpg)
Coming up next!
Understanding and Monitoring TempdbVicky Harp
![Page 32: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/32.jpg)
Thank You for Attending
Follow @pass24hop
Share your thoughts with #pass24hop & #sqlpass
![Page 33: Session 7-24HOP DmitriKorotkevitch The Ambiguous Case …aboutsqlserver.com/blog/sqlsat/24HOP/2016/24hop2016.pdf · The Ambiguous Case of Off-Row Storage in In- ... Wolter Cologne.](https://reader031.fdocuments.us/reader031/viewer/2022030802/5b0b40777f8b9a99488d8615/html5/thumbnails/33.jpg)