VLDBs: Lessons Learned Bob Pusateri

Database Administrator Northwestern University

Thanks to our sponsors

2

these companies made this event possible

visit them with your raffle tickets

Connect with the Community

3

Event staff, volunteers and speakers are here to help and answer questions. Scan the QR code on their badges to connect and network with them.

Feedback!

4

We like it

We love it

We want more of it!

About Me

@SQLBob

What is “Very Large”?

• 100GB?

• 500GB?

• 1TB?

http://tinyurl.com/9cv8y6z

What is “Very Large”?

• Larger than an Excel spreadsheet can hold

• Large enough that your standard DBA tasks need to change.

Why are we here?

• Amount of data we have is ever-growing

• VLDB skills are in demand

• Best Practices Good Ideas I’ve gathered over 3.5 years of working with VLDBs

https://flic.kr/p/26JrAU

Instance Layout

Databases give…

• Consistency

• Backup/Restore/Recovery

• Security

• Configurability

Backups

Backup Tuning

• Timing

• Backup Compression

• Multiple Backup Files

• BUFFERCOUNT

• MAXTRANSFERSIZE

https://flic.kr/p/gyRmUR

SAN Snapshots

• Instantaneous!

• Unsafe!

.mdf

.ldf

.mdf (snapshot)

.ldf (snapshot)

SAN Snapshots

IT’S NOT A BACKUP UNTIL IT’S ON

SEPARATE STORAGE!

IO Errors

823 – OS I/O operation failed (4 retries) 824 – OS I/O operation succeeded SQL Server determined data is corrupt (4 retries) 825 – An 823/824 error occurred BUT it succeeded on one of the retries AND no error gets raised

Integrity Checks

• These are slow on VLDBs too!

• Running them elsewhere can help • Replicas can be part of a solution • Still need to check physical consistency on primary

Integrity Checks

• Primary Server • DBCC CHECKDB (DB_NAME) WITH PHYSICAL_ONLY;

• Replica Server

•DBCC CHECKDB (DB_NAME);

Filegroups

• More than 1 = a great idea!

• Why? • Tier data by performance • Backup performance • Restore performance • Partial Restores • Read-Only Filegroups*

https://flic.kr/p/2UcCH

Filegroups

Easy!

Not!

• Enforcing “correct” filegroup usage

• Moving data between filegroups • Clustered/Nonclustered Indexes • Heaps • LOB Data

Create new table on new FG

Copy data

Copy indexes/permissions/triggers/FKs/etc

Swap Names

Drop old table

Drive Letters & Mount Points

• More <> better performance

• Way simpler ways to fix most performance issues

• Investigate other problems first

High(er) Availability

• Disk = $$$

• Mirroring/Replicas/Log Shipping all duplicate data

• Failover cluster instances don’t

Principal Mirror Active Node Passive Node

Restores

• Partial Restores can be your friend!

• Do you need the entire DB right away?

https://flic.kr/p/cAgEYo

Piecemeal

Partitioning

• Great for loads

• Otherwise try partitioned views

• Really only good if you are always filtering on partition

key

• May end up being more work to merge results together

from different partitions

Where It Occurs

DISK MEMORY ACCESS METHODS

USER

COMPRESSED DECOMPRESSED

Compression

Compression

• Compresses data on disk & in memory

• 2 flavors: Row and Page

http://flic.kr/p/6hSai9

Statistics

• Bigger tables = less frequent updates

• TF 2371 can fix this

• Or run stats more often

• Is it really improving your performance?

Parallelism

• Are you getting all the CPU you’ve paid for?

• Scalar UDFs kill parallelism

• See Paul White’s post on sqlblog.com for others:

http://tinyurl.com/PaulWhiteParallelExecution

Security

• Roles

• Event Notifications

• Auditing

Read-Only Mode

• No concurrency issues!

• Statistics stored in TempDB

• Disruptive to switch between RO & RW

Questions?

Email: Bob@BobPusateri.com Twitter: @SQLBob Resources: www.bobpusateri.com I’m here to help!

Thank You

PASS • www.sqlpass.org

Oregon SQL • www.OregonSQL.org

Oregon SQL LinkedIn • www.linkedin.com/groups?gid=1800923

12/11/2014 | 31 |