Heart of Data Modeling Webinar: The Ticking Timebombs in Your Data Model
-
Upload
dataversity -
Category
Technology
-
view
1.023 -
download
0
Transcript of Heart of Data Modeling Webinar: The Ticking Timebombs in Your Data Model
Ticking Timebombs in Your Data Models
Thomas LaRock, Solarwinds
Karen Lopez, InfoAdvisors
Know them, fix them, prevent them
Karen López
Karen has 20+ years of data and information architecture experience on large, multi-project programs.
She is a frequent speaker on data modeling, data-driven methodologies and pattern data models.
She wants you to love your data.
A Little About Me…
Thomas has over 15 years experience in roles including programmer, developer, analyst, and DBA.
He is a frequent speaker, published author, and avid blogger on data related technologies.
He enjoys working with data, probably too much to be healthy, really.
Yes, Please do Tweet/Share today’s event
@datachick #heartdata
Use Q&A for
formal questions
Get them in now!
Use chat to discuss with each
other
We have a great community
Yes!
Slides
Recording
…next week…
POLL: Who Are You?
Aug 2014
POLL: On Call?
Aug 2014
Agenda
1.Issue
2.Background
3.Story Telling
4.What to do
AD-322-S
MichaelJSwart.com
Primary Scope: Transactional Designs
Time constraints
Designs differ depending on usage of data
AD-322-S
What’s a time bomb?
Perfectly fine design choice
Can be misused
Often leads to 3 AM on-call event
Often misunderstood
Needs monitoring and care
Ready?
Identities
Issue: How people use them
AD-322-S
25532,767
9,223,372,036,854,775,807
2,147,483,647
IDENTITY [ (seed , increment) ]
What could go wrong?
Message: Arithmetic overflow error converting IDENTITY to data type int.
Uncommittable transaction is detected at the end of the batch. The transaction is rolled back.
Identities
Issue: How people monitor them
IDENTITY [ (seed , increment) ]
AD-322-S
How do you fix it if it’s already gone off?
RESEED
GAPS
Not UNIQUE?
Change DataType
We have a script (for SQL server)!
AD-322-S
http://thomaslarock.com/2015/11/sql-server-identity-values-check/
What About SEQUENCEs?
CREATE SEQUENCE [schema_name . ] sequence_name[ AS [ built_in_integer_type | user-defined_integer_type ] ] [ START WITH <constant> ] [ INCREMENT BY <constant> ] [ { MINVALUE [ <constant> ] } | { NOMINVALUE } ]
[ { MAXVALUE [ <constant> ] } | { NOMAXVALUE } ] [ CYCLE | { NOCYCLE } ] [ { CACHE [ <constant> ] } | { NO CACHE } ] [ ; ]
How do you fix it if it’s already gone off?
RESTART
GAPS
Not UNIQUE?
Change DataType
RESTART
ALTER SEQUENCE [schema_name. ] sequence_name
[ RESTART [ WITH <constant> ] ]
[ INCREMENT BY <constant> ]
[ { MINVALUE <constant> } | { NO MINVALUE } ] [ { MAXVALUE <constant> } | { NO MAXVALUE } ]
[ CYCLE | { NO CYCLE } ] [ { CACHE [ <constant> ] } | { NO CACHE } ]
[ ; ]
AD-322-S
How do you prevent it?
Proper Design
Do the math
SEED Value
Increment Value
Use another feature
Monitor
Max allowable value
Max current value
Gaps
Predict* countdown timer
Team Data Script
Know your hottest tables
Prioritize
Datatypes
• Issue: Are you real-world proof?
AD-322-S
CREATE TABLE dbo.customer(CustomerID int PRIMARY KEY NOT NULL,
EmailAddress varchar(25) NULL)
What could go wrong?
Wrong datatype
ZIPCode
López
Account Number
Wrong Length/Precision
karenmariadeguadalupeannlopezymartinez@torontolab.contractor.restricted.reallylongdomainnameherereal
lyreally.com
How do you fix it if it’s already gone off?
ALTER Column
Extended Alter
How do you prevent it?
Proper Design
Do the math
Do research
User Engagement
Go long
Data Profiling
Legacy data
Statistical analysis
Edge cases
Get out a little
DeNormalization
Issue: Denormalizations blow up
•Repeating columns instead of child table
•“Manually” Calculated data
•Roll up, Roll down
AD-322-S
Simple Denormalization
• CREATE TABLE customer(CustomerID int,LastName varchar(255),FirstName varchar(255),EmailAddress varchar(50),
• EmailAddressTwo varchar(255),
• EmailAddressThree varchar(255));
What could go wrong?
SELECT *
Issue: How people use them
• SELECT * FROM ….
AD-322-S
What could go wrong?
SELECT *
Issue: How people monitor them
GovernanceReviewsScriptsTools
How do you fix it if it’s already gone off?
Fix the source table
Fix the query
Fix the target table
Throw out your ORM
Do you have a McGuyver Moment?
Defuse a bomb at 3AM?
Your Time Bombs
What went wrong? How did you fix/prevent it?
Other Time Bombs
• Collations
• Running out of storage space
• Bad Backups (ergo, no restores)
• Mutually dependent tables
• Using deprecated features in new designs
• Accepting a “just for now server”
• Triggers Firing
• Real world scaling (Dev vs. Prod)
• Linked Servers & Nested Views
• Bucket Columns (Bad DEV!)
• VM Snapshots
• Wrong SQL Server License…or “over installed”
• Installing external script without understanding license.
• Server Settings (whole other hour!)
• Faux NULLs
• GenerationNEXT installation process
• Multi-tenant DB Design
• Clustering on Unique Key
• GUIDs when SEQUENCES might do
• SELECT *
• Use of ORMs (generated DML)
Finally…
These are perfectly fine design decisions, but..
• Fully implemented
• Monitored and managed
• Fully understood
• Emergency response plans needed
• Sometimes a paper clip and a stick of gum is not enough (Sorry McGuyver)
AD-322-S
Thank you, you were great.
Let’s do this next month!
Karen Lopez @datachick
#heartdata
Resources
IDENTITY Script:
http://thomaslarock.com/2015/11/sql-server-identity-values-check/
• Karen’s Blog: www.datamodel.com
• Tom’s Blog: www.thomaslarock.com
Thank you!
You were GREAT!
….We should do this again….