Presented by:Jeff Garbus

jeff@soaringeagle.biz

Advanced SQL Programming and Optimization

Case studies in Re-writing SQL Code

About Soaring Eagle

Since 1997, Soaring Eagle Consulting has been helping enterprise clients improve their overall system performance at the database tier, arguably the most volatile and critical component of distributed application architecture. Our clients range in size from fledgling startups through Fortune 100 companies and leading financial institutions.

Soaring Eagle has been a leader in software development, architecture, performance and tuning databases, while promoting mentoring and training all over the world for over a decade. Many of our employees, and partners have written books, speak at seminars about leading edge technologies. We have expertise in all business tiers, financial; health, manufacturing, government agencies and many ecommerce businesses.

Consulting• Performance & Tuning• Data Performance

Management• Emergency Triage• Performance &

Security Audits• Staff Augmentation• Project management• Database

architecture• Scalability

assessment and planning

Training• Onsite/Web based

• Microsoft• Sybase• Oracle• APM• Six Sigma

Software• Application

Performance Management

• Database performance accelerator

• Database performance management

• Database Security

Managed Services

• Remote Database Management

• Performance management

• Emergency db Service

• Proactive mitigation• Problem notification • Problem resolution

9

Microsoft SQL server, SQL EM, Query Analyzer are all trademarks of Microsoft Inc.This presentation is copyrighted. This presentation is not for re-saleThis presentation shall not be used or modified without express written consent of Soaring Eagle Consulting, Inc.

Acknowledgements

Page 6 - 3

Why other people’s code stinks

• Poor performance• Poor documentation• Inadherence to standards

• From a real-life problem• Your task is to migrate from a source

database to a new target, which has incremental values on the cost sheet detail information.

• Your task is to make fun of the first two options, and to understand the third

Case Study #1: Very Bad, Bad, Not so Bad

CREATE TABLE #WKCostSheet(      WKCostSheetHeaderID int NOT NULL,      Sequence int identity,      ItemDescription varchar(50) NULL,      Value money NULL,      GiftInKind money NULL,      KWNAmount money NULL,      CheckNumber varchar(50) NULL)

Very Bad

CREATE TABLE #WKCostSheet2(      WKCostSheetHeaderID int NOT NULL,      Sequence int,      ItemDescription varchar(50) NULL,      Value money NULL,      GiftInKind money NULL,      KWNAmount money NULL,      CheckNumber varchar(50) NULL)

  

Very Bad (continued)

INSERT INTO #WKCostSheet           (WKCostSheetHeaderID           ,ItemDescription           ,Value           ,GiftInKind           ,KWNAmount           ,CheckNumber          )

Very Bad (cont’d)

SELECT WKCostSheetHeaderID, exp ,val ,gik ,kwn ,chkNum  FROM

KWN_Access_Admin.dbo.CostSheet c join   KWN_dev.dbo.WKCostSheetHeader h

on  h.WKID = c.KID andh.WKCostSheetTypeID=  c.WishNum

    where KID in (select WKID from dbo.WKs)

    order by WKCostSheetHeaderID

 

Very Bad (cont’d)

insert #WKCostSheet2 select * from #WKCostSheet

create clustered index a on #WKCostSheet2 (WKCostSheetHeaderID,

Sequence)

Very Bad (cont’d)

while exists (select * from #WKCostSheet2)begin      INSERT INTO KWN_dev.dbo.WKCostSheet           (WKCostSheetHeaderID           ,Sequence           ,ItemDescription           ,Value           ,GiftInKind           ,KWNAmount           ,CheckNumber          )

Very Bad (cont’d)

select * from #WKCostSheet2 where

WKCostSheetHeaderID = (select MIN (WKCostSheetHeaderID)

from #WKCostSheet2)

 delete from #WKCostSheet2 where WKCostSheetHeaderID =

    (select MIN (WKCostSheetHeaderID) from #WKCostSheet2)

update #WKCostSheet2 set Sequence = Sequence –

(select MIN ( sequence) from #WKCostSheet2 ) + 1 end -- While

Very Bad (cont’d)

• Multiple temp tables• Enormous amount of IO from second temp

table, heavily due to the large quantity of deletions & multiple passes through the temp table

Very Bad – summary

CREATE TABLE #WKCostSheet(      WKCostSheetHeaderID int NOT NULL,      Sequence int identity,      ItemDescription varchar(50) NULL,      Value money NULL,      GiftInKind money NULL,      KWNAmount money NULL,      CheckNumber varchar(50) NULL)

Less bad

INSERT INTO #WKCostSheet          /* (SAME) */SELECT WKCostSheetHeaderID               /* (SAME) */

create clustered index a on #WKCostSheet(WKCostSheetHeaderID, Sequence)

Less bad (cont’d)

INSERT INTO WKCostSheet           (WKCostSheetHeaderID           ,Sequence           ,ItemDescription           ,Value           ,GiftInKind           ,KWNAmount           ,CheckNumber          )/* Continued */

Less bad (cont’d)

Less bad (cont’d)

select a.WKCostSheetHeaderID           ,a.Sequence - b.minseq + 1           ,a.ItemDescription ,a.Value ,a.GiftInKind           ,a.KWNAmount ,a.CheckNumber

           from #WKCostSheet a join                      (select c.WKCostSheetHeaderID,

MIN (c.sequence) as minseq from #WKCostSheet c group by WKCostSheetHeaderID) as b

on a.WKCostSheetHeaderID = b.WKCostSheetHeaderID

• Still need a temp table• Join is far less costly than the repeated

deletions

Less bad – summary

SELECT WKCostSheetHeaderID,  row_number() over

(partition by WKCostSheetHeaderID order by val)

      ,exp ,val ,gik ,kwn ,chkNum       FROM KWN_Access_Admin.dbo.CostSheet c join   KWN_dev.dbo.WKCostSheetHeader h on  h.WKID =

c.KID and h.WKCostSheetTypeID=  c.WishNum    where KID in (select WKID from dbo.WKs)    order by WKCostSheetHeaderID

Not so bad

• No temp table, no join, no trouble

Not so bad – summary

• We’re fans of data-driven design; anything that keeps us from having to push code back through QA is a good thing. But, we’re going to make fun of the code that accesses the hierarchical data (note: It was written in SQL Server 2005, prior to hierarchy IDs being available), starting with naming conventions. The below character string is a table name.

• [_SynComs.Orders.OrderItem.product->SynComs.Products.PrinterCartridge]• Nontrivial to type, contains special characters… not a lot right with this.• The interesting thing, from their perspective, is that the same query is used for every

single database call. That’s right, one query only for every access. The catch is, there’s an unlimited number of recursive calls to get the database results, and the structure was set up to put real (data) information into the physical schema, a nifty way to create extra contention in the system tables.

• For the record, the CTE changed approach brought query time from 9.5 seconds down to .23 seconds.

Case Study #2:From horrid code to CTE

SELECT0 as generation, major_id as tableId

into #tblguidFROM

sys.extended_properties WHERE (value in

( 'SynComs.Orders.Order, SynComs' )

and name = 'ParentType') 

create clustered index CItablID on #tblguid(tableId)

Original Code

declare @generation intselect @generation=0

while (1=1) beginselect @generation=@generation+1insert into #tblguid (generation, tableId)SELECT @generation, parent.major_idFROM#tblguid tbl JOIN sys.extended_properties child on tbl.tableId = child.major_id and child.name = 'ChildType' and generation = @generation -1JOIN sys.extended_properties parent on child.value = parent.value and parent.name = 'ParentType'

Original Code (cont’d)

where not exists(select * from #tblguid lookitup

where parent.major_id = lookitup.tableId)

if (@@ROWCOUNT=0) breakendselect

name as tableName from

sys.tables join #tblguid on object_id = tableId

Original Code (cont’d)

/* Do you like this? We’re about to recursively

create / execute a large view… good candidate for rewrite / approach change

 */ 

Original Code (cont’d)

declare @string varchar(max)select @string = '

create view my_view asselect * from

[_SynComs.Orders.Order.billingAddress->SynComs.Customers.CustomerAddress]

union all /* At least it’s “union all” here */ select * from [_SynComs.Orders.Order.discounts-

>SynComs.Orders.Discounts.Discount]/* … for brevity, I’ve removed about 12 more of these */exec (@string)go  

Original Code (cont’d)

select 0 as generation, parentObjectGuid, childObjectGuid, fieldName, parentType, childType, _guid_, _pk_ into

#guidsfrom

my_view where

parentObjectGuid IN ('3ee588d1-2096-4ddb-adc6-

d5a140725721',/* about 70 more removed */);

 

Original Code (cont’d)

update @guids28927 set generation=0 create clustered index CI_GUID on #guids (_guid_)create nonclustered index NCI_childobject_generation on #guids (generation,childObjectGuid,parentObjectGuid) declare @generation intselect @generation=0

Original Code (cont’d)

while (1=1) begin

select @generation=@generation+1insert into #guids

(generation, parentObjectGuid, childObjectGuid, fieldName, parentType, childType, _guid_, _pk_)

select @generation, parentObjectGuid, childObjectGuid, fieldName, parentType, childType, _guid_, _pk_

from my_viewwhere parentObjectGuid in (select childObjectGuid from #guids

where generation=(@generation-1))andnot exists (select * from #guids where my_view._guid_ =#guids._guid_ )

Original Code (cont’d)

if (@@ROWCOUNT=0) breakend Select parentObjectGuid, childObjectGuid, fieldName,

parentType, childType, _guid_, _pk_ from #guids

Original Code (cont’d)

WITH RecursionRelationship (generation, parentObjectGuid, childObjectGuid,fieldName, parentType,

childType, [_guid_], [_pk_] )AS(-- Anchor Query

select 0 as generation, parentObjectGuid, childObjectGuid, fieldName,parentType, childType, _guid_, _pk_ from

dbo.ObjectRelationshipwhere

parentObjectGuid IN ('3ee588d1-2096-4ddb-adc6-d5a140725721', /* same list as above

*/)

Revised Code

UNION ALL-- Recursion QuerySelect r_r.generation +1, o_r.parentObjectGuid, o_r.childObjectGuid,

o_r.fieldName, o_r.parentType, o_r.childType, o_r._guid_, o_r._pk_from dbo.ObjectRelationship o_r JOIN RecursionRelationship r_r on

o_r.parentObjectGuid = r_r.childObjectGuid) select parentObjectGuid, childObjectGuid, fieldName, parentType,

childType, _guid_, _pk_ from

RecursionRelationship option (maxrecursion 32767)

Revised Code (cont’d)

Jeff Garbus – Email me for a copy! jeff@soaringeagle.guru813.641.3434mssqlperformance.blogspot.comhttp://www.youtube.com/user/soaringeagledba/

Microsoft Transact - SQL, The Definitive Guide

– 35% off from jblearning.com. Use code "GARBUS"

Upcoming WebinarsCheck our web site: www.soaringeagle.guru

Find us on Social Media @SoaringEagleDBALike us on Facebook: Facebook.com/SoaringEagleDBA

Thank You! – Questions?

33 - 60 © Soaring Eagle Consulting 8/19/2013

Soaring Eagle Flight Center FREE! For 3 months

• Why Try Soaring Eagle Flight Center?• See your environment health in seconds• The hundreds or thousand email alerts are

corralled into alerts that you control• Flight provides Predictive Analysis Tools

Email sales@soaringeagle.guru to take advantage

Purchase SQL Diagnostic Manager and get SQL Doctor FREE!

Limited Time Offer!

Try Any IDERA Products Free for 14-Days!