SQL Server High Availability and Disaster Recovery for SAP Deployment at QR: A Technical Case Study

8/10/2019 SQL Server High Availability and Disaster Recovery for SAP Deployment at QR: A Technical Case Study

1/28

SQL Server High Availability and Disaster Recovery forSAP Deployment at QR: A Technical Case Stdy

Writer:Danny Tambs (Microsoft)

Contributor:Sanjay Mishra (Microsoft)

Technical Reviewers: Prem Mehra (Microsoft), Juergen Thomas (Microsoft), Tim Robinson

(QR), Chris Lubey (QR)

Published:Se!tember "#$#

Applies to:SQL Ser%er "##&

Summary:This !a!er escribes the high a%ai'abi'ity an isaster reco%ery so'ution

im!'emente at QR to su!!ort the mission critica' SP a!!'ication

Note: This document is current as of its most recent publish date of September 15, 2010. Besure to check www.sqlcat.com to obtain the most recent ersion of this paper.


2/28

Co!yright

The information containe in this ocument re!resents the current %ie* of Microsoft Cor!oration

on the issues iscusse as of the ate of !ub'ication +ecause Microsoft must res!on to

changing maret conitions, it shou' not be inter!rete to be a commitment on the !art of

Microsoft, an Microsoft cannot guarantee the accuracy of any information !resente after the

ate of !ub'ication

This *hite !a!er is for informationa' !ur!oses on'y M-CR.S./T M01S 2. 3RR2T-1S,

14PR1SS, -MPL-1D, .R STT5T.R6, S T. T71 -2/.RMT-.2 -2 T7-S D.C5M12T

Com!'ying *ith a'' a!!'icab'e co!yright 'a*s is the res!onsibi'ity of the user 3ithout 'imiting the

rights uner co!yright, no !art of this ocument may be re!rouce, store in, or introuce into

a retrie%a' system, or transmitte in any form or by any means (e'ectronic, mechanica',

!hotoco!ying, recoring, or other*ise), or for any !ur!ose, *ithout the e8!ress *ritten

!ermission of Microsoft Cor!oration

Microsoft may ha%e !atents, !atent a!!'ications, traemars, co!yrights, or other inte''ectua'

!ro!erty rights co%ering subject matter in this ocument 18ce!t as e8!ress'y !ro%ie in any

*ritten 'icense agreement from Microsoft, the furnishing of this ocument oes not gi%e you any

'icense to these !atents, traemars, co!yrights, or other inte''ectua' !ro!erty

5n'ess other*ise note, the e8am!'e com!anies, organi9ations, !roucts, omain names, e:

mai' aresses, 'ogos, !eo!'e, !'aces, an e%ents e!icte herein are fictitious, an no

association *ith any rea' com!any, organi9ation, !rouct, omain name, e:mai' aress, 'ogo,

!erson, !'ace, or e%ent is intene or shou' be inferre

; "#$# Microsoft Cor!oration '' rights reser%e

"


3/28

ContentsSQL Ser%er 7igh %ai'abi'ity an Disaster Reco%ery for SP< Technica' Case Stuy$

-ntrouction&

uience &

So'ution Re=uirements &

>oa's for the So'ution ?

Reco%ery Point .bjecti%e ?

Reco%ery Time .bjecti%e ?

Site Disaster Reco%ery?

Reuction in Mainframe Su!!ort Costs@

Performance@

7igh:Le%e' Resu'ts Post Migration@

Performance Measure Post:Migration@

Su!!ortabi'ity an Tuning A

/'e8ibi'ityA

Database Si9eA

Dea'ing *ith a Site /ai'o%erA

Running the So'ution on .ne 2oe B

SP Lansca!e BPremigration Legacy -m!'ementation$#

2e* Design 3ino*s Ser%er >eo:C'ustering an SQL Ser%er$#

Mu'ti!'e Data Centers$$

Logica' C'uster Design Layout$"

S2 Data Re!'ication$

>eo:C'uster -nstance /ai'o%er Time $?

So'ution /ai'ure Moes Sustainab'e$?

Loss of 2et*or Lins$@Loss of a Site or Database C'uster 2oe$A

Logica' Design of SP ?C on SQL Ser%er "##& $A

Logica' Design of SP 1RP? on SQL Ser%er "##&$B

Design 1'ements"#

C'ustering S!ecifics"#

E


4/28

Database Ser%er Configuration""

S2 Storage ""

Post Migration >o:Li%e Practices"E

Monitoring SQL Ser%er an SP Performance"E

SQL Ser%er Monitoring %ia System Monitor"E

Monitoring 5sing SP Transactions"&

ST#E "&

ST# or D+ Coc!it "?

D+ Coc!it "?

Security an 7arening"?

!!'ying 5!ates to 3ino*s Ser%er an SQL Ser%er"@

Conc'usion "A

!!eni8< 7ar*are an Soft*are 7igh Le%e' +i'' .f Materia's"A


5/28

!ntrodctionQueens'an Rai' (QR) has been running trains in the ustra'ian state of Queens'an since the

$A?#s Queens'an Rai' unertoo a migration from a 9F.S mainframe running D+" to host

SP 1ar'y %ersions of SP for e8am!'e RF" on'y ran on mainframesG ho*e%er, ne*er %ersionsof SP RFE run on a number of ifferent har*are an soft*are !'atforms, !ro%iing more

o!tions for SP customers QR engage Microsoft Ser%ices to assist in migrating the atabase

!'atform uner!inning SP to MicrosoftH SQL Ser%erH "##& by June "##@ This timeframe

*ou' enab'e QR to u!grae SP to 1RP? running on SQL Ser%er "##& running in a high'y

a%ai'ab'e configuration by "##A The change of !'atform to the 3ino*s Ser%erH an SQL

Ser%er !'atforms *ou' !ro%ie a much more cost:effecti%e tota' cost of o*nershi!, as *e'' as

the high a%ai'abi'ity bet*een ata centers that *as a re=uirement for QR -m!ro%e

!erformance hearoom *as a'so critica', because the 'oa on the system *as e8!ecte to gro*

QR stresse that achie%ing ma8imum a%ai'abi'ity an near:immeiate reco%ery from a isaster

*as of highest im!ortance for QRIs o!erations QRIs esign goa's *ere to e'iminate a'' sing'e!oints of fai'ure in the architecture, an no*, *ith *e'' thought:out !roceures an !ractices, QR

has BBBB8 !ercent u!time, inc'uing both !'anne an un!'anne o*ntime

This !a!er !ro%ies insight into the etai's of the high'y a%ai'ab'e architecture Microsoft

esigne to host a mission:critica' SP so'ution

AdienceThis !a!er is intene for architects an so'ution integration s!ecia'ists *ho *ant to 'earn more

about SQL Ser%er high a%ai'abi'ity esign !atterns that customers use in !rouction to host

mission:critica' a!!'ication en%ironments The focus of this !a!er is to !ro%ie insight into higha%ai'abi'ity for a SP im!'ementation !rojectG ho*e%er, the best !ractices !resente here can be

reuse to satisfy other atabase high a%ai'abi'ity !roject re=uirements, not just those that in%o'%e

SP

Soltion Re"irementsSP is a mission:critica' a!!'ication *ithin QR, secon in critica'ity on'y to the systems that

contro' signa'ing an junction bo8es for the rai' trac in the fie' -n the !remigration

configuration, SP a!!'ication ser%ers running %ersion ?C *ere connecte to D+" %@$ for

9F.S %er $ on the mainframe The mainframe running the atabase an su!!orting the SP

system *as acti%e in on'y one ata center The isaster reco%ery (DR) site, 'ocate se%era'

i'ometers a*ay, containe on'y SP ser%ers an a DR mainframe in a co' state -f the main

ata center containing the atabase *ere to fai', the entire SP so'ution *ou' be off'ine unti'

the co' mainframe cou' be acti%ate (starte) an a!!ro!riate net*or changes mae in orer

to simu'ate an en%ironment in *hich the DR mainframe 'ooe 'ie the main ata center

mainframe This startu! !roceure *as 'abor:intensi%e, an it too a!!ro8imate'y four hours

*hen the a!!ro!riate staff members *ere a%ai'ab'e onsite The QR team neee to minimi9e

&


6/28

any unscheu'e o*ntime, as *e'' as im!ro%e !erformance significant'y in orer to co!e *ith

the increase atabase emans im!ose by the u!coming u!grae to SP 1RP ? The

mainframe ha many scheu'e outages (on Sunay nights) that a'so cause SP outages

SP a'so s!ecifie that su!!ort for SP +P an Ja%a *or !rocesses *ou' en *ith SP

2et3ea%er "##s on the 9F.S !'atform /or more information about SP su!!ort for o!erating

system !'atforms, see the SP !rouct a%ai'abi'ity matri8, *hich is a%ai'ab'e from SP

(htt!


7/28

en%ironment 7o*e%er, if the entire ata center *ent off'ine, the SP system *ou' remain

una%ai'ab'e unti' the a!!'ication ser%ers an SP atabase ser%ers *ere brought u! manua''y

on the DR site mainframe, *hich, as mentione ear'ier, too aroun four hours s !art of this

migration !roject QR *ante to ensure that the so'ution *as im!er%ious to site fai'ures

Redction in &ainframe Spport CostsThe maintenance costs for su!!orting SP amounte to hunres of thousans of o''ars !ermonth This *as a'so e8!ecte to gro* significant'y ue to the 'arge number of u!coming

!rojects The ne* !'atform neee to !ro%ie a configuration for running SP at reuce costs,

because the mainframe *ou' be ecommissione s a se!arate business e8ercise, the cost

of the re!'atforming !roject *as *eighe against the running costs for the e8isting !'atform, an

ca'cu'ations re%ea'e that the entire SP migration !roject *ou' cost 'ess than si8 months of

the su!!ort costs of the mainframe en%ironment The return on in%estment (R.-) of this

atabase migration !roject *as 'ess than si8 months, counting both interna' QR costs an

e8terna' har*are an consu'ting costs

PerformanceQR sti!u'ate that !erformance base'ine for the ne* so'ution shou' be the same or better -t is

im!ortant to note ho*e%er that the un*ritten e8!ectation *as that the ne* !'atform shou'

!erform better in most areas so that QR cou' u!grae from SP ?C to 1RP? (*hich im!oses

more 'oa on the atabase an a!!'ication infrastructure) *ithout further har*are u!graes

Current:generation ser%er har*are 'e%erages the benefits of commoity !rices as *e'' as

increase techno'ogy im!ro%ements o%er the years +ecause the mainframe -F. !ath

techno'ogy is an o'er im!'ementation, its !erformance *as not com!arab'e to current:

generation har*are, an it i not offer the same range of !erformance o!tions The M-PS

rating of the mainframe *as $@##

The fo''o*ing tab'e 'ists the success measures QR ientifie for this !roject

Success measure Criteria

RP. #RT. & minutes tota' !er year for un!'anne

o*ntime

%ai'abi'ity sco!e Site isaster

Performance Same or better

/'e8ibi'ity bi'ity to run entire so'ution on one noe

P'anne o*ntime P'anne as neee for SP u!graes

Table 1:Summary of goa's an success criteria

High'Level Reslts Post &igrationfter the migration !roject *as com!'ete, the QR team re%ie*e the !roject re=uirements to

%a'iate that those re=uirements *ere met

@


8/28

Performance &easred Post'&igrationQRIs management ser%ice:'e%e' agreement (SL) re!orting !rocesses trac the !erformance of

the SP so'ution %ia ear'y *atch re!orts These re!orts ha%e inicate that the ia'og res!onse

times for users (at !ea ia'og ste!s !er hour !erios) im!ro%e from initia' ranges of # to

#&& secons !er ia'og ste! *ith D+" to a range of #"& to #E& secons !er ia'og ste! *ith

SQL Ser%er fter the D+" to SQL Ser%er migration, res!onse time !erformance im!ro%e morethan ?# !ercent uring the !ea 'oa !erio

CP5 uti'i9ation is on trac s of this *riting, the same har*are is sti'' in use (The migration

occurre in June "##@) .ne c'uster noe su!!orts main'y the SP 1RP? 'oa *hi'e the other

c'uster noe su!!orts main'y the SP +3 'oa The har*are configuration is co%ere in more

etai' 'ater in this !a!er There ha%e been a number of occasions *here QR has o!erate a''

the SP atabases on the one noe *ith more than ae=uate !erformance 2orma' atabase

'oa *hi'e the so'ution is running is bet*een "# an E# !ercent CP5 uti'i9ation on both noes

Spportability and Tning

QR e8!ene a 'ot of time an effort on tuning the D+" system an running the mainframeinfrastructure This *as rastica''y reuce ue to the introuction of SQL Ser%er "##&, *hich is

esigne to re=uire 'ess hans:on tuning an su!!ort QR estimates that this has sa%e a'most

a fu'':time e=ui%a'ent (/T1) resource in atabase tuning acti%ities a'one

(le)ibilityThe SQL Ser%er so'ution has great'y im!ro%e f'e8ibi'ity s emonstrate in a 'arge number of

!rojects that ha%e occurre since the con%ersion, QR can far more easi'y u!'icate systems

from a base system (either e%e'o!ment or !rouction) to assist in !roject e8ecution or

!rototy!ing

Database Si*e-nitia' estimates !ro%ie by other %enors suggeste that atabase si9e *ou' increase

significant'y after a mo%e from D+" to SQL Ser%er -n fact, the re%erse *as true in QRIs caseeo:c'uster esign for high a%ai'abi'ity

-n a geo:c'uster, s!ecia' S2:s!ecific ri%ers an soft*are are neee in orer to ensure that

the L52s are ma!!e across to the other S2 just before the instance of SQL Ser%er an any

fi'e shares fai' o%er to, an then start on, the remaining noe 'so, in a sing'e:site

3ino*s Ser%er "##E fai'o%er c'uster, the share is becomes a %oting member in the

arbitration !rocess to etermine *hich noe o*ns the iss, if the noes can no 'onger

communicate *ith each other -n a geo:c'uster, this is not !ossib'e because there are t*o S2s

$"


13/28

an the iss must be synchroni9e an then mounte on the estination noe before the SQL

Ser%er RD+MS is starte u! -n this case a *itness or thir noe can !ro%ie this arbitration

ca!abi'ity The thir noe on'y e8ists to !ro%ie the arbitration %ote an oes not ha%e to be a

!o*erfu' ser%er, because it oes not nee to actua''y host an instance of SQL Ser%er s an

a'ternati%e to a thir noe, a remote fi'e share can be use for this !ur!ose This enhancement

o!tion *as mae a%ai'ab'e in Ser%ice Pac $ of 3ino*s Ser%er "##E

Related Article

rtic'e -D< B"$$A$ n u!ate is a%ai'ab'e that as a fi'e share *itness feature an a

configurab'e c'uster heartbeats feature to 3ino*s Ser%er "##E Ser%ice Pac $:base ser%er

c'usters

htt!


14/28

The fai'ure of either any t*o net*or 'ins or a sing'e ser%er fai'ure can be mitigate in this

esign as the arbitration bet*een the ser%er noes is maintaine 2et*or 'ins to user an

a!!'ication connections must a'so be maintaine, ho*e%er this is beyon the c'uster an not

co%ere in this ocument The net*or connecti%ity bet*een a'' of the sites *as E by $ gigabits

because the com!any a'reay o*ne the sufficient fiber strans -t shou' be note ho*e%er

that gigabit s!ee for the connecti%ity to the thir site is not re=uire 7o*e%er, the net*or

'atency must be 'ess than # ms (# ms is an u!!er 'imit, an it is s!ecific to 3ino*s

Ser%er "##E fai'o%er c'uster net*or -F., not to the S2 site:to:site fiber synchronous 'atency

that is co%ere 'ater in this !a!er) 3e recommen that net*or -F. 'atency uner norma'

situations be at 'east an orer of magnitue better than the #:ms 'imit

SA, Data ReplicationS2 ata re!'ication bet*een sites is a critica' !art of this so'ution S2 %enors ty!ica''y

su!!ort t*o re!'ication moe's : asynchronous an synchronous ata re!'ication geo:c'uster

im!'ementation re=uires synchronous ata re!'ication to ensure no ata 'oss synchronousata re!'ication cannot be use if the e8!ecte RP. %a'ue is 9ero, because if any ata has not

yet been sent to the remote site at the time of fai'ure, it *i'' be 'ost

The main ifference bet*een the t*o S2 re!'ication moe's is in the 'atency of the ata that is

!ersiste on the remote site synchronous re!'ication oes not *ait for the ata to be !ersiste

on the remote S2, an the ata 'atency e!ens on the 'in s!ee That means that

uncommitte -F.s bet*een the S2s are a''o*e, an the ser%er oes not ha%e to !ause -F.

through!ut .n the other han, synchronous ata re!'ication must ensure that e%ery -F. is

committe on the remote site before the *ho'e -F. o!eration is eeme successfu' The

im!'ication here is that there is no ata 'oss uring a site fai'ure an that the -F. res!onse times

may suffer if the site to site ban*ith is not sufficient This can be seen in the fo''o*ing figure,*hich sho*s the -F. !ath iagram

i!ure %< Synchronous -F. re!'ication bet*een S2s

$


15/28

$ The -F. *rite re=uest is !asse to the 'oca' S2

", E These t*o ste!s ha!!en together -F. is !ersiste to the 'oca' S2 cache or is

an a'so sent to the remote S2

-F. is !ersiste on the remote S2 cache or is

& cno*'egement that the -F. *as !ersiste successfu''y is sent to the originating

S2? The 'oca' S2 acno*'eges that the -F. has been !ersiste on the S2 fabric

The !roject team at QR teste the 'in bet*een sites to ensure that the s!ee an ban*ith

*as sufficient for e%en the most intensi%e tass The team use too's 'ie SQL-. an

SQL-.Sim, *hich can simu'ate ranom an se=uentia' -F., *ith simi'ar -F. si9es to *hat SQL

Ser%er can generate to test the -F. through!ut of the so'ution These test !rograms are

a%ai'ab'e from Microsoft to stress:test is -F. subsystems

7ere are some high 'e%e' ata !oints from -F. !erformance %a'iation testing using SQL-. that

QR conucte before they commissione the so'ution

Log /i'e -F. TestingType 32KB sequential I/O Write

No of Files 1

Threads 1

IOs/sec 442!"

#Bs/sec 13!4

$%&! 'atency()s* 1

IO +isto&ra),

'atency ()s* - 1 2 3 4 . 0 " 1- 11 12 13 14 1. 1 10 1 1" 2- 21 22 23 24

of IO requests 34 3 21 2 1 - - - - - - - - - - - - - - - - - - -

Data /i'e -F. Testing

Type KB rando) I/O WriteNo of Files

Threads

IOs/sec 413!"

#Bs/sec 32!33

$%&! 'atency()s* 14

IO +isto&ra),

'atency ()s* - 1 2 3 4 . 0 " 1- 11 12 13 14 1. 1 10 1 1" 2- 21 22 23 24

of IO requests .1 23 2 1 - - - - - - - - - - - - - - - - - - - - 22

Type KB rando) I/O ead

No of Files

Threads

IOs/sec 4"3-!

#Bs/sec 3!.2

$%&! 'atency()s* 12

IO +isto&ra),

'atency ()s* - 1 2 3 4 . 0 " 1- 11 12 13 14 1. 1 10 1 1" 2- 21 22 23 24 of IO requests - - - - 1 3 . 1- 1- " " 0 . 4 3 3 2 2 1 1 1 .

Related Articles

rtic'e -D< "E$?$B< 7o* to use the SQL-.Sim uti'ity to simu'ate SQL Ser%er acti%ity on a is

subsystem

$&


16/28

htt!


17/28

Soltion (ailre &odes SstainableThe net*or an c'uster noes mae u! the so'ution 1ach noe communicates *ith the others

o%er the net*or 'ins to ensure that in the case of any sing'e fai'ure the so'ution can sti''

maintain u!:time an a%ai'abi'ity -n orer for the c'uster to be a%ai'ab'e, =uorum must be

maintaine, *hich means that any t*o out of the three noes must a'*ays be on'ine an ab'e to

communicate *ith the other noe or noes

i!ure +:2et*or 'ins an sites

Loss of ,et+or/ Lin/s

-n this configuration, the so'ution can remain a%ai'ab'e e%en if any t*o net*or 'ins are off'ine

-f one net*or 'in is remo%e, as 'ong as that traffic can be !asse bet*een sites, the so'utioncan maintain high a%ai'abi'ity (abbre%iate to 7 in the fo''o*ing tab'e) -f t*o se!arate intra:site

net*or fai'ures occur, the so'ution *i'' sti'' be a%ai'ab'e but not ab'e to sustain high a%ai'abi'ity if

a further fai'ure occurs The fo''o*ing tab'e i''ustrates ho* ifferent ty!es of fai'ures are han'e

/ai'ure (of) Quorum Resu't

2et*or Maintaine bet*een

noes $, ", an E

.n'ine : 7 maintaine

2et*or + Maintaine bet*een

noes $,", an E


2et*or C Maintaine bet*een

noe $, ", an E


2et*ors an + Maintaine bet*een

noes $ an "

.n'ine : 2o further fai'ure a''o*e

2et*ors an C Maintaine bet*een

noes " an E


$@


18/28

2et*ors + an C Maintaine bet*een

noes $ an E


2et*ors , +, an C 2o =uorum .ff'ine : 2o communicationTable #:2et*or fai'ure moes

Loss of a Site or Database Clster ,ode

Simi'ar'y the c'uster can sustain a 'oss of one ser%er noe an sti'' remain acti%e This is

because the c'uster is bui't u!on the !rinci!'e of a share:nothing c'uster *here a majority of the

noes must be a%ai'ab'e an ab'e to communicate in orer to form =uorum -n this case, if any

noe fai's the atabase is sti'' a%ai'ab'e 7o*e%er, if more than one noe fai's, the entire c'uster

*i'' go off'ine, because the majority of noes are no* off'ine

The fai'o%er c'uster a%ai'abi'ity is go%erne by the !rinci!'es of noe majority That means that

the c'uster can to'erate the fai'ure of ha'f the number of noes minus one

Logical Design of SAP 012C on SQL Server 3445

i!ure ,:Logica' esign

The iagram sho*s the 'ogica' esign of the SP !rouction an non!rouction en%ironments

/o''o*ing best !ractices, the De%e'o!ment, Test, an 5CT en%ironments are a'so base on a

$A


19/28

geo:c'uster configuration so that coe changes, SP trans!orts, an u!graes are !romote

'ogica''y into the !rouction en%ironment an change contro' is maintaine This a'so enab'es

!rouction su!!ort staff to test a'' !ro!ose !rouction changes in an en%ironment that e8act'y

mirrors !rouction an a'so gain intimate fami'iarity *ith 3ino*s Ser%er an SQL Ser%er

features that the !rouction en%ironment uses

The so'ution uses a number of instances of SQL Ser%er that can be running on either of the t*o

c'uster noes hosting mu'ti!'e SP atabases The SP centra' instance !ro%ies critica' SP

Message an 1n=ueue Ser%er functiona'ity for the other SP a!!'ication ser%ers The SP

centra' instance !rocess is a sing'e !oint of fai'ure because it is not a c'ustere resource This

situation has been im!ro%e *ith the u!grae to SP 1RP? from SP ?C an using SCS

that !ro%ies SP high a%ai'abi'ity at a SP a!!'ication 'e%e' This configuration is further

iscusse 'ater in this !a!er

The atabase noes can host the a!!'ication 'oa from a'' of the SP a!!'ication ser%ers The

tiebreaer ser%er at Site E is on'y use to maintain =uorum an cast a %ote to etermine *hich

noe *i'' be use to create =uorum in the e%ent of a system fai'ure of one of the noes Thenoe at Site E is a sma''er ser%er an not configure to host SQL Ser%er

-n this configuration, the c'uster noes ha%e the fo''o*ing soft*are insta''e

SQL Server High Availability and Disaster Recovery for SAP Deployment at QR: A Technical Case Study

Documents

Transcript of SQL Server High Availability and Disaster Recovery for SAP Deployment at QR: A Technical Case Study