Getting Started: DBA Basics (90 minutes)

1

GettingStarted:DBABasics(90minutes)Congratulations!You'reOurNewProgressDBA!NowWhat?TomBascom,WhiteStarSoftwareAbstract:PeopleoftenaskwhattasksanewlymintedOpenEdgeDBAshouldbeperforming?Whatshouldmydaily,weekly,monthlyetcchecklisthaveonit?What'sgood?What'sBAD?WheredoIevenlooktofindout?WhatdoIneedtopayattentionto?WhatcanIignore?WhatshouldIlosesleepover?WhenshouldIupdatemyresume?Whocanhelpme?Inthissessionwewillshowyouthewaythroughthechallengesthatlieinfrontofyouandhelpbringpeacetoyourweekendsandevenings!

GettingStarted:DBABasicsCongratulations!You'reOurNewProgressDBA!NowWhat?

TomBascom,WhiteStarSoftware

[email protected]

AFewWordsabouttheSpeaker

•  TomBascom:Progressuser&roamingDBAsince1987•  Partner:WhiteStarSoftware&DBAppraise,LLC

– ExpertconsultingservicesrelatedtoallaspectsofProgressandOpenEdge.– RemotedatabasemanagementserviceforOpenEdge.– Authorof:– Simplifyingthejobofmanagingandmonitoringtheworld’sbestbusinessapplications.

–  [email protected]

3

AudienceSurvey

•  HowmanydoatleastsomeProgressDBAwork?

5

AudienceSurvey

•  HowmanydoatleastsomeProgressDBAwork?•  Howmanyhavebeendoingthatworkforlessthanayear?

6

AudienceSurvey

•  HowmanydoatleastsomeProgressDBAwork?•  Howmanyhavebeendoingthatworkforlessthanayear?•  IsanyoneonaversionofProgressthatisasingledigitnumber…i.e.V9,V8,etc.

7

AudienceSurvey


•  WhoisonOpenEdge10?

8

AudienceSurvey


•  WhoisonOpenEdge10?•  Howabout11.x?

9

AudienceSurvey


•  WhoisonOpenEdge10?•  Howabout11.x?•  12?

10

AudienceSurvey


•  WhoisonOpenEdge10?•  Howabout11.x?•  12?•  Arethereanycodersintheroom?

11

Now What?!?

13

ResourcesToGetStartedWith

OnlineDocumentation

14

https://community.progress.com/community_groups/openedge_general/w/openedgegeneral/1329.openedge-product-documentation-overview

LocalDocumentation•  Windows“CHM”DocsStart->OpenEdge->Help->

15

TherearegoodMacappsforreadingCHMfiles:

Knowledgebasehttp://knowledgebase.progress.com

Knowledgebase

PANS–DailyTechnicalBulletinsStep1:Exerciseyourknowledgebasesearchskills:

ProgressCommunityhttp://community.progress.com

ProgressTalkhttp://progresstalk.com

StackOverflowhttps://stackoverflow.com/questions/tagged/progress-4gl%20or%20progress-db%20or%20openedge

Slackhttps://join.slack.com/t/openedge/shared_invite/enQtMTgwODYyODI4MDM2LTg3MDFlMGVkODY2NTdiYjQ0ZDE4MzAwZTUxMDcxMWI0NzMwYTU5M2JjY2ZmMWRiZGIyNzZlNWViNzk3MDJiNDY

Or:https://goo.gl/oTwneQOr:https://tinyurl.com/slackOpenEdge

PUGChallenge

Don’tforget!

•  YourApplicationVendor– Documentation&recommendations– Mayhavetheirownforums– Knowledgebase– Userconferences– Andsoforth!

25

SettingUpYourCommandLine

Environment

PROENV

•  Providesacommandlineenvironmentthatisproperlyconfiguredtorunalladministrativecommands.– DLC&WRKDIRenvironmentvariableswillbeset.– PATHwillinclude$DLC/binandotherusefulbits.

•  IfyoudonotusePROENVthenyouneedtoensurethattheDLCvariableisexportedandthat$DLC/binisinyourPATH!

26

PROENVonWindows

ProTip!IfyouarerunningonWindowsmakesuretoadjustthewidthandheightoftheproenvwindow!Thepre-oe11defaultis300linesby24columns–andthatdoesnotworkwellforanyone.Modernmonitorscaneasilysupport160columnsby64lineswithacomfortablefontsize.Andtherearemuchbettercoloroptionsthanwhiteonblack.

28

StartingaDatabase

StartingaDatabase

•  PROSERVE•  DBMAN•  Exploder

29

PROSERVEPro•  CommandLine•  EasytoScript•  LotsofControl•  Repeatable•  Sequenceofactionsisguaranteed

Con•  MustProvideDetailslike“dbname”•  Dinosaurstigma?

30

$proservedbname–n500–spin3149–B1000000–L50000$probiwdbname$proaiwdbname$prowdogdbname$proapwdbname

DBMANPro•  CommandLine•  Workswithconmgr.properties•  EasytoScript•  StartsPageWriters&Watchdog•  Runsasa“Service”onWindows(avoidsWindows“userlogoutclosesbackgroundwindows”issue).

Con•  Usesconmgr.properties•  AdminServermustberunning•  Orderofoperationsisnotcontrollable.

•  AdminServerusesJava

31

proenv>dbmandbName-start

ExploderPro•  ExercisesHealthCareBenefits

Con•  GraphicalInterface•  RequiresAdminServer•  Requiresconmgr.properties•  Ugly•  Confusing•  Incomplete•  Unreliable(Java)•  Butifyoulikethatsortofthing…

32

ExploderProTip

•  AlwaysmakesurethattheadminserverisrunningbeforedecidingthatExploder(ordbman)isn’tworkingproperly!

proenv>proadsv-queryOpenEdgeRelease10.2BasofMonDec1417:02:01EST2009AdminServerisalive.(8545)proenv>

34

StoppingaDatabase

StoppingaDatabase

•  PROSHUT•  DBMAN•  Exploder

35

$proshut-bysports2000Shutdownisexecuting.(1613)Shutdowncomplete.(1614)

StoppingaDatabase


36

•  Deletethe.lkfile…•  Kill-9•  Rebootserver…•  Tripoverplug…

StoppingaDatabase


38

•  Deletethe.lkfile…•  Kill-9•  Rebootserver…•  Tripoverplug…

OnthebrightsideProgresscrashrecoveryworksvery,verywell–noneoftheabovewillcorruptanOpenEdgedatabase.(DoNOT,however,trythesetechniqueswithcertainotherbignamedatabases…)

PROSHUTPro•  CommandLine•  EasytoScript•  LotsofControl•  Fast

Con•  What’saCommandLine?

39

$proshut–bydbname

DBMANPro•  CommandLine•  Workswithconmgr.properties•  EasytoScript

Con•  CommandLine•  Usesconmgr.properties•  AdminServermustberunning

40

$dbmandbName-stop

ExploderPro Con

•  GraphicalInterface•  RequiresAdminServer•  Requiresconmgr.properties•  Ugly,Confusing,Incomplete•  Unreliable•  Cannotcontrolsequence•  Easytoshutdownthewrongdb

41

42

HowDoIMakeaBackup?

HowDoIMakeaBackup?

•  PROBKUP–thepreferredmethodendorsedbyexperiencedProgressDBAseverywhere.

43

HowDoIMakeaBackup?

•  PROBKUP–thepreferredmethodendorsedbyexperiencedProgressDBAseverywhere.

•  MethodsusedbyFutureJob-Seekers:– DiskMirroring– OSBackup– 3rdPartyTools– VMorSAN“snapshots”

44

PROBKUP

•  Knowswhereallofthepartsofthedatabaseare(eventhepartsinmemoryorinunusualdisklocations).

•  Canbeexecutedwiththedatabaseonline.•  Canskipemptyspaceinthedb.•  Canbeusedtochange“physical”DBStructure(extentsize&location)

•  Canturnonafter-imagingonlineifyouforget!

45

$probkuponlinedbnamedbname.pbk-com

“AllThePartsoftheDB”

46

#sports.st#b/bi/sports.b1#d"SchemaArea":6,32;1 /db/sports.d1d"InfoArea":7,32;1 /db/sports_7.d1d"Customer/OrderArea":8,32;8 /db/sports_8.d1d"PrimaryIndexArea":9,1;8 /db/sports_9.d1d"CustomerIndexArea":10,1;64 /db2/sports_10.d1d"OrderIndexArea":11,32;64 /db/sports_11.d1#a/ai/sports.a1a/ai/sports.a2a/ai/sports.a3a/ai/sports.a4

OSBackup&3rdPartyTools

•  Thedatabasemustbeofflineorina“quiescentstate”.•  Youareresponsibleformakingcertainoftheabove.•  Youmustalsoensurethatallpartsofthedatabaseareincludedinthebackup.

47

VMorSAN“Snapshots”•  Thesnapshotmustbeaconsistent,pointintimeimageoftheentiredatabase.

•  Thedatabasewillbeinacrashedstatewhenrecovered.•  Notallsnapshotproductsarecreatedequally!•  ProgresshastestedandcertifiedEMC’sSRDF.•  SomeusersreportsuccesswithVMWare’sVMotion(butbewareofpossibleperformanceissues).

•  OtherusersreportVMotionhorrorstories–CaveatEmptor!

48

OtherStufftoBackup

•  $DLC/properties•  DBLogFiles•  Structure(.st)Files•  Parameter(.pf)Files•  $DLC/startup.pf•  repl.propertiesetc.•  $DLC/certs

49

50

NoneofMySQLKnowledgeWorks

NoneofMySQLWorks!?!

•  ProgressisNOTSQL.•  Period.FullStop.EndofStory.•  ThinkingaboutProgresslikeSQLwillonlyleadtopainandagony.

52

NoneofMySQLWorks!?!

•  ProgressisNOTSQL.•  Period.FullStop.EndofStory.•  ThinkingaboutProgresslikeSQLwillonlyleadtopainandagony.

•  SomeofuslikeitthiswayJ

53

4glvsSQL•  Thereare2“languages”–4glandSQL-92thatshareacommon“storageengine”.

•  ThestorageengineiswhatDBAsmanage.•  VirtuallyallProgressapplicationsarewrittenwiththe4gl.•  SQLisusedalmostexclusivelyforexternalreportingordataextracts.

•  Thelanguagesarenotawareofeachother.– Triggersandstoredproceduresinonelanguagearenotknowntotriggersintheother!

54

EmbeddedSQL-89–IsNOTSQL-92!!!•  The4GLlanguagehassomeveryoldandverylimitedSQL-89syntaxembeddedinit.

•  Thispermitstrivialad-hocquerieswithina4glsessionsuchas:selectcount(*)fromcustomer.

•  Thismakesagooddemo.

55

SQL-89

AttemptingtousetheembeddedSQL-89inapplicationcodewillonlyleadtopain,

sufferingandagony.

Donotgothere.Youhavebeenwarned.

56

ProgressisNOTSQL!Progressismuchbetterthanthat.•  The4GLlanguagedoesnotrespect“fieldwidth”–alldataisvariablewidth.

•  Afieldformatisjustadefaultsuggestionusedwhentheprogrammerprovidesnootherinputordisplayformat–itisnotaconstraint.

•  Forinstanceacharacterfielddefinedas“x(30)”mightbe“overstuffed”withastringoflength2,000.

•  Thisisnormalandverycommonpracticewithin4glapplications.Itisnotanerror.

57

NewSQLWidthOptions!ADTandASUcanbeenabledforspecificLoginBrokers,Forexample:proserve<dbname>-S<port>-SQLTruncateTooLargeON-SQLWidthUpdateONServerTypebothproserve<dbname>-m3-S<port>-SQLTruncateTooLargeON-SQLWidthUpdateONServerTypeSQLWhendataistruncatedthereisamessageinthedatabaselogsimilarto:SQLSRV22:(-----)13authorizeddatatruncationaction(s)performed.IfADTorASUisnotneededmayalsobeenabledforindividualclientsessionsviaODBCorJDBC.Example:sqlexp-url"jdbc:datadirect:openedge://localhost:<port>;databasename=<databasename>;truncateTooLarge=output;"RefertoArticle000068442,HowtoenableAuthorizedDataTruncationinaJDBCorODBCconnection.ForinteractionbetweenADTandASURefertotheWhitepaper:AutonomousSchemaUpdateFunctionality:https://community.progress.com/community_groups/openedge_rdbms/f/18/t/19534

SQL-92Interface

•  SQLEXP•  ODBC/JDBC•  DBTOOL•  UPDATESTATISTICS•  SQLDUMP

59

SQLEXP

•  RunSQLscripts•  GRANTandREVOKEperms(savethescriptsbecausetheSQLpermissionsarenotincludedinadump&load!)

60

$DLC/bin/sqlexp-useruserName-passwordpassWord\-dbdbName-SservicePort\–infilescript.sql–outfilesqlexp.log

ODBC/JDBC

61

DBTOOL

62

$dbtoolsports DATABASETOOLSMENU ------------------------------------------- 1.SQLWidth&DateScanw/ReportOption 2.SQLWidthScanw/FixOption 3.RecordValidation 4.RecordVersionValidation 5.ReadorValidateDatabaseBlock(s) 6.RecordFixup 7.SchemaValidation 9.Enable/DisableFileLogging Q.Quit

Choice:

http://knowledgebase.progress.com/articles/Article/P24496

UPDATESTATISTICS

63

/*genUpdateSQL.p**mprodbName–pgenUpdateSQL.p–param"updstats.sql"**sqlexp-useruser-passwordpassWord-dbdbName-Sport-infileupdstats.sql–outfileupdstats.log*/outputtovalue(session:parameter).foreach_fileno-lockwhere_hidden=no:putunformatted"UPDATETABLESTATISTICSANDINDEXSTATISTICSAND""ALLCOLUMNSTATISTICSFORPUB."'"'_file._file-name'"'";"skip"commitwork;"skip.end.outputclose.

DumpingYourSQLConfiguration

Thursday:10:30–11:30BrusselsPaulKoufalis

Dump&Load:MorethanjustafewProutilCommands

65

TheDBA’sBestFriend

After-Imaging

After-Imaging

•  Roll-forwardrecovery.•  Ajournaloftransaction“notes”thatcanbereplayedagainstabaselinebackuptorestoreadatabasetothelastcompletedtransactionorapointintimeoraspecifictransactionnumber.

•  Thisisthesameconceptthatsomeotherdatabasesrefertoasthe“redolog”.

66

WhydoIneedafter-imaging?

•  Protectionfrommedialoss--suchasbadtapes,acrasheddisk,adestroyeddatacenterorstolenservers…

WhyelsedoIneedafter-imaging?

•  Protectionfromhumanerrors:

•  Humanerrorisatleastasbigariskashardwareproblems.

for each customer: delete customer. end.

$ cd /db $ rm *

for each order: delivered = yes. end.

$ vi dbname.db … :x

After-ImagingBestPractices•  Enableafter-imagingonallupdateabledatabases.•  Placeafter-imageextentsonseparatedisksfromdataextents.•  Use8to16variableextentswith“largefiles”enabled(moreforOER).•  RunanAIW.•  Switchextentsasoftenasthebusinessneedsyouto.•  Usethesequencenumberwhennamingarchivedlogs.•  CopyarchivedlogstoanexternallocationASAP.•  Verifyyourprocessbycontinuouslyrollingforward.•  Monitorbothyour“empty”and“full”extents.•  Keepatleast30+daysofarchivedafter-imagelogs.•  Establishdedicatedbackupandrecoveryfilesystems.

70

StartupParameters

WheretoFindThem?

•  Onthecommandline.•  Inascript.•  In“.pf”files.•  In$DLC/properties/conmgr.propertiesL•  Verifybyexaminingthe“dbname.lg”file.

– Searchfor“(333)”andexaminethenext50-75lines…

71

72

ImportantConfigurationOptions

ImportantConfigurationOptions

•  Connections•  BIClusterSize•  BlockSizes•  StorageAreas•  RowsPerBlock

73

Connections

•  -nisconnectionsNOTlicenses,-nshouldalwaysbelargerthanyourlicensecount!

•  -Mi/Ma/Mn/Mpb:networkbrokerconnectionparameters•  -m3/ServerType:SQLvs4GLserver•  -minport,-maxport:portrangeusedbyservers

Connections•  -n:isNOTlicencecountsogiveyourselfagoodbuffer•  -Mi1–Ma5isagoodstartformostpeople•  -Mpb=maxconcurrentusers(4GLorSQL)/-Maplusafew•  -Mn=Sumof–Mpbplusagenerousfewextra•  -ServerType:segregate4GLandSQLconnectionsonseparate–m3brokers

•  Usenon-overlapping–minportand–maxportrangesforeachbroker

BIClusterSize

•  TheDefaultvarieswithrelease,512KBiscurrent.•  For“Workgroup”Licensessmallerisbetter.

– More,butsmaller,delays.

•  For“Enterprise”Licensesbiggerisbetter.– Mayincreasecrashrecoverytimeabit.– Butreducestheriskofpainfuldelaysduringheavyprocessing.– 32768KBis“agoodstart”.

•  proutildbname–Ctruncatebi–bi32768

76

BlockSizes•  DBBlock(MustDump&LoadtoChange)

–  4KBor8KB–  NOT1KBor2KB

•  BIBlock–  16KB–  proutildbname–Ctruncatebi–biblocksize16

•  AIBlock–  16KB–  rfutildbname–Caimagetruncate–ai16–  AImustbedisabledsodoit‘early’

77

StorageAreas•  Type2storageareasarethefoundationforalladvancedfeaturesoftheOpenEdge

database.•  Type2areashaveclustersizesof8,64or512.•  DatablocksinType2areascontaindatafromjustonetable.•  UseMany(Type2)StorageAreas(andneveruseType1Areas).•  DoNOTassigntablestoareasbasedon“function”.•  Insteadgroupobjectsbycommon“technicalattributes”(RowsPerBlock,size,activity

level).•  PutLargeObjects(LOBs)inseparateAreas.•  DoNOTstoredata,indexesorLOBsinthe“SchemaArea”.

78

Storage•  RAID5isEVIL!•  Soarethevariantsspawnedfromit:

–  RAID6–  RAIDDP(DoubleParityorDefinitelyPutrid)–  Etal–ParitybasedRAIDprovidesaParodyofPerformance(allRAIDotherthanRAID10isparitybased).

•  NAS<>SAN•  IfatallpossibleavoidNetApp•  SANsexisttomakelifeeasyforstorageadmins–theyarenottheretomakeyourdatabaserunfaster.

79

Just2050slidestogo!

80

81

WhereAretheLogFiles?

Wherearethelogfiles?•  dbname.lg•  AdminServer:

$WRKDIR/admserv.log•  AppServers:

$WRKDIR/appsrv.broker.log$WRKDIR/appsrv.server.logLocationof$WRKDIRcanbefoundin$DLC/bin/proenv

•  OSLogs– /var/log,/usr/adm

82

83

Monitoring

Butfirst…

•  Alwaysset–tablerangesizeand–indexrangesizeondbstartup!

•  Towhatvalues?

Butfirst…•  Alwaysset–tablerangesizeand–indexrangesizeondbstartup!•  Towhatvalues?

/*range.p--mprodbName-p./range.p*/definevariabletasintegerno-undolabel"tables".definevariableiasintegerno-undolabel"indexes".foreach_fileno-lockwhere_hidden=no:t=t+1.end.foreach_indexno-lock:i=i+1.end.displaytiwithside-labels.

dbname.lg:WhattoIgnore

86

(452)Loginbyrooton/dev/pts/6.(453)Logoutbyrooton/dev/pts/6.(708)Useridisnowtom.(8873)Loginusernum2547,remoteSQLclient.(14658)Previousmessagesentonbehalfofuser2542,serverpid22516,brokerpid5778.(12699)DatabasexyzOptions:

Andafewsquillionmore“noise”messages… Msg Type Client Type | | Date Time Process-Id v v Usr# Msg# Message Text------------------------------------------------------------------------------------------------[2014/08/23@09:34:36.572-0400] P-336 T-1240798976 I ABL 5: (452) Login by tom on /dev/pts/1.[2014/10/24@13:07:54.129-0400] P-11073 T-1226471168 I ABL : (334) Single-user session end.

dbname.lg:WhattoPayAttentionTo

87

(2248)Beginnormalshutdown(2249)BeginABNORMALshutdown(542)Servershutdownstartedbyrooton/dev/pts/6.(5292)SYSTEMERROR:Thebrokerisexitingunexpectedly,beginningAbnormalShutdown.(1384)Thedatabaseisbeingshutdown.(915)Locktableoverflow,increase-Lonserver(1081)Toomanyusersrequestedsemaphoreundo,increaseSEMMNU.(358) SYSTEMERROR:Toomanysubprocesses,cannotfork.Errno=12.

Note:notacomprehensivelistorerrorsbutyougettheidea

dbname.lg:WhattoPanicOver!

88

(37)Yourdatabasewasdamaged.Dumpitsdataandreloadit.(43)Cannotfindoropenfile<filename>,errno=<number>.(1124)SYSTEMERROR:Wrongdbkeyinblock.Found5512,shouldbe1458inarea15.(886)Thedatabasewaslastused<date/time>.(887)Thebefore-imagefileexpected<date/time>.(888)Thosedatesdon'tmatch,soyouhavethewrongcopyofoneofthem.SYSTEMERROR:…

Note2:notallSYSTEMERRORsareworthgettingoutofbedfor

dbname.lg:WhatNotToMiss

•  Thelackofthesemessagesmeansthatbackupsarenotexecuting.•  Orthatafter-imagingisnotfunctioning.

89

(1362)Fullbackupstarted.(1364)Fullbackupsuccessfullycompleted.(7129)Usr61setnametoAimagefull.(3778)Thisisafter-imagefilenumber270sincethelastAIMAGEBEGIN(3777)Switchedtoaiextent/ai/sports.a4.

MonitoringTools&Services•  PROMON(free,included)•  ProTop(freeorpaid)

16:30–17:30Copenhagen;[WSS]ProTop:TheBestOEMonitoringDashboardintheGalaxy

•  RollyourownwithVirtualSystemTables•  OEManagement(“FathomManagement”)•  ProgressManagedDBAService•  DBAppraise(supportservicesadd-onforProTop)

90

WhattoMonitor

•  IstheDBup?•  BackupAge•  Numberofconnections•  Oldestactivetransaction•  Commits/sec•  LogicalReads/sec•  After-image#offullextents•  Busyusers,tablesandindexes

•  Latchtimeouts•  Locksinuse•  Blockedusers•  IOresponse•  CPUperformance•  DiskSpace

91

DatabasePerformanceTuningTalks

Wednesday10:00–11:00Amsterdam;[AdamBackman]Topnperformancetips

15:45–16:45Amsterdam;[DanForeman]Captain,WhereshouldIgo?WhatshouldIdo?

Thursday11:45–12:45Brussels;[AdamBackman]Buildyourownluck

13:45–14:45Brussels;[TomBascom]OpenEdgeDatabasePerformanceTuning16:30–17:30Copenhagen;[WSS]ProTop:TheBestOEMonitoringDashboardintheGalaxy

Friday

9:00–10:00Brussels;[DanForeman]promonforDummies&Geniuses

93

It’sNot“Just”aDatabase


•  AnyAmountofTuningcanalwaysbedefeatedbylousycode.

94


•  AnyAmountofTuningcanalwaysbedefeatedbylousycode.

•  …andthereisaLOToflousycodeoutthereL

95

PickYourBattles

Theperformanceenhancementpossiblewithagivenimprovementislimitedbythefractionofthe

executiontimethattheimprovedfeatureisused. --Amdahl’sLaw

Inotherwords:

•  Tryingtoimprovesmallthingsthatnobodynoticesprobablyisn’ttheroadtofameandfortune.

•  Bigqueriesthatreturnlotsofdataandwhicharefrequentlyusedbylotsofuserswillbemuchmorenoticeable.

Users!

•  Sometimesyouneedtoknowwhattheyaredoing:– UNIX: kill–USR1<pid>– Windows: %DLC%\bin\proGetStack<pid>

•  Mustowntheprocessorberoot/administrator•  Createsaprotrace.<PID>inworkingdirectory•  Theprotracecontainsa4glstacktrace!

98

protrace.<PID>

99

PROGRESSstacktraceasofTueDec411:28:002012Commandlineargumentsare/progress/dlc/bin/_progres-pmls.p-pf/los_prod/develop.pf-T/dbtmpStartupparameters:-pf/progress/dlc/startup.pf,-T/dbtmp,-clientlog/los_logs/debug/client_f474458_20121204_112720_45302816.log...(snip)+++PARALLELTOOLSCONSORTIUMLIGHTWEIGHTCOREFILEFORMATversion1.0...(snip)**4GLStackTrace**-->obj/mnu/menu(/los_prod/obj/mnu/menu.r)atline416obj/src/startup(/los_prod/obj/src/startup.r)atline2042mls.p(/los_prod/mls.r)atline26**Persistentprocedures/Classes**HandleFileName001010/los_prod/obj/tools/proclib.r000000/los_prod/config.r(STATIC)001000/los_prod/obj/prt/genBarcode.r

ApplicationCodePerformanceTuningTalksWednesday

14:00–17:30Prague;[PaulKoufalis,PeterJudge]Workshop:4GLCodePerformance-ProfilingandIndexing15:45–16:45Cologne;[Panel]DebuggingandTrouble-ShootingPanel

Thursday

10:30–11:30Noordwijk;[DanForeman]TEMP-TABLEMonitoring&PerfTuning13:45–14:45Manchester;[DanForeman]NeedleinaStackTrace

16:30–17:30Copenhagen;[WSS]ProTop:TheBestOEMonitoringDashboardintheGalaxy

Friday

9:00–10:00Brussels;[DanForeman]promonforDummies&Geniuses11:45–12:45Cologne;[PaulKoufalis,PeterJudge]Mastering4GLCodePerformance-Profiling&Indexing10:15–11:14Brussels;[TomBascom]EmbeddedApplicationPerformanceInstrumentation16:30–17:30Manchester;[TomBascom]4GLCodingWorstPractices

101

AFewWordsAbout“killing”Users

About“killing”Users…

•  IfyouareonUNIX:– “kill-9”isdangerous!Youwillcrashthedatabase!– “kill-9”doesnot“alwayswork”– “kill-1”(“hangup”)issafeandeffective

– proshutdbname–Cdisconnectusr#

102

About“killing”Users…

•  IfyouareonWindows:– We’resorryL– TaskManager“endtask”isequivalenttokill-9.– Loggingoutcanterminateallofyourbackgroundwindows–including“scheduledtasks”.Whichwillactlikeakill-9.

103

104

Checklists

Mornings

105

Mornings

•  Verifysuccessfulbackup•  Verifythatafter-imagingisenabledandproperlyswitchingextents•  Verifythatwarmspareisavailableanduptodate•  Verifythatmonitorsarerunningandthatalertsareflowing•  Verifysufficientfilesystemfreespace

Mornings

•  Checkbifilesize•  Checkfixedextentfreespace•  Checkfreespaceinaiarchivefilesystem•  Checkfreespaceinbackupfilesystem•  Checkfor“runaway”processes•  Checkforovernightprocessesthatmaystillberunning(butshouldnotbe)

•  Checkforleftoverlongopentransactions

Mornings

•  Reviewdblogfileforovernightmessages•  Ifyouareusingthealternatebufferpool--reviewB2,ensurethattherearefreeblocksandthatlru2isdisabled

•  Checkmonitoredmetricsfortrendsthatareapproachingactionablethresholds

•  ReadProgressPANSalerts

Mornings

•  ReviewOSlogs•  ReviewOSfreememory•  Reviewsummaryofpreviousday’sCPUanddiskutilization•  CheckOSconfigurationforunwelcomechanges

DuringtheDay

110

DuringtheDay

•  AIswitching&warmspareapply•  Numberofusers/connections•  HighdiskIOrates,lowbufferhitratio•  Unusuallyactivetablesorindexes•  Unusuallogfilemessagesandalerts•  LockTableHWM,activelocks•  Timebetweencheckpoints•  OSbottlenecksandconstraints

DuringtheDay•  LongopenTRX&BIfilegrowth

–  Findtheoldesttransaction(usuallyacodeproblem)•  Blockedusers/connections

–  REC=recordlocking,codingissue–  BK*,TX*etcindicatesystemresourceconstraints

•  Excessivelyactiveconnectionsor“rapidreaders”–  Whataretheydoing?–  Isitlegitimate?–  Isthereabetterway?

•  Usethe“clientstatementcache”or“proGetStack”toidentifyspecificcodecausingaproblem.–  Workwithdevelopmenttogetitfixed.

Weekly

113

Weekly

•  Rotate/Truncatethe.lgfile•  Cleanuptrashindbdirectories

–  protrace,leftoverscratchfiles,corefiles,etc– Don’tforget–T!

•  Refreshdbanalys•  Refresh“prostrctlist”•  RefreshDEV/TEST/QA/Trainingetc.

–  ThismayinvolverestoringaPRODbackupwhichwillverifythatbackupsaregood.

Weekly

•  Afterrefreshingdbanalysreview:–  Indexutilization,identifyidxcompacttargets– Checkrowsperblocksettings– CheckRMchains– Fragmentation– Scatter

•  Scheduleappropriateremediationactivities

Monthly

116

Monthly

•  Outagesummary:plannedandunplanned•  CapacityPlanningReports:

– BasicCRUD&TRXtrends– OverallDBgrowth– User/Connectiontrends–  IOresponse– Projectdiskspaceneeds– Projectdiskthroughputneeds– Projectmemory&CPUutilization

Quarterly

118

Quarterly•  Reviewallstartupparametersandconfigoptions•  Reviewstorageareaconfiguration•  IfallowingSQL-92connections:

–  RundbtooltoadjustSQL-width–  Run“updatestatistics”fortheoptimizer

•  IfusingSSLetc–reviewcertificatevalidity&expiration•  ReviewOSconfiguration,kernelparamsetc.

–  TestIOthroughput:randomreads&synchronouswrites•  ReviewProgressreleaselevel•  Reviewmonitoredmetricsandalerts•  Reviewanynewbusinessgrowthplans

Annually

120

Annually

•  DRTest•  Licensereview&“true-up”•  ReviewHWlandscapeandpotentialupgrades•  Reviewbusinessgrowthplans&projections•  PUGChallenge/Exchange

121

SpecialEvents

122

Upgrades&ServicePacks

•  Shutdown•  Truncatethebi•  Backup•  InstalltheupgradeorSP(orchange$DLC)•  proutil–Cupdatevst•  proutil–Cupdateschema•  Restart

123

Pre-ReleaseorUpgrade•  Reviewanyonlinechangesthatshouldbemadepermanent:-spin,-L,etc.•  Run“proutil–Cdescribe”andconfirmthatyouhavetheconfigoptionsthatyouneed(largefiles

etc).•  Reviewallstartupparameters&configoptions:

–  General:-n,-L,-B,-B2,-lruskips,-lru2skips,-spin,-M*(ifused)–  Schemarelated:-omsize,-*rangesize–  OtherConfig:bicluster&blocksize

•  Reviewstorageareas–areanynewareasneeded?(Perhapsatableshouldbesplitout?)•  Review.dfforproblems–i.e.RECIDfields,nostorageareaetc•  ReviewProgressservicepacks–shouldaSPbeapplied?

Post-Release

•  Checkschemaareaforstrayobjects•  CheckthatnoRECIDfieldshavesnuckin•  Verifythattables,indexesandLOBsareallinproperstorageareas•  Verify–omsize,-*rangesizeetc.•  VerifyB2assignments

Post-Outage(unplanned)

•  Rootcauseanalysis•  Remediationplan•  Lessonslearned

– Neworimprovedalerts– Additionalinstrumentation–  Improvedprocedures– Additionaltraining

126

FreeTime

127

Planning,Testing&Optimizing

•  Benchmarkingandstresstesting•  Alternativeconfigurations•  NewOpenEdgereleasesandfeatures•  Reducingrequireddowntime

Questions?129

ThankYou!

130

Getting Started: DBA Basics (90 minutes)

Documents

Transcript of Getting Started: DBA Basics (90 minutes)