The Looming Complexity Catastrophe in Large (and Ultra ... · Customer Repair Tracking I35 Early...
Transcript of The Looming Complexity Catastrophe in Large (and Ultra ... · Customer Repair Tracking I35 Early...
Slide 1 © Carliss Y. Baldwin and Kim B. Clark, 2005
The Looming ComplexityCatastrophe in Large (and Ultra-Large) Systems
Carliss Y. BaldwinHarvard Business School
August 15, 2005
Slide 2 © Carliss Y. Baldwin and Kim B. Clark, 2005
IT system of a large retailer, 1999Boxes=Systems; Lines=Standards
E01-EDI
Data Warehouse(Interfaces to and from the
Data Warehouse are notdisplayed on this diagram)
DRAFT Best Buy - Application Diagram V4 DRAFTNovember 10, 1999
G02 - General
Ledger
A05 - AP
S01 - SalesCorrections
I01 PO
Receiving
I03 Return to
Vendor
I06 Warehouse
Management
M a inframe apps - Blue
PC/NT apps - G reen
Unix apps - Yellow3rd party interface - Orange
Lines: Colors have no special meaning.
They are to help make the diagram easier toread.For More Information: See the database
containing information about eachapplication: Application V4.mdb
S06 - Credit App
P15 EES EmployeeChange Notice
OTHER APPS - PC
AP - Collections/Credit
TM - Credit Card DB
ACCTS REC APPS - PC
990COR
Bad Debt
Benefical Fees
Beneficial Reconcil
JEAXF
JEBFA
JEBKA
JEDVA
JESOA
JEVSA
JEVSF
NSF
TeleCredit Fees
INVENTORY CONTROL APPS - PC
Code Alarm
Debit Receivings
Devo Sales
Display Inventory
In Home
Junkouts
Merchandise Withdrawl
Promo Credits
RTV Accrual
Shrink
AP Research - Inv Cntrl
AP Research-Addl Rpts
Book to Perpetual Inventory
Close Out Reporting
Computer Intelligence Data
Count Corrections
Cross Ref for VCB Dnlds
Damage Write Off
Debit Receivings
DFI Vendor Database
Display Inventory Reconcil
Display Inventory Reporting
INVENTORY CONTROL APPS - PC
DPI/CPI
IC Batching
Inventory Adj/Count Correct
Inventory Control Reports
Inventory Levels
Inventory Roll
Merchandise Withdrawl
Open Receivings
PI Count Results
PI Time Results from Inv
Price Protection
Sales Flash Reporting
Shrink Reporting
SKU Gross Margin
SKU Shrink Level Detail
USM
VCB Downloads
Journal Entry Tool Kit
Scorecard - HR
L02-ResourceScheduling
(Campbell)
P09 - P17Cyborg
M02 - Millennium
M03 - Millennuim 3.0
Banks - ACH and Pos toPay
Cobra
B01 - StockStatus
S03-Polling
P14 On-line NewHire Entry
CTS
Plan Administrators(401K, PCS, Life,
Unicare, Solomon
Smith Barney)
D01 Post LoadBilling
I04 HomeDeliveries
I02 -Transfers
Arthur Planning
I07 Purchase
Order
I12 Entertainment
Software
I05Inventory Info
E13E3 Interface
S04 - Sales Posting
V01-Price ManagementSystem
I10 Cycle PhysicalInventory
I55 SKUInformation
K02Customer Repair
TrackingI35 Early Warning
System
B02 MerchandiseAnalysis
I13- Auto
Replenishment
U18 - CTO
Intercept
I09 Cycle Counts
E02-EmployeePurchase
Texlon 3.5
ACH
Stock Options
I17 Customer PerceivedIn-Stock
U16-Texlon
SiteSeer
C02 - CapitalProjects
F06 - FixedAssets
US Bank Recon
File
Star Repair
EDICoordinator
Mesa Data
NEW Soundscan
NPD Group
AIG Warranty Guard
Resumix
Optika
Store Budget
Reporting
P16 - Tally Sheet
Cash Receipts/Credit
S05 - HouseCharges
Ad Expense
L01-PromoAnalysis
V02-PriceMarketing
Support
BMP - Busperformance Mngt
StoreScorecard
I11 PriceTesting
Valley Media
P09
Bonus/HR
I15 Hand ScanApps
Roadshow
POS
S08 - VertexSales
Tax
A04 - Cust
Refund Chks
Equifax
ICMS Credit
CellularRollover
S09 - Digital
SatelliteSystem
NPD,SoundScan
Sterling VANMailbox (Value)
I 18SKU Rep
X92-X96Host to AS400
Communication
S02 -Layaways
Washington,
RGIS,Ntl Bus Systems
V04-SignSystem
I14 Count CorrectionsNARM
P01-
EmployeeMasterfile
I06 - CustomerOrder
Frick
Co
UAR - Universal AccountReconcilliation
Depository
Banks
S07 - Cell
Phones
S11 - ISP
Tracking
AAS
Fringe PO
Cash Over/Short
L60 MDF
CoopSKU Selection
Tool
SKU
Performance
SupplierCompliance
1
I35 - CEI
ASIS
Misc Accounting/Finance Apps - PC/NT
COBA (Corp office Budget Assistant)PCBS(Profit Center Budget System)
Merchandising Budget
AIMS
Merch Mngr ApprovalBatch ForcastingAd Measurement
AIMS Admin
AIMSReportingAd
Launcher
V03- MktReactions
SpecSource
CTO2.Bestbuy.com
RebateTransfer
SignSystem
CopyWriter'sWorkspace
ELTPowerSuite
StoreMonitor
AIS Calendar
Stores & Mrkts
Due Dates
Smart Plus
Insertions
Orders
BudgetAnalysis Tool
Print CostingInvoice App
AIS Reports
Broadcast
Filter
Smart PlusLauncher
General
Maintenance
Printer PO
PrinterMaintenance
VendorMaintenance
Vendor Setup
Connect 3
Connect 3Reports
Connect 3
PDF Transfe
Spec SourceSKU Tracking
S20-SalesPolling
Prodigy
PSP
In-Home
Repair
WarrantyBilling
System
Process Servers(Imaging)
Prepared by Michelle Mills
Page 1 of 2
Slide 3 © Carliss Y. Baldwin and Kim B. Clark, 2005
Another appalling hairball from banking—3-year plan to do merger integration
Universal VRU-ContactCtr/Einstein E2
Desktop
#3: NC/SC Deposits,WIA/CAP, CD's, IRA,
OD Protect. . .
Comml Credit CardsConv'n FDR to MBNA
Fixed Assets GEACFAE Series
SignageChange (finish)
Key dates which will cross Lines of Business. Lines indicate point in time.
Size of milestone indicates complexity of event - small, medium or large
Key dates - specifically conversion dates, which will cross Lines of Business. Lines indicate a point in time.
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
CorporateRisk
Commercial Credit & RiskProducts (CA Tools, Viper, MFAand Polar)
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
E-Commerce
Internet Merge tonew Wachovia.com
Internet online billpay -FUNB CheckFree toMetavante
Internet Online Banking& Bill Payment
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Executive
Accounts Payable toOracle Financials
Systems DataMapping
LawPacCase Mngmt
TN - Close
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Finance
Purchasing -CLARUS to OSCAR
GeneralBank
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Divestiture DeconversionDep & Loans
ATM Pilot
SBA Loan Conv'n to Strategy/McCracken (AS400)
WB Mortg Loans via Mortg Mngr
Retail Loan Coll'n and Recovery
All Mortg LoanOrig'ns Close to
Alltel/CPI Credit Card Conv'n:First USA to MBNA
Retail Consmr Loans:ACLS Conv'n 4.x to 5.0
ATM-Base 24ATM MachineConv'n-HWMigration
Commercial Res EstateConstruct'n Loans CLCS PEL's-Home
Equity Linesof Credit
ATM Card andCheckCard
Retail ConsumerLoans: AHInterface to ACLS
Retail ConsumerLoans ExistingFTU Port'o
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
HR/Training
HR-Payroll/BenefitsConversion
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Operations
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Item Processing(Crossover Logic)
InformationTechnology
EBSA/DRCore Doc Mngmt System
(Image File Folder)
Commercial Loans(CLS to AFS, TES)
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
TreasuryServices
Reg#1 Unit test
#1: Simulation
#1: RegionalConversion Live
Reg #1-4: DepositEliminate Duplicate AcctNums for all States
NGN Stress Test: Virginia
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
WealthManagement
Kinnexus Aggreg'nSoftware
Personal TrustSEI to NCS
Trust PortfolioOnline via Internet
Insurance ADS toPivot Life/Health
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
MergerProjectOffice
CapitalManagement
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Key Accounts toCap/Brokerage
BrokerageComplianceand Finance
Institutional Custody of4,500 accounts fromNCS (FTU) to SEI (FTU)
InstitutionalInvestments
OMPM
EB/401KCustody &
Transf. AgentConv'n: WB to
Evergreen StBk
M.F. WBFund toEvergreenSys.
WIA's to CAP (foreach region)
Int'l DOC toEXIMBILLS
Int'l LOC toEXIMBILLS
Leveraged Fin. Asset-Based Lending Conv'n
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
Corporate &Investment
Balance Sheet MngmtPortfolio
Fixed Income Salesand Trading
ForeignExchange Conv'nBSM Custodian Conv'n
REFS CPConduit Conv'n
Int'l GFI &Payments Investm't Banking Loan
Syndication Conversion
Int'l: ConsolidateLondon Branches Leveraged Fin.
Leasing Conv'n
International BrandingSafekeeping Conv'n
2/1/02Jan 1, 2002 Dec 31, 2003
3/1/02 4/1/02 5/1/02 6/1/02 7/1/02 8/1/02 9/1/02 10/1/02 11/1/02 12/1/02 1/1/03 2/1/03 3/1/03 4/1/03 5/1/03 6/1/03 7/1/03 8/1/03 9/1/03 10/1/03 11/1/03 12/1/03
RegionalDeposit
Conversions
Legal NameChange
MergerConsummationof Bank Charters
FirstNet/ETNVideo Sat. Dish
RolloutNorth Region PennDel
and Atlantic ExternalSignage Change
T&EExpenseReporting
Compas/OFSA Profitability
EIM - Data Warehousecombine WB & FTU
Network StressTest Florida
Network StressTest Georgia
NGN Network
EIM - Sold/ProConversion (lead
generation)
Network StressTest N/S Carolina
Treasury Svcs FileTransmissions w/Existing Customers
Data CenterConsolidation CIC-Silas
InterofficeMail CodeConversion
OCCIF - CISCustomer Conv'n ACH (Prior to first
regional dep conv'n)
Wire Transfer - MTS& IPX to Intranet
ARP
Lockbox - WholesaleWILIS to DMP
ACH - PEP + 5.0 to 5.0.3DB2 Major Release
Denovo Branch Opens: Stuart, FL
Interim Mortg Servc'gto Raleigh
Dealer Financial Services(DFS) Interface Review
PCs in FinancialCtrs - FL (545)
Common Prod. Conv'n
PCs in Financial Ctrs- GA (255)
New Retail Consmr LoansOriginat'n via AH
Teller Argo for Win. - FL
#1: ReadinessReview
#1:Go/No-GoDecision
Teller Argo for Win. - GA
PCs in Fin Ctrs -N/S Carolina (442)
PCs in Fin Ctrs -VA (70)
Teller Argo forWin. - NC/SC
Teller Argo for Win (foreach region - Legacy
FTU must be 90 daysprior)
#4: VA Deposits, WIA/CAP, CD's, IRA, ODProtect. . .
Platform (witheach regional)
Treasury ServcsCustomer SoftwareRollout
Account Analysis (witheach regional conv'n)
Corp Phone Single Directory withnew Org. (follows PeopleSoft)
OFFITBANK TrustSEI to Sungard/NCS
Year-end 2001ProcessingMoratorium
GAP/EnhancementsFinal Approval
Year-end 2002ProcessingMoratorium
Year-end 2003 - Allconversions complete
2002-2003 Conversion Timeline(based on 05/14/02 Project Plan)
SignageChange (Start)
Today
Charitable TrustSIEBEL ContactManagment
CommercialInsurance Agencies
Contact Center -Columbia Call
Center
Updated: 05/20/02
EIM - Base Warehouse InfoIntegration-Pre Legacy
Conversions/Joint Householding -Grouping all Relationships
EIM - Legacy/Source ApplicationConversions - Data Warehouse,
Marts, COMPAS
EIM - Consumer Risk Management InfoIntegration - Pre Legacy Conversions
CIS/OCCIF and Relationship(with each regional dep conv'n)
Teller Argo for Win. - VA
Platform - Argo Salesand Service (IncludesGUI for SOLD)
SOLD-Prospecting
Early AnnuitySales Rollout
#2: GA Deposits, WIA/CAP, CD's, IRA, OD
Protect. . .
DUC - Statement PrintMove: WEC to DUC
Wire Tr. Upgrade VAX toUNIX on Intranet Wires
Region #1-#4Deposit: Account
Number Generationfor all States
SOLD-Sales Tracking
Divestiture-Man'l ConvWIA to CAP
SOLD-Book ofBusiness
Community Banking-First Contact
Re-Issue LWB FL-ATMCards
Trust SEI Data Merge
Day 2 ExceptionProcessing-Vector 5to GEMS
1/2004 12/2004
Data Center Move/Consolidations
ImageDEO/PODPlatformConv'n
Business Requirements
File Retention ConversionLockdown
EnhancementLockdown
IntegratedTest
Integ.Testday2 proc.
Integ.Testday 3
Mock Conversion
Day 1 Mock
Start IntegrationTesting
Data CenterMove- Go/No-go
Data CenterMove Simulation
Integ.Testday1 proc.
Emerald-new version
Credit Card Conv'n:EDS to MBNA/TSIS
Data Center Move:Baymeadows toSilas
Lockbox-DallasWILIS to DMP
New WachoviaBrand/LogoAnnouncement
AcctsPayable toOracle Finan
SBLOriginat'n
CommonProductConv
RIGSelf-
DirectedIRA
Image Archive -SDC - Jacx toSilas
Image ArchiveRetrieval Mgr
Lockbox-Charlotte WILISto DMP
Lockbox-AtlantaWILIS to DMP
CommunityBanking-Firstcontact
Internet eMail Address Conversion
Internet Email Conversion(@firstunion.com to @wachovia.com
EIM - Consumer Risk Management InfoIntegration - Pre Legacy Conversions
Credit Card -Merchant
Slide 4 © Carliss Y. Baldwin and Kim B. Clark, 2005
Goals for an Ultra-Large System
Complete interoperability of components(mix-and-match)
Seamless upgradability Unlimited extensibility
No First or Second Level Hairballs!Internet is good at the First Level but has
Second Level Problems (e.g. Windows)
Slide 5 © Carliss Y. Baldwin and Kim B. Clark, 2005
To avoid complexity catastrophesin Ultra-Large Systems we need:
Tools/techniques to map actualinterdependencies in large, changingcodebases
Formal measures of design option value exante and ex post
“If you can’t measure it, you can’t manageit.”
Slide 6 © Carliss Y. Baldwin and Kim B. Clark, 2005
Complexity = Interdependency There are ex ante architectectural representation
languages (UML)– Like architectural drawings
But very few code maps– Maps are ex post, updatable, show what’s really there– Like plumber’s/electrician’s diagrams– Example: Rusovan, Lawford, Parnas (2005) critique of
Linux was based on ONE sourcefile! Architecture of functionally similar codebases can
be very different
Slide 7 © Carliss Y. Baldwin and Kim B. Clark, 2005
Mozilla Before Redesign Mozilla After Redesign
Browsers
Slide 10 © Carliss Y. Baldwin and Kim B. Clark, 2005
These architectural differencesaffect design option value Design options have “technical potential”,
denoted σ Technical potential, σ, varies
– By system– By module– Over time
Slide 11 © Carliss Y. Baldwin and Kim B. Clark, 2005
Global Design Rules v.1
Version 1.0Version 1.2
Version 1.5Version 1.8
σ = Low Medium Zero High
Evidence of Option Value Successive, improving versions are evidence of
option values being realized over time—after thefact
Designers “see” option values before the fact
Slide 12 © Carliss Y. Baldwin and Kim B. Clark, 2005
Measuring Option Value Measure unpredicted residuals from
performance data Example: TPC-C is a benchmark for
transactions processors– 153 submissions from 1/1/01 - 1/25/05– Use regression to cull out predictable
performance:
TpmC = a0 + a1(time) + a2(CPUs) + a3(FrEnds) + ε
Slide 13 © Carliss Y. Baldwin and Kim B. Clark, 2005
Measuring Option Value (cont)TpmC = a0 + a1(time) + a2(CPUs) + a3(FrEnds) + εStdDev(ε) is a proxy for σ—
02.0e
-06
4.0e
-06
6.0e
-06
Dens
ity
-500000 0 500000 1000000e
σ= ± 124814
σ= ± 69% ofMean TpmC