Technical Debt

University of Southern California

Center for Systems and Software Engineering

Technical Debt

CS 510 Software Engineering

Supannika Koolmanojwong



Outline

• What is Technical Debt? • Technical Debt Patterns• Technical Debt Calculation & Estimation• Technical Debt – Industrial perspective

2



Technical Debt Example

3

“Guys, we don’t have time to dot every I and cross every t on this release. Just get the code done. It doesn’t have to be perfect. We’ll fix it after we release.”

“We don't have time to reconcile these two databases before our deadline, so we'll write some glue code that keeps them synchronized for now and reconcile them after we ship.”Ref: Steve McConnell – ICSE 2013



Total cost of ownership• a financial estimate whose

purpose is to help consumers and enterprise managers determine direct and indirect costs of a product or system.

– including the costs to research, develop, acquire, own, operate, maintain, and dispose of a system

4

www.comindwork.com

http://www.comindwork.com/daily/2013-04-08/productivity/total-cost-of-ownership-tco-for-project-management-software



Total Cost of Software Ownership

5

www.dtbusiness.com

newtecharticles.com

http://www.dtbusiness.com/index-1.html

http://newtecharticles.com/software-engineering-introduction/



Potential total cost of ownershipComputer hardware and

programsOperation expenses Long term expenses

• HW and SW• Network• Server• Workstation• Installation & integration

• Purchasing research• Warranties and licenses• License tracking -

compliance• Migration expenses• Risks: susceptibility to

vulnerabilities, availability of upgrades, patches and future licensing policies, etc.

• Infrastructure• Electricity• Testing costs• Downtime, outage and

failure expenses• Diminished performance• Security (including

breaches, loss of reputation, recovery & prevention)

• Backup and recovery process

• Technology training• Audit• Insurance• IT personnel

• Replacement• Future upgrade or scalability

expenses• Decommissioning

6



Common causes of technical debt

• Business pressures• Lack of process or understanding• Lack of building loosely coupled

components (hard-coded)• Lack of documentation• Parallel Development• Delayed Refactoring

http://en.wikipedia.org/wiki/Technical_debt7

http://en.wikipedia.org/wiki/Technical_debt



8

Single System Development Perspective

Choices driven by potential– Market share– Future opportunities– Technical debt– Cost of failure to provide needed

capability



Technical DebtCode Debt

DESIGN DEBT

Deficit programming

Technical Inflation

CodeSmell

Software Decay

coined by Ward Cunningham

TOXIC CODE



10Ref: Zadia Codabux and Byron J. Williams. Managing Technical Debt: An Industrial Case Study

http://www.sei.cmu.edu/library/abstracts/presentations/codabux-mtd2013.cfm



Technical Debt Observations“Agile Project Management”, Jim Highsmith, second edition

11



Types of Debt“Managing Software Debt: Building for Inevitable Change”,

Chris Sterling

• Technical Debt– These are activities that a team or team members choose not to do well

now and will impede future development if left undone• Quality Debt

– There is a diminishing ability to verify the functional and technical quality of software

• Configuration Management Debt– Integration and release management becomes more risky, complex and

error-prone• Design Debt

– The cost of adding features is increasing toward the point where it is more than the cost of writing from scratch.

• Platform Debt– The availability of people to work on software changes is becoming

limited or cost-prohibitive.

www.psmsc.com12



Technical Debt

• “is a measure of how untidy or out-of-date the development work area for a product is”

• Not the deferred requirements

http://www.c2.com/cgi/wiki?TechnicalDebt13

http://www.c2.com/cgi/wiki?TechnicalDebt



Technical Debt• “I don’t know what happened, I just

changed one line”• “We can’t upgrade, It will break”• “We can’t upgrade the code, we don’t have

time”• “We can’t upgrade the code, no one

understands it” • “Just put in the comment XXX, we will do it

later” • “Just put in the TODO comment”

http://petdance.com/perl/technical-debt 14

http://petdance.com/perl/technical-debt



Technical Debt

• intentional technical debt– cost to taking short cuts

• unintentional technical debt– making mistakes– Conscious decision to optimize for the “present”

rather than the “future”• cost of not dealing with these short cuts and

mistakes will increase over time.

Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phjvB5A915

http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html



http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf16

http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf



http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf17




18http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf




19http://pkruchten.files.wordpress.com/2011/10/kruchten-111027-techdebt.pdf




Outline

• What is Technical Debt? • Technical Debt Patterns• Technical Debt Calculation & Estimation• Technical Debt tool – Industrial perspective

20



Technical DebtArchitecture or the platform technology mistake

• Scalability, reliability• Foundations are poorly design• you don’t find out until too late (operations phase)• no choice but to start again or rewrite big chunk

to keep it working

Read more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUV 21




Technical DebtError-prone code

• “ .. If I change X, it is going to break Y, I think ..”• “ Don’t touch that code, last time we did, we

spent a week fixing it…”• 20% of the code where 80% of bugs are found• Hard to understand• Dangerous to change because done poorly one

in the first place• Not rewriting this code is one of the most

expensive mistakes that developers makeRead more: http://www.javacodegeeks.com/2012/02/technical-debt-how-much-is-it-really.html#ixzz1phnGFlUVhttp://petdance.com/perl/technical-debt/technical-debt.007.html

22


http://petdance.com/perl/technical-debt/technical-debt.007.html



Technical DebtNot easily tested

• “ .. I thought we had a test for that ..”• Don’t have good automated tests• Tests keep falling apart when you change the

code• Testing costs tend to go up over time as you write

more code





Technical DebtCode that mysteriously works

• nobody is sure how or why• Might be written by the geek who left the

company• if nobody on the team understands it, it’s a time

bomb





Technical DebtOthers

• Forward and backward compatibility– Short term debt

• Duplicate, copy-and-paste code– How many ? Trackable ?

• Hard coding• Out of date documentation

– “We just lost the drive, where are the backups” – If nobody is using it, get rid of it. If people are using it,

why isn’t it up to date?





Analysis Design Implementation Test Integration

Development Cost(Perfect World)%

Effo

rt p

er P

hase

27




Defect Insertion Defect detection & Removal

DesignDefects

IntegrationDefects

Typical Defect Profiles

ImplementationDefects

Classification of Defects

28




Development Cost(Real World)%

Effo

rt p

er P

hase

29




Development Cost%

Effo

rt p

er P

hase

Real world Perfect World 30




Development Cost%

Effo

rt p

er P

hase

TechnicalDebt?

TechnicalDebt?

TechnicalDebt?

TechnicalDebt?





Development Cost%

Effo

rt p

er P

hase

TechnicalDebt?

TechnicalDebt?





COTS Integration%

Effo

rt p

er P

hase

TechnicalDebt?




Outline


35



Financial debt vs Technical Debt

Interest payments extra effort in future development because of the quick and dirty design choice

continue paying the interest

suffer for performance, problem is still there

pay down the principal refactoring the quick and dirty design into the better design; also reduce interest payments in the future

http://martinfowler.com/bliki/TechnicalDebt.html36

http://martinfowler.com/bliki/TechnicalDebt.html



Technical Debt Cost Estimation

37

Principal = Repair EffortInterest = Maintenance Effort

[Nugroho et al 2011]

[Curtis et al 2012]

Ref: Nugroho et al 2011, An Empirical Model of Technical Debt and Interest, MTD 2011Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software

Debt(in man days)

cost_to_fix_duplications + cost_to_fix_violations + cost_to_comment_public_API + cost_to_fix_uncovered_complexity + cost_to_bring_complexity_below_threshold + cost_to_cut_cycles_at_package_level



38

Conservative - Assume all violations would be fixed within one hour

Weighted - varied the hours needed for fixing within each severity category,

More realistic - based on data observed in several ITorganizations.

• 700 applications• 158 organizations• 357 MLOC• No sample less than 10KSLOC

Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software



39

Average Technical Debt per LOC = $3.61

Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software



40Ref: Curtis e al 2011, Estimating the Principal of an Application's Technical Debt. IEEE Software



Technical Debt within each technology

41http://www.castsoftware.com/research-labs/technical-debt-estimation



Outline


42



Sonar

• Mainly manage the software quality• SQALE – Software Quality Assessment

based on Lifecycle Expectations

43



Nemo - An open source tool – by sonarhttp://nemo.sonarqube.org

44



Sonar – SQALE plug-inhttp://www.sonarsource.com/products/plugins/governance/sqale/

45



Developers’ 7 Deadly Sins

• Bad Distribution of Complexity• Duplications• Lack of Unit Tests• No Coding Standards• Not Enough or Too Many Comments• Potential Bugs• Spaghetti Design

46http://docs.codehaus.org/display/SONAR/Developers%27+Seven+Deadly+Sins



Developers’ 7 Deadly Sins• Bad Distribution of Complexity

• Duplications




Developers’ 7 Deadly Sins• Lack of Unit Tests

• No Coding Standards




Developers’ 7 Deadly Sins• Not Enough or Too Many Comments (time consuming

maintenance)– Documented complex or not straightforward pieces of code– Readable unit tests scenarios that will help the developers understand the code– Documented API that will help external developers understand how to use those API– Respect of coding standard such as naming conventions

• Potential Bugs




Developers’ 7 Deadly Sins• Spaghetti Design

– Hunting for cycles– Checking your own architecture rules– Managing your libraries– Checking the single responsibility principle with LCOM4 (Lack of

Cohesion of Methods)– RFC (Response for Class) – checking coupling (complexity of class in

terms of method calls)




51http://www.moovweb.com/blog/technical-debt-infographic/

x

=

x =



Back up charts

53



Fixing technical debt• Big Bang

– no new features for a year? Really? – spend some time cleaning up the mess– Good ?

• Dedicated Team– Have another team dedicated – Good ? 80/20 rule ?

• Boy Scout– remove technical debt little and often– If no tests, add some. If poor test, improve them. If bad code,

refactor it– The boy scout rule – leave the camp cleaner than you found it

http://www.javacodegeeks.com/2011/11/dealing-with-technical-debt.html#ixzz1pjQ8bQpF 54

http://www.javacodegeeks.com/2011/11/dealing-with-technical-debt.html%23ixzz1pjQ8bQpF



Fixing technical debt

• Think time & risk & $$$• No gold-plating

– Unnecessary task that no one wants • BigResearchUpFront vs BigDesignUpFront

55



Technical Debt vs Agile : Bad

• Quick and dirty approach• Scalability vs DoTheSimplestThingThatCouldPossiblyWork

– Just-in-time Scalability• change the architecture in response to actual

customer demand– Wasted Code

http://www.startuplessonslearned.com/2009/07/embrace-technical-debt.html 56

http://www.startuplessonslearned.com/2009/07/embrace-technical-debt.html



Technical Debt vs Agile : Good

• Get feedback faster• Smaller batches





Technical Debt vs Lean : Good

• Less waste, less debt• Just-in-time nature





Technical Debt vs NDI : Bad

• Product Development Leverage– Your product is fortified by others

• Open-source, COTS, Services• Pros: Faster• Cons

– Code understanding– Different coding styles, architecture, quality– No direct control



Technical Debt

Documents

Transcript of Technical Debt