Post on 26-Jan-2017
Crushed by Technical Debt? Disciplined Agile Strategies to Dig Your Way Out
The primary goal of this presentation is to help you to understand the issues surrounding technical debt. It overviews a collection of strategies for avoiding, finding, fixing, and funding technical debt.
© 2015 Disciplined Agile Consortium 2
Let’s explore several important questions…
What is technical debt?
How can we avoid technical debt?
How do we identify technical debt?
How do we fix technical debt?
When should we accept technical debt?
How can we fund technical debt removal?
© 2015 Disciplined Agile Consortium 3
The Survey Results Shared in This Presentation
• All surveys were performed in an open manner
• The questions as they were asked, the source data, and a summary slide deck can be downloaded free of charge from Ambysoft.com/surveys/
© 2015 Disciplined Agile Consortium 4
What is Technical Debt?
Technical debt is the accumulation of defects, quality issues (such as difficult to read code or low data quality), poor architecture, and poor design in existing solutions
Types of technical debt: – Code – Data – Documentation – Architectural – User Interface (UI) – Operational infrastructure – Middleware – Tests (or lack thereof) – …and many more
© 2015 Disciplined Agile Consortium 5
Why Does Technical Debt Occur?
• Business pressures • Lack of process • Lack of alignment of implementation to requirements • Lack of architectural thinking • Lack of a regression test suite • Lack of documentation • Lack of collaborative development • Poorly coordinated parallel development • Delayed refactoring • Lack of alignment to standards • Lack of knowledge • Lack of effective governance
© 2015 Disciplined Agile Consortium 6
Source: Reworked from Wikipedia
How Aware Are People About Technical Debt?
-1.8
1.3 2.3 2.5
3.9 4.9
6.7
Copyright 2015 Scott Ambler + Associates
Source: SA+A 2015 Q1 Agile State of the Art Survey
Observation: Development teams are often far more aware of technical debt than key decision makers are
Technical Debt Quadrant
© 2015 Disciplined Agile Consortium 8
Reckless Prudent
Deliberate
Inadvertent
“We don’t have time for architecture”
“We don’t have time
for design”
“We must ship now and deal with the
consequences later”
“What is layering?”
“What is data normalization?”
“What is UX design?”
“Now we know how we should
have done it”
Source: Martin Fowler
Disciplined Agile Delivery (DAD) is a process decision framework The key characteristics of DAD:
– People-first – Goal-driven – Hybrid agile – Learning-oriented – Full delivery lifecycle – Solution focused – Risk-value lifecycle – Enterprise aware
© 2015 Disciplined Agile Consortium 9
Claim: DAD motivates you towards the prudent end of the
spectrum
Disciplined Agile 2.0: The Agile IT Department
10 @scottwambler
Avoiding Technical Debt
© 2015 Disciplined Agile Consortium 11
Explore the Initial Scope
Strategy: – At the beginning of an endeavor
you should invest some time to identify what your stakeholders expect you to deliver
– This should be done in a light-weight manner
Why do this? – Reduces risk of injecting
technical debt caused by misunderstanding the domain
– Improves productivity during Construction
© Disciplined Agile Consortium 12
Build Quality In From the Beginning
© Scott Ambler + Associates 13
Strategy: – At the beginning of an
endeavor you should identify the quality of service (QoS)/non-functional requirements
Why do this? – Reduces risk of
injecting technical debt caused by missing key QoS needs
Observation: Most architectural debt results from missed QoS requirements
© Disciplined Agile Consortium 14
Goal: Explore Initial Scope
Observation: There’s more to agile
requirements than writing user stories
Identify the Initial Technical Strategy
Strategy: – At the beginning of an endeavor
you should invest some time to identify how the solution is to be built
– This should be done in a light-weight manner
Why do this? – Reduces risk of injecting
technical debt caused by inappropriate technical decisions
– Improved productivity during Construction
© Disciplined Agile Consortium 15
Architecture Owner
• Guides the creation and evolution of the solution’s architecture
• Mentors and coaches team members in architecture practices and issues
• Understands the architectural direction and standards of your organization and ensures that the team adheres to them
• Ensures the system will be easy to support by encouraging appropriate design and refactoring
• Ensures that the system is integrated and tested frequently
• Has the final decision regarding technical decisions, but doesn’t dictate them
• Leads the initial architecture envisioning effort • Works closely with enterprise architecture team (if one
exists) • Responsible for technical risk mitigation
16 © Disciplined Agile Consortium
Observation: Often leads the avoidance and
removal of technical debt
Goal: Identify Initial Technical Strategy
© Disciplined Agile Consortium 17
Observation: A little bit of agile architecture
modeling can avoid a lot of future rework
Be Enterprise Aware
Strategy: – Disciplined agilists work closely with
enterprise professionals, including enterprise architects
– Your architecture owner may be a member of the enterprise architecture (EA) team
– Leverage your existing ecosystem to reuse existing assets such as services, components, data sources, and more
Why do this? – Improves consistency and
collaboration between teams, leading to greater quality and thereby lower technical debt
© Scott Ambler + Associates 18
Enterprise Architecture Teams
© Scott Ambler + Associates 19
Observation: A collaborative, light-weight approach to EA increases the
chance that delivery teams will understand and follow the roadmaps
© Disciplined Agile Consortium 20
Observation: Common architectural roadmaps
and guidelines can help to avoid injection of new technical debt
© Disciplined Agile Consortium 21
Observation: The greater the level of
reuse, the lower overall technical debt
Following Organizational Guidelines
© Disciplined Agile Consortium 22
Strategy: – Teams adopt and follow organizational guidelines
Examples of guidelines: – Development, database, user interface, and coding guidelines – Testing guidelines (e.g. unit test coverage) – Governance guidelines – Architecture guidelines – Asset reuse guidelines – Documentation guidelines
Why do this? – Following guidelines can improve quality, consistency, supportability,
and consumability of solutions across the organization
© Disciplined Agile Consortium 23
Goal: Align with Enterprise Direction
Observation: A lot of technical debt is caused by development teams that don’t understand the “big picture”
Technical Debt Avoidance Strategies
Detailed up-front architecture modeling
Team members trained in technical debt
Team works with Enterprise Architects
Team includes Architecture Owner/Agile Architect
Tech debt considered when designing
Lightweight up-front architecture
12%
16%
19%
39%
49%
53%
Copyright 2015 Scott Ambler + Associates
Source: SA+A 2015 Q1 Agile State of the Art Survey
Finding Technical Debt
© 2015 Disciplined Agile Consortium 25
Regression Testing
© 2015 Disciplined Agile Consortium 26
Strategy: – Have a full regression test suite for
all of your IT assets – This includes systems/applications,
purchased packages, legacy databases, services, and other IT infrastructure components
– Run it often
Why do this? – Enables teams to safely evolve their
solutions – Enables teams to find potential
problems at the point in time that they inject them
Test-First Development (TFD)
© Disciplined Agile Consortium 27
Strategy: – Write a single test before you write
just enough production code to fulfill that test
Why do this?
– Forces you to think before you code – Results in development of higher-
quality assets – Reduces technical debt associated
with poor requirements or poor design
Automated Tooling to Detect Technical Debt
Strategy – Include static and dynamic code analysis
in your continuous integration (CI) strategy
Many potential code analysis tools: – CAST – SonarQube – Checkstyle – FindBugs – Database schema tools such as TOAD are
emerging
Why do this? – Straightforward and easy way to identify
potential quality problems
© 2015 Disciplined Agile Consortium 28
Continuous Integration (CI)
29 © Disciplined Agile Consortium
Strategy – Automatically build, test, and
measure your work whenever you check a file in
Why do this? – Enables previously mentioned
techniques, including regression testing, TFD, and static/dynamic analysis
Measure Technical Debt
Strategy – Make measurement of technical a common practice across
your organization
Subjective measures: – We know it when we see it
Potential objective measures: – Quality metrics from code analysis tools – Defect trends – Cycle time – Coverage testing
Why do this? – Provides insight into the extent of your technical debt
problem, thereby motivating the need to invest in its removal
© 2015 Disciplined Agile Consortium 30
Observation: What gets measured gets improved
Technical Debt Identification Strategies
Explictly measure tech debt across IT
Continuous integration strategy includes the database
Measure tech debt within teams
Continous integration includes code analysis
We know technical debt when we see it
8%
10%
20%
35%
61%
Copyright 2015 Scott Ambler + Associates
Source: SA+A 2015 Q1 Agile State of the Art Survey
Observation: If you can’t consistently
identify technical debt then you can’t fix it
Fixing Technical Debt
© 2015 Disciplined Agile Consortium 32
Refactoring
Strategy – Make simple changes to your design that improves the quality
without changing its semantics (in a practical manner)
Examples – Rename operation – Introduce variable – Rename column – Align entry fields
Why do this – Enables us to safely improve the quality of our work, including
legacy systems
© 2015 Disciplined Agile Consortium 33
Database Refactoring
Strategy – Fix data quality problems via
simple refactorings of your database
Why do it? – Data is a corporate asset, or at
least it should be – Significant data technical debt
exists in your legacy data sources
© Scott Ambler + Associates 34
Challenge: Refactoring is still considered a radical concept within the data management community
Technical Debt Removal Strategies
Database refactoring
User interface refactoring
Code refactoring
38%
53%
90%
Copyright 2015 Scott Ambler + Associates
Source: SA+A 2015 Q1 Agile State of the Art Survey
Accepting Technical
Debt
© 2015 Disciplined Agile Consortium 36
Explicitly Accept Technical Debt
Strategy – Explicitly accept existing technical debt, or even accept
injection of new technical debt – This is a business decision that is the responsibility of the
Product Owner – The team, in particular your Architecture Owner, must
educate the Product Owner on the implications of accepting technical debt
Why do this? – Sometimes it makes sense to accept technical debt in the
short term, particularly given schedule constraints
© 2015 Disciplined Agile Consortium 37
Architecture Owner
Product Owner
Recommendation: Document the reasons in your architecture handbook as to why you accepted the technical debt
Challenges
• Business doesn’t understand technical debt
• Business wants new functionality
• Time-to-market concerns are often allowed to override long-term sustainability concerns
• Acceptance of technical debt is typically the result of short-term tactical thinking, not long-term strategic thinking
© 2015 Disciplined Agile Consortium 38
Funding Removal of Technical Debt
© 2015 Disciplined Agile Consortium 39
Explicitly Fund Removal of Technical Debt
Strategy – Choose a strategy to fund the removal of technical debt
Why do this?
– It requires investment to remove technical debt
© 2015 Disciplined Agile Consortium 40
Option Advantages Disadvantages Built-Into Team Budget
• Easy to estimate costs • Often insufficient • TD removal often dropped in
favor of new development • Seen as an overhead
Technical Stories
• Easy to estimate costs • Explicit funding strategy for
small-scale TD removal
• Clear impact on team velocity motivates deprioritization of such stories
Projects • Explicit funding strategy for large-scale TD removal
• Becomes a big ticket item in your IT budget
Govern Technical Debt
Strategy – Explicitly include technical debt in your IT
governance activities
Potential activities – Monitor and track technical debt – Educate, coach, and mentor people in
technical debt skills and thinking – Set organizational and team goals related to
technical debt
Why do this? – Gets senior management behind the
avoidance and removal of technical debt
© 2015 Disciplined Agile Consortium 41
Technical Debt Funding Strategies
Cost of addressing tech debt is tracked
Value of addressing tech debt is tracked
Specific projects to pay down tech debt
Paying down tech debt automatically funded
Specific requirements to pay down tech debt
5%
14%
23%
48%
49%
Copyright 2015 Scott Ambler + Associates
Source: SA+A 2015 Q1 Agile State of the Art Survey
© 2015 Disciplined Agile Consortium 43
Technical Debt Requires a Culture Change
• Make technical debt awareness part of your culture
• Educate people in technical debt and the implications of it
• Help people recognize the tradeoffs that they’re making
• Communicate, communicate, communicate
© 2015 Disciplined Agile Consortium 44
You Can Address Technical Debt
• This presentation shared a collection of strategies to: – Avoid technical debt – Find and fix technical debt – Find and accept technical debt – Fund technical debt related efforts
© 2015 Disciplined Agile Consortium 45
Adage: The best time to plant a tree was 20 years ago. The next best
time is today.
If You Don’t Address Technical Debt…
© 2015 Disciplined Agile Consortium 46
Thank you – Questions?
• Scott Ambler + Associates – ScottAmbler.com
– scott@scottambler.com @scottwambler
• Disciplined Agile Delivery: A Practitioner’s Guide, by Scott Ambler & Mark Lines
• Introduction to Disciplined Agile Delivery: A Small Team’s Journey, by Mark Lines and Scott Ambler
• DisciplinedAgileDelivery.com • DisciplinedAgileConsortium.org • DAD LinkedIn Discussion Group:
– linkedin.com/groups/Disciplined-Agile-Delivery-4685263
47
Scott Ambler + Associates is the thought leader behind the Disciplined Agile Delivery (DAD) framework and its application. We are a boutique IT management consulting firm that advises organizations to be more
effective applying disciplined agile and lean processes within the context of your business.
Our website is ScottAmbler.com
We can help
© 2015 Disciplined Agile Consortium 48
For the Certified Disciplined Agilists among us…
This webinar counts as one hour towards your education time to maintain your certification
See DisciplinedAgileConsortium.org/certification for details
© 2015 Disciplined Agile Consortium 49
Shuhari and Disciplined Agile Certification
At the shu stage you are beginning to learn the techniques and philosophies of
disciplined agile development. Your goal is to build a strong foundation from which
to build upon.
At the ha stage you reflect upon and question why disciplined agile strategies work, seeking to understand the range of strategies available to you and when they
are best applied.
At the ri stage you seek to extend and improve upon disciplined agile techniques,
sharing your learnings with others.
© 2015 Disciplined Agile Consortium 50
Disciplined Agile Delivery (DAD)
Disciplined Agile Delivery: The Foundation for Scaling Agile
© 2015 Disciplined Agile Consortium
Scrum Lean Kanban
Unified Process Agile Modeling
And more… “Traditional” DevOps
Team Size Geographic Distribution Compliance
Domain Complexity
Technical Complexity
Organizational Distribution Team Culture Organizational
Culture
DAD leverages proven strategies from several sources, providing a decision framework to guide your adoption and
tailoring of them in a context-driven manner.
51