RAL14114USEN
-
Upload
adarsh-mishra -
Category
Documents
-
view
212 -
download
0
Transcript of RAL14114USEN
-
A Forrester Consulting
Thought Leadership Paper
Commissioned By IBM
October 2014
The New Software
Imperative: Fast Delivery
With Quality Eight DevOps Practices Are The Key To Success
-
Table Of Contents
Executive Summary ........................................................................................... 1
The Race Is On To Deliver In Under Three Weeks With Quality .................. 2
The Challenge: Achieve Consistently Fast Cycle Times .............................. 4
To Deliver Fast With Quality, Adopt Eight Key DevOps Practices .............. 5
Working Smarter Achieves Rapid Delivery Cadences .................................. 8
DevOps Eliminates The Tradeoff Between Cycle Time And Risk ............... 8
Key Recommendations ................................................................................... 10
Appendix A: Methodology .............................................................................. 12
Appendix B: Demographics/Data ................................................................... 13
ABOUT FORRESTER CONSULTING
Forrester Consulting provides independent and objective research-based
consulting to help leaders succeed in their organizations. Ranging in scope from a
short strategy session to custom projects, Forresters Consulting services connect
you directly with research analysts who apply expert insight to your specific
business challenges. For more information, visit forrester.com/consulting.
2014, Forrester Research, Inc. All rights reserved. Unauthorized reproduction is strictly prohibited. Information is based on best available resources. Opinions reflect judgment at the time and are subject to
change. Forrester, Technographics, Forrester Wave, RoleView, TechRadar, and Total Economic Impact are trademarks of Forrester Research, Inc. All other trademarks are the property of their respective companies. For additional information, go to www.forrester.com. [1-Q4RGU3]
-
1
Executive Summary
The pressure to quickly develop and deliver quality software has entered a phase of
new intensity. Most shops report being pressured to cut their cycle times.1 And its no
wonder: Demands for new customer-oriented software, new systems to counter digital
competitors, and new software-driven business innovations make application
development crucial to business success.
IBM commissioned Forrester Consulting to conduct research on development
practices and cycle times within businesses to help evolve its DevOps maturity model.
Through a survey of 600 IT professionals with application development responsibilities in the US, Canada, France, Germany,
and the UK, Forrester found that about a third of teams can consistently deliver at cadences of one to three weeks. The
fastest teams generated far better business satisfaction than slower teams, an indication they dont sacrifice quality to deliver
fast. Our study reveals how these teams were able to break through to achieve consistently fast software delivery with high
business satisfaction: by adopting eight DevOps practices to replace waterfall methods and surpass Agile methods.
DevOps practices will be the new normal as software development teams respond to business needs for more software
delivered faster at an efficient cost and with high quality. Without adopting DevOps practices, teams will fail to deliver quality
software consistently on fast business schedules.
KEY FINDINGS
Forresters study yielded five key findings:
Development teams that consistently deliver at the fastest cycle times enjoy the highest business satisfaction. We found a high correlation between fast cycle times and business satisfaction in our survey results. In this study, high
business satisfaction is an indicator of software quality. The reverse is also true teams delivering at the slowest cycle
times self-reported the lowest business satisfaction with their projects.
Incremental improvements to waterfall methods run out of steam at one- to two-month delivery cycles. Our findings strongly suggest that sustained cycle times of two months or less are impossible to achieve with incremental
improvements to waterfall methods. The overhead of those regimes even when supplemented by Agile methods
eats up too much of a multiweek delivery window to succeed. To consistently deliver quality software at the fastest cycle
times, teams adopt new methods and software designs and leave waterfall behind.
Eight DevOps/continuous delivery practices are the key. DevOps/continuous delivery relies on eight practices: 1) deliver small increments of functionality; 2) use dedicated, cross-functional teams; 3) use loose architectural coupling; 4)
automate environment provisioning; 5) continuously integrate code; 6) continuously test; 7) continuously fund; and 8)
provide real-time transparency.
DevOps practices address the top reasons for project disappointment. DevOps practices directly address the reported causes of business dissatisfaction with software projects. We found the practices used by the fastest teams to
achieve high business satisfaction were a mirror image of the practices the slowest teams reported caused low business
satisfaction.
DevOps practices reduce cycle time and the risk of failure at the same time. Conventional wisdom and intuition say that faster release must come at the expense of higher risk of project failure. DevOps techniques turn this logic on its head.
DevOps practices enable faster and safer releases because they are smaller, less complex, and more thoroughly tested.
Automating provisioning and deployment further reduces configuration errors and speeds up delivery.
Development teams
that delivered software
fastest enjoyed far
better business
satisfaction than
slower teams.
-
2
The Race Is On To Deliver In Under Three Weeks With Quality
Software development and delivery has always been a difficult race against time. Now, that race has entered an even more
challenging phase. Software delivery schedules measured as cycle time from project start to delivery of working product
have followed a downward curve since the 1960s, dropping further with the advent of each major era in our industry. The
latest era focuses on software for and about customers, including consumer apps, connected products, B2B sites and apps,
and all manner of systems to enable and support citizens, students, soldiers, parents, and so on.
FIGURE 1
The Age Of The Customer Demands Faster Software Delivery
Years
1960s
Data Processing Batch automation of accounting, back office
IT Database, online systems-of-record,and PCs automate front office
The Internet eBusiness brings external(Web) access to internal business processes
The Customer Mobile and socialempower customers; systems of engagementwin, serve, and retain them
1980s
1990s
2010
RequiredCycleTime
Year
Days
Source: Forrester Research, Inc. 2014
-
3
Why? To start, customer-facing software must keep pace with changing tastes, needs, and competition. And theres a lot of
demand for customer-facing software. Consumers are as fickle as ever, but now have more choices for products and
services. Your hot offering last week will shortly be old news and youll have to build new software or modify existing
software to create a new advantage.
But theres more (see Figure 2):
Digital competitors firms that use software to disrupt established markets are everywhere, even in staid industries like power generation/distribution, travel, and banking. Digital businesses move faster than hardware- or
people-based businesses because software can evolve faster than physical things. The result: escalating demands for
new digital services.
Software success is increasingly indistinguishable from business success. All business innovation requires new software, changes to software, or both. And business innovations cant wait for long software cycles to finish. The result:
demands to test business ideas in software within days or hours at a low risk and low cost.
No shop is immune from these business pressures. As a software delivery community, we need to raise our game to deliver
faster. How much faster? Our research suggests that one- to three-week cycle times are the gold standard. Software
delivery will utterly change to hit that mark.
FIGURE 2
Whats Driving Fast Delivery Requirements And How Well Respond
New App Delivery Models
As a service APIs
Mobile apps
Apps composed of services
DevOps/continuous delivery
New App Delivery Platforms
Cloud platforms
SaaS
Continuous Integration DevOps platforms
New Business Models
Open source
Subscriptions
On demand
Deliver fast to keep pace with fads, market shifts, opportunities
Deliver more digital business products, services, channels
Deliver quickly to prove out business ideas and innovations
Empowered,demandingcustomers
Increasingdigital
competition
Increasingexpectationsof software
Source: Forrester Research, Inc. 2014
-
4
The Challenge: Achieve Consistently Fast Cycle Times
Some teams in our study are producing software at the fastest cycle times today, which in our study we defined as one to
three weeks. We can learn from their experiences. As Figure 3 illustrates, about a third of our sample delivers all or some of
its work within one- to three-week cycles a group we call Delivering Fast. This group was also the most highly motivated
to reduce cycle times, and that is no coincidence. A second sizable group can sometimes deliver within three weeks, but not
consistently.
FIGURE 3
We Found Four Development Team Segments
High
Motivationto cut cycle
times
Ability to maintain fast cycle timesLow High
Delivering Fast32%
Improving Speed37%
Struggling forSpeed19%
DeliveringSlowly13%
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Consulting on behalf of IBM, May 2014
-
5
As well see, the delivering fast group succeeds because its development and delivery practices are tuned to delivering
quality software quickly. Anecdotally, introducing these new practices comes with the pain of any big change in the way
people work. Our studys results indicate that the pain is justified by the results.
Among the four groups of development teams in our study, when asked how successful they were at meeting the needs of
their individual, or companywide, business leaders, the delivering fast group enjoyed the highest rates of business
satisfaction with its work and the lowest negative ratings at the two fastest cycle times by far. The contrast between success
and failure is highest at the fastest cycle times, highlighting the need to change practices to consistently achieve those cycle
times (see Figure 4).
Delivery success is difficult to measure. Our study relies on self-reporting of how cycle times and dozens of software
development practices correlated with the project meeting the demands of the business leaders associated with the
respondents team.
FIGURE 4
The Fastest Teams Achieve High Business Satisfaction At Fast Cycle Times; The Others Do Not
Does the work you deliver at these cycle-times tend to be more
successful or less successful at meeting the demands of your business leaders?
Less successful at meetingbusiness demands
More successful at meetingbusiness demands
Delivering Fast
Delivering Slowly
Struggling for Speed
Improving Speed
1-3 weeks
1-3 weeks
1-2 months
1-3 weeks
1-3 weeks
1-2 months
1-2 months
1-2 months
25%
42%
30%
40%
21%
22%
6%
14%
15%
11%
27%
8%
26%
27%
53%
48%
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Consulting on behalf of IBM, May 2014
-
6
To Deliver Fast With Quality, Adopt Eight Key DevOps Practices
How do the teams that consistently deliver in two months or less do so? Their software practices show the way to fast
delivery with quality. Moreover, their experience suggests that to deliver consistently in two months or less, software teams
must adopt eight key practices:
1. Deliver in small increments of functionality. Fast delivery means building on a minimum viable product by delivering
value in the smallest increments possible. Small releases provide the fastest path to customer value realization. Small
releases mean fast feedback, which focuses effort on building the right things. Small releases also minimize risk by
reducing complex release dependencies.
2. Form dedicated, cross-functional teams. Sharing resources between teams wastes time, complicates scheduling,
weakens collaboration, reduces team throughput, and dilutes business expertise. Overspecialization and role silos kill
productivity and reduce delivery effectiveness.
3. Use loose architectural coupling within and between applications. Loose coupling allows straightforward change of
whole applications or components within applications. It can greatly reduce complexity and is one of the key enablers of
delivery in small increments.
4. Automate environment provisioning. Lack of consistent environment configurations causes hard-to-diagnose
production outages resulting from the it works fine on my machine syndrome. Long delays in getting suitably configured
development and test environments cause delay and increases cost. Automating provisioning dramatically reduces cost,
improves the effectiveness of development and testing, and reduces the number and severity of production incidents.
5. Continuously integrate code. Continuous Integration (CI) the building and testing of code every time a change is
delivered promotes faster user feedback. Early and continuous integration the building and testing of change sets
together exposes the most uncertain issues first and enables feedback on the most valuable usage models earlier.
6. Continuously test. Continuous testing results from extending CI with API-driven test automation that executes every
time code is changed. When enabled by loosely coupled architectures, continuous testing can automate the execution of
all functional and nonfunctional testing, leaving only UAT and exploratory testing for manual testers. Continuous testing
dramatically improves delivery speed, quality, and testing accuracy, while dramatically reducing cost.
7. Continuously fund. Organizations that deliver at rapid release cycles create a stable basis for funding a continuous
stream of innovation. They use customer feedback and business priorities to continuously prioritize the capabilities that will
result in the greatest business value.
8. Provide real-time transparency. The overhead of periodic and manual status reports and status meetings absorbs
valuable time and distracts focus from delivery value. Increasing transparency through instrumentation of the evolving
code and test base improves visibility into progress, and risk without decreasing productivity.
The survey results show a close correspondence between the practices of the delivering fast group and the barriers to
business success reported by the slowest delivering group (see Figure 5). The delivering slowly groups two biggest
barriers team members working on too many projects at the same time and too much status reporting and too many
meetings are the mutually reinforcing consequences of waterfall processes. No amount of incremental process
improvement will eliminate those barriers.
-
7
FIGURE 5
Delivering Fasts Strengths Are Delivering Slowlys Weaknesses
What are the characteristics of the projects that are most successfulat meeting the expectations of your business leaders?
(Delivering Fast only)
What are the characteristics of the projects that are least successfulat meeting the expectations of your business leaders?
(Delivering Slowly only)
53% 53% 52% 51% 51%48% 46%
20%
RequirementsTeam-memberscope
DesignUnits ofwork
Specialization& handoffs
Projectvisibility
Managementoverhead
Testing
26%
50%
23%28% 28% 28%
50%
33%
RequirementsDone In
Increments
TeamMembersAssignedTo OneStream
LooselyCoupledDesigns
SmallBatch
Releases
Cross-FunctionalTeams,Few
Handoffs
HighVisibility
intoProgress
FewStatusReports
&Meetings
AutomatedTesting
AllRequirementsDone Upfront
TeamMembersAssigned
ToManyProjects
TightlyCoupledDesigns
LargeBatch
Releases
SiloedSpecialists,
ManyHandoffs
LowVisibility
IntoProgress
ManyStatusReports
&Meetings
ManualTesting
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Consulting on behalf of IBM, May 2014
-
8
Working Smarter Achieves Rapid Delivery Cadences
Working more frantically to meet release dates will improve cycle times only incrementally. To achieve fast and predictable
releases consistently requires simplifying the work and automating everything possible. Hybrids of Agile methods and
traditional approaches (often called water-SCRUM-fall) can produce one- to two-month releases sometimes, but achieving
three-week cycle times consistently, inexpensively, and with high quality requires consistent use of DevOps practices and
supporting tools (see Figure 6). Traditional methods simply cant meet the mark.
FIGURE 6
DevOps Essential To Leap The Fast-Delivery Chasm
Delivery Cycle Times
Degree ofRisk
12+months
6-11months
3-5months
1-2months
1-3weeks
Days
Waterfall andisolated Agilepractices cantcross the chasm
DevOps practices,automation, and toolingare essential to fastdelivery with low
failure risk
Source: Forrester Research, Inc.
-
9
DevOps Eliminates The Tradeoff Between Cycle Time And Risk
Conventional wisdom and intuition say that a faster release must come at the expense of higher risk. Organizations steeped
in conventional delivery approaches assume that the only way to deliver faster is to cut corners, and the corner they most
often cut is testing. The result? Increased production incidents and lower quality. DevOps techniques turn this on its head;
faster releases, enabled by DevOps practices, are safer because (see Figure 7):
Faster releases demand smaller and less complex increments. Smaller, less complex releases are easier to deploy and easier to roll back when needed. They also have fewer dependencies because they use loose coupling.
Faster releases demand more test automation, which results in more thorough and continuous testing. DevOps practices like continuous integration and continuous testing produce higher quality because they find defects early, when
they are easier to fix, and they are more thoroughly tested. Continuous testing and service virtualizations enable
comprehensive testing from the start of the delivery cycle, shattering the conventional risk-speed tradeoff.
Automated environment provisioning can prevent and minimize configuration errors. Preventing configuration errors prevents production incidents that testing cant catch. Eliminating the root cause for these errors not only reduces
risk and improves stability, but it also eliminates costly incident response.
FIGURE 7
DevOps Practices Have A Counterintuitive Effect On Risk
HighIn waterfall practices,the risk of failurequickly rises asdelivery speedincreasesDevOps
practices enablefast delivery andfalling risk offailure at thesame time
Risk ofFailedRelease
Delivery Speed
Low
Years
This is a conceptual diagram.
Days
Source: Forrester Research, Inc.
-
10
Key Recommendations
DevOps/continuous delivery is a departure in practices for most teams and larger application development
organizations. We describe it as a revolution, but it is a revolution that unfolds over time. The danger is that application
delivery leaders will confuse incremental improvement of practices incapable of ever meeting the need for speedy
delivery of quality software with progress toward a new fast-delivery regime.
The start of your journey will depend on what kind of shop you are now. The delivering fast teams have already
arrived; they need to extend their success into the full measure of DevOps. The three other segments usually need to
start with the basics. If your teams are in the delivering slowly, struggling for speed, or improving speed categories,
here is how to start your journey (see Figure 8):
Start with consistent environments. Effective, automated on-demand environment provisioning provides the foundation for nearly all subsequent improvements. It reduces production incidents by preventing configuration
drift between development, test, and production environments. A closely related practice is the use of service
virtualization, which enables services and applications to be simulated without actually provisioning them. All
delivery cadences and all software life cycles, both waterfall and agile, benefit from these practices.
Reduce management overhead. Project status reporting overhead is often an empty ceremony that decreases the probability of success by diverting valuable resources to non-value added overhead activities. Providing
transparent visibility into progress and priority information, to anyone who needs it, eliminates pointless ceremony
while providing better access to more accurate information. Everyone knows the status at all times.
Stop multitasking. Focusing staff on one initiative at a time reduces task-switching time, eliminates schedule coordination, reduces unproductive wait time, and frees people to do productive work. Spreading resources
across projects reduces throughput due to the overhead of context switching and wait time.
Automate the integration and release process. Continuous integration and release automation provide the engine for the delivery process that eliminates manual errors, dramatically reduces manual effort, instruments the
product delivery pipeline and enables further lean transformation. Manual integration and release processes
simply cannot keep pace with the rapid flow of work.
Automate testing using APIs. Manual testing is expensive, inconsistent, hard to manage, and slow. Automated UI-based testing is expensive and usually happens too late in a release to achieve desired coverage rates. API-
based testing can start as soon as code is delivered in the CI process; it achieves high coverage rates and, once
developed, is virtually free. Shifting testing left is one of the key practices enabling faster delivery.
-
11
FIGURE 8
Where Should You Start?
DeliveringSlowly
1. Paperwork, management overhead
2. Too much manual testing
3. Too many concurrent projects
1. Project status visible to anyone, anytime
2. Automate most testing
3. Small batches4. 1 team on 1 project at a time
Strugglingfor Speed
1. Too much manual testing
2. Too many concurrent projects
3. Paperwork, management overhead
1. Automate most testing
2. Small batches3. 1 team on 1 project at a time
4. Project status visible to anyone, anytime
IncreasingSpeed
1. Paperwork, management overhead
2. Too much manual testing
3. Too many concurrent projects
1. Project status visible to anyone, anytime
2. Automate most testing
3. Small batches4. 1 team on 1 project at a time
Segment Top Pains DevOps Practices Priorities
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Consulting on behalf of IBM, May 2014
-
12
Appendix A: Methodology
In this study, Forrester conducted a global online survey of 600 IT professionals with application development responsibilities
from organizations with an IT budget of $10 million or more from the US, Canada, UK, France, and Germany. Survey
participants included those in manager level or higher, with oversight of or direct involvement with application development
responsibilities. Respondents were offered a small incentive as a thank you for time spent on the survey. The survey began
in April 2014 and was completed in May 2014.
Its important to understand our use of terms in the survey and in this paper. In some cases, Forrester uses different
terminology to: 1) elicit information from respondents and 2) analyze and explain survey results. Why?
Our survey tested the hypothesis that certain key development and delivery practices contribute to a teams ability to deliver software at fast cycle times and other practices prevent fast cycle times. We phrased our
questions using generic and self-evident terms, avoiding either leading respondents toward foregone conclusions or
confusing respondents with industry jargon and unfamiliar terms.
We asked respondents for the same information about their practices in three different ways. First, we asked them directly about their practices across the project life cycle, starting with funding. Second, we asked them which practices
lead to project success or failure. Third, we asked them what theyd change if they could to improve business satisfaction
with their teams software project(s).
DevOps Practices Supporting Survey Questions (Examples)
Deliver in small increments of functionality How often do the following characteristics apply to the definition of your requirements? [Choice: Team
members initially define a minimal product concept and
seek to validate the concept as quickly as possible.]
Form dedicated, cross-functional teams What are some of the characteristics of the projects that are most successful at meeting the expectations of
your business leaders? [Choice: We dedicate team
members to one project at a time.]
Use loose architectural coupling within and between applications
How often do these [design/structural] characteristics apply to your applications? [Choice: The applications
are composed of loosely coupled components or
services.]
Automate environment provisioning Given total authority, what would you start doing /do more of to better meet the demands from your
business leaders? [Choice: We would automate more
of the production deployment process.]
Continuously integrate code What are some of the characteristics of the projects that are most successful at meeting the expectations of
your business leaders? [Choice: Each function is an
atomic unit of work that passes as quickly as possible
through the SDLC and into production.]
Continuously test How often do these characteristics apply to your testing processes? [Choice: Testing begins as soon as there is
some code to test and proceeds in parallel with
-
13
development.]
Continuously fund How often do the following budgeting/funding characteristics apply to the initiation of your software
projects? [Choice: Funding is based on an investment
stream for the product were building.]
Provide real-time transparency Given total authority, what would you start doing /do more of to better meet the demands from your
business leaders? [Choice: We would make the entire
development process visible to team members, other
teams, and all stakeholders/levels of management.]
Appendix B: Demographics/Data
We identified the four segments used in this paper using a cluster analysis of our survey responses. See Figure 9 for the
results of that analysis.
FIGURE 9
Cluster Analysis Groupings
Deliver most/ somework in 1-3 week cycle
Pressured to make deliverydates more predictable
Pressured to reducedelivery cycle time
Pressured to reducedelivery cost
DeliveringFast(32%)
100%
100%
78%
74%
Improvingspeed(37%)
54%
0%
56%
50%
Strugglingfor Speed(19%)
0%
100%
80%
54%
DeliveringSlowly(13%)
0%
45%
0%
100%
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Research on behalf of IBM, May 2014
-
14
FIGURE 10
Country Distribution
Canada
0%
15%
16%
55%
14% 5%
14%
12%
26%
42%Delivering
FastStrugglingFor Speed
DeliveringSlowly
ImprovingSpeed
1%
22%
12%
22%
44%
2%
20%
23%
23%
32%
France
Germany
United Kingdom
United States
Country Distribution Of Respondents
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Research on behalf of IBM, May 2014
-
15
Appendix C: Endnotes
1 In our survey, conducted during May 2014, only 13% of respondents reported having no pressure to reduce cycle times.
FIGURE 11
Industry Distribution
Manufacturing, energy
DeliveringFast
DeliveringSlowly
ImprovingSpeed
Retail and wholesale
Utilities, telecoms
Business services, construction
Media, entertainment, leisure
Financial services (all)
Public sector, healthcare
Other
StrugglingFor Speed
19%
23%
7%13%
6%
17%
14%
2%
14%
7%
9%
11%
12%
30%
7%
10%12%
4%
12%
10%
4%33%
17%
9%
14%
11%
12%
14%
1%
23%
12%
13%
Industry Distribution Of Respondents
Base: 600 IT professionals with application development responsibilities from the US, Canada, UK, France, and Germany
Source: A commissioned study conducted by Forrester Research on behalf of IBM, May 2014