The impact of agile principles on market-driven software product development

SOFTWARE PROCESS IMPROVEMENT AND PRACTICESoftw. Process Improve. Pract. (2009)

Published online in Wiley InterScience(www.interscience.wiley.com) DOI: 10.1002/spip.420

The Impact of AgilePrinciples onMarket-Driven SoftwareProduct Development

Research SectionNina Dzamashvili Fogelstrom,1* Tony Gorschek,1

Mikael Svahnberg,1 and Peo Olsson2

1 Department of Systems and Software Engineering, Blekinge Institute ofTechnology, PO Box 520, S-372 25 Ronneby, Sweden2 Ericsson AB, Business Unit Multimedia Olandsgatan 1, Box 518 S-371 23Karlskrona, Sweden

Agile development methods such as extreme programming (XP), SCRUM, Lean SoftwareDevelopment (Lean SD) and others have gained much popularity during the last years. Agilemethodologies promise faster time-to-market, satisfied customers and high quality software.While these prospects are appealing, the suitability of agile practices to different domains andbusiness contexts still remains unclear. In this article we investigate the applicability of agileprinciples in the context of market-driven software product development (MDPD), focusing onpre-project activities. This article presents results of a comparison between typical propertiesof agile methods to the needs of MDPD, as well as findings of a case study conducted atEricsson, an early adopter of agile product development. The results show misalignmentbetween the agile principles and needs of pre-project activities in market-driven development.This misalignment threatens to subtract from the positive aspects of agile development, butmaybe more importantly, threaten the overall product development by disabling effectiveproduct management. Copyright 2009 John Wiley & Sons, Ltd.

KEY WORDS: market-driven software development; software product management; agile methods; case study

1. INTRODUCTION

Agile methods are attractive to software companiessince they promise shorter time-to-market, as wellas higher flexibility to accommodate changes in therequirements and thereby increased ability to react

∗ Correspondence to: Nina Dzamashvili Fogelstrom, Depart-ment of Systems and Software Engineering, Blekinge Institute ofTechnology, PO Box 520, S-372 25 Ronneby, Sweden†E-mail: [email protected]

Copyright 2009 John Wiley & Sons, Ltd.

to changing customer (market) needs (Williams andCockburn 2003, 2005).

The major feature of agile methods is to useiterations of small development projects developinglimited sets of the most important functionality at agiven point in time. The common property of agilemethods is reactive rather than proactive tactics,encouraging quick adaptation to change ratherthan planning for it. The goal of agile methodsis to eliminate long pre-study and analysis ofrequirements and start feature production as soonas possible. The positive aspect of this approach isthat it speeds up the production of functionality that

Research Section N. Dzamashvili Fogelstrom et al.

is considered important at present; the downside isthat it limits the possibility to control and plan thecontent of a release (Boehm 2002, Williams andCockburn 2003).

The flexibility and ability to quickly respond tomarket fluctuations makes agile/lean developmentmethods attractive for companies operating in amarket-driven context, despite the fact that the long-term impact of adopting these principles and theirapplicability in the market-driven context are to alarge extent unknown. Existing studies and experi-ence reports from application of agile methods aremostly isolated to evaluating the performance ofthese methods on software development activities,such as increasing developers’ efficiency, producingbetter quality code and so on (Dyba and Dingsøyr2008). Isolated experience articles on increased cus-tomer satisfaction exist, but then the studied projectwas developed according to bespoke practices, i.e.in a customer–developer relationship with a con-tractually bound effort and not in a market-drivencontext (Mannaro et al. 2004, Mann and Maurer2005).

To the best of our knowledge, currently thereare no studies analyzing the impacts of agile/leandevelopment principles on either pre-project activ-ities, such as product planning/management inMDPD, or the long-term impact of optimising theproject perspective rather than that of products orproduct portfolios.

The most important aspects of MDPD are require-ments triage (initial selection), requirements pri-oritisation and planning (release planning), i.e.deciding which of the thousands of potentialrequirements should be implemented, and when(Carlshamre 2002, Karlsson et al. 2003, Regnell et al.2005). The main task of product management isto weigh all types of criteria ranging from marketdemands, trends, key-customer demands, to tech-nical and long-term product impact, and arrive atdecisions supported by the company and productstrategy. Selecting the best sub-set of requirementsfor implementation has to take both the long-termand short-term perspectives into account as well asmarket-pull and technology-push.

In light of the effect of pre-project activities on thesuccess of any company practicing market-drivendevelopment, it becomes important to know whatkind of impact agile principles have on productmanagement and critical pre-project activities suchas initial requirements triage, prioritisation, and

release planning. This is important in order todetermine to what extent agile principles can beadopted in market-driven organisations, and whatproduct management and pre-project practicesshould be in place to maximise the positive aspectsof lean development in projects (Carlshamre 2002,Karlsson et al. 2003, Regnell et al. 2005).

This article investigates the impact of agile princi-ples on pre-project activities in MDPD by studyingan industry case. The study is performed in collab-oration with Ericsson AB in Sweden, where a largeeffort to move from ‘traditional’ pre-study intensivedevelopment to lean development (agile) has beenongoing for over 3 years. An early evaluation of themove to lean development and its potential waspresented by Tomaszewski et al. (2008). The casestudy in this article presents a postchange evalua-tion at the same company, but from the perspectiveof pre-project activities and the potential misalign-ment between the needs of product managementand agile principles. In addition to this, the arti-cle also qualifies the challenges with future researchdirections and suggestions as how the misalignmentcould be handled.

The rest of this article is outlined as follows:Section 2 provides a background to agile softwaredevelopment and market-driven software productdevelopment (MDPD) where the common aspectsof agile methods are put against the specialneeds in a market-driven context. Further, a shortoverview of empirical evidence on agile methods isprovided. Section 3 gives details on study designand execution. Case study results, analysis, anddiscussion can be found in Section 4 and Section 5respectively. The report is finalised by conclusionsand future work in section 6.

2. BACKGROUND AND RELATED WORK

2.1. Common Aspects of Agile SoftwareDevelopment Methods

The introduction of extreme programming (XP) inthe late 90s is commonly acknowledged as a startingpoint, coining the term agile in software develop-ment. In 2001 other agile initiatives such as DynamicSystems Development Method (DSDM), Crystal,Feature Driven Development, and SCRUM joinedXP and the ‘Agile Manifesto’ was formed describingthe ideas and agile commonalities (Abrahamssonet al. 2003, Cohen et al. 2004).

Copyright 2009 John Wiley & Sons, Ltd. Softw. Process Improve. Pract., (2009)

DOI: 10.1002/spip

Research Section Agile Methods in MDPD

Nowadays agile methods are quite popular andare seen as an alternative to the traditional plan-driven methods, which thus far have been thebasis for accepted practices in software engineer-ing (Abrahamsson et al. 2003, Cohen et al. 2004,Tomaszewski et al. 2008). Plan-driven methods arecharacterised by heavy upfront planning, focus ondocumentation, predictability, and repeatable pro-cesses (Boehm 2002). Lately these methods havebeen criticised for their inability to accommodatechange and the high costs that are associatedwith updating documentation and plans (High-smith and Cockburn 2001, Poppendieck and Pop-pendieck 2003, Leffingwell 2007). On the otherhand, agile methods are designed to accommodaterapid change (Beck 2000, Williams and Cockburn2003). In order to be flexible, agile methods relyon individuals and their knowledge rather thanprocesses, value working piece of software overcomplete requirements specifications and designplans; and encourage responding to a change ratherthan planning for it (Manifesto for Agile SoftwareDevelopment 2008).

The latest contribution to the agile commu-nity, Lean Software Development (Lean SD) (Pop-pendieck and Poppendieck 2003), is based on acollection of attitudes and principles originatingfrom the successful concepts of lean manufacturingdeveloped by Toyota known as the Toyota pro-duction system (Womack et al. 1991). The originallean manufacturing principles at Toyota (Elimi-nate waste, Amplify learning, Decide as late aspossible, Deliver as fast as possible, Empower theteam, Build integrity, etc.) are based on continuousimprovement and elimination of waste. Since theseprinciples are rather general they can be interpretedand adapted to suit different applications and busi-ness domains, for example software development.This can be seen as an advantage as the appli-cability of agile methods are generally consideredto be limited to small- to medium-sized enterprisedeveloping medium-sized non-critical applications(Boehm 2002, Williams and Cockburn 2003, Sillittiet al. 2005). However, the current translation of leanprinciples to software development (Poppendieckand Poppendieck 2003) is fully compliant with, oreven designed after, existing agile principles. Thismight imply a limitation of this potential, limitingthe flexibility of the origin ideas behind the leandevelopment.

In general, agile methods may vary when itcomes to the specific application of certain practices.However, they all follow the main principle ofproducing working software in small iterations,and utilising small teams of dedicated individualsthat focus on software development. The commonaspects of agile methods can be summarised bythe following principles [hereafter called AgileProperty (AP)] (Highsmith and Cockburn 2001,Abrahamsson et al. 2003, Cohen et al. 2004):

• AP1. Feature orientation: The main focus is onthe production of features as soon as possible.The goal is to deliver working functionality thatis perceived to have the most value for thecustomer, and this is done in small and frequentiterations.

• AP2. Reactive development: Reactive develop-ment is about responding to a change instead ofplanning ahead, and delaying decisions as longas possible. Agile methods promise flexibilityand follow the ideology that one can not con-trol the world, thus the strategy is to respondto a change rather than plan for it. Examplesof reactive practices are refactoring, adjustingrequirements’ priorities, and release scope aftereach iteration and delaying decisions for as longas possible.

• AP3. Evolving project (release) scope: Perhapsone of the main distinguishing features of agiledevelopment is the change from a fix-scopeapproach to a more open-ended approach. Inthe traditional fix-scope approach much effort isspent on defining and planning the content ofa product release of a project upfront. In agilethe release scope is emerging in the processof development rather than planned ahead. Aprioritised list of requirements (product backlogin SCRUM and prioritised user stories in XP)serves as an initial input which is a kind ofwish-list of a release scope. The release scopeis expected to be refined and updated at theend of each iteration in order to accommodatechanges and new information that was learnedin the latest iteration. This practice is in line withthe reactive development property describedin AP2.

2.2. Specific Needs of MDPD

In MDPD software products are often developedas product families that are offered to a mass


DOI: 10.1002/spip


market rather than a specific customer. The productdevelopment has an iterative character where newversions of the product are delivered to the marketvia established release cycles (Dahlstedt et al. 2003,Gorschek and Wohlin 2006).

The fact that the product is not ordered anddeveloped to suit the wishes and needs of onespecific customer creates special needs distinguish-ing the characteristics of this type of develop-ment from customer-specific (bespoke develop-ment)(Potts 1995). In a bespoke situation the devel-opment organisation can rely on the customer todecide what functionality should be included inthe product and this information can be eliciteddirectly from the customer. However, in market-driven development this is not feasible as thereare any number of potential customers. Insteadthe development organisation itself decides whatfunctionality should be delivered to what marketsegment and when (Potts 1995, Karlsson et al. 2003,Ebert 2005, Regnell et al. 2005, van de Weerd et al.2006). This creates a special focus on pre-projectactivities usually associated with product planningand management operations (van de Weerd et al.2006). The specific needs of MDPD are summarisedbelow:

• N1. Balancing different requirements types:Selecting a set of requirements that will beincluded in future releases of a product is oneof the most important strategic decisions inMDPD. This decision is based on comprehen-sive analysis of the company’s business intel-ligence, product planning and road-mapping,marketing and sales information, and strate-gic assets such as engineering know-how andinvestments in product architecture (Greer andRuhe 2004, Ebert 2005, Regnell et al. 2005,Saliu and Ruhe 2005). The specifics of thistask require that the development organisationmust find an optimal balance between differentrequirement types such as commercial require-ments, innovations, requirements connected toarchitecture improvements and also balancequality attributes like maintainability, perfor-mance, scalability, and usability across differentreleases.

• N2. Trade-off between market-pull and tech-nology-push: Studies in relation to MDPD haveshown that the development organisation notonly needs to respond to the current consumer

needs but also anticipate future needs as well asinfluence the needs of the consumers via inno-vations and technology breakthrough. Findinga balance between market-pull and technology-push is considered one of the challenging aspectsfor companies operating in this context as itindirectly involves balancing low- and high-riskas well as long- and short-term considerations(Karlsson et al. 2003, Regnell et al. 2005).

• N3. Release content planning (requirementsselection): Planning the content of a specificrelease is recognised as one of the most chal-lenging parts of MDPD (Carlshamre et al. 2001,Carlshamre 2002, Greer and Ruhe 2004). Thisactivity occurs after a large number of initialrequirements have been filtered through thesteps of requirements triage/screening and pri-oritisation. The reduced amount of requirementsis then considered for inclusion for a specificrelease, which is the task of release scope plan-ning. The ultimate goal of this activity is to finda selection of requirements that will give thebest possible return on investment (ROI) andwill be feasible to implement within a giventime- and resource-frame. The main character-istics of this task require finding an appropriatebalance between relative cost and value of eachrequirement, technical and business dependen-cies between requirements, and available budgetand resources at hand. The outcome of thisactivity results in a list of requirements that con-stitutes the planned scope of a release. Having anestablished release scope is important from threemain aspects. One, it enables decision makers toestimate and plan for the total value of a release,and a comparison to the total value needed fora specific market segment at a certain time. Thegoals of a release may be combinatory in nature,i.e. several projects in collaboration over timecan together offer a specific value for the mar-ket. Two, a set release scope provides a baselinethat can be used to effectively evaluate emergingchanges with respect to how this change affectsthe total value and cost of a release. This pro-vides a basis for informed decisions and helpsmanagers to not only react but also find the bestway to react on a given change. Three, it providesa possibility to plan and implement long-termproduct development and innovation that arenot based on current customer wishes but focus


DOI: 10.1002/spip


on, for example, system evolution goals that gobeyond the project perspective.

2.3. Applying Agile Principles in MDPD

In order to understand the possible impact of agilemethods on pre-project activities, the propertiesof agile methods described in Section 2.1 are putagainst the needs of MDPD described in Section 2.2.The result is presented in Table1.

In Table 1, AP1 is put against N1 since fea-ture orientation by definition favors feature-typerequirements thereby being the opposite of creatinga balance between different requirement types suchas feature and non-functional requirements. AP2:Reactive development is associated with N2, i.e.the need of finding trade-off between market-pulland technology-push since there is a risk that theagile view of adopting and reacting to customerneeds results in the company becoming too focusedon responding to current market needs and thusbeing unable to push innovative ideas that do notdirectly originate from the market or key-customersat present. This can have devastating effects on thelong-term, invisible to the project perspective whichis inherently focused on delivering the requirementsrelevant from the project perspective.

Using an evolving release scope (AP3) is bydefinition at odds with planning the release scope(N3) as it limits the decision maker’s ability toactively control and influence release content. Aspresented in Section 2.2 in MDPD there is aneed for an established release scope, where thedecision on what is included in a release is basedon careful analysis of all candidate requirements(Carlshamre 2002, Greer and Ruhe 2004, Saliu andRuhe 2005, Ngo-The and Ruhe 2008). In contrast,an agile project release scope is loosely definedand is based on rough estimates and the initialfeelings of the customer about the requirements.The intention in an agile project is not to have

Table 1. Agile properties vs. MDPD needs

Agile property Affected MDPD need

AP1. Featureorientation

N1. Balancing differentrequirements types

AP2. Reactivedevelopment

N2. Trade-off betweenmarket-pull and technology-push

AP3. Evolvingrelease scope

N3. Release content planning

the best selection but to refine the scope duringthe course of development (Beck and Fowler 2001,Poppendieck and Poppendieck 2003, Williams andCockburn 2003, Leffingwell 2007).

The lack of alignment between the propertiesof agile methods and the needs of MDPD can befurther explored by studying certain assumptionsmade in agile development methods and mappingthem to the product development practices inmarket-driven context. These assumptions are listedin Table 2. Each is elaborated upon below.

• Development context: Agile methods are bes-poke in origin (Karlstrom and Runeson 2005)and largely focus on a specific project ratherthan long-term product development commonto MDPD where the focus is on the productor product families offered to any numberof potential customers. Success in a bespoke(project) perspective is the successful completionof the project fulfilling the needs of the customer.

Table 2. Agile assumptions and MDPD reality

Agile methods MDPD

Developmentcontext

Onecustomer – focuson a specificproject. Theproject is central.

Many potentialcustomers – focuson a product orproduct family.Development isfocused on theproduct, and anynumber ofprojects cancontribute to arelease.

Business context Customer ownsthe product andfinances thedevelopmenteffort.

Developmentorganisation ownsthe product andfinances thedevelopmentactivities itself.

The releasescope can beundefined andnegotiated.

The release scopeneeds to bedefined and is noteasilyrenegotiable.

Understanding ofvalue

Simple. Complex.

Assumptions aboutrequirements

Requirementsare at largeunknown.

Requirements areknown throughmarketinvestigations andbusinessintelligence.


DOI: 10.1002/spip


Success from a market-driven perspective ismeasured in sales, revenue, market growth, andthe ability to create and maintain a flexibleproduct architecture that will support futureproduct releases. Thus the difference betweenagile and MDPD context is both in the scaleand the complexity of the projects. For examplecreating a maintainable product architectureis not as critical in a small bespoke projectcompared to a project that is part of a large-scale product development effort of a productthat will have a long life span.

• Business context: Agile methods assume thatthe customer is responsible for identifying valu-able requirements and financing developmentactivities, as well as that the scope of the releaseor a project is negotiable. In contrast in MDPDthe development organisation itself is respon-sible and has to decide what requirements arevaluable for a current release and for findingfinancial resources to develop the selected setof functionality. The contents of the release can-not be negotiable in the same sense as in agileprojects as there is no specific customer, ratherthe decision of what to develop is arrived atafter complex deliberation involving productmanagement, marketing, key-customers, salesand upper management, as well as long-termplan influence. In fact, the scope of the releasehas to be defined since it is quite common thatproduct management need to produce a mar-keting message on what is to be expected inthe next release of the product long before thedevelopment activities are finished (or even ini-tiated). Applying the concept of evolving releasescope has some economical consequences as wellwhere the developing organisation (which inMDPD stands for the development costs itself)must evaluate how much updating, refactor-ing and other correcting after-the-fact activitiescan be afforded by a company that has todeliver a certain value at a defined marketwindow time-point. In MDPD time-to-marketis generally seen as being of paramount impor-tance.

• Understanding of value: Agile methods focuson creating rapid value for a specific customer. InMDPD the definition of what constitutes valueis more complex and the potential value fora specific customer is just one component ofa value measure (this relates to N1 and N2

presented in Section 2.2). Also, agile methodsassume that the value of rapidly developed func-tionality can be confirmed by a customer or anon-site customer representative almost immedi-ately. In MDPD the validation of perceived valueof developed functionality requires longer lead-times as well as different tools, for example GAPanalysis, customer value analysis (CVA) andfocus groups (Gorschek 2006). An agile projectmember cannot go to an on-site customer asthis representative does not exist. The conceptof value not only transcends any one customerbut also the project itself. For example, a projectcan be a part of a long-term development plan tooffer infrastructure to subsequent developmentdown the line. From a project perspective mostof the project contents might seem as waste, butfrom a product perspective the view is another(Gorschek and Davis 2008).

• Assumptions about requirements: Agile meth-ods assume that at the beginning of the develop-ment project requirements are largely unknown.This is not the case for MDPD where large effortshave been devoted to collecting large amountsof potential requirements, which undergo triage,and subsequent refinement and selection (andpackaging) prior to the project being created(Potts 1995, Karlsson et al. 2003, Ebert 2005, Reg-nell et al. 2005, van de Weerd et al. 2006). Thisdoes not mean that the requirements given to aproject are perfect or even complete, but it doesmean that there has been a deliberate and criticaleffort to select and deliver a set of requirementsto the project.

As a consequence to these differences in assump-tions and properties of agile development methodsand MDPD needs, it seems reasonable to assumethat applying agile principles in this context willhave certain consequences on pre-project activitiesconnected to requirements selection and productplanning. This is the focus of this article and thecase study presented in sections 3 and 4.

First a background to agile development is givenwith focus on reporting experiences from industryusage.

2.4. Empirical Studies on Agile Methods

Agile software development models are relativelynew and have not been tested to the same


DOI: 10.1002/spip


extent as traditional software development models(Abrahamsson et al. 2003). A large part of theexisting empirical evidence regarding agile meth-ods is focused on XP. Here research efforts aremainly focused on studying isolated practices, suchas pair programming or test-driven development(Erickson et al. 2005). When it comes to the impactof agile methods on software development activitiesa number of case studies exist. These studies reportimprovements in the quality of the developed code,increased effectiveness and productivity of softwaredevelopers, and customer satisfaction (Middleton2001, Ilieva et al. 2004, Layman et al. 2004, Man-naro et al. 2004, Mann and Maurer 2005). However,a closer examination of these studies shows thatmost of them focus on identifying the effect of agileprinciples on in-project activities, and that observedeffects are primarily identified in the context of smalldevelopment teams developing small- to medium-sized applications. These results are in line withthe findings of Abrahamsson et al. (2003), who intheir survey concluded that there is a lack of soundempirical evidence to support the applicability ofagile methods in different development contextsand application domains.

Empirical evidence in relation to the applicationof agile practices in large-scale contexts is quitescarce. In a recently conducted systematic reviewof empirical studies on agile software development270 reports were identified as containing empiricalresults on agile development methods (Dyba andDingsøyr 2008). After assessment of these accordingto the principles of what constitutes good practicesof empirical research, only 33 of the original 270were considered as credible empirical reports.Out of these only a small fraction, two reports(Dagnino et al. 2004, Karlstrom and Runeson 2005),report on application of agile principles in largeorganisations. Both of these studies focus on in-project activities and report positive results ofapplying agile principles. It is important to noticethough that in the works of Karlstrom and Runeson(2005) the application of XP practices was donein the context of small pilot teams. Informationabout the context of development size of developingproject and which XP practices had been appliedwas unfortunately missing. In the works of Dagninoet al. (2004) a rather conservative version of agiledevelopment methodology was used to produce aprototype for an internal customer.

2.4.1. Empirical Studies on the Impact of AgileMethods on MDPDWhen reviewing existing knowledge and researchon agile methods it is easy to notice that most ofthe empirical reports focus on analyzing the impactof agile methods on in-project activities such assoftware design, development and test activities,when studies focusing on pre-project activities andproduct management in general are quite rare (seeSection 2.4).

Recently there has been interest in investigatingthe application of agile methods for product lineengineering (PLE) (Clements and Northrop 2002),which have some similarities with product man-agement activities. At the current stage howeverthe research in this area is quite new, focusing onstudying in what way agile principles and PLEactivities can be combined (Tian and Cooper 2006,Hanssen and Fægri 2008), or the application of theselected principles of agile methods (such as col-laboration and focus on individuals) in some PLEactivities (Noor et al. 2008).

When it comes to MDPD, to the best of ourknowledge and for the time being, studies focusingon understanding of the impact of agile principleson pre-project, product management activities donot exist. This situation indicates a critical gapin understanding, e.g. the long-term effects ofagile methods on product development. This isespecially relevant when considering MDPD, wherepre-project activities are critical (Regnell et al. 2005).

Studies investigating the impact of agile princi-ples on pre-project activities are needed in orderto determine to what extent agile principles can beadopted in the MDPD context. This can involve e.g.how agile models need to be tailored, or what otherpractices need to be in place in order to take fulladvantage of positive aspects of agile developmentwithout negatively effecting MDPD.

The case study described in this article inves-tigates the characteristics of decision making inEricsson AB that has worked with adopting agileprinciples over the last years. The goal of the casestudy is to study and understand the character-istics of the pre-project decision-making processwhen applying agile development principles. Theidentified characteristics and specifics of the pre-project process are analyzed in order to see if thestudy findings can support the expected impact ofagile principles on pre-project activities in MDPDas described in Section 2.3.


DOI: 10.1002/spip


3. CASE STUDY SETTING AND DESIGN

The case study was conducted during the fall of 2007at Ericsson AB in Sweden (called Ericsson hereafter).Ericsson is one of the world’s leading companiesin telecommunication, providing a wide range ofproducts and solutions. The company operates ina market-driven context where the products aresold as generic solutions offered to an open market,although customised versions of the products arealso developed and sold to key-customers.

With the goals of achieving higher process per-formance and increased flexibility when it comesto accommodating changing requirements Ericssonmoved from a traditional development process withrelatively extensive development projects whichlasted about a year or even more (following theclassical waterfall development steps) to lean devel-opment. One of the main motivations for movingto a new development process was to minimisethe number of change requests. Change requestswere associated with high development costs sincethey often resulted in wasting development effort inform of detailed analysis of requirements that werenot implemented in the final product or expensesassociated with rework.

According to Tomaszewski et al. (2008), the tradi-tional waterfall development projects at Ericssonsuffered from large number of change requestsmainly caused by long development cycles andextensive analysis of the pre-defined release projectscope. The new agile development process calledStreamline (Tomaszewski et al. 2008) promised tocut down the costs associated with the changerequests substantially. A recent evaluation of theapplication of Streamline at Ericsson reports reduc-tion of requirements volatility in the projects mainlycaused by application of small development pack-ages and shortening the lead-time of developmentprojects (Petersen and Wohlin 2008). The Stream-line development process (called only Streamlinehereafter) is strongly influenced by agile and espe-cially Lean SD, and a summary of it can be found inSection 4.1.

3.1. Research Roadmap

The goal of this study was to identify how theintroduction of Streamline has affected productmanagement [pre-project requirements engineering(RE)] activities practiced at Ericsson. In order to

achieve this goal it was decided to first identifyand study the process and the characteristics of pre-project decisions at Ericsson. Collected informationwas then analyzed using the structure seen inTable 1. The intention here was to identify how wellthe study findings support the anticipated impactof agile principles on MDPD outlined in Section2.3.

Information about the practiced pre-project activ-ities in Streamline was gathered by means ofstudying the pre-project RE decision-making pro-cess applied in the development of three separateproducts at Ericsson. Product 1 and product 2are large mature systems with a large number ofreleases operating all over the world at many cus-tomer sites. Products 1 and 2 are over 9 years oldand have undergone seven and four releases respec-tively. Product 3 is a smaller product with only oneprevious release since its creation in 2006.

In order to get an overview of the RE decision-making process three main sources were used toachieve a triangulation (Gorschek and Wohlin 2004)of sources as can be seen in Figure 1.

Part A consisted of the study of official processdocumentation, Part B was interviews with prac-titioners allowing for a balanced view where theofficial process was weighed against the one actu-ally practiced, and Part C consisted of observations(e.g. sitting at the meetings). The use and compari-son between different sources is at the central core ofthe triangulation (Gorschek and Wohlin 2004, Pet-tersson et al. 2008). The details of study executionare provided in Section 3.2, and the results obtainedare detailed in Section 4.

3.2. Study Execution

The initial stage was focused on studying theofficial process documentation in order to identify

Figure 1. Study overview of mapping pre-project REdecisions


DOI: 10.1002/spip


pre-project decisions, roles and responsibilitiesassociated with the decisions, and mapping thedecision characteristics. In addition, the study ofthe official process allowed a mapping of meetingsand forums for decision-making.

3.2.1. Process Documentation Study (Part A)The documentation study was conducted by meansof reviewing official process descriptions for pre-project activities in Streamline. A senior expert,responsible for process improvement activities wasused as a guide to the formal process in orderto identify relevant process documents. An impor-tant selection criterion for the document study wasto consider those process descriptions and work-flow guidelines that were common for developmentactivities.

The information provided in the process docu-mentation was analyzed in order to identify themajor steps/activities and the decisions pointsinvolved in pre-project RE. For each identifieddecision point the characteristics were mapped.Characteristics in this case are the (i) goal(s)for an activity, (ii) the decision criteria/methodused, and (iii) the decision support material usedand/or produced. The identified roles and decisionforums were later used to plan the interviews andobservations.

3.2.2. Interviews with Practitioners (Part B)The intention with the interviews was to elicit infor-mation on the pre-project decision-making processnot evident from the formal process descriptions inPart A. Semi-structured interviews were used in aninformal setting.

At the beginning of each interview the participantwas asked to describe the process of workingwith pre-project requirements. In most cases theinterviewees would draw a figure of the overallprocess and map the major steps/decisions in theprocess.

Once the major steps and decisions were identi-fied the interview continued detailing them and theparticipants were asked a set of questions designedto retrieve the characteristics of the decision points.In order to allow a comparison between the formaland practiced process, for each mentioned decision,the interviewees answered questions on goals of adecision, how the decisions were taken, what infor-mation was important/necessary in order to take

a decision (decision support material used) as out-lined in Section 3.2.1. At the end of each interviewparticipants were asked to identify major challengesand improvement suggestions in relation to theprocess and the characteristics.

The results of the interview were transcribedinto session summary sheets (Robson 2002) directlyafter the interview session. Information placed inthese sheets followed the same template, providingdata on interviewed person, issues covered, andresearcher’s own notes regarding the informationobtained in the interview.

Once the interviews were completed, the infor-mation from the individual interview was codedin different data categories related to the prac-ticed pre-project requirements decisions and theircharacteristics. The utilised data categories coveredtopics such as practices requirements decision steps,requirement types considered at each decision,practiced approach for taking decision, requireddecision material, experienced challenges, etc. Thecoded data from each interview was then placed inan excel sheet where it was analyzed again in orderto find relations to the agile properties and MDPDneeds presented in Table 1.

The selection of the interview participants wasbased on the process documentation where rolesinvolved in pre-project RE and decision-makingwere identified.

Further, the case study consisted of the studyof three different products at Ericsson to makesure that the process and the characteristics studiedwere representative for the organisation. Thereforethe interviewees were also selected to representthe important roles for each of the products (seeTable 3).

In total 14 interviews were conducted. Each inter-view was conducted individually. To avoid validityissues such as the subjects feeling uncomfortablewith speaking their mind none of the interviewswere recorded. Detailed notes were taken duringthe interviews, and during the subsequent refine-ment of these notes supplemental questions wereposed to the interview subjects post-interview ifneeded. The detailed and refined notes were thenused to validate understanding and avoid misinter-pretations as they were shared with the subjects forvalidation purposes. The distribution of the inter-views was the following: Product Managers – 5;System Experts – 3; Line Managers – 3; Project Man-agers – 3.


DOI: 10.1002/spip


Table 3. Roles identified in pre-project RE

Name(number interviewed)

Description Responsibility

Product Manager (5) This role has thestrategic productresponsibility anddecides the overallproductdevelopmentdirection

Select, prioritiseand planrequirements forfuture releases

System Expert (3) This role ispopulated bypeople with expertknowledge of thesystems and theirarchitecture

Responsibility ofsystemarchitecture andsystemdevelopmentplan

System expertsprovide decisionmaterial for productmanagement sincethey are responsiblefor providinganalysis ofpre-projectrequirements, in theform of feasibility,impact andtechnicaldependencies

Line Manager (3) Line managers haveoverallresponsibility ofDevelopment andTesting (R&D)resources. Theyplan for R&Dutilisation indevelopmentprojects and allocateresources todeveloprequirementsselected by productmanagement

R&D capacityoverview,planning ofdevelopmentprojects

Project Manager (3) Responsible for thedetailed planning,execution andfollow-up ofdevelopmentprojects for aspecific release

Planning andexecution of aspecific project

3.2.3. Observations (Part C)This part of the study was focused on attendingforums and meetings where pre-project decisionswere discussed. These forums were identifiedinitially in the process study (Part A) and later

confirmed by the process expert at the company.Identified decision forums are summarised inTable 4.

In total four meetings were attended; Pre-projectRequirements Council (2), Release Scope refinementBoard (1) and Release Planning Board (1). In order tominimise any potential effect caused by the presenceof a researcher the meetings were not recorded.Instead a researcher observed the character andcontent of discussions and decision-making processduring the meetings and notes were taken.

4. EMPIRICAL RESULTS

4.1. Process Documentation Study (Part A)

Figure 2 presents the major pre-project activitiesin Streamline as described in the process docu-mentation. The activities are displayed in chrono-logical order (order of execution in the company).

Table 4. Decision forums

Name(number visited)

Purpose

Pre-projectRequirementsCouncil (2)

This council consists of strategicproduct managers, technicalproduct managers and systemexperts. Decision forum is usedfor decisions on whichrequirements will be approvedfor further analysis and how theselected requirements will beallocated.

PrioritisationWorkshop

Workshop participants includeproduct managers, systemexperts and different requirementrepresentatives.Decisions on which requirementswill be assigned highest priorityfor the next release are taken.

Release ScopeRefinement Board (1)

The board consists of productmanager, line manager andsystem experts. Here thedecisions on approvingimplementation proposals andassociated cost of a requirementare taken.

Release PlanningBoard (1)

The board consists of productmanager, line manager, systemexpert and project manager. Herethe decisions regarding releasescope planning and initiating ofdevelopment projects are taken.


DOI: 10.1002/spip


Figure 2. Pre-project decision making process in Streamline

As shown in the figure, pre-screening of incom-ing requirements, requirements prioritisation andrelease scope analysis and refinement are the majoractivities performed prior to initiating developmentprojects.

4.1.1. Pre-screeningAccording to Streamline, initial requirements areelicited from different sources in a continuousmanner. Incoming requirements are then insertedin the repository after which they pass through thepre-screening step. The goal of this activity is toidentify a sub-set of requirements that is interestingto include in a certain product. The processinvolves two sub-steps: selection and analysis.During selection requirements are analyzed inorder to determine suitability of a requirementfor a certain product. In this step requirementsthat are wrongly allocated to a product, duplicaterequirements and other requirements that areconsidered not suitable for a product are filteredaway. During analysis requirements are analyzedto determine their business value and technicalimpact. This information together with directivesfrom product plans and roadmaps are used toallocate requirement to a certain release of a

product. At the end of the pre-screening step aninitial requirement is either allocated to a futurerelease, rejected, or considered so urgent that isallocated to the release under definition.

4.1.2. PrioritisationDuring this step requirements that are allocated toa certain release are prioritised in order to producean initial release scope. At the end of a prioritisationstep only a small sub-set of all requirements thatare allocated to a certain release are selected tobe included in the list of prioritised requirementsfor a certain release which in Streamline is called‘Priority directive’ of a release. At the end of theprioritisation step requirements included in the‘Priority directive’ are handed over to the R&Dorganisation (development and testing) for analysisand realisation.

4.1.3. Release Scope Analysis and RefinementThe goal of this activity is to analyze and packagerequirements included in the ‘Priority directive’ intosmall development projects each in size approx-imately 3 months. Activities in this step have acontinuous and iterative character where require-ments are analyzed and assigned to development


DOI: 10.1002/spip


projects following the order of their priority in the‘Priority directive’.

Once the priority directive is received by R&Dthe initial step is to create a project anatomy plan.The purpose of the anatomy plan is to identifyand describe functional dependencies between therequirements in the priority directive, which aresubsequently used to update the priority ordertaking the dependencies into account, resulting in afunction container list. This list provides a base forthe planning of development projects as it dictatesin which order the requirements will be analyzedand assigned to the development projects.

When the first version of the function containerlist is available, R&D can start conducting techni-cal analysis of requirements following the priorityorder dictated by the list. Requirements are ana-lyzed independently and development projects areinitiated as soon as the line management in theR&D organisation can find and allocate resourcesnecessary for realisation of the selected solutionproposal.

The technical analysis of each requirement shouldtake a maximum of twenty (20) person-hours andproduce a report detailing which components ofthe existing system will be affected by the newfunctionality, possible implementation solutions,and a cost estimate of each solution. The resultsof the technical investigation is presented at theRelease Planning Board (see Table 4) and thesolution that provides the best trade-off betweenproduct management priorities, resource capacity,and system architecture impact is selected. If anacceptable trade-off cannot be reached, for examplethe estimated cost of requirements implementationis higher than expected by product management,or the resource capacity is inadequate the priorityorder of this requirement must be renegotiated.

4.1.4. SummaryStreamline is a process that tries to apply agileprinciples in an MDPD context. The process hasclear characteristics of market-driven development,where requirements are collected from differentsources and filtered through the pre-screeningand prioritisation steps to produce the ‘Prioritydirective’ of a release.

Streamline also has a strong connection toagile and lean development principles, which isevident in the step of release content analysis andrefinement. As shown in Section 4.1.3, the scope of a

release if not fixed and development activities do notwait until analysis of all requirements allocated to arelease are finalised, instead development projectsare initiated as soon as analysis of top priorityrequirement allows for identifying what resourcesare necessary and these resources can be allocated.This approach is in line with the agile propertyAP3 – Evolving release scope. In Streamline thisproperty is intended to ensure that the developmenteffort is focused on top priority requirements andto minimise waste connected to change requestsmentioned in Section 3.

4.2. Interviews and Observations (Part Band Part C)

In this section we describe study results obtainedvia interviews with the practitioners and conductedmeeting observations. The results are groupedaccording to major pre-project steps identified inthe process documentation study (see Section 4.1).

4.2.1. Results for Pre-screening (Step 1 in Figure 2)4.2.1.1. Finding 1: Practiced Approach for Pre-screeningInterviews with the practitioners showed that ingeneral practiced activities followed the steps of thedocumented process. However some differenceswere identified.

As shown in Table 5, for product 1 and prod-uct 2 activities in pre-screening mainly focus ondefining if a requirement is appropriate for theproduct, rather than evaluating requirement’s busi-ness value and technical impact as prescribed bythe process documentation. This type of analysiswas performed in an informal way and no reportswere produced. Product managers for this productexplained: ‘‘we receive hundreds of requirements; there-fore the first step is to lower the volume by removingduplicates and requirements that are wrongly allocatedto our product’’.

In the case of product 3 the pre-project activitiesconsidered both technical analysis and value evalu-ation. However compared to the technical analysis,evaluation of the requirements value received littleattention. Moreover, mechanisms used for evaluat-ing and determining value of a requirement werenot obvious. Observation of Pre-Project Require-ments Council meetings for product 3 confirmedthis since most of the meeting time was spenton clarifying technical details around the require-ments. Technical evaluation was mostly based on


DOI: 10.1002/spip


Table 5. Practiced approach for pre-screening

Approach/criteriafor pre-screening

Reqirementtypes

Product 1 andProduct 2

Practiced approachfor pre-screening isfocused onidentifying:(i) duplicates, (ii)requirements thatare wronglyaddressed (notapplicable for aproduct), (iii)unreasonablerequirements like‘‘you should haveno downtime’’

Commercialrequirementsfrom influentialcustomers andmarket units

Product 3 Focus on evaluatingimpact and cost of arequirement

Commercialrequirementsfrom influentialcustomers andmarket units

expert opinion. For requirements that were con-sidered complex an official technical report wasordered and produced.

4.2.1.2. Finding 2: Balance between Different Require-ment Types at Pre-screening During the interviewsproduct managers of each product were asked to listthe different types of the requirements consideredby them during pre-screening. All product man-agers listed commercial requirements originatingmainly from influential customers and marketingunits, while other types of requirements like tech-nical and architectural improvements or productlife-cycle requirements were not mentioned.

4.2.2. Results for Requirements Prioritisation (Step 2in Figure 2)When it comes to requirements prioritisation,interviews with practitioners provided detailedinformation on the applied prioritisation techniqueand evaluation criteria, which was missing in theformal process descriptions. Findings identifiedthrough analysis of interview results are presentedbelow:

4.2.2.1. Finding 3: Practiced Approach for PrioritisationFor all the products studied the requirementsprioritisation of requirements in the overall productplan is performed every 6 months. The prioritisation

is performed in the form of a workshop whereexperts with different competences participate (seeTable 4). The prioritisation is conducted using pair-wise comparisons and supported by the commercialtool Focal Point (Telelogic Focal Point – Softwarefor Product Management and Product PortfolioManagement 2009).

The components used as evaluation criteriaare presented below following the order of theirimportance as defined by practitioners: (i) benefit(value) to company and alignment with productstrategy; (ii) benefit (value) for potential customers;(iii) cost; (iv) how well the requirement improvesnon-functional characteristics.

For all three products the prioritisation andqualifying the criteria, such as requirements costand value (for the organisation itself and potentialcustomers), were mainly based on expert opinion(knowledge). Occasionally for estimating cost ofa requirement additional material such as technicalreports were used. When it came to estimating valueit was completely based on expert opinion. Thepractitioners in general found it difficult to explainthe mechanism behind estimation of requirementsvalue. One of the product managers commented‘‘we estimate requirements value based on the knowledgeof our product and our customers’ needs’’.

4.2.2.2. Finding 4: Balance between Different Require-ments Types at Prioritisation At this stage of thepracticed process all products considered both com-mercial and technical requirements. When it comesto the question of balance between the commer-cial and technical requirements the interviewedsystem experts expressed their concern that sys-tem and architectural improvements were receivingless attention compared to commercial require-ments from e.g. market units and key-customers.‘‘Internal requirements connected to architectural andsystem improvement issues have a hard time compet-ing with commercial features and often end up lower inthe prioritised list of requirements’’ mentioned one ofthe system experts. Another expert noted: ‘‘systemrequirements receive attention when we can show thatthey are necessary for implementation of some importantcommercial requirement’’. System experts anticipatedthat this development could in the long run causedeterioration of the system architecture. The inter-views system experts also expressed that describingthe value of system requirements was not easy and


DOI: 10.1002/spip


that they needed guidelines for formulating the‘business case’ for system requirements.

4.2.3. Results for Release Scope Analysis andRefinement (Step 3 in Figure 2)Analysis of the interviews with involved rolesin this step, and data collected via observationsof Release Scope refinement Board and ReleasePlanning Board meetings (see Table 4) revealedthat involved parties made a real effort to followthe agile way of working described in the processdocumentation. However, following lean principlesto 100% when planning the contents of the futurerelease proved to be problematic.

4.2.3.1. Finding 5: In Practice It Was Not Realistic toFollow Lean Development Principles Fully The studyresults indicated that following the practices ofevolving release scope (AP3) as described in theofficial documentation for Streamline (see Section4.1.3) made it difficult to know which of therequirements included in the priority directivewould be included in the final release. This createddisturbances for product managers who were inneed to have a good overview of release plan andits contents. As one product managers explained‘‘if we follow agile then I can not tell market units andinfluential customers what functionality will be deliveredin the next release’’.

According to the interviewed line managers andproduct managers it was usual that the number ofrequirements included in the priority directive wasmore than the capacity of the organisation, thusnot all requirements could be implemented. Conse-quently the product managers required confirma-tion from the R&D department on how many ofthe requirements in the priority directive would bepossible to include in the release. This in turn meantthat all requirements in the priority directive hadto be analyzed to produce a reliable cost overviewof the requirements included in the priority direc-tive (according to interviewed line managers theavailable information on the cost from the previouspre-project steps was too high level to be trusted).This situation created problems since analyzing theentire scope of the release was against the leanphilosophy of Streamline and line and project man-agers at the R&D department were afraid to fallback into the waterfall way of working.

In order to find a middle ground between leanprinciples and the needs of product management

a commitment model was suggested according towhich the R&D organisation would commit up to50% of requirements in the priority directive fromthe start. This meant that the scope of the releasewould be defined ahead of development projectsand it would contain the first half of all prioritisedrequirements. The remaining requirements wouldremain as candidates for in-scoping and would beconsidered only after the analysis of the committedscope was finished and necessary resources forproducing the requirements assigned to the scopewere secured.

4.2.3.2. Finding 6: Related topic: Different AttitudesTowards Agile in R&D and Product ManagementInterviews revealed a difference between the atti-tudes towards agile principles in the R&D organisa-tion and in the product management organisation.Line and project managers who are representingR&D were in general positive and enthusiastictowards agile, while product managers in generalexpressed concern about agile. As one product man-ager said ‘‘I must feel confident about the contents ofthe release in order to be able to market our product andhandle competition’’. It is interesting to note that theR&D organisation perceived the product managersconcern about agile principles to be due to ‘‘the oldway of thinking’’. As some of the project managerssaid ‘‘Product managers just have to find different waysto sell our products.’’

4.2.4. SummaryIn this section the findings identified through inter-views and observations will be analyzed withregard to the extent the practiced process followsthe official Streamline process described in Section4.1. After this, the findings from the interviewsand observations are organised in practiced pro-cess symptoms, providing ground to discuss theimplication of agile properties on MDPD needs.

Comparison between official and practiced pro-cesses: Finding 1 and Finding 3 show that forthe steps of pre-screening and prioritisation thepracticed process closely followed the steps out-lined by the official process (see Section 4.1). Somedifferences were identified showing misalignmentwhen it comes to the focus of the activities (forexample in pre-screening the activities were lessfocused on value analysis than expected). In addi-tion, the produced decision support material wasnot always written down as official reports on


DOI: 10.1002/spip


technical and business analysis (as mandated bythe formal process), but often consisted of informaldecisions based on expert opinion.

According to Finding 5 the largest differencebetween practiced and official process was identi-fied in the step of release planning and refinement,where the practice of evolving release scope asprescribed by Streamline was in reality difficult tofollow. It is important to notice that the step ofrelease scope analysis and refinement representsthe actual change from the waterfall approach tothe new agile way of working with the require-ments. According to the existing research on theextent of methodology adoption (Fitzgerald 1998) itis not uncommon that the prescribed developmentmethodology in reality is not followed rigorously.However, in this case the interviews and obser-vations provided clear indications that due to theproblems associated with the old way of working,the practitioners have made every effort to followthe prescribed process. It was due to the misalign-ment with the needs of decision makers that theadaptations to the prescribed way of release contentanalysis and planning was necessary.

4.2.4.1. Practiced Process Symptom 1: Pre-projectActivities are Focused on Commercial RequirementsAnalysis of the findings for the pre-screening andprioritisation steps shows clearly that these initialpre-project activities are mainly driven by commer-cial requirements (see Finding 2 and Finding 4).Additionally according to Finding 4, commercialfeatures are often prioritised over other types ofrequirements like architectural improvements.

4.2.4.2. Practiced Process Symptom 2: Evolving ReleaseScope Was Found Wanting in Relation to the Needs ofDecision Makers Finding 5 indicates that applyingthe concept of evolving release scope (AP3) wasdifficult for release planning purposes in a market-driven context. Thus there was a need to find asolution where agile properties would be adjustedto the needs of pre-project decisions. The introducedapproach of a commitment model is an example offinding a middle ground between agile principlesand the company’s needs. Further, Finding 6indicates a lack of understanding of the challengesand business constraints of pre-project decisions ina market-driven context.

4.2.4.3. Practiced Process Symptom 3: RequirementsValue is Defined through Expert Judgement Findings1 and 3 reveal that practitioners do not have aclear mechanism for describing, evaluating andcomparing value of different requirement types.According to Finding 3 evaluation of requirementsvalue is mainly a tacit process based on expectopinion, knowledge of product’s business caseand customer needs. Moreover, data presented inFinding 4 indicate a need to define clear proceduresand guidelines that can be used for representingdifferent requirement types, for example systemrequirements.

5. DISCUSSION

In Table 6 the symptoms of the practiced processidentified through the case study are connected tothe MDPD needs and agile properties that werepresented in Section 2.3.

As shown in the table, Symptom 1 and Symptom3 of the practiced process is associated with theagile properties AP1 and AP2 and is consideredto impact the need to find an appropriate balancebetween different requirement types, such as com-mercial requirements, requirements connected totechnology innovation and architectural and systemimprovements (N1 and N2). Symptom 2 is connectedto AP3 and is considered to impact the need to effec-tively plan the content of the future release (N3). Thefollowing sections elaborate on the implications ofapplying agile properties on pre-project activities inthe MDPD context, and discuss possible solutionsto decrease the gap between agile properties andneeds in MDPD.

5.1. Feature Orientation vs. Balancing DifferentRequirement Types

In the MDPD context balancing commercial, archi-tectural and quality requirements are extremelyimportant (Ebert 2005, Regnell et al. 2005). The sys-tem architecture is seen as one of the assets onwhich future releases will be built and maintainabil-ity issues are connected to the cost of supportingdifferent versions of the products in the marketover time (Tomaszewski et al. 2008). The problemsof finding a balance between features and architec-ture improvements have been acknowledged andreported by other researchers (Wohlin and Aurum


DOI: 10.1002/spip


Table 6. Alignment of study results with agile and MDPD properties

Agile property MDPD need Observed symptom Possible impact

AP1. Feature orientation N1. Balancing differentrequirements types

Symptom 1: Pre-projectactivities are focusedon commercialrequirements

- Short-term thinking

AP2. Reactive development N2. Trade-off betweenmarket-pull andtechnology-push

Symptom 3:Requirements value isdefined through expertjudgement

- Possible architecturedeterioration

- Product integrationproblems- Limited ability forpre-emptive release offeatures and thusinfluencing markets

AP3. Evolving release scope N3. Release contentplanning

Symptom 2: Evolvingrelease scope wasfound unfitting for theneeds of decisionmakers

- Limited possibility toinfluence and planrelease content andrelated developmentcost- Limitedunderstanding of totalrelease value- Change managementbecomes difficult (nobaseline to compare)

2005) and are common to Market Driven Require-ments Engineering (MDRE).

The results of the conducted case study haveshown the dominance of commercial requirementsin Streamline, threatening to create an imbalancebetween features and other types of requirementsnot driven by the perceived immediate needs to thecustomer or market.

In the case studied in this article we can notclaim that a perfect balance existed prior to theintroduction of Streamline. However there is a riskthat due to the agile properties of feature orientationand reactive development (AP1 and AP2), findingthe balance might be even more difficult. Thisassumption partly is motivated by a lack of supportfor handling non-functional requirements in agilemethods (Paetsch et al. 2003, Sillitti et al. 2005), aswell as existing research where feature orientationand reactive development in agile methods isnamed as a threat for long-term goals such asmaintaining a clean system architecture and otherquality aspects (Boehm 2002, Tomaszewski et al.2008).

The risk of increased dominance of commercialrequirements when applying agile methods can be

explained by examining the definition of require-ments value and requirements priority in agilemethods.

As discussed in Section 2.3, agile methods focuson delivering rapid value for a customer, whererequirements value and consequently its priorityis strongly associated to the customer value of arequirement (Beck and Fowler 2001, Poppendieckand Poppendieck 2003, Williams and Cockburn2003, Leffingwell 2007). This definition of a require-ment’s value in agile methods in combination withunclear procedures for defining requirements value(Practiced Symptom 3) can easily encourage pri-oritisation of commercial requirements which havea direct link to existing customers and overlookthe value of requirements which cannot be easilylinked to the customer, for example system andarchitecture related requirements.

The latest studies of requirements prioritisationpractices applied in industry have revealed thathaving unclear procedures when it comes to def-inition of requirements value is not uncommon(Barney et al. 2006, Lehtola and Kauppinen 2006).Thus for companies aiming to combine market-driven development with agile practices and still


DOI: 10.1002/spip


maintain a reasonable balance between the commer-cial and other types of requirements it is extremelyimportant to (i) establish a clear picture of whatis/should be perceived as valuable for a company,as opposed to value based only on the current needsof customers and how commercial and other type ofrequirements fit in this picture; and (ii) define clearprocedures and guidelines for expressing the valueof non-commercial requirements.

5.2. Evolving Release Scope vs. Release ContentPlanning

Previously in this article it was argued that applyingthe concept of evolving release scope as defined inAP3 fundamentally limits product managers’ abil-ity to preemptively plan and control the contentsand value of a release. The potential limitationsof applying the evolving release scope concept inMDPD context can be explained by the assumptionsof agile methods regarding development and busi-ness context in which a company operates (Section2.3), as well as the simplistic view of agile meth-ods when it comes to deciding the priority order ofrequirements and managing the release scope.

In agile methods all development is orchestratedby the priority order of requirements, makingthe priority list central and very important. Thequestion is then how this priority is decided.Agile methods provide quite a simple solutionto this where the task is left to the customerwho decides priority based on the business valueof a requirement. In order to help the customerprioritise, the developers provide some rough costestimates, but mainly the order is decided by thebusiness value (Beck and Fowler 2001, Poppendieckand Poppendieck 2003, Williams and Cockburn2003, Leffingwell 2007).

In MDPD the priority and implementation orderof a requirement is a function of the expected valueand cost of a requirement, technical dependenciesof a requirement and value dependencies betweenthe requirements (Carlshamre 2002, Greer and Ruhe2004, Saliu and Ruhe 2005, Ngo-The and Ruhe 2008).The mentioned difference in definition of priorityimplies that for the same set of initial requirementsagile and MDPD will produce different priorityorderings of these. More importantly, the definitionof requirements priority in agile, in combinationwith the concept of the evolving release scope affectsthe possibilities to plan and follow up the contents

and expected ROI of a release. This point is furtheron exemplified in Figure 3.

Figure 3 shows typical process steps of releasecontent planning and evolution in an agile (tothe left) and market-driven (to the right) contextrespectively and inputs and outputs for each step.Looking at the figure it is easy to notice that bothscenarios follow the same process steps. In bothof the scenarios there are collections of prioritisedrequirements that represent the initial release scopeand both of the scenarios are iterative, where thedevelopment activities follow the priority order ofrequirements placed in the release scope. After eachiteration there is a process step allowing analysis ofdelivered results and progress, as well as handlingof changes, for example dealing with newly elicitedrequirements (Req. x in the figure). This analysis mayresult in re-prioritisation and an updated releasescope.

A closer look at Figure 3 however uncoverscritical differences between the two scenarios, mostof which is caused by the difference in how thepriority and implementation order of a requirementis defined in agile vs. MDPD.

The differences between the scenarios are sum-marised below: (i) priority order of requirements inthe initial scope is different between the scenarios,resulting in differences in development activities.For example in the agile scenario the first iterationfocuses on Req. a, and in the MDPD scenario on Req.c; (ii) requirements analysis and re-prioritisationstep results in different content and requirementspriority order in the updated scope of each scenario;where for example the newly arrived requirementReq. x in the agile scenario receives high priorityand in the market-driven scenario does not. (iii)And finally in MDPD scenario pre-project decisionsare tightly connected with defining and followingup the total value and expected ROI of the finalrelease (in the figure this is indicated by assigning arbi-trary values for total release ROI and total release value)whereas in agile scenario this kind of analysis isdifficult (denoted by having question marks for the samevariables).

The definition of the priority order of the require-ments in MDPD demands a proper understandingof all requirements which are assigned to the scope.This way it provides a base for estimating the targetROI and total value provided by the requirementsselection (Saliu and Ruhe 2005, Ngo-The and Ruhe2008). This base is then used in order to plan, control,


DOI: 10.1002/spip


Figure 3. Differences in approach towards release planning between Agile and MDPD

and follow up how well the selected list of require-ment satisfies target business goals of a release. Inaddition it is important to gauge the consequencesof how removing or adding a new requirement toa release scope affects the total value and expectedROI of a release, making it possible to deal with achange in an appropriate way.

The simple definition of the priority order ofthe requirements in agile methods combined withthe property of an evolving release scope (AP3)does not allow an in-depth analysis of all candidaterequirements (Beck and Fowler 2001, Poppendieckand Poppendieck 2003, Williams and Cockburn2003, Leffingwell 2007). This makes it difficultto define the target ROI and total value to bedelivered by the release and creates a risk of abusingthe practice of ‘delaying the decisions’ where thedecision on what will be included in the release istaken after the fact, and thus the target ROI and thetotal value provided by the release is not clear untilafter the delivery.

Results of the conducted case study have pro-vided two important insights regarding this issue.

Firstly the results uncovered difficulties to followthe practice of evolving release scope, mainly dueto posed limitations to product management oper-ations (see Finding 5). This result relates well to theMDPD characteristics presented in Section 2.2, andprovides support for above-outlined discussion onmisalignment between product management needsand limited possibility of managing and controllingrelease scope in agile methods.

Secondly the results have indicated differentattributes towards agile methods in the organisa-tion, showing that while R&D part of the organisa-tion was in general positive to application of agilepractices in development project, there was lim-ited understanding of how agile properties affectedproduct management (see Finding 6).

The above mentioned findings are importantsince they show that while agile is appreciatedin separate development projects (Karlstrom andRuneson 2005, Svensson and Host 2005, Petersenand Wohlin 2008) they are not directly suitedor designed for long-term product planning andproduct management, thus there is a need to find


DOI: 10.1002/spip


a way to apply agile methods in developmentprojects without disrupting product managementoperations which are in need of more long-termplanning and control.

The need for finding a compromise betweenagile and plan-driven approaches and tailoringagile methods to meet specifics of different prod-uct development and business contexts has beensuggested by other researchers as well (Boehm2002, Abrahamsson et al. 2003, Cohen et al. 2004,Fitzgerald et al. 2006), however currently a concretesuggestion on how to accomplish this in MDPDcontext has not been suggested. The commitmentmodel described in Finding 5 represents one exam-ple of such a compromise. However, it is importantto notice that in order to determine the fist selectionof functionality committed to by R&D, somewhatreliable information on the relative value and thecost of all requirements assigned to the release scopeshould be available. This implies that some pre-study work is still necessary. The question is whatis good-enough, and how to support informed deci-sion making without investing too much effort orover analysing the requirements (Fricker et al. 2007).

5.3. Validity

Issues connected to construct validity; internalvalidity and external validity describe commonlyknown validity threats associated with empiricalstudies (Yin 1994, Wohlin et al. 2000).

Construct validity is concerned with establishingthe correct operational measures in order to assurethat the proper information will be collected. Thisincludes decisions on how the information sourcesare selected. The usual way to handle this threat is touse multiple sources of information. The case studyat Ericsson was designed to use multiple sourcesof information and multiple ways of collectionfrom documentation analysis and interviews toobservation sessions during meetings. All thesesources allowed for data triangulation. As shownin Section 3.1, studied documents, interviews, andmeetings were carefully selected to achieve a correctrepresentation.

When conducting investigations that are based oninterviews, the amount and form of the interviewsare important. In this case study budget and timeconsiderations as well as availability constraintsallowed for a total of 14 interviews. The interviewswere semi-structured. Notes and conclusions made

during the interview sessions were also confirmedby sending them to the interview subjects, whereadditional questions for clarification were attachedwhere relevant. In some cases an extra interviewwas arranged, however these extra interviews arenot included in the total count of the interviews inthe study.

Internal validity threats are usually importantwhen examining the causal relationships. Thecase study presented in this article is mainlyfocused of studying the characteristics of pre-project activities. It has an exploratory characterthus the internal validity threats can be consideredas minimal. In the discussion section study findingsare used to explore how well the study findingssupport the anticipated misalignment betweenagile properties and MDPD needs as describedin Section 2.3. For example the dominance ofcommercial requirements in pre-project activities(Symptom 1) is connected to the agile propertiesfeature orientation (AP1) and reactive development(AP2) and difficulties with applying the concept ofevolving release scope (Symptom 2) is associated tothe agile property of evolving release scope (AP3).

External validity is concerned with generalisabil-ity issues and is a threat that is common to casestudy research. Even though the case study in thisreport is conducted in a concrete industrial setting,we hope that the study results should be generalis-able beyond the studied case since the investigatedpre-project activities are not specific only for Erics-son but are common for most companies operatingin a market-driven context.

6. CONCLUSIONS

In this article we have investigated the applicabilityof agile methods for MDPD, focusing particularlyon pre-project activities such as requirements pre-screening, prioritisation and release planning. Thepresented work is considered to add value sinceit is one of the first contributions that investigatesthe applicability of agile practices in this context,providing extensive analysis and comparison ofagile practices and needs in MDPD, as well as resultsfrom a large empirical study at Ericsson. This area isespecially relevant considering the current hype andpopularity of agile methods, and the contents andconclusions of this article should be interesting for


DOI: 10.1002/spip


any software company operating in a market-drivencontext planning to adopt agile practices.

The initial analysis and comparison of agile prop-erties and MDPD needs (see Section 2.3) indicateda misalignment between the two. This was furtherconfirmed by the results of the case study, wherethe impact of agile properties was studied for threedifferent products developed at Ericsson. The out-come of the case study has provided confirmation ofthe misalignment between agile property ‘evolvingrelease scope’ and the needs for release planningin the market-driven context. Further, case studyfindings also provided indications that ‘feature ori-entation’ and ‘reactive development’ might be athreat when it comes to balancing commercial andother types of requirements, and achieving a trade-off between market-pull and technology-push asthese aspects are not central in agile development.

Looking at the origin of agile/lean develop-ment the misalignment can be partially explained.Agile methodology having bespoke software devel-opment origin was developed with customer–developer relationships in mind and has a projectfocus. Thus agile methods such as Scrum and XPare lacking support for an overall long-term productdevelopment focus, normally associated with soft-ware product management activities in companieslike Ericsson.

The main conclusion drawn is that agile methodsdo not directly support the needs of pre-projectactivities in MDPD. Application of agile propertiesin an organisation operating in market-drivencontext places limitations on product managementactivities, and may have a detrimental effect onlong-term product development.

Research results presented in this article indicatethat agile principles in their current form should notbe forced on product management tasks since theyare not fit for it. In MDPD product managers needto maintain long-term focus, differentiate betweencustomer and enterprise value and plan and controlthe scope of a release. However, this does not meanthat product management can not benefit fromagile/lean principles at all. The ideas from LeanSD can be reused to find ways for producing good-enough decision material for product management,enabling informed decisions far beyond the scopeof any individual project. For example companiesshould benefit from practices which will allowminimising time and effort spent on analysingpre-project decisions. These practices should also

guarantee that minimising effort does not comeat cost of jeopardising product managers’ abilityto take informed decisions or increased costs inlater stages of product development. The point hereis that while it is important to produce fast, itis also important to maintain the big picture andunderstand how the achievements of individualdevelopment projects benefit both short- and long-term business goals of the company developing theproduct.

6.1. Future Work

Through the investigation and uncovering of theimpact of agile practices on software productmanagement activities the authors of this articlehope to create a foundation for finding waysto incorporate agile projects in MDPD withoutcompromising long-term product goals.

The initial steps towards solving this issue willbe connected to resolving the uncovered misalign-ment points between agile principles and productmanagement needs. The future research will evolvein two directions: (i) the first direction will focus onhelping software companies maintain an appropri-ate balance between different types of requirementseven when following agile development princi-ples. In order to achieve this we plan to workon separation of customer value from enterprisevalue and finding approaches for defining andcomparing value of commercial and architecturalrequirements. (ii) The second direction intends tofind ways of applying lean development ideas insoftware product and release planning situations.The key here is in finding just-enough level of anal-ysis of the entire release scope without requiringexcessive details but also allowing for the presenceof decision support material (e.g. in the form ofgood-enough requirements). In this way the authorshope to facilitate defining practices for Lean SD.

ACKNOWLEDGEMENTS

We would like to express our gratitude to Ericsson,an industrial partner of the BESQ (Blekinge Instituteof Technology, Sweden; http://www.bth.se/besq)research centre, for making this study possible.In the search for a middle ground between agileand plan-driven models Ericsson has taken animportant step to integrate agile methods into their


DOI: 10.1002/spip


product development. Through the experience ofusing these methods, Ericsson started adding tostate-of-the-art by investing effort in finding newand improved ways of applying agile principlesto large-scale product development. These effortsare a part of joint ongoing research projects. Lastbut not least we would like to thank all the skilledand dedicated professionals at Ericsson for seeingopportunities and not problems.

REFERENCES

Abrahamsson P, Warsta J, Siponen M, Ronkainen J. 2003.New directions on agile methods: a comparative analysis.25th International Conference on Software Engineering(ICSE’03). IEEE Press.

Barney S, Aurum A, Wohlin C. 2006. Quest for aSilver Bullet: Creating Software Product Value throughRequirements Selection. 32nd EUROMICRO Conferenceon Software Engineering and Advanced Applications(EUROMICRO’06), IEEE Computer Society. Cavtat,Croatia.

Beck K. 1999. Extreme Programming Explained: EmbraceChange. Addison-Wesley: Massachusetts, Boston MA.

Beck K, Fowler M. 2001. Planning Extreme Programming,The XP series. Addison-Wesley: Boston, MA.

Boehm B. 2002. Get ready for agile methods, with care.Computer 35(1): 64–69.

Carlshamre P. 2002. Release planning in market-driven software product development: provoking anunderstanding. Requirements Engineering 7(3): 139–151.

Carlshamre P, Sandahl K, Lindvall M, Regnell B, NattochDag J. 2001. An industrial survey of requirementsinterdependencies in software product release planning.Proceedings of the Fifth IEEE International Symposiumon Requirements Engineering, Los Alamitos, CA, IEEE.

Clements P, Northrop L. 2002. Software product lines:practices and patterns. The SEI Series in SoftwareEngineering. Addison-Wesley: Boston, MA; 608.

Cohen D, Lindvall M, Costa P. 2004. An introduction toagile methods. Advances in Computers 62: 1–66.

Dagnino A, Smiley K, Srikanth H, Anton AI, Williams L.2004. Experiences in applying agile software developmentpractices in new product development. 8th IASTEDInternational Conference on Software Engineering andApplications: Cambridge, MA.

Dahlstedt A, Karlsson L, Persson A, NattochDag J,Regnell B. 2003. Market-driven requirements engineering

processes for software products – a report on currentpractices. International Workshop on COTS and ProductSoftware RECOTS, held in conjunction with the 11th IEEEInternational Requirements Engineering Conference, LosAlamitos, CA, IEEE.

Dyba T, Dingsøyr T. 2008. Empirical studies of agilesoftware development: a systematic review. Informationand Software Technology 50(9–10): 833–859.

Ebert C. 2005. Requirements BEFORE the requirements:understanding the upstream impacts. 13th IEEEInternational Conference on Requirements Engineering(RE’05), IEEE Paris, France.

Erickson J, Lyytinen K, Siau K. 2005. Agile modeling,agile software development, and extreme programming:the state of research. Journal of Database Management 16(4):88–100.

Fitzgerald B. 1998. An empirical investigation into theadoption of systems development methodologies. Journalof Information & Management 34: 317–328.

Fitzgerald B, Hartnett G, Conboy K. 2006. Customizingagile methods to software practices at Intel Shannon.European Journal of Information Systems 15: 200–213.

Fricker S, Gorschek T, Myllyperkio P. 2007. Handshakingbetween software projects and stakeholders usingimplementation proposals. Requirements Engineering:Foundation for Software Quality, (REFSQ’07). SpringerBerlin/Heidelberg: Trondheim.

Gorschek T. 2006. Requirements engineering support-ing technical product management. School of Engineer-ing – Department of Systems and Software Engineering.Blekinge Institute of Technology: Ronneby. 342.

Gorschek T, Davis A. 2008. Requirements engineering: insearch of the dependent variables. Information and SoftwareTechnology 50: 67–75.

Gorschek T, Wohlin C. 2004. Packaging software processimprovement issues – a method and a case study.Software: Practice and Experience 34(14): 1311–1344.

Gorschek T, Wohlin C. 2006. Requirements abstractionmodel. Requirements Engineering 11(1): 79–101.

Greer D, Ruhe G. 2004. Software release planning: anevolutionary and iterative approach. Information andSoftware Technology 46(4): 243–253.

Hanssen GK, Fægri TE. 2008. Process fusion: an industrialcase study on agile software product line engineering.Journal of Systems and Software 81(6): 843–854.

Highsmith J, Cockburn A. 2001. Agile softwaredevelopment: the business of innovation. IEEE Computer34(9): 120–122.


DOI: 10.1002/spip


Ilieva S, Ivanov P, Stefanova E. 2004. Analyses of an agilemethodology implementation. 30th Euromicro Conference.IEEE Computer Society Press.

Karlsson L, Dahlstedt A, Nattoch Dag J, Regnell B,Persson A. 2003. Challenges in market-drivenrequirements engineering – an industrial interviewstudy. Proceedings of the Eighth International Workshopon Requirements Engineering: Foundation for SoftwareQuality (REFSQ’02). Essen, Germany, UniversitatDuisburg-Essen.

Karlstrom D, Runeson P. 2005. Combining agile methodswith stage-gate project management. IEEE Software 22(3):43–49.

Koch AS. 2005. Agile Software Development Evaluating theMethods for Your Organization. Artech House Publishers:London.

Layman L, Williams L, Cunningham L. 2004. Exploringextreme programming in context: an industrial casestudy. Agile Development Conference, Salt lake city, UT;published by IEEE Computer Society Press.

Leffingwell D. 2007. Scaling Software Agility. Addison-Wesley: Boston, MA.

Lehtola L, Kauppinen M. 2006. Suitability of require-ments prioritization methods for market-driven softwareproduct development. Software Process: Improvement andPractice 11(1): 7–19.

Cunningham W, Manifesto for Agile SoftwareDevelopment. 2001. [cited 2008-09-30]; Available from:http://www.agilemanifesto.org.

Mann C, Maurer F. 2005. A case study on the impactof scrum on overtime and customer satisfaction. AgileDevelopment Conference (ADC’05). IEEE Computer Society:Denver, CO; published by IEEE Computer Society Press.

Mannaro K, Melis M, Marchesi M. 2004. Empiricalanalysis on the satisfaction of IT employees comparingXP practices with other software developmentmethodologies. Extreme Programming and Agile Processes inSoftware Engineering. Lecture Notes in Computer Science,Springer, Berlin.

Middleton P. 2001. Lean software development: two casestudies. Software Quality Journal 9(4): 241–252.

Ngo-The A, Ruhe G. 2008. A systematic approach forsolving the wicked problem of software release planning.Soft Computing – A Fusion of Foundations, Methodologies andApplications 12(1): 95–108.

Noor MA, Rabiser R, Grunbacher P. 2008. Agile productline planning: a collaborative approach and a case study.Journal of Systems and Software 81(6): 868–882.

North American and European Enterprise Softwareand Services Survey. 2005. Business Technographics.Survey conducted by Forrester Research inc. Availableat http://www.Forrester.com/ER/Research/Survey/Excerpt/0, 5449, 436,00.html.

Paetsch F, Eberlein A, Maurer F. 2003. Requirementsengineering and agile software development. 12th IEEEinternational Workshops on Enabling Technologies:Infrastructure for Collaborative Enterprises, IEEE, Linz,Austria.

Petersen K, Wohlin C. 2008. Issues and advantagesof using agile and incremental practices. The EighthConference on Software Engineering Research andPractice in Sweden (SERPS’08), Karlskrona, Sweden.

Pettersson F, Ivarsson M, Gorschek T, Ohman P. 2008.A practitioner’s guide to light weight software processassessment and improvement planning. Journal of Systemsand Software 81(6): 972–995.

Poppendieck M, Poppendieck T. 2003. Lean SoftwareDevelopment – An Agile Toolkit. Addison-Wesley: Boston,MA.

Potts C. Invented requirements and imagined customers:requirements engineering for off-the-shelf software. 1995.Proceedings of the Second IEEE International Symposiumon Requirements Engineering, Los Alamitos, IEEE.

Regnell B, Brinkkemper S. 2005. Market-Driven Require-ments Engineering for Software Products. In Engineeringand Managing Software Requirements, Wohlin C, Aurum A(eds). Springer: Heidelberg, Berlin 287–308.

Robson C. 2002. Real World Research: A Resource for SocialScientists and Practitioner-Researchers, 2nd edn. BlackwellPublishers: Oxford.

Saliu O, Ruhe G. 2005. Supporting software releaseplanning decisions for evolving systems. 29th AnnualIEEE/NASA Software Engineering Workshop (SEW’05),IEEE, Greenbelt, MD.

Schwaber K, Beedle M. 2001. Agile Software Developmentwith Scrum. Prentice Hall: Upper Saddle River, NJ.

Sillitti A, Succi G. 2005. Requirements engineering foragile methods. In Engineering and Managing SoftwareRequirements, Wohlin C, Aurum A (eds). Springer: NewYork, NY; 309–326.

Svensson H, Host M. 2005. Introducing an agile processin a software maintenance and evoluition organization.9th European Conference on Software Maintenance andReengineering (CSMR 2005). Manchester, UK.

Telelogic Focal Point – Software for Product Managementand Product Portfolio Management. 2009. [cited


DOI: 10.1002/spip


2009-01-22]; Available from: http://www.telelogic.com/corp/products/focalpoint/index.cfm IBM Corporation.

Tian K, Cooper K. 2006. Agile and software productline methods: are they so different?. 1st InternationalWorkshop on Agile Product Line Engineering: collocatedwith the 10th International Software Product LineConference (SPLC), Baltimore, MD.

Tomaszewski P, Berander P, Damm L-O. 2008. Fromtraditional to streamline development – opportunitiesand challenges. Software Process: Improvement and Practice13(2): 195–212.

van de Weerd I, Brinkkemper S, Nieuwenhuis R,Versendaal J, Bijlsma L. 2006. Towards a referenceframework for software product management.Proceedings of the 14th IEEE International RequirementsEngineering Conference (RE’06), IEEE Computer Society,Minneapolis, MN.

Williams L, Cockburn A. 2003. Agile softwaredevelopment: it’s about feedback and change. Computer36(6): 39–43.

Wohlin C, Aurum A. 2005. What is important whendeciding to include a software requirement in a project orrelease?. International Symposium on Empirical SoftwareEngineering, IEEE, Noosa Heads, Australia.

Wohlin C, Runeson P, Host M, Ohlson MC, Regnell B,Wesslen A. 2000. Experimentation in Software Engineering:An Introduction, The Kluwer International Series in SoftwareEngineering. Kluwer Academic: Boston, MA; 204.

Womack JP, Jones DT, Roos D. 1991. The Machine ThatChanged the World : The Story of Lean Production. HarperPerennial: New york.

Yin R. 1994. Case Study Research. SAGE Publications:Thousand Oaks, CA.


DOI: 10.1002/spip

The impact of agile principles on market-driven software product development

Documents

Transcript of The impact of agile principles on market-driven software product development