Theoretical Reflections on Agile Development Methodologies [1]

&Can We Manage Agile in Traditional

Project Environments?[2]

Chandrika Seenappa

23rd February 2015

Professor: Hossein Saiedian

2

Agenda [1]

• Introduction• Strategic Thinking• Design Metaphor• Conceptualization of Software Development• Holographic Organization• Conclusions

3

Agenda [2]

• Introduction• Agile - Unique Project Management Concept• Principles of Agile• Agile vs Conventional Project Management• Research• Results• Conclusions

4

Theoretical Reflections on Agile Development Methodologies

5

Introduction

• In the past decade, there has been much debate among software developers over contradictory perspectives of the traditional plan-driven approaches vs. the agile philosophy of software development

• Identify theoretical and conceptual support for the emerging epistemology (a branch of philosophy that investigates the origin, nature, methods, and limits of human knowledge)

• Enriches our understanding of the agile phenomenon by examining its theoretical roots

• Software development is a complex undertaking beset with many problems, called “wicked problems” termed by Horst Rittel, an urban planner who pioneered the concept of issue-based information systems to facilitate the formulation and clarification of complex administrative decisions

6

Wicked Problem

• Problem are unique and difficult to formulate • Solutions evolve continually as the designer gains a greater appreciation of

what must be solved • Need to satisfy multiple conflicting viewpoints makes it difficult to devise

a test to determine the effectiveness of solutions• “Argumentation” critical for problem solving

7

Wicked Problem Characteristics

• There is no definitive formulation of a wicked problem• Wicked problems have no stopping rule• Solutions to wicked problems are not true-or-false, but good or bad• There is no immediate and no ultimate test of a solution to a wicked problem• Every solution to a wicked problem is a "one-shot operation"; because there is no

opportunity to learn by trial and error, every attempt counts significantly• Wicked problems do not have an enumerable set of potential solutions, nor is there

a well-described set of permissible operations that may be incorporated into the plan

• Every wicked problem is essentially unique• Every wicked problem can be considered to be a symptom of another problem• The existence of a discrepancy representing a wicked problem can be explained in

numerous ways. The choice of explanation determines the nature of the problem's resolution

• The social planner has no right to be wrong

8

Strategies to Tackle Wicked Problems

• Cannot be tackled by the traditional approach in which problems are defined, analyzed and solved in sequential steps

• No clear problem definition of wicked problems• In a paper published in 2000, Roberts identifies the following strategies to

cope with wicked problems– Authoritative : Tame wicked problems by vesting the responsibility for

solving the problems in the hands of a few people– Competitive: Attempt to solve wicked problems by pitting opposing

points of view against each other, requiring parties to come up with their preferred solutions. Advantage is that different solutions can be weighed up against each other and the best one chosen.

– Collaborative: Aim to engage all stakeholders in order to find the best possible solution for all stakeholders

9

Design Concepts in Architecture

• Chris Alexander, a professor of architecture at the University of California, Berkeley, pioneered in the 1970s work on patterns in architectural design

• Inspired development of software design patterns• Design patterns are proven design experiences and best practices for

solving problems across domains• Early model conceived design as an instrumental process dictated by

technical rationality• Aimed to identify the optimal means to a predictable end by employing

scientific methods used in problem solving in the engineering disciplines• Reduced design to a linear sequence of well-defined steps, with analysis

preceding synthesis• Key activity in analysis- systematic problem decomposition, hierarchical

top-down functional decomposition

10

Design Concepts Development

• As early as 1963, key aspects of design are– Continuous feedback among phases, communication, and iterative

cycles of analysis and synthesis• Argumentation involving stakeholders became an essential part of the

design process inspired by wicked problem• Need to integrate multiple viewpoints did not explicitly include the concept

of learning• Model of design process viewed design as a learning process in which the

solution evolves through repetitive cycles of problem formulation, solution evaluation, and documentation

• Designers use their own personal expanding knowledge to continually reframe the problem and devise an appropriate solution

11

Insights of Nature of Design

• Complexity– Problems are recognized as wicked, or not easily tractable

• Emergence– Design conceived as a repetitive emergent process interlacing

argumentation, introspective reflection, learning, and continual problem and solution refinement

• Integrative– Integration of multiple perspectives is critical to problem resolution

12

Strategic Thinking

• The dominant paradigm for strategy formulation is a formal process to identify the best fit between an organization and its environment

• Predicated on the assumption of a foreseeable and unchanging world• Mintzberg, conceptualizes strategy formulation as a craft, likening

strategists to craftspeople whose minds work in concert with their hands as they learn and improvise in shaping an artifact

• Effective strategies evolve through a learning process involving skills, experience, and insights gained through the dynamic interplay among formulation, implementation, and critical reflection

• Strategy is an emergent process in which strategy formulation and implementation are inseparable

13

Design Metaphor

• Design metaphor incorporates learning and acknowledges the connectedness of knowing and doing (thought and action), the interwoven nature of means and ends, and the need to reconcile multiple worldviews.

• Approach is “substantially rational” facilitating learning and adaptation through the constant questioning of assumptions

• Constantly reexamining assumptions and evaluating discrepancies against existing norms and values fosters double-loop learning , which is “generative” by increasing both learning and the ability to innovate and use change to one’s advantage

• The traditional school of design is more in the spirit of single-loop learning or “adaptive learning” , in which routine problem solving parameters and preset rules, norms, and contrasts determine goal-seeking behavior

14

Evolutionary Shift in Design Thinking

15

Conceptualization of Software Development• Trend in management thinking, moving from a deterministic/mechanistic

view of problem solving to a dynamic process, characterized by iterative cycles and the active involvement of all stakeholders, is reflected in software development

• Agile methods are people-centric, recognizing the value competent people and their relationships bring to software development

• Focuses on providing high customer satisfaction through three principles: quick delivery of quality software; active participation of concerned stakeholders; and creating and leveraging change methods

• Important features of this approach include evolutionary delivery through short iterative cycles of planning, action, reflection intense collaboration, self-organizing teams, and a high degree of developer discretion

16

Traditional and Emerging Perspectives of Design

17

Holographic Organization

• Holographic organization draws its inspiration from the fact that every fragment or piece of a broken holographic film contains the information required to completely construct the image represented in the whole film

• Can be accessed through any of its parts, as each one is a reflection of the whole

• Agile philosophy facilitates formation of holistic teams through a culture that encourages the interchangeability of roles or jobs based on autonomy

• Builds both specialized and generalized skills among members so they become multifaceted and exhibit overlapping knowledge so they can self-organize in response to emergent requirements

18

Holographic Organization (Cont.)

• Agile practices include – Continuous code integration– Refactoring to improve design and code– Reflection workshops and stand-up meetings to determine what

worked and what didn’t– Instant feedback from participating stakeholders

• Agile practices facilitate double-loop learning, supporting the principle of “learning to learn”

19

Alignment of Agile Practices with Holographic Principles

20

Conclusions

• Agile methods depart from the traditional orthodoxy of software development

• Metaphor of design and the theory of holographic organization offer a strong theoretical basis for the conceptual foundation of agile methods

• Efforts to understand the theoretical roots of software development practices by examining the evolution of design ideas in architecture, strategic management, and other disciplines are more relevant as system domains extend beyond simpler to the complex social aspects of software applications

21

Can We Manage Agile in Traditional Project Environments?

22

Introduction• Agile project management is the execution of projects in a highly flexible

and interactive manner, in opposite to the waterfall model• Agile has gained much attention and controversy as a contemporary

project management approach• Great benefits and advantages over traditional execution methods• Others experienced disappointments or outright failures• Results of a three-year field study into the practices of agile project

management at 37 technology-intensive companies

23

Introduction (Cont.)

• Agile principles are applicable to most projects which are– Independent of their nature– Size or IT-orientation– Improving resource effectiveness– Project execution time – Overall project success

• Large and highly complex projects– Carefully modified to fit the organizational processes and cultures of

the enterprise

24

How did Agile Project Management Philosophy Emerge?• Up to 1980s the waterfall model was the prevailing method for project

execution• Dramatic changes in the business environment

– Required project management to be reengineered – Deal effectively with new challenges– Balance efficiency, speed, and quality

• New technologies, in computers and communications– Created enormous opportunities and challenges– Transformed global economy into a hypercompetitive enterprise

system

25

How did Agile Project Management Philosophy Emerge? (Cont.)• To survive and prosper project leaders had to

– Deal effectively with time to-market pressures– Innovation, cost, and risks in an increasingly fast-changing global

business environment • Iterative, incremental project management gained momentum in the 1990s

– Norm for developing and introducing new products, systems, and services gradually replacing the traditional waterfall model

• Agile plus its derivatives, such as Extreme, Lean and Guerilla project management were introduced in 2001

• Iterative lifecycle or adaptive project life cycle or change-driven methods– Facilitate change and require a high degree of ongoing stakeholder

involvement

26

Agile - A Unique Project Management Concept• Agile is an extension of the multi-phased approach to project management• Promotes concurrent execution of tasks segments

– Requires more intense and effective interaction among overlapping task teams

– Need for strong cross-functional cooperation, integration and team involvement

– Creates both managerial benefits and challenges the operational and strategic values

27

Agile - A Unique Project Management Concept (Cont.)• Wide spectrum of benefits to the enterprise

– Validating functionality of work-in-progress – Better communication and information sharing among all stakeholders

to total project lifecycle thinking and strategic alignment• Effective cross-functional collaboration, and full integration of the project

management process with the total enterprise and its supply chain• Systematic approach to integrated project execution

– Emphasizes parallel, integrated execution of project phases– Replaces the traditional linear process of serial engineering and

expensive design-build-rollout rework

28

Principles of Agile

• Customer satisfaction by rapid delivery of useful software• Welcome changing requirements, even late in development• Working software is delivered frequently• Working software is the principal measure of progress• Sustainable development, able to maintain a constant pace• Close, daily cooperation between business people and developers• Face-to-face conversation is the best form of communication (co-location)• Projects are built around motivated individuals, who should be trusted• Continuous attention to technical excellence and good design• Simplicity the art of maximizing the amount of work not done is essential• Facilitate self-organizing teams• Regular adaptation to changing circumstances

29

Agile vs Conventional Project Management• Agile project management relies on the iterative, incremental project

execution– Involvement and collaboration of all project stakeholder groups– Top management, suppliers, partners, users and customers

• Deal with changes and uncertainty by aiming for quick, short-term results, called releases

• New product or service system developed in short activity cycles called sprints

• Shared power and responsibility concept between product owner and scrum master

30

Typical Stages of Agile Project Life Cycle

31

Action Research• Summarizes the last three years (2010-2013) of this investigation with

focus on Agile management practices and experiences of 35 project management teams in 17 high-technology enterprises dealing with major product developments

• Qualitative methods for data retrieval– Participant observation, in-depth retrospective interviewing,

questionnaires• The questionnaires were personally introduced to 230 team members and

35 team leaders, yielding an overall return of 75%• The focus was on four interrelated sets of variables

– Risk– Team– Team leader– Project environment

32

Summary of Field Sample• Projects were on information system, financial services, automotive,

airplane, computer and pharmaceutical products• Three project sizes

– Small medium-sized projects with 50 team members or less– Large projects with over 50 team members– Mega projects with over 1000 team members

• Data evaluated via content analysis and standard statistical methods

33

Results

• Findings of study organized into three sections– First, a simple summary of Agile usage across project categories is

presented– Second, the reaction of managers to the 12 Principles of Agile is

summarized and discussed with focus on benefits, challenges, limitations and performance impact of managing agile

– Third, the lessons learned from this study are summarized based on effective agile management

34

Intensity of Agile Application on Project Type• One size does not fit all project

35

Experiences from Agile with Focus on twelve Agile Principles

36

Experiences from Agile with Focus on twelve Agile Principles (Cont.)

37

Agile Principles in Large Projects• To be truly agile, management process must be

– Iterative, that is take several cycles to complete– Incremental, delivering the product and/or service in workable pieces– Rely on self-organizing teams– Evolve regarding its work structures and processes as needed during

the project lifecycle• Difficult to realize in large projects

– Require more execution formality and discipline to deal with specific complexities, contractual requirements and project interfaces

– Airplane developments, advertisement campaigns, Olympic Games or aerospace missions, cannot be delivered incrementally

– Hybrid approaches: Simulations, emulations, focus grouping, user-centered design and computer modeling achieve objective of early validation of deliverables and flexibility in changing requirements

38

Agile Challenges in Projects with Long Development Cycles• Major challenge to agile methodology is rapidly (or immediately)

delivering functionality• Difficult to achieve without compromising long-range optimization of

overall results

39

Integrating Agile Approaches into Traditional Project Systems• Featured benefits of agile

– Flexibility– Responsiveness to changing environments– Active user involvement– Iterative requirements verification

• Difficult to realize in traditional or large project environments with established client relations and specific contractual procedures

• One of the biggest challenges relates to process execution conflict

40

Challenges to Established Standards, Norms and Processes of Professional Project Management

• Agile often is unworkable in projects that need to be executed to meet specific results within given time and resource constraints

• Difficult in large projects and special categories of projects, such as government contracts, where the overall requirements and project scope must be established up-front and becomes the basis for performance measurements

• Despite these challenges, some organizations are able to use agile at least in part or in principle, even for very large and complex projects.

• Agile is not a “do as you want process”• Hard work and skillful leadership required to introduce a general process

template to a specific work environment

41

Not all Project Activities fit Agile

• Documentation, training or other logistics support not part of agile, but require major adjustment

• Agile can often be adapted• As stated by a project manager responsible for implementation of large

training program:

“I hear people saying all the time that Agile is not meant for this kind of project. The reality is that no method will guarantee you success. You have to fine-tune it to your specific situation. I select the pieces that are appropriate for my projects. Agile gives me a framework for interacting with my user community, testing out deliverables before we go to a major training session.”

42

Not all Challenges are Limitations• Challenges and barriers to agile are perceptions, but not necessarily true

limitations. • The culture or process is fixed and not under the control of the project

manager• The process of agile is not a rigid template, but a guideline that must be

adapted to a specific project situation, especially if the situation is outside the framework for which agile was originally designed for

• As explained by a senior manager of a large transport system development: “Just because you're following the process doesn't mean you're doing it right. You really need to think creatively and have talented people who understand the objectives of the agile principles to adapt the ideas to work in our environment, and to do it right. If you’re not involving your team, and constantly evaluating your practices what works, what doesn't and hat needs to be fine-tuned, you're likely to fail.”

43

Lessons Learnt

• Introduce Agile to your organization incrementally. Start with agile practices that already support existing processes and organizational cultures

• Examine the various components of your project management process• for opportunities for applying Agile principles without compromising

existing management processes and controls; identify compatible and incompatible assumptions, and encourage synergism between agile and traditional methods

44

Lessons Learnt (Cont.)

• Examine customer interfaces and contracting practices, redesign procedures to support agile execution

• Examine progress measurement, reviews and payment practices for realignment with agile execution

• Win the hearts and minds and support of senior management, convincing them that Agile is a more effective way of executing projects leading to higher success rates and customer satisfaction

• pay attention to people issues, such as empowerment, collaboration, commitment and ownership which are at the backbone of Agile and crucial to its successful practice

45

Conclusions• Agile has great potential for improving project management effectiveness

in our ultra-competitive, fast-changing business environment• In spite of the challenges, the principles and basic philosophy of agile can

be integrated into traditional project management processes• In order to be effective, change leaders must go far beyond issuing new

procedures and guidelines• Each component of agile, including underlying assumptions, must be

carefully examined to determine the degree of adaptability, redesigned and fine-tuned if necessary to fit into the existing management system

• Companies are combining, integrating and synchronizing agile and traditional processes to create a unique agile hybrid system which fits specific project situations, together with their people, organizational cultures and the total enterprise business environment

46

References

• Nerur, S., & Balijepally, V. (2007). Theoretical reflections on agile development methodologies. Communications of the ACM, Vol. 50(3), pp.79-83.

• Thamhain, H.J. (2014). Can We Manage Agile in Traditional Project Environments?. 2014 Proceeding of PICMET (Portland International Conference on Management of Engineering and Technology): Infrastructure and Service Integration, pp. 2497-2505.