Systems Engineering Advancement Research...

8
Systems Engineering Advancement Research Initiative RESEARCH BULLETIN DECEMBER 2007 Vol. 2, Issue 3 About SEAri The Systems Engineering Advancement Research Initiative (SEAri) brings together a set of sponsored research projects and a consortium of systems engineering leaders from industry, government, and academia. SEAri is uniquely positioned within the Engineering Systems Division (ESD) at the Massachusetts Institute of Technology (MIT), a new kind of interdisciplinary academic unit that spans most departments within the School of Engineering, as well as the School of Science, the School of Humanities, Arts, and Social Sciences, and the Sloan School of Management. This setting offers a robust research and learning environment for advancing systems engineering to meet the contemporary challenges of complex socio-technical systems. SEAri has strategic relationships with several educational and research programs at MIT, including the MIT System Design & Management Program (SDM) and the Lean Advancement Initiative (LAI) Research Program at MIT. Find out more at http://seari.mit.edu/ SEAri First Annual Research Summit The concept of an annual summit involves inviting together a group of parties with mutual interests for sharing perspectives on a topic. A research summit provides an opportunity for sharing knowledge and for crafting a shared research agenda for a research initiative. With this goal in mind, on 16 October 2007, SEAri held its first annual research summit at the MIT faculty club in Cambridge, MA. Providing a venue for interaction between students, research staff, and sponsors, the summit supported one of SEAri’s primary goals of disseminating information to the broader Systems Engineering community. Structured as a single full day event, the summit featured research overview and project briefings on advanced systems engineering topics. SEAri research portfolio topics were described by researchers, with alternating short student research summary presentations. The agenda sought to create opportunities for discussion and feedback with audience members. The agenda topics included: motivations and challenges; designing for changeability; socio-technical decision making; architecting principles for “ilities”; systems engineering in the enterprise; and systems engineering economics. The presentations are now available for download on the SEAri website: http://seari.mit.edu . A Message from the Leadership Team Fall is always an exciting time at MIT as new students begin their degree programs and new research projects get underway. A particular highlight of the semester for SEAri was our annual research summit held on October 16 th at the MIT Faculty Club. The event featured briefings by research staff and graduate students on SEAri and its research portfolio, with progress updates on ten sponsored research projects. When asked what motivated their participation in the summit, one sponsor remarked, “I’m here because I want to see the fun put back into systems engineering.” While on the surface a light-hearted comment, the reality is that this remark resonated with all attendees, and is at the heart of what is needed to advance systems engineering practice and revitalize the profession. Classical systems engineering has served the systems community well, and continues to do so for solving certain classes of problems. Yet, we live in a world where systems and their contexts grow increasingly complex, and where enterprises require unprecedented levels of responsiveness to dynamic needs. As a result, we have seen classical approaches and methods of systems engineering stretched beyond their intended capabilities. The over-extension of these practices has had a negative outcome on programs and on the profession. For example, systems engineering planning has too often become about creating an overly prescriptive planning document that satisfies a checklist of topics, rather than a strategic and dynamic planning endeavor. Further, systems engineering leaders struggle with new issues and decisions in the modern global engineering environment and in a world where systems and enterprises are often no longer created in the same Table of Contents Message from the Leadership Team ................. 1-2 Research Spotlight 1 ......................................... 3 Research Spotlight 2 ......................................... 5 SEAri News ........................................................ 7 Upcoming Events............................................... 8

Transcript of Systems Engineering Advancement Research...

Page 1: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

Systems Engineering Advancement

Research Initiative

RESEARCH BULLETIN DECEMBER 2007 Vol. 2, Issue 3

About SEAri

The Systems Engineering Advancement Research Initiative (SEAri) brings together a set of sponsored research projects and a consortium of systems engineering leaders from industry, government, and academia. SEAri is uniquely positioned within the Engineering Systems Division (ESD) at the Massachusetts Institute of Technology (MIT), a new kind of interdisciplinary academic unit that spans most departments within the School of Engineering, as well as the School of Science, the School of Humanities, Arts, and Social Sciences, and the Sloan School of Management. This setting offers a robust research and learning environment for advancing systems engineering to meet the contemporary challenges of complex socio-technical systems. SEAri

has strategic relationships with several educational and research programs at

MIT, including the MIT System Design & Management Program (SDM) and the Lean Advancement Initiative (LAI) Research Program at MIT. Find out more at http://seari.mit.edu/

SEAri First Annual Research Summit

The concept of an annual summit involves inviting together a group of parties with mutual interests for sharing perspectives on a topic. A research summit provides an opportunity for sharing knowledge and for crafting a shared research agenda for a research initiative. With this goal in mind, on 16 October 2007, SEAri held its first annual research summit at the MIT faculty club in Cambridge, MA. Providing a venue for interaction between students, research staff, and sponsors, the summit supported one of SEAri’s primary goals of disseminating information to the broader Systems Engineering community. Structured as a single full day event, the summit featured research overview and project briefings on advanced systems engineering topics. SEAri research portfolio topics were described by researchers, with alternating short student research summary presentations. The agenda sought to create opportunities for discussion and feedback with audience members. The agenda topics included: motivations and challenges; designing for changeability; socio-technical decision making; architecting principles for “ilities”; systems engineering in the enterprise; and systems engineering economics. The presentations are now available for download on the SEAri website: http://seari.mit.edu.

A Message from the Leadership Team

Fall is always an exciting time at MIT as new students begin their degree programs and new research projects get underway. A particular highlight of the semester for SEAri was our annual research summit held on October 16

th at the MIT Faculty Club. The event featured

briefings by research staff and graduate students on

SEAri and its research portfolio, with progress updates on ten sponsored research projects.

When asked what motivated their participation in the summit, one sponsor remarked, “I’m here because I want to see the fun put back into systems engineering.” While on the surface a light-hearted comment, the reality is that this remark resonated with all attendees, and is at the heart of what is needed to advance systems engineering practice and revitalize the profession.

Classical systems engineering has served the systems community well, and continues to do so for solving certain classes of problems. Yet, we live in a world where systems and their contexts grow increasingly complex, and where enterprises require unprecedented levels of responsiveness to dynamic needs. As a result, we have seen classical approaches and methods of systems engineering stretched beyond their intended capabilities. The over-extension of these practices has had a negative outcome on programs and on the profession. For example, systems engineering planning has too often become about creating an overly prescriptive planning document that satisfies a checklist of topics, rather than a strategic and dynamic planning endeavor.

Further, systems engineering leaders struggle with new issues and decisions in the modern global engineering environment and in a world where systems and enterprises are often no longer created in the same

Table of Contents Message from the Leadership Team................. 1-2 Research Spotlight 1 ......................................... 3 Research Spotlight 2 ......................................... 5 SEAri News........................................................ 7 Upcoming Events............................................... 8

Page 2: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 2 of 8

manner as in the past decade. Decision making and leadership require an ability to effectively consider that many modern systems are ‘composed’ through the integration of legacy systems, with culture, economics, politics and many other socio-technical factors coming into play. The challenges are significant, and evolving engineering to address these has proven to be difficult and at times very discouraging as evidenced by continued reports of failures of programs and organizations. So how then, can we put the fun back into systems engineering in this very complex and dynamic world? This is a significant topic to explore, so let’s consider two examples. The first relates to professional growth and vitality of engineering leaders, and the second is an example of putting more ‘real’ engineering into engineering practice.

First, systems engineering will be invigorated if engineering professionals have adequate opportunity to learn and professionally explore topics of interest at all stages of their careers. SEAri believes we can help put the fun back into systems engineering by creating opportunities for meaningful dialogue. We see great value in getting sponsors together to review our research, and this also results in a vital dialogue among the group participants on the issues and motivations for the research. While systems engineering degree programs and training are important, engineering leaders learn the most through direct experience and by sharing these experiences with others. Senior leaders most often engage with their peers in review meetings or when dealing with program issues, rather than having the chance for reflection and exploration of new ideas. This is why we believe that consortia and exchanges in a ‘neutral venue’ are so important, and this was underscored by the enthusiastic interchanges between sponsors that we observed in the summit event.

A second example of putting more ‘fun’ in systems engineering relates to innovations in architecting and design methods. Approaches such as model-based systems engineering are engaging practicing engineers in a more interactive and analytic manner. A new MIT developed methodology, Multi-attribute Tradespace Exploration (MATE), provides engineers with the ability to extensively explore concept tradespaces in a rigorous, yet ‘playful’ manner. The exploration of concepts through this method allows consideration of how combinations of design variables address desired stakeholder value which is elicited during tradespace exploration. This method engages the mind of the engineer and creates an enriched dialogue between decision makers and designers. In our research, we are collaborating with several sponsors to evolve and validate this method on real world problems. We believe that advanced architecting and design methods are necessary to address modern engineering challenges,

and that the limitations of prior methods have inhibited the engineer’s ability to adequately perform good systems engineering. MATE is an innovation that enables engineers to perform extensive analyses and explore ‘what if’ scenarios on complex programs, particularly during the concept phase where such decisions have significant influence on future outcomes and the opportunity to ‘design for change’ can be leveraged.

The desire to put the fun back into systems engineering is shared by many today, and is an attainable goal through advancing the practices and revitalizing the profession. Creating opportunities for meaningful dialogue and innovations in architecting methods are examples of how we can progress toward this goal.

Donna H. Rhodes, Ph.D.

SEAri Community

Leadership Dr. Donna Rhodes, Principal Dr. Adam Ross, V-STARS lead Dr. Ricardo Valerdi, R-STARS lead

Internal Advisory Board Professor Warren Seering, Faculty Co-Lead Professor Daniel Hastings, Aero/Astro, ESD Professor Debbie Nightingale, Aero/Astro, ESD Mr. Pat Hale, SDM Fellows Program Director

Affiliated Faculty Professor Missy Cummings, Aero/Astro Professor Richard de Neufville, Civil E., ESD Professor Olivier de Weck, Aero/Astro, ESD Professor Daniel Frey, Mech E., ESD Professor Daniel Hastings, Aero/Astro, ESD Professor Christopher Magee, Mech E., ESD Professor Debbie Nightingale, Aero/Astro, ESD Professor Daniel Roos, Civil E., ESD Professor Warren Seering, Mech. E., ESD Professor Joseph Sussman, Civil E., ESD Professor Annalisa Weigel, Aero/Astro, ESD

SEAri and Affiliated Graduate Research Assistants David Broniatowski Deb Chattopadhyay Caroline Lamb Tsoline Mikaelian Julia Nickel Matthew Richards Christopher Roberts Nirav Shah Lauren Viscito Jennifer Wilds

Sponsored Research Partners Singapore DSTA US AFOSR NSF/IGERT PoET MIT-Portugal Program LAI Draper Laboratory US Government

Page 3: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 3 of 8

Methodology to Identify Opportunities

for Flexible Design

Research by Jennifer Wilds, S.M. Student in Technology & Policy, Aeronautics & Astronautics

Rapidly changing technologies and undefined requirements for new capabilities necessitate consideration of the uncertainties in developing and defining requirements. Identification and management of the uncertainties can allow decision-makers to create real options to improve the flexibility of the design to accommodate uncertainty. Key challenges in developing changeable systems include understanding where to insert real options to improve flexibility and how to value those real options in a flexible system. This research, attempts to identify potential opportunities for real options within the sociotechnical system to improve the system value in response to uncertainty. The research is based on a collaborative engagement using case studies to test a methodology to identify opportunities for changeability and the potential to exploit results within the Department of Defense Acquisitions process. Bartolomei [1] proposed the following methodology for identifying opportunities for changeability using the Engineering System Matrix (ESM) framework:

Bartolomei [1] developed the ESM in response to the limitations of existing modeling frameworks to sufficiently represent the environmental interactions and influences of time to provide a more holistic representation of the system. Within the ESM methodology, there are six domains—1) environmental or system drivers, 2) social or stakeholders, 3) objectives, 4) functions, 5) physical or objects, and 6) process or activities—that are important to describe the engineering system. The ESM organizes this information using a matrix structure that

can be used to facilitate network and graph theoretic analysis and in identifying system uncertainties. Uncertainty can be quantified in terms of likelihood of occurrence and probable outcomes, and then modeled as a probability distribution with upper and lower limits bounding the potential outcome. Uncertainties provide a measure of how likely something is to change. Then, it is important to ask the question, if “A” does change, how sensitive is the system to the change? This is an example of a defining a change scenario. Suh [2] recommends considering possible change scenarios that are likely to occur given the recognition of the uncertain system parameters. To define change scenarios, consider what changes are likely as a result of the potential outcomes of the uncertainties. Then using the changes scenarios, observe how the change impacts the system. The proposed methodology combines two types of analysis tools to do this: 1) Change Propagation—pertaining to how an induced change flows through the connectivity of the system; and 2) Sensitivity Analysis—pertaining to the magnitudes of change propagating through the system. The results of this analysis are observation of system “hotspots” and “coldspots”. Hotspots indicate elements within the system which are strongly impacted by change. Therefore, decision-makers may consider designing the system to accommodate the possible outcome for the identified uncertainty. Coldspots indicate system elements that are less sensitive to change and/or act as absorbers of change propagation. Clustering the coldspots may indicate a method of platforming system elements. [3] System elements that have a greater degree of change propagation are hotspots, while those that have a low degree of change propagation are coldspots. System elements that are very sensitive to change are defined as hotspots, and those elements that are less sensitive to change are defined as coldspots. System hotspots may also be identified by high switch costs associated with the cost of the change. Although system elements may not significantly change when perturbed or propagate throughout the entire system, high switch costs can make future modifications extremely costly, possibly even cost prohibitive. However, given the uncertainty a change to a high switch cost element may be inevitable. Therefore, it is important to consider the ramifications of these potential hotspots as well.

Research Spotlight 1

Initial Proposed Methodology 1. Construct ESM of a particular system. 2. Identify sources of uncertainty driving change. 3. Define change scenarios. 4. Identify change modes for each scenario (eg change

propagation method [2]). 5. Calculate how change modes affect objectives for each

scenario (eg sensitivity analysis [3]). 6. Calculate the cost of change for each scenario (eg cost

analysis [2]). 7. Identify hotspots/coldspots for each change scenario. 8. Examine hotspots/coldspots across scenarios. 9. Formulate real options from hotspots/coldspots. 10. Value changeability using Real Options Analysis

Page 4: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 4 of 8

Note that sensitivity hotspots may or may not correspond to the same change propagation hotspots. Therefore it is important not to neglect either analysis in considering all potential hotspots and coldspots. A system element may be very sensitive to change, yet limit the propagation of the induced change throughout the overall system. Likewise, a system element may be

insignificantly impacted by a change ∆x, however the small reaction may have significant implications downstream for greater change propagation. Furthermore, the hotspots/coldspots may vary for each change scenario. For example, an operational uncertainty regarding the performance of the system may have a range of desired outcomes, creating a

change in requirement, ∆x1. A technical uncertainty corresponding to the availability of a new technology may have probability distribution associated with the likelihood of use in the system, creating a change in a

system component, ∆x2. Because the CPI is a measure of the degree of change the element itself propagates, related to the connectivity of individual elements, the solution is unique to the change scenario that defines

∆x2. Likewise, the sensitivity of the system to a changing requirement, ∆x1 will not have the same impact (magnitude) due to the relationships that are necessary to achieve the new performance measure. Due to the uniqueness of the solutions to the individual metrics and change scenarios, the hotspots and coldspots must be first assessed individually and then aggregately. Sensitivity hotspots that also correspond to change propagation hotspots are likely system hotspots. The magnitude of the propagated change and the number of system elements affected by the change must be considered to determine the system hotspots. Once the system hotspots and coldspots have been determined, this information can be used to formulate the real options to provide system flexibility to accommodate the changes associated with the identified hotpsots. The real options may include modifications to system elements or substitution of alternative elements to improve flexibility. As hypothesized by Suh (2005), “if a critical subset of elements within the product platform is made flexible, it can make the whole [system] flexible to a specified set of uncertainties.” Each real option should be evaluated to assure that the alternative systems still meets the required (constant) objectives, as well as responds to the identified uncertainties. This set of potential solutions defines the design space. The final step requires knowledge of the base case costs, including the initial capital investment and variable costs. If these cost estimates are undetermined, the analysis can proceed, however the accuracy of the switch costs

is critically important. The initial capital investment cost includes start-up costs for fabrication and assembly lines, required tooling, and developmental costs. The variable cost is the total recurring cost, typically measure per unit produced. Recall that the switch cost is incurred when the system changes due to the uncertainties. Therefore, the cost of the real option can be calculated, which translates to the cost of flexibility. Real Options Analysis (ROA) provides a framework to value options, where value of the option may reflect improvements in one, some, or all of the “ilities” to meet needs. The value of flexibility is determined by implementing ROA methods, including Decision Tree Analysis, Net Present Value, and/or the Lattice Method. The ROA method is selected based on the ability to adhere to the assumptions of the models. A flexible system is measured with respect to a base case, which is defined as the system prior to the improved flexibility. The research outcomes anticipated over the next nine months include a demonstration of the ESM framework for creating flexible in systems and a methodology to assist in deciding where to focus efforts for flexible design. This research will further the use of real options for designing flexible systems that leverage uncertainty. References Bartolomei, J. E. (2007). Qualitative Knowledge Construction

for Engineering Systems: Extending the Design Structure Matrix Methodology in Scope and Procedure. Cambridge, MA. Massachusetts Institute of Technology, Engineering Systems Division. Doctoral Dissertation.

Suh, E.S. (2005). Flexible Product Platforms. Cambridge, MA. Massachusetts Institute of Technology, Engineering Systems Division. Doctoral Dissertation.

Kalligeros, K. (2006). Platforms and Real Options in Large-Scale Engineering Systems. Cambridge, MA. Massachusetts Institute of Technology, Engineering Systems Division. Doctoral Dissertation.

Jennifer Wilds Jennifer Wilds is a graduate student at MIT pursuing a S.M. in Technology and Policy and S.M in Aeronautics and Astronautics. As a member of MIT’s Systems Engineering Advancement Research Initiative (SEAri), Jennifer’s current research focuses on evaluating and valuing real options for changeable

sociotechnical systems, supported by Synexxus Corporation and US Special Operations Command. She holds a B.S in Aerospace Engineering (2002) from Texas A&M University at College Station, Texas and is a civil servant for the US Air Force Research Laboratory, Munitions Directorate at Eglin Air Force Base, Florida.

Page 5: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 5 of 8

Optimism Bias in Software Schedule

Estimation

Research by Ricardo Valerdi, Ph.D., Research Associate in SEAri, R-STARS Team Lead

Introduction Optimism, the opposite of pessimism, is the perspective that people and events are generally good and that things will work out favorably in the end. It comes from the Latin word optimus meaning “best”, and hinges on the dependence of internal conditions of a person rather than external good fortune. In the context of project management, it is likely beneficial to be an optimist since many negative events could damage an individual’s motivation. In the end, optimism can be easily reaffirmed as long as the products being developed bring value to critical stakeholders. The case in which optimism can be damaging is when it biases the planning of project budgets and schedules. Referred to as optimism bias, it is observed as the systematic tendency for people to be over-optimistic about the outcome of planned actions. The role of this bias has been traced to two cognitive biases: anchoring and competitor neglect (Lovallo and Kahneman 2003). Anchoring is the tendency to rely too heavily, or “anchor”, on one trait or piece of information and competitor neglect is the tendency to focus on a company’s capabilities and plans and thus neglect the potential abilities and actions of rivals. Optimism Bias in Large Projects A study of 258 transportation infrastructure projects worth US$90 billion and representing different project types, geographical regions, and historical periods, found with overwhelming significance that the cost estimates used to decide whether such projects should be built were highly and systematically misleading (Flyvbjerg, Holm et al. 2002). This study cited economic, psychological, political reasons that led to optimism bias in situations such as the Panama Canal, Sydney Opera House, and the Eurotunnel; finding that costs are underestimated in 9 out of 10 transportation infrastructure projects. Federal governments have also identified optimism bias as a major concern. The UK government explicitly acknowledges that it is a problem in government budgeting (HM-Treasury 2003). In addition, The Treasury, the UK’s economics and finance ministry, in conjunction with the Department of Transport developed

a guidance document to help organizations deal with optimism bias in transport planning (BDoT 2004). The U.S. General Accountability Office has published studies on the unrealistic cost estimates which have led to unrealistic budgets and unexecutable programs. One study found that the space acquisition system is strongly biased to produce unrealistically low cost estimates throughout the process (GAO 2006). During program formulation, advocacy tends to dominate and a strong motivation exists to minimize program cost estimates. Moreover, proposals from competing contractors typically reflect the minimum program content and a “price to win.” Putting a high priority on affordability makes it important for program sponsors to provide cost estimates that will fit within the funding constraints. Optimistic cost estimates are encouraged because they help gain program approval and attract budgetary resources. The consequences of cost growth are not directly felt by an individual program because they are accommodated through delivery delays and quantity changes and by spreading the cost impact across many programs. Example: Software Schedule Estimation With Parametric Models Schedule pressure is often faced by project managers and software developers who want to quickly deploy software systems. Typical strategies to compress project time scales might include adding more staff/personnel, investing in development tools, improving hardware, or improving development methods. This makes the tradeoff between cost, schedule, and performance one of the most important analyses performed during the planning stages of software development projects. Since cost models are deterministic, users can easily manipulate the inputs to produce the desired output that fits within a desired range. This practice, however, overlooks many important risks that can set the project up for failure since unrealistic estimates can lead to expectations that are impossible to satisfy. General Approach to Estimating Schedule Compression in Cost Estimation Models A project’s estimated schedule significantly influences its success. If the schedule is under estimated, planning inefficiencies are introduced to the project. Invariably this can lead to delays in the project and increase the actual schedule. If the schedule is over estimated, Parkinson's Law can come into effect. The law claims that work expands to fill the time available for its completion. Allowing for extra time may also endanger

Research Spotlight 2

Page 6: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 6 of 8

the project with unexpected functions and unnecessary gold plating, potentially leading to increased schedules.

Figure 1. SEER-SEM Schedule Effects The effect of schedule compression and expansion can be evaluated using parametric cost models. One example is the SEER-SEM model which considers any schedule that is less than the optimal schedule impossible (SEER-SEM 2002) as shown. On the other hand, the optimal effort is considered to be too long for the planned project. The ideal estimate falls between these two extremes but allows for tradeoffs that fall within the region of realistic projects. Conclusion Possible solutions to optimism bias should be considered since highly accurate cost models are not immune from manipulation by users to obtain the desired answer. One approach is to take an outside view when performing cost estimates (Kahneman and Tversky 1979). This requires users to consider their estimate as part of a distribution of estimates and assess the reliability of their estimate in the context of a probability distribution. Another approach is to introduce fiscal incentives against cost overruns through requiring local co-financing of project cost escalation where possible (BDoT 2004). Finally, eliminating the motivation for quick prediction may also help reduce or eliminate this bias. Whichever approach is employed, the intent should be to promote unbiased practices so that reliable estimates can be produced.

References BDoT (2004). Procedures for Dealing with Optimism

Bias in Transport Planning - Guidance Document, The British Department for Transport.

Flyvbjerg, B., M. S. Holm, et al. (2002). "Underestimating Costs in Public Works Projects: Error or Lie?" Journal of the American Planning Association 68(3): 279-295.

GAO (2006). Space Acquisitions: DoD Needs to Take More Action to Address Unrealistic Initial Cost Estimates of Space Systems, GAO Report to Subcommittee on Strategic Forces, Committee on Armed Services, House of Representatives.

HM-Treasury (2003). The Green Book: Appraisal and Evaluation in Central Government. Stationery Office, Great Britain H.M. Treasury.

Kahneman, D. and A. Tversky (1979). "Intuitive Predictions: Biases and Corrective Procedures." TIMS Studies in Management Science 12.

Lovallo, D. and D. Kahneman (2003). "Delusions of Success: How Optimism Undermines Executives' Decisions." Harvard Business Review 81(7): 56-63.

SEER-SEM (December 2002). SEER-SEM Manual, version 6.0.28, http://www.galorath.com.

Note: For interested readers, a longer version of this article is available upon request to [email protected]

Ricardo Valerdi Dr. Ricardo Valerdi is a Research Associate at the Lean Advancement Initiative (LAI) and the Systems Engineering Advancement Research Initiative (SEAri) at MIT and a Visiting Associate at the Center for Systems and Software Engineering (CSSE) at USC. He earned his BS in Electrical Engineering from the University of

San Diego, MS and PhD in Industrial and Systems Engineering from USC. He was a Senior Member of the Technical Staff at the Aerospace Corporation in the Economic & Market Analysis Center. Previously, he worked as a Systems Engineer at Motorola and at General Instrument Corporation. He is on the Board of Directors of INCOSE.

Page 7: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 7 of 8

Pat Hale to Take Office as INCOSE

President in January

Mr. Hale, member of SEAri’s internal advisory board and Director of MIT’s System Design and Management Fellows Program (SDM), is currently INCOSE President-Elect and will become President in January 2008. Prior to joining MIT, he was Director of Systems Engineering at both Draper Laboratory and Otis Elevator Company. SDM educates technically grounded professionals in using systems thinking to tackle the complexities of competing in global markets while leading their organizations to success. SEAri is the intellectual home for SDM students interested in pursuing thesis research on advanced systems engineering topics.

INCOSE Doctoral Student Network

Event to be held in April

The International Council on Systems Engineering (INCOSE) doctoral student network, SEANET, will hold its 3

rd annual workshop in conjunction with the upcoming

Conference on Systems Engineering Research (CSER). The SEANET event will be held on USC campus in Los Angeles on April 3, 2008. The workshop is open to doctoral students (or soon to be) from universities around the world who are researching systems related topics. The workshop format, similar to the prior two workshops, will feature speakers and presentations, and will include birds-of-a-feather sessions on topics such as publication strategies, research methods, converging on a dissertation topic, and mixed methods research. In the evening, a reception featuring student research posters will be held at the CSER Conference hotel as an opportunity for doctoral students to showcase their research. Reception attendees will include incoming attendees for the CSER conference, and many university faculty who are members of the Council on Engineering Systems Universities (CESUN, http://www.cesun.org). CESUN will be holding a meeting concurrent with the SEANET workshop. The INCOSE Journal of Systems Engineering Volume 10, Issue 4, includes an article on the SEANET entitled “Enabling Research Synergies through a Doctoral Research Network for Systems Engineering” authored by SEANET leaders Dr. Donna Rhodes and Dr. Ricardo Valerdi. The call for participation in the event can be found at http://incose-la.org/cser2008. For more information, contact [email protected]

SEAri to Teach Course in MIT

Professional Institute in June 2008

The MIT Professional Institute, founded in 1949 as the MIT Summer Institute, brings more than 600 participants to campus each year. Its courses are “designed to allow participants to absorb working knowledge fast”. This year the program will include a four day course Value-Driven Tradespace Exploration for System Design, taught by SEAri researchers Dr. Adam Ross, Dr. Donna Rhodes and Dr. Hugh McManus. The course presents a new method for tradespace exploration based on a value driven perspective, allowing designers to better understand and meet both present and future stakeholder needs and expectations. The Multi-Attribute Tradespace Exploration (MATE) methodology was developed at MIT for exploring tradespaces of possible architectures rather than settling quickly on an optimum. The power of the method comes primarily from the ability to quantitatively assess many design choices very early in the design process, along with gaining insight into how stakeholder preferences relate to technically feasible design alternatives. This ability allows system architects, designers and analysts to explore many design options, comparing many system designs on a common basis, to ensure “right” decisions are made at the beginning of system development. This capability enables quantitative assessment of factors such as variability in technical performance and cost, how they relate to stakeholder expectations, and the impact of changes in markets or government policy on the potential for system success by allowing exploration of a large number of possible situations, including speculative (“what if”) scenarios. The course will cover several advanced topics including system design for selected “ilities” (flexibility, adaptability, survivability, scalability, modifiability, and robustness) into the architecture; a systematic method for handling many types of uncertainties (requirements, technical, economic etc.); and integration with policy and product development issues. Strategic issues such as temporal considerations in tradespace exploration, use of real option approaches, and exploring complex tradespaces of families of designs and systems of systems will also be discussed. The course is scheduled for June 9-12; more information can be found at: http://web.mit.edu/mitpep.

Crosstalk Article Garners SE Interest

SEAri researchers Ricardo Valerdi, Adam Ross, and Donna Rhodes published an article entitled “A Framework for Evolving System of Systems Engineering” in Crosstalk—The Journal of Defense Software Engineering, October 2007, which has garnered much interest in the Systems Engineering community. A link to the article is on the SEAri web site: http://seari.mit.edu.

SEAri News

Page 8: Systems Engineering Advancement Research Initiativeseari.mit.edu/documents/SEAri-Research-Bulletin-v2n3.pdf · A research summit provides an opportunity for sharing knowledge and

SEAri Research Bulletin Volume 2, Issue 3 December 2007 http://seari.mit.edu © 2007 Massachusetts Institute of Technology Page 8 of 8

Spring 2008

3: SEANET Event, Los Angeles, CA

4-5: Conference on SE Research, Los Angeles, CA

7-10: IEEE Systems Conference, Montreal, Canada

28-1: Responsive Space 6, Los Angeles, CA

April

March

February

27-30: INCOSE Int’l Workshop, Albuquerque, NMJanuary

3: SEANET Event, Los Angeles, CA

4-5: Conference on SE Research, Los Angeles, CA

7-10: IEEE Systems Conference, Montreal, Canada

28-1: Responsive Space 6, Los Angeles, CA

April

March

February

27-30: INCOSE Int’l Workshop, Albuquerque, NMJanuary

New Cost Estimation Course

This Fall, Dr. Ricardo Valerdi taught a new MIT course called Cost Estimation & Measurement Systems. Material was divided into five major sections: cost estimation fundamentals, parametric model development/calibration, economic principles, measurement systems, and government/policy issues. Guest speakers included Capers Jones (Software Productivity Research), Paul Garvey (MITRE), and David Seaver (PRICE Systems), who shared their insights. Student final projects ranged from a description of a cost model for a healthcare deployment system to an analysis of the cost estimation processes used on the F/A-18 fighter jet.

SEAri Submits Over a Dozen Papers to

Upcoming Spring Conferences

The 6th Annual Conference on Systems Engineering (CSER) will be held 4-5 April 2008 in Los Angeles; the 2

nd Annual IEEE Systems Conference will be held 7-10

April 2008 in Montreal, Canada; and the 6th Responsive

Space Conference will be held 28 April – 1 May 2008 in Los Angeles. SEAri students and staff have submitted over a dozen papers to these events. This year, CSER 2008 will feature among the plenary speakers Dr. Donna Rhodes, SEAri Director, and Mr. Pat Hale, INCOSE President and member of SEAri’s MIT Advisory Board.

CSER 2008: [1] Broniatowski, D.A., Magee, C.L., Coughlin, J.F., and Yang, M.C., The Influence of Institutional Bias on System Review and Approval. [2] Chattopadhyay, D., Ross, A.M., and Rhodes, D.H., A Framework for Tradespace Exploration of Systems of Systems. [3] Lamb, C.T., Rhodes, D.H., and Nightingale, D.J., Evolving Systems Engineering for the Realization of Effective Enterprises.

[4] Mikaelian, T., Rhodes, D.H., Nightingale, D.J., and Hastings, D.E., Managing Uncertainty in Socio-Technical Enterprises Using a Real Options Framework. [5] Nickel, J., Ross, A.M., and Rhodes, D.H., Cross Domain Comparison of Design Factors in System Design and Analysis of Space and Transportation Systems. [6] Richards, M.G., Hastings, D.E., Rhodes, D.H., and Weigel, A.L., Systems Architecting for Survivability: A Review of Current Methods and Implications for Space Architecture. [7] Roberts, C.J., Nightingale, D.J., and Rhodes, D.H., Design for Harmony: Methods for Complementary Enterprise and System Design. [8] Shah, N.B., Viscito, L., Wilds, J.M., Ross, A.M., and Hastings, D.E., Quantifying Flexibility for Architecting Changeable Systems.

IEEE 2008: [1] Lamb, C.T., and Rhodes, D.H., Systems Thinking as an Emergent Team Property: Ongoing Research into the Enablers and Barriers to Team-level Systems Thinking. [2] Rhodes, D.H., Lamb, C.T., and Nightingale, D.J., Empirical Research on Systems Thinking and Practice in the Engineering Enterprise. [3] Richards, M.G., Ross, A.M., Hastings, D.E., and Rhodes, D.H., Empirical Validation of Design Principles for Survivable System Architecture. [4] Ross, A.M., and Rhodes, D.H., Using Attribute Classes to Uncover Latent Value during Conceptual Systems Design. [5] Ross, A.M., and Rhodes, D.H., Architecting Systems for Value Robustness: Research Motivations and Progress.

RS6: [1] Richards, M.G., Viscito, L., Ross, A.M., and Hastings, D.E., Distinguishing Attributes for the Operationally Responsive Space Paradigm.

Voice of the Expert -- What Do You

Think?

SEAri is continuously shaping our research agenda and seeks input from experts from the systems community.

The following are examples of some potential research questions that SEAri researchers have proposed.

What do you think?

1. How does the issue of equity arise when negotiating the distribution of costs and benefits in system design, development, and operation?

2. How can a System of System architect compensate for loss of “control” in order to ensure SoS capabilities over time?

3. What changes to the profession and practice can put the “fun” back in Systems Engineering?

We encourage your feedback on these or other research questions of interest that fall within the research scope of SEAri. (Please email [email protected] with your thoughts.)

Upcoming Events & Activities

Member Input Request