Measuring Complexity of

Aerospace Systems

Shashank Tamaskar1 Kartavya Neema

2 Tatsuya Kotegawa

3 Daniel

Delaurentis4

School of Aeronautics and Astronautics

Purdue University, West Lafayette

stamaska@purdue.edu

In this paper, a framework for measuring system complexity is proposed, marrying structural

and functional representations of the system and addressing cross-domain interactions. We

identify seven different aspects of complexity in aerospace systems, namely, levels of

abstraction, type of representation, size, heterogeneity of components and interactions, network

topology factors such as coupling, modes of operation and off-design interactions. A system is

decomposed into different levels of abstraction and within each level the system is represented

using structural (network of components and interactions) and functional graphs (network of

functions). Networks provide a natural framework for system representation and facilitate use

of topological metrics such as size, coupling and modularity. Synthetic examples are developed

to study how the different aspects of network complexity aggregate and the relationship

between structural and functional graphs. The authors investigate the relationship between

different levels of abstraction. We also study how cross-domain interactions, modes of

operation and off-design interactions affect complexity. We also generate some preliminary

results to show the relevance of the proposed framework by applying it to a spacecraft design

problem and highlight some of its advantages and disadvantages.

1 Introduction

Over the past several decades, many industries have seen a significant improvement in the performance and capability of their products. One such example is the fifth generation F-35 tactical fighter which offers roughly three to eight times the operational capability of a fourth generation aircraft such as the F-16 or F-18. This superior capability comes at the cost of a substantial increase in technical complexity. For example, the F-35 has roughly 130 subsystems, ~10

5 interfaces and 90% of its

903

functions managed by software. In comparison, the F-16 has 15 subsystems, ~103

interfaces and 40% of its function managed by software (Arena, 2008). Automotive and electronics industry reflect similar trend as well (Eremenko, 2009). The increase in technical complexity has resulted in greater time being spent on design, verification and validation of these systems. Another important consequence of increasing complexity is the appearance of emergent behaviors, which are not apparent during the design and development phase but become visible during system validation or operation. These potentially undesirable behaviors, can lead to expensive redesign and result in cost and schedule overruns (Eremenko, 2009). An analysis conducted by the US Government Accountability Office (GAO) of major defense acquisition program found that in the recent years, the research and development costs are on average 42 % higher than what was originally estimated and the average time required in delivering the initial capability has increased by 22 months (Assessments of Selected Weapon Programs, March 2009). Thus, complexity not only leads to higher cost and effort for a system but also decreases our ability to predict its development cost and schedule accurately.

A brief literature survey reveals that numerous approaches for measuring

complexity have been proposed. These techniques address one or more aspects of this

multifaceted problem and can be classified into four different categories-

1. Metrics based on software complexity metrics

2. Metrics based on information entropy

3. Network theoretic metrics

4. Traditional/ Empirical metrics

Many metrics to measure the complexity of software architectures are available

from the realm of computer science. Software architectures define the flow of

information and control between the different modules of the software. Flow of

materials in the field of manufacturing is similar to the flow of information within the

software. Consequently, several metrics, which measure the complexity of

manufacturing plans, are derived from the field of computer science (Phukan, 2005).

Further, some software complexity metrics are used for measuring complexity of

manufacturing processes (Ahn, 1994). Models of many manufacturing processes

closely resemble the graphical representations of software (directed flow-graph). The

software complexity metrics employed include cyclomatic, nesting level, program

complexity, and cyclomatic flow. Cyclomatic complexity measures the difference

between the number of the program statements (design tasks) and the number of the

control flows (connections between the design tasks) (Oviendo, 1980). Nesting level

complexity focuses on the depth (distance) the design actor is from the final solution

and is found by summing the scope of each of the design actors, providing an insight

into the depth to which each actor has an influence in the entire process network

(Harrison, 1981). Program complexity measures the coupling between the design

actors with respect to the number of times that a declaration is used as an input or

definition for another actor, thus measuring the functional relations between the data

elements involved in the solution process (Stetter, 1984). The cyclomatic flow

904

complexity measures the complexity of the program or process by combining the

control flow and data relations into a single metric.

The information axiom of Suh’s axiomatic design process states that a good

design is one, which has minimal information content (functional requirements and

design parameters). In this context, complexity is defined as the measure that relates

the desired objective against what is known and unknown about the system (Suh,

2001). This forms the basis of application of information-based approaches for

measuring the complexity of the design. While some of the methods suggest that

complexity is fundamentally a characteristic of the information content within a

design, others try to incorporate the designer’s lack of understanding while measuring

the complexity of system (Braha, 1996). Haik and Yang further extend the idea of

information entropy as a measure of system complexity to highlight some of the

components of system complexity of design process such as variability, vulnerability

and correlation (Yang, 1999). Braha and Maimon describe the measurement of

structural and functional aspects of design complexity (Maimon, 1998). Gell-Mann

and Lloyd provide information measures for measuring the effective complexity and

total information content of the system (Lloyd, 1996). Hornby adopts a similar

approach to measure modularity, reuse and hierarchy within the system (Hornby,

2007). While the information-based methods are good for measuring the size aspect

of system complexity, they do not provide an intuitive understanding of the

complexity due the topology of interactions.

Network theoretic approaches are useful for measuring coupling aspect of system

complexity. In these techniques, we represent a complex system as a network of

nodes and links with nodes representing the components and links representing

interactions between the components. Complexity values are assigned to nodes and

links and a roll-up operation is performed which generates a single number

representing the complexity of the entire network. Different authors adopt different

approaches for the roll-up operation. Most recent among this effort is by Zeidner,

Reeve and Khire from United Technologies Research Center (UTRC) who describe a

metric to capture the size, heterogeneity and connectivity aspects of system

complexity. UTRC has also proposed different approaches for roll-up operation based

on graph energy and spectral graph partitioning (Abstraction based complexity

management, 2010). Barabasi and Ravasz have used the network approach to

describe the characteristics of networks having hierarchy and modularity (Barabasi,

2003). DeLaurentis and Ayyalasomayajula (DeLaurentis, 2009)

looked at

complexity's role in system-of-systems engineering for sustainable air transportation,

using network topologies as a source of practical information for complexity metrics,

yet a generic metric was not put forward. One of the shortcomings of the network

theoretic metrics is the lack of a consistent scheme for system representation.

Apart from these more formal developments, many traditional design examples

exist where the complexity metrics is qualitatively and subjectively estimated. These

metrics typically measure the coupling between the performance parameters and

design variables. The most notable in this regard is the work of Bearden (Bearden,

2003). This metric is based on empirical data collected from small satellites

developed between 1990 and 1999. The approach used by the author is the following:

905

Identify the parameters that drive or otherwise contribute to a spacecraft design,

quantify the identified parameters; and combine the parameters into an aggregate

complexity index. While the author has confined himself to space domain the same

basic approach can be applied to other domains as well. A limitation of the

classification is its failure to address the design process, which might include the

number, type, and repeatability of tasks. Furthermore, the complexity of the

individual tasks (process elements) was not addressed. In addition, this might not

work for designs that are radically different from conventional designs.

The literature survey reveals the lack of a comprehensive metric, which measures

the complexity of a design. Recently there has been some effort to unify these

numerous complexity metrics. The efforts of Shah and Summers (Jami J. Shah, 2010)

are notable in this regard. The authors have proposed three measures of design

complexity: solvability, size and coupling. The authors suggest that these measures

capture different aspects of design and hence should be kept independent of each

other. Metrics are developed to capture problem, process and product aspects of the

design. However, the analysis assumes a binary un-weighted network for calculating

coupling and assumes the independence of size, coupling and solvability.

This paper represents another small step in unifying different aspects of

complexity. Seven different aspects of complexity in aerospace systems are identified

namely, levels of abstraction, type of representation, size, heterogeneity of

components and interactions, network topology factors such as coupling, modularity,

modes of operation and off-design interactions. A framework for measuring system

complexity that accounts for these aspects is proposed. Some of the unique aspects

about this framework are its focus on marriage of structural and functional

representations of the system and a unified treatment of the cross-domain

interactions. A system is decomposed into different levels of abstraction and within

each level; the system is represented using structural (network of components and

interactions) and functional graphs (network of functions). Networks provide a

natural framework for system representation and facilitate use of network topology

metrics to capture size and coupling aspects of complexity. Synthetic examples are

developed to study how the different aspects of network complexity aggregate and

the relationship between structural and functional graphs. The authors investigate the

relationship between different levels of abstraction and study how cross-domain

interactions, modes of operation and off-design interactions affect complexity. We

also generate some preliminary results to show the relevance of the proposed

framework by applying it to a spacecraft design problem and highlight some of its

advantages and disadvantages.

2 Approach

2.1 Different Aspects of System Complexity

Based on the literature survey described in the previous section, it can be concluded

that some of the important aspects that govern system complexity, are-

Level of Abstraction

System Representation: Structure/Function

Size which is indicative of number of components and interactions

906

Heterogeneity of components and interactions

Network Topology

Dynamics involving different modes of operation

Off-design interactions: Off-design interactions are the interactions, which

occur due the operation of components, and interactions outside their design

range. They are different from un-modeled interactions (interactions that

cannot be modeled) which occur due to emergent behavior within the

system.

Figure 1: Different aspects of System Complexity

Figure 1 is a pictorial representation of the different aspects of system complexity,

which are considered in the proposed framework.

2.2 Proposed framework for measuring system complexity

Based on consideration of the aforementioned aspects of system complexity our

framework is described below-

1. Representing the system at appropriate level of abstraction: The choice of the

level of abstraction depends on the fidelity of analysis specified by the designer.

In the component level of abstraction, the functions in functional representation

of the system are assumed to have unit complexity. Complexity of components

and interactions at the higher level of abstraction are a result of aggregation of

complexity at the lower level of abstraction. Complexity responsible for cost

and schedule overruns of modern aerospace system is attributed to the complex

interactions between the simple components and functions, which give rise to

emergent behaviors.

2. Generating the structural and functional system representation: At each level of

abstraction, the system is represented via structural and functional graphs. A

CAD model would likely represent the lowest level of structural representation.

907

The functional graph represents the relationship between the system functions.

Fig. 2 describes the structural and functional graphs for an aircraft at the

subsystem level of abstraction.

3. Mapping the functional representation onto the structural graph: In this step,

each function within the functional graph is mapped to components and

interactions in the structural graph. The complexity of components and

interactions in the structural graph is defined by the number and complexity of

functions associated to them. In case, groups of components perform a function,

the complexity associated with this function is divided uniformly between the

components. In a similar fashion, the complexity associated with off-design

interactions is also mapped on to the structural graph. The result of this

mapping is a weighted network where the nodes represent the components and

the links represent the interactions within the system.

Figure 2: Marriage of Structural and Functional Representations for an aircraft represented in

the subsystem level of abstraction

908

4. Calculate the complexity of the weighted network: After generating the

weighted network in step 3, system can be analyzed using a variety of metrics,

which capture different aspects of complexity of networks. Some of the

aspects that are captured in our framework are size and coupling. Coupling

complexity is the product of the topology of interactions within the system.

Summers and Shah (Jami J. Shah, 2010) have proposed an algorithm for

measuring coupling by testing the decomposability of an entity relation graph.

One of the disadvantages of their approach is that it assumes a binary,

undirected graph for calculating coupling, which is a restrictive assumption for

analysis of engineering systems. In addition, it ignores the presence of

feedback loops or cycles, which play an important role in determining the

system complexity. Our proposed approach for measuring coupling

complexity accounts for directivity and weights within a network. It also tests

for the presence of feedback looks within the system. Another key feature of

the proposed metric for coupling is its ability to measure coupling between

nodes, which are not directly connected. Sometimes, design changes in one

part of the system often affect other parts of the system not directly connected

with it. Our proposed metric apart from accounting for direct connectivity

between the components also accounts for all the indirect interactions, which

are associated with a particular link. The metric for coupling is as follows:-

1. Represent the system using directed and weighted structural graph of the

system.

2. To capture coupling between indirectly connected nodes, we propose to

calculate how frequently each links of the network are used while finding

all the paths to connects all the pairs of the nodes in the network. Any

indirectly connected node of the network can only interact using the paths

laid between them. This step will ensure higher weightage to links that are

used larger number of times and thus capture coupling between indirectly

connected nodes.

3. Define refined weights of the system by multiplying the weights of the

network by the frequency of each link calculated in step 2.

4. Calculate coupling complexity using the following formula.

𝐶𝑜𝑢𝑝𝑙𝑖𝑛𝑔 𝐶𝑜𝑚𝑝𝑙𝑒𝑥𝑖𝑡𝑦 = 𝑗 𝑊𝑖 + 𝑊𝑘𝑚𝑘=1

𝑛𝑖=1

𝑐𝑗=1 (1)

Where: c and j denotes the number of cycle and size of the cycle

respectively. Wi denotes the weights of the links of the cycle. m is the

number of links which are not the part of any cycle. Wk denotes the

weights of those links that are not the part of any cycle.

Structural graph of an artificial network is shown in figure 3 (a). We are trying to

calculate the coupling complexity of the network. Therefore, we calculate the

frequency of each links being used when finding all the paths to connects all the

pair of nodes. Frequencies of the links are shown in red color in figure 3 (b).

909

Using the equation given in step 4, the coupling complexity of the network

comes out to be 171.

Figure 3: (a) Artificial Network (b) Frequency of the links

Size is an excellent measure for providing a rough measure for system

complexity. A metric for measuring size complexity is described in Shah and

Summers (Shah, February 2010). It is based on the information content of the

system, modeled in a particular representation. The information content of a

system is related to size of the vocabulary (λ) necessary to describe the system

and the number of language instances (Λ). The number of unique components

and relationships is a measure of the size of the vocabulary of the system, while

the total number of components and interactions gives the number of language

instances. The complexity of the system is defined as-

C=Λln (λ) (2)

However, since the metric for coupling proposed above already accounts for number and heterogeneity of interactions hence these are omitted from (2) to avoid double counting.

5. Aggregate different aspects of complexity into a single measure for system

complexity: After computing the size and coupling of the weighted network, they

are combined to produce a single number, which represents an overall

complexity score for the system. For the sake of simplicity, it is assumed that the

overall system complexity is a weighted sum of size and coupling aspects. An

important aspect to note is that aggregating size and coupling aspects into a

single numeric score is important as it facilitates easy comparison between two

designs. This aggregation however, can lead to loss of valuable insights about the

sources of complexity in a design. We are investigating a potential approach for

developing a heat map, which can be used to identify the regions of high and low

complexity in a design.

3 Analysis

In this section, some examples, which illustrate some of the aspects of the framework

mentioned in the previous section, are described. Fig. 3 shows synthetic examples,

(a) (b)

910

which have been created to demonstrate the effectiveness of the size and coupling

metrics. For the ease of demonstration, nodes and link are assumed to be of unit

complexity.

Figure 3: Synthetic Examples

Table 1 describes the complexity scores for each of the networks shown in Fig. 3.

Increasing the number of nodes will result in increase in size and total complexity.

Another important observation is that as the networks with feedback have

significantly higher coupling than those without feedback. In addition, complexity

significantly increases with increase in number of components within a feedback

cycle. Thus, network B has significantly higher coupling than networks E or F. This

is expected, as increasing the number of components in a feedback path makes it

increasingly difficult to accurately predict the behavior of the system.

Table 1: Calculating complexity of synthetic examples

Networks Size Complexity Coupling Complexity Overall Complexity

A 8.04 4 12.04

B 13.62 1029 1042.62

C 13.62 56 69.62

D 13.62 64 77.62

E 13.62 150 163.52

F 13.62 208 221.62

While the synthetic examples demonstrate the reasonableness of the metrics and the

proposed scheme for the aggregation of different aspects of complexity, a real

aerospace design example is required to demonstrate some of the merits and demerits

of the proposed framework. Fig. 4, 5 shows a conceptual sketch of two satellites

which have been used to illustrate the framework. As described in the previous

section, any complexity analysis begins with choosing an appropriate level of

911

abstraction. For this problem, we have chosen to analyze the system in the component

level of abstraction. The choice of component level of abstraction is dictated by the

fact that both satellites look almost the same at subsystem level of abstraction. Fig. 6

shows the unified system representation for satellite B, which is obtained after

mapping the functional graph on the structural graph. Since the analysis is carried out

at the component level, hence the weights of the complexity of the components and

interactions can be assumed to be one.

Figure 4: Satellite A

Figure 5: Satellite B

912

Fig. 6 shows that our analysis considers four different types of interactions, namely-

matter, energy, force and information. For sake of uniformity, all these interactions

have been assumed to be of unit complexity.

Figure 6: Unified system representation for satellite B (component level)

Table 2: Complexity of the Satellite Example

Satellite Size Complexity Coupling Complexity Overall Complexity

A 184.2 349 533.2

B 212.0 584 796.1

Table 2 shows the complexity calculations for satellites A and B. From this

analysis, it can be seen that satellite B is more complex because not only it has more

components, but also because it has significantly higher coupling than satellite A.

Table 2 also describes complexity of individual modules. This analysis is important

as it helps us identify the distribution of complexity across different subsystems.

For large networks, which are observed in most modern aerospace systems, it is

difficult to draw inferences by visualizing the interactions between the components.

By color coding the components as shown in fig. 6, it can be easily observed that the

system has a hub and spoke architecture for data handling and storage. All the

components act as sources or sinks of information.

913

3 Conclusions

In this paper, a comprehensive framework for measuring system complexity is

proposed. Some of the salient features of the proposed framework are:

Greater components/interactions should imply more complexity

Heterogeneity of components/interactions should result in greater system

complexity

Network topology, which is a measure of coupling within a system, should

be related to the complexity of the system

Complexity of a system should depend on the level of abstraction

The metric should account for coupling between different subsystems

The metric should account for un-modeled interactions happening within the

system

One of the unique aspects of this framework is its intuitive mapping of functional

graph on the structural to generate a unified system representation. This weighted

network is then analyzed to evaluate the size and coupling aspects of complexity,

which are aggregated to calculate the overall complexity score. Synthetic examples

are described to demonstrate the effectiveness of coupling and size metrics and a

satellite example is presented to help us evaluate the merits and demerits of the

framework. Future work will focus on incorporating the effect of uncertainty in the

framework.

Bibliography

[1] Abstraction based complexity management. (2010).

[2] Ahn, J. a. (1994). Complexity Analysis of Computational Engineering. 68.

[3] Arena, M. V. (2008). Why Has the Cost of Fixed Wing Aircraft Risen. RAND

Corporation.

[4] Assessments of Selected Weapon Programs. United States Government

Accountability Office. (March 2009).

[5] Balazs, M. a. (2002). Design Simplification by Analogical Reasoning. From

Knowledge Intensive CAD to Knowledge Intensive Engineering (pp. 29-44). U.

Cugini and M. Wozny, eds., Kluwer.

[6] Bashir, H. a. (2001). Models for Estimating Design Effort and Time. Des. Stud ,

Volume 22 issue 2 pages 141-155.

[7] Bearden, D. A. (2003). A complexity based risk assessment of low-cost planetary

missions: when is the mission too fast and too cheap. 52 (2-6).

[8] Braha, D. a. (1996). On the Complexity of the Design Synthesis Problem. 28.

[9] DeLaurentis, D. A. (2009). Exploring the Synergy between Industrial Ecology

and System-of-Systems to Understand Complexity: A Case Study in Air

Transportation. 13 (2).

[10] Dixon, J. D. (1988). A Proposed Taxonomy of Mechanical Design Problems.

Computers in Engineering Conference, (pp. 41-46). San Francisco, CA.

[11] El-Haik, B. a. (1999). The Components of Complexity in Engineering Design.

IIE Trans , 925-934.

914

[12] Eremenko, P. (2009). META, Novel Methods for Design & Verification of

Complex Systems. DARPA.

[13] Fitzhorn, P. (1994). Engineering Design as a Computable Function. Artif. Intell.

Eng. Des. Anal. Manuf , Volume 8 page 35-44.

[14] Harrison, W. a. (1981). A Complexity Measure Based on Nesting Level. 16.

[15] Hornby, G. S. (2007). Modularity, reuse, and hierarchy: Measuring complexity

by measuring structure and organization. 13 (2).

[16] Jami J. Shah, J. D. (2010). Mechniacal Engineering Design Complexity: Size,

Coupling and Solvability. 132.

[17] Kolmogorov, A. (1983). Combinatorial Foundations of Information Theory and

the Calculus of Probabilities. Russ. Math. Surveys, , 29-40.

[18] Lloyd, M. G.-M. (1996). Information Measures, Effective Complexity and Total

Information. 2 (1).

[19] Maimon, D. B. (1998). The Measurement of a Design Structural and Functional

Complexity. 28 (4).

[20] Oviendo, E. (1980). Control Flow, Data Flow and Program Complexity.

Chicago,IL: Proceedings of the COMPSAC80.

[21] Pahl, G. B. (2007). Engineering Design: A Systematic Approach. London:

Springer-Verlag.

[22] Phukan, A. (2005). Complexity metrics for manufacturing control architectures

based on software and information flow. 49 (1).

[23] Sedgewick, R. (1998). Algorithms in C++. Menlo Park, CA: Addison-Wesley,.

[24] Simon, H. (1998). The Sciences of the Artificial. Cambridge, MA: MIT Press.

[25] Stetter, F. (1984). A Measure of Program Complexity. 9.

[26] Suh, N. (2001). Axiomatic Design: Advances and Applications. NewYork, NY:

Oxford University Press.

[27] Varma, D. a. (1990). On the Estimation of Logic Complexity for Design

Automation Applications. International Conference on Computer Design: VLSI in

Computers and Processors. Cambridge,MA: IEEE.

[28] Yang, B. E.-H. (1999). The components of complexity in engineering design. 31

(925-934).

915