Complex Systems Modeling and Networks Summary of NECSI Course CX202 11 February 2009 John M....
-
Upload
elijah-bradford -
Category
Documents
-
view
216 -
download
0
Transcript of Complex Systems Modeling and Networks Summary of NECSI Course CX202 11 February 2009 John M....
Complex Systems Modeling and Networks
Summary of NECSI Course CX202
11 February 2009
John M. Linebarger, PhDSandia National Laboratories
Interactive Systems Simulation & Analysis (06344)
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
Outline
• Overview• What are Complex
Systems?• Complex Networks
– Metrics– Types of Networks
• Modeling and Simulation of Complex Systems– How to Model– Types of Models
• Further Reading• Class Project• Overview of Design for
Tractable Analysis (DTA) framework
Overview
• I attended a Complex Systems Modeling and Networks (CX202) course from 23-27 June 2008– I used all 32 of my FY 2008 Strategic Education Initiative
(SEI) hours on the course; the balance was covered by the Systems Analysis LDRD.
• The course was held at the New England Complex Systems Institute (NECSI) on the campus of the Sloan School at MIT, in Cambridge, MA
• The course prerequisite is CX201, Complex Physical, Biological and Social Systems– I fulfilled the prerequisite by reading Making Things Work
by Yaneer Bar-Yam, and Linked by Albert-László Barabási, before the course began
• The follow-on course is CX203, Methods for the Study of Complex Systems, which is mathematically-oriented
Overview (cont’d)
• Over 30 attendees from all over the world, predominately PhD students and academics– Only three or four people from “industry”– My project group consisted of a Norwegian PhD student, a
South Korean health care researcher (working in the Boston area), an Iranian PhD student (studying in Maryland), and a Bulgarian PhD student (studying in France)
• The course was divided into two parts – The first part was complex network analysis, taught by Dan
Braha of NECSI and UMass Dartmouth – The second part was modeling and simulation of complex
systems, taught by Hiroki Sayama of NECSI and SUNY Binghamton
• Project groups met each evening, often until quite late, to work on their course project
• Projects (and the associated papers) were presented to the class on Friday, followed by the final exam for the course
NECSI Course Participants
What are Complex Systems?• Complex systems are characterized by emergent, self-
organizing, collective behavior– Such emergent collective behavior often arises due to the
combination of simple individual behavior patterns
• The components of a complex system are interdependent• Complex systems exhibit multiscale variety
– The structure of the system differs depending on the level at which it is viewed
– A common pattern is a mixture of competition and cooperation at different levels of the system
• Complex systems arise due to evolutionary, not deterministic, processes– Engineered systems should be implemented this way too
• Examples of complex systems include the health care system, the education system, and recent military conflict
From Making Things Work by Yaneer Bar-Yam
Another Characterization of Complex Systems
• Complex systems contain many constituents interacting nonlinearly
• The constituents of a complex system are interdependent
• A complex system possesses a structure spanning several scales
• A complex system is capable of emerging behavior
• Complexity involves an interplay between chaos and non-chaos– “Complex systems dance on the edge of chaos”
• Complexity involves an interplay between cooperation and competition
From “Chaos, Complexity, and Entropy” by Michel Baranger
Wicked Problems
• Every wicked problem is essentially unique• There is no definitive formulation of a wicked problem• Wicked problems are never solved• A wicked problem is not understood until after the
formulation of a solution• Solutions to wicked problems change the problem itself• Solutions to wicked problems are not true or false, but
better or worse• Stakeholders have radically different world views and
different frames for understanding the problem• Conclusion
– Wicked problems are a proper subset of complex systems
– All wicked problems are complex systems, but not all complex systems are wicked problems
Dancing on the Edge of Chaos• Two fundamental assumptions of most
systems analysis are that the system is continuous, and that if it is decomposed far enough it becomes simple
– It can then be analyzed without loss of fidelity
• These assumptions may be driven by the predominant analytical tool of 20th century physics, which was calculus
– For both derivatives and integrals, if you zoom in far enough a curve becomes a straight line
– Thus a calculus problem is transformed into a geometry problem, which has been well understood since Euclid
• However, chaotic systems are often discontinuous, and the decomposition of chaotic systems generally yields more complexity
– e.g., fractals, Sierpiński triangles• “The arrogance of analysis”• Yet chaotic systems often are simple,
not in terms of their decomposition, but in terms of the rules that govern their behavior
– e.g., “boids,” where simple rules lead to complex flocking behaviors
Complex Networks• The interdependencies of complex systems are often modeled by
networks• Networks can be analyzed using several important metrics from
graph theory– Directed (nodes and arcs) vs. undirected graphs (nodes and
edges)– Number of nodes (N): The number of nodes in a network– Degree (k): The number of edges or arcs incident on a node
• Average degree is denoted as <k>• Can distinguish between in degree and out degree
– Node density is actual connections between nodes divided by potential connections that could exist between nodes
• Note that real-world networks are quite sparse, and have an average density of 10-4
– Characteristic path length (L): The average length of the shortest path in the network between two nodes
– Clustering coefficient (C): Number of actual links between neighbors of a node divided by the number of possible links between those neighbors
• This can be thought of as local density, the probability that two neighbors know each other
• The clustering coefficient is a measure of modularity
Random Networks
• For years (since the late 1950’s) the dominant model of networks was the random network, otherwise known as the Erdős-Rényi model, even though it is rarely seen in nature
• Random graphs have a fixed number of nodes which are connected randomly by undirected edges. The number of edges depend on a specified probability.
• The seminal paper is “On Random Graphs,” by Erdős and Rényi (1959)
Small-World Networks
• Adding a small number of links to a regular ring network, in which only the nearest neighbors are connected, can have dramatic effects
• The characteristic path length becomes quite small with only a small number of such cross-neighborhood links
• Small-world networks are pervasive in nature: Friend networks, power grids, predator-prey networks, and even neural networks of the brain.
• The seminal paper is “Collective Dynamics of ‘Small-World’ Networks,” by Watts and Strogatz (1998)
Scale-Free Networks
• Scale-free networks are so-called because they have no characteristic scale, or average number of links per node
• Instead, they are characterized by numerous hubs that hold together highly connected regions of the network
• However, direct connections between hubs are rare (for reasons of resilience)
• Scale-free networks are also pervasive in nature: Disease propagation, terrorist networks, and the World-Wide Web
• The seminal paper is “Emergence of Scaling in Random Networks,” by Barabási and Albert (1999)
Scale-Free Networks (cont’d)
• Scale-free networks seem to follow a power law distribution (which is a variant of a Pareto distribution, or the 80/20 rule)
• Earlier models (such as Erdős-Rényi) described networks using a bell curve, with many nodes with the same number of links and no highly connected nodes
• The scale free model describes networks as having many nodes with only a few links, and a few nodes with many links
• Power law functions appear as straight lines on a log-log plot
Most nodes have the same number of links
No highly connected nodes
Many nodes with few links
A few nodes with many links
# of links (k) # of links (k)
# o
f n
od
es
with
k li
nks
# o
f n
od
es
with
k li
nks
Bell Curve Power Law Distribution
Scale-Free Networks (cont’d)
Pk ~ e-k Pk ~ k-γ
Scale-Free Networks (cont’d)
• In a more formal sense, a scale-free network is characterized by two properties– Continuous growth (the
other network models did not consider growth)
– Preferential attachment (e.g., “the rich get richer”)
• The number of nodes with exactly k links follows a power law for any value of k– P(k) ~ k-γ
– Often 2 < γ < 3• However, many real-world
networks actually exhibit “stretched exponential” behavior, becoming exponential past a threshold– Cost of adding new links– Aging
From Faloutsos, M., P., & C., “On Power-Law Relationships of the Internet Topology” (1999)
Topology is Destiny!
• Characterizing a network quantitatively gives insight into the dynamics (performance) and vulnerabilities of the network
• Random graphs have a short path length (good), but also a small clustering coefficient (not good)
• Regular ring networks have a large clustering coefficient (good) but long path length (not good)
• Small-world networks are a balance between those two extremes, but do not have the robustness of scale-free networks
• Scale-free networks (which are similar in some ways to small-world networks) are robust against random attacks (good) but not against targeted attacks (not good)– The hubs are vulnerable when directly targeted– The network can become disconnected fairly rapidly in this kind
of attack• The moral of the story: When it comes to networks (and to
everything else in life), you can’t have it all …
How Complex Network Research is Done• A network under study is represented and analyzed
mathematically• A randomly generated “ensemble” of networks is
created with which to compare to the network under study– Ensembles of several types of networks are
generated, such as random graphs, small world, regular ring, and scale-free
• The network ensembles are analyzed mathematically and compared to the network under study
• Both the network under study and the generated ensembles are subjected to the same process (e.g., attack), generally via simulation, and the results are compared
Tools for Complex Network Analysis
• Pajek, an open source tool that generates and visualizes networks– A useful book is
Exploratory Social Network Analysis with Pajek, by de Nooy et al.
• UCINET, a commercial tool from Analytic Technologies that quantitatively analyzes networks
• Python with the NetworkX package• MATLAB, a commercial tool from MathWorks,
which can generate, visualize, and quantitatively analyze networks after “a simple matter of programming” – Free network analysis packages are available for
MATLAB
Modeling and Simulation of Emergent Behavior
• Fundamentals of Modeling • Modeling and Simulation of Dynamical Systems (complex
interdependent systems that can exhibit emergent behavior)– Non-Spatial Modeling– Cellular Automata– Agent-Based Models– Network Models
• Other Simulation Paradigms for Dynamical Systems (not covered in the course)– Systems Dynamics
• Numerous commercial packages exist, such as Vensim• Can be described in terms of difference or differential
equations, so can be addressed by non-spatial modeling– Spatial models with partial differential equations– “Soft computing” (e.g., evolutionary computation, artificial
neural networks, or support vector machines)
Fundamentals of Modeling
• Scientific “laws” are really hypotheses, models of observed behavior
• There is no way to prove anything about nature, although it is easy to disprove something
• A model is a simplified representation of a system
• Rule-based modeling allows the system’s possible or future states to be extrapolated, unlike descriptive modeling, which just specifies the system’s actual or past states
• A good model is simple, correct, and robust
Modeling Procedure
• In general– Observe the system of interest– Reflect on the possible rules that might cause the
system’s characteristics– Derive predictions from those rules and compare them
with reality– Repeat to modify the rules until satisfied
• For complex systems– Define the key questions and choose the right scale of
modeling– Identify the structure (the parts and their connections) of
the system– Define the possible states for each part (state space)– Describe how the state of each part changes over time
through interactions with other parts• Models must be evaluated and validated, in terms of
simplicity, correctness, and robustness
Tools Used for Modeling and Simulation
• Python• NumPy and SciPy (MATLAB-like numerical
packages for Python)• matplotlib (MATLAB-like visualization library for
Python)• NetworkX (network graphing package)• Agent-based modeling toolkits, such as NetLogo,
StarLogo, or Swarm (see the Swarm Wiki for other tools)
• And, of course, MATLAB …
Non-Spatial Modeling• The simplest mathematical model
– Called “non-spatial” because the spatial location of the elements of the system is unimportant
• A dynamical system is a system whose state is uniquely specified by a finite set of variables and whose behavior is uniquely determined by predetermined rules – Examples are simple population growth, simple pendulum
swing, motion of celestial bodies • Two schools of modeling dynamical systems
– Discrete-time model (using difference/recurrence equations or iterative maps)
– Continuous-time model (using differential equations)• Non-autonomous, higher-order equations can always be
converted into autonomous, first order equations, by introducing additional variables
• Non-linear systems are generally simulated because they cannot be solved analytically
• Exercise: Logistic growth (logisticgrowthmodel.py)
Cellular Automata
• Adds spatio-temporal dynamics, especially for locally connected networks, to the modeling process
• Cells in a regular grid interact with their nearest neighbors according to simple interaction rules– Transition from one state to another depends on
simple rules based on interactions with neighbors
– The resulting system is a linear system
• Example: Simple diffusion, panic in a gym• Exercise: Decide whether or not to panic based
on the state of a simple majority of your neighbors (majority.py)
Agent-Based Models
• The only way to study the behavior of agent-based models is to simulate, since no global equation exists, only equations at the agent level
• Agent characteristics (what makes an agent an agent)– Local state– Spatial localization– Local interaction with other agents based on
predefined rules– No central supervisor– May learn autonomously– May produce non-trivial collective behavior
Agent-Based Models (cont’d)• To develop an agent-based model
– Identify the agents and get a theory of agent behavior– Identify the agent relationships and get a theory of agent
interaction– Get the requisite agent-related data– Validate the agent behavior models in addition to the model as
a whole (thus you must justify your model relative to the real-world, including emergent behavior)
– Run the model and analyze the output• Exercise: Garbage collection by ants (ants.py)
– Eventually, only one pile of garbage remains– No simulation of the environment is needed, just tracking of
the environment; however, if pheremones were emitted and decayed with time, the environment would need to be simulated as well, based on a model of pheremone decay
• An excellent two-part tutorial was presented by Macal and North at the Winter Simulation Conference in 2005 and 2006, respectively
Network Models• Simulation of the types of complex networks we
learned about in the first part of the course• Uses the NetworkX package from LANL
– Has methods to create random networks, small-world networks, and scale-free networks
• Exercise: A disease epidemic on a random network (networkepidemics.py)– Two parameters: Infection probability and recovery
probability– Red means infected; yellow means susceptible– Note that the disease is never completely
eradicated from a social network; this is called the fixation of a disease
– Can experiment with different network topologies
Further Reading
• Complexity and Chaos– Making Things Work, by Yaneer Bar-Yam (2005)– Ubiquity: The Science of History … or Why the World is Simpler th
an We Think (a.k.a. “Ubiquity: Why Catastrophes Happen”), by Mark Buchanan (2001, 2002)
– Chaos: Making a New Science, by James Gleick (1988)– The Dynamics of Complex Systems, by Yaneer Bar-Yam (1997)
• Complex networks– Linked, by Albert-László Barabási (2003)– Small Worlds: The Dynamics of Networks between Order and Rand
omness, by D. J. Watts (2003)
– Exploratory Social Network Analysis with Pajek, by de Nooy, et al. (2005)
• Modeling and simulation of dynamical systems– Cellular Automata, by Andrew Ilachinski (2001)– Python Scripting for Computational Science, 3rd ed., by Hans Petter
Langtangen (2006)– Simulation Modeling and Analysis, 4th ed., by Averill Law (2007)
Class Project