Predicting Spatial Self-Organization withStatistical Moments

Linge Bai∗, Robert Gilmore† and David E. Breen∗∗College of Computing and Informatics †Department of Physics

Drexel University, Philadelphia, PA 19104Email: {lb353,david}@cs.drexel.edu, robert.gilmore@drexel.edu

Abstract—We have developed a self-organizing shape forma-tion system based on locally interacting agents whose behaviorsare inspired by living cells. Given a predefined macroscopicshape, genetic programming is used to find a finite field functionthat defines the agents’ interactions. By following the gradientof the cumulative field the agents form into a desired shape.It has been seen that the self-organization process may formtwo or more stable final configurations. In order to control theoutcome of the shape formation process, it is first necessary toaccurately predict the outcome of the dynamic simulation. Thispaper describes an approach to predicting the final configurationsproduced by our spatial self-organization system at an earlystage in the process. The approach calculates statistical momentsof the coordinates of the agents, and employs Support VectorMachines to predict the final shape of the agent swarm based onthe moments and their time derivatives.

I. INTRODUCTION

Motivated by the ability of living cells to form into specificshapes and structures, we have developed chemotaxis-inspiredsoftware agents for self-organizing shape formation [1], [2].The actions of the agents, which we call Morphogenetic Prim-itives (MPs), are based on the behaviors exhibited by livingcells. Cells emit chemicals into the environment. Neighboringcells detect the overall chemical concentration at their surfacesand respond to the chemical stimulus by moving along thechemical field’s gradients [3]. Similarly, in our system theagents emit a virtual chemical, with its concentration definedby an explicit mathematical expression. All agents start with aninitial random configuration and stochastically follow the gra-dient of the cumulative concentration field. These chemotaxis-based local interactions can direct the agents to self-organizeinto user-specified shapes (Figure 1), a capability that couldbe used to control a robotic swarm.

In some cases, we have observed that the agents do notspatially self-organize into a unique shape, but instead formtwo or more stable final configurations. It would be useful tocontrol the outcome of these divergent spatial self-organizationprocesses. This would allow us to guarantee that all of our MPsimulations would produce a single, desired shape. The firststep toward providing the guarantee involves predicting, at anearly stage of the evolution, which final shape will be formed.This will then make it possible to apply changes that direct theagent swarm towards the correct spatial configuration. Towardsthis end, we have discovered features, based on statisticalmoments and their derivatives of the agents’ positions, that

Fig. 1. MPs self-organizing into a pinwheel shape.

can be computed early in the swarm’s evolution and may beused to predict the final shape of the self-organizing system.Analyzing these features with a Support Vector Machine[4], a supervised machine learning technique, classifies theagent system, i.e. provides a prediction for the system’s finalconfiguration. Given a prediction that a swarm is aggregatinginto an “undesired” configuration, it may be possible to perturbthe positions of the agents to redirect their shape evolution. Weaim to design biased perturbations that can change the path ofthe evolution towards a “desired” shape.

II. RELATED WORK

Multi-agent approaches have been widely utilized to modelself-organizing systems [5], [6], [7]. Our current work focuseson how to analyze and predict the outcome of such systems.To date, a variety of methods have been used to analyze thesesystems. These methods usually require identifying globalvariables [8] and/or deriving quantitative properties of thewhole system [9]. Differential equations also are used tocapture state transitions of a system [10], [11]. These meth-ods require identifying global variables and the relationshipsbetween them, which can be obscure and quite complex.

Various analytical methods have been used to predict theoutcomes of self-organizing systems. Time series analysis hasbeen applied to predict self-organizing maps [12]. The analysisaims to predict long-term trends, rather than predicting specificfinal states. Hamann [13] presents a data-driven, iterativeprocess, based on a swarm urn model, Markov chains andabductive reasoning, for determining, testing, and refining

hypotheses about how self-organizing decision-making systemoperate. The process is able to identify influential feedbackloops and important subsections of the configuration space inthese types of systems.

Machine learning algorithms have also been applied toself-organizing systems. An unsupervised learning algorithmis utilized to construct a self-organizing map for informa-tion retrieval [14]. Support Vector Machines have been usedtogether with self-organizing maps for shape-based imageretrieval [15].

Statistical moments are measures that quantify the shapeof distributions. In statistics, the method of moments is usedto estimate population parameters [16]. In our work, we gaininsights from the moments of the coordinates of moving agentsin a 2D arena and predict the final outcome of the systembased on the moment information.

III. SELF-ORGANIZING SHAPE FORMATION

Our previous work [2], [17] is inspired by developmentalbiology [18] and morphogenesis [19] and defines softwareagents with behaviors similar to living cells. This work buildsupon a chemotaxis-based cell aggregation simulation sys-tem [20]. Morphogenesis is the process that develops the shapeor structure of an organism through cell shape change, move-ment, attachment, growth and death. The motions inducedby chemotaxis (one of the mechanisms of morphogenesis)may produce patterns or sorting of cells [21]. These naturalphenomena provide the proof-of-concept that chemotaxis-based cell aggregation offers a promising approach to guidingspatial self-organization processes.

A. Morphogenetic Primitives

In our system, virtual cells (agents) are initially placedinside a 2D environment with a random uniform distribution,and interact with each other via a chemotaxis paradigm.This interaction produces movements that lead the agents toaggregate into a single user-specified shape. We call these self-organizing agents Morphogenetic Primitives (MPs).

Each MP is represented by a small disc and emits a‘chemical’ into the environment within a fixed distance. EveryMP emits the identical local chemical field. An MP detects thecumulative chemical field at eight receptors on its surface, andcalculates the field gradient from this input. MPs move in thedirection of the field gradient with a speed proportional tothe magnitude of the gradient. By employing these relativelysimple chemotaxis-inspired behaviors MPs are able to self-organize into specific macroscopic shapes.

B. Local Interactions

While MPs’ fundamental interactions are based on achemotaxis-inspired paradigm, we do not limit their behav-iors/properties to be physically realistic or completely con-sistent with biology. Instead, developmental biology providesa motivating starting point for MPs. As a way to customizechemotaxis-inspired agents for shape formation, we alter thechemical concentration fields around individual cells. Instead

of the chemical concentration dropping off as a function ofdistance, e.g. 1/r, in our system we define the concentrationfield with an explicit function of cell-cell distance d, one cell’sangular location θ in another cell’s local coordinate system andsimulation time t.

Currently, there is no prescriptive way to specify a par-ticular local field function that will direct MPs to forma specific macroscopic shape, we therefore employ geneticprogramming [22] to produce the mathematical expressionthat explicitly specifies the field function. In order to meetthe substantial computational requirement imposed by ourevolutionary computing approach, we have implemented amaster-slave form of the distributed genetic programming pro-cess [1]. The fitness measure associated with each individualfield function is based on the shape that emerges from thechemical-field-driven aggregation simulation, and determineswhich functions will be passed along to later generations. Thegenetic process stops once an individual (i.e., a mathematicalexpression) in the population produces the desired shape via achemotaxis simulation, or after a certain number of generationshave been produced and evaluated.

With this algorithm, we have successfully evolved local MPchemical field functions for a number of simple shapes. Theseresults support the proposition that biological phenomena offerparadigms for designing cellular primitives for self-organizingshape formation. Additionally, evolutionary computing tech-niques, specifically genetic programming, have been crucialfor discovering the detailed local interactions that lead to theemergence of macroscopic shapes and structures.

However, given the MPs’ initial random configurations andthe stochastic nature of the self-organization process, theoutcomes of the simulations with a specific field function arenot always the same. We have found that the shape formationsimulations can generate bifurcating results. For some fieldfunctions, if we run numerous simulations each starting witha different random distribution of MPs, two sets of finalconfigurations will be formed. In most cases an equal numberof each configuration is formed, but in a few cases the ratio ofthe numbers is not one. Since it would be useful to control theoutcomes of the self-organization process, we are developingmethods for predicting the final configuration of a bifurcatingsimulation at an early stage of the process.

IV. MOMENT ANALYSIS

In order to predict the final outcome of a self-organizingshape formation simulation, we first extract features thatcapture the spatial distribution of the MPs. Moments providea quantitative way to describe a distribution. Since MPs aredefined as small discs, we use the center of each disc torepresent each MP’s location. We therefore can simplify thecollection of MP locations as a set of 2D points, and applymoment analysis to this set over the duration of the MPsimulation.

A. Moment Calculation

We calculate the mean (first moment), variance (secondcentral moment), skewness (third central moment) and kurtosis(fourth central moment) from the x and y coordinates of theMP centers. We analyze the locations Xi of all points (MPs)as a whole, rather than tracking the location and movementof each individual point. The population size of the agentsis denoted as n, (n = 500), and the formulas of the fourmoments M1 to M4 are given in Equations 1 to 4,

M1 =1

n

n∑i=1

Xi, (1)

M2 =1

n

n∑i=1

(Xi −M1)2, (2)

M3 = [1

n

n∑i=1

(Xi −M1)3]/(M2)

3/2, (3)

M4 = [1

n

n∑i=1

(Xi −M1)4]/(M2)

2. (4)

These statistical moments provide quantitative informationabout the shape of histograms/distributions. When computedfor the x and y coordinates of the MPs, these moments capturethe symmetry and asymmetry of the spatial distribution of thewhole population. Since the x and y coordinates of the pointschange over time, so do the four moments of the distribution ofthe x and y values. The change of the moments as a function ofsimulation time also provides insight into the dynamic natureof a particular MP simulation.

We define the time derivative of the moments as the slope ofa linear interpolating function of consecutive moment values.A series of values are obtained by sampling the simulationtime with a fixed interval τ . At each simulation time t, thefour moments Mi(t) (i = 1 to 4) of the overall distribution arecalculated. For each moment Mi, we consider an interval of4τ which containing 5 samples, that is, Mi(t−2τ), Mi(t−τ),Mi(t), Mi(t+ τ) and Mi(t+2τ). A straight line is fit to thefive points with a least squares method. For each moment Mi

at time t, we find the ki and bi that minimize Equation 5,

min

2∑j=−2

|ki · (t+ jτ) + bi −Mi(t+ jτ)|2. (5)

The slope ki of the fitted line is then used as an approximationof the time derivative of Mi(t)(i = 1 to 4).

By calculating the moments and their time derivatives forboth the x and y coordinates of the point set, at a given time t,we obtain a 16-dimensional vector to represent the distribution,

[Mx1(t),My1(t),Mx2(t),My2(t),

Mx3(t),My3(t),Mx4(t),My4(t),

kx1(t), ky1(t), kx2(t), ky2(t),

kx3(t), ky3(t), kx4(t), ky4(t)] (6)

Our simulations consist of (n = 500) 2D points and therefore

have a configuration space of dimension 2n. By distilling anMP spatial distribution down to this feature vector, we reducethe dimensionality of the simulation from 1000 to 16.

Given the sensitivity of non-linear dynamical systems toinitial conditions [23], it makes it extremely difficult, if notimpossible, to predict the outcome of our complex, self-organizing system from its initial, random spatial configura-tion. We therefore attempt to predict the final spatial config-uration at an early stage of the aggregation, usually before itis visually evident what shape will emerge from the process.We have found that applying our analysis at a time in thesimulation that is a small percentage of the total time neededfor the final aggregated shape to form (e.g. 5% to 10%)produces acceptable results (81% to 91% accuracy).

B. Using Support Vector Machines

We consider prediction of the bifurcating outcomes as aclassification problem and utilize support vector machines(SVMs) [4] to solve it. An SVM applies kernel methods tolabeled training data and transforms the data into a high dimen-sional space. It computes the optimal separating hyperplane,which has the largest margin between the two classes. Doingso produces a trained SVM model with kernel and constraintparameters. Classification is performed by mapping new datainto the same high dimensional space and determining onwhich side of the hyperplane the new data lie.

Since our data are not linearly separable, we apply aRadial Basis Function (RBF) kernel in our SVM. We useLIBSVM [24] to perform SVM training and testing. To trainthe SVM, we employ a leave-one-out cross-validation method.We use grid search to find the best cost parameter C for theSVM and γ parameter for the RBF kernel. Once the optimal Cand γ values were identified, accuracy statistics were gatheredfrom numerous simulations, each utilizing different randominitial conditions.

To obtain training data for a specific target shape, we run acertain number (200) of MP simulations starting with differ-ent, random spatial configurations. Each simulation consists ofa fixed number of time steps (10, 000 to 35, 000), dependingon how long it takes for the target shape to form. We savethe spatial distribution of the agents every 100 steps of thesimulation, that is, τ = 100. Each simulation will producean aggregated structure at the end of the simulation, and wemanually label all final structures with one of two categories.

V. RESULTS

Our system is able to produce a large number of interestingand stable shapes. However, it occasionally (∼8% of inter-action functions) will produce two or more different shapesfrom the same agent interaction function. In these cases wedivide the results into two “bifurcated” groups, one groupwith the dominant, i.e. the most prevalent, configuration, andthe second ”other” group with the remaining less numerousconfigurations. Given this grouping scheme, we have appliedthe proposed prediction method to a number of bifurcatingshape evolutions. The target shapes are a quarter-moon, an

Quarter-moonTotal Time Prediction Time Positive Instances Negative Instances Kernel

35,000 3,500 100 100 RBFCost (C) gamma (γ) Accuracy Sensitivity Specificity

512 0.03125 91.5% 92.0% 91.0%

EllipseTotal Time Prediction Time Positive Instances Negative Instances Kernel

10,000 500 100 100 RBFCost (C) gamma (γ) Accuracy Sensitivity Specificity

2048 4.883e-04 89.0% 89.0% 89.0%

Four DiscsTotal Time Prediction Time Positive Instances Negative Instances Kernel

15,000 1,500 100 100 RBFCost (C) gamma (γ) Accuracy Sensitivity Specificity

512 1.221e-04 83.0% 84.0% 82.0%

Parallel Line SegmentsTotal Time Prediction Time Positive Instances Negative Instances Kernel

10,000 1,000 508 100 RBFCost (C) gamma (γ) Accuracy Sensitivity Specificity

4 0.125 (80.9± 2.5)% (80.4± 3.5)% (81.4± 2.6)%

TABLE IDATASET INFORMATION AND PREDICTION ACCURACIES FOR BIFURCATING SHAPES AT 10% (5% FOR ELLIPSE) OF SIMULATION TIME.

ellipse, a set of four discs, and a set of two parallel linesegments. Once the SVM parameters with the best overallaccuracy are determined through grid search and leave-one-outcross-validation, we calculate overall accuracy, sensitivity andspecificity for each example. These results are summarized,along with the dataset descriptions and SVM parameters, inTable I. The quarter-moon, ellipse and four-discs datasets eachcontains 200 simulations, with 100 belonging to each class.The parallel line segments dataset is unbalanced, with 508instances belonging to one class and 100 belonging to theother. We have seen that the SVM approach yields higheraccuracies when the bifurcation is balanced.

The quarter-moon dataset contains 100 left-facing structuresand 100 right-facing ones. The quarter-moon interaction func-tion did produce a few shapes of a third type (a diagonal shape)(∼9% of simulation runs), but these were not included in thisstudy. Instead we only used the balanced left- and right-facingoutcomes. A typical shape evolution for this dataset is shownin Figure 2. The simulation reaches a stable state by 35, 000simulation steps. We extract feature vectors in the form ofVector 6. At step 3, 500 (10% of the total simulation time) weare able to predict the left-facing or right-facing outcome withan overall accuracy of 91.5%.

The ellipse dataset contains 100 single ellipses and 100non-single ellipses. The non-single ellipse group includedboth two ellipses and single malformed structures. Figure 3demonstrates a typical ellipse shape evolution. The simulation

reaches a stable state by 10, 000 steps. At 500 steps (5% ofsimulation time) we are able to predict whether the simulationwill produce a single ellipse or not with an overall accuracyof 89%.

The four-discs dataset contains 100 four discs structures and100 structures not containing four discs, with a typical shapeevolution shown in Figure 4. The second group contains avariety of disc numbers, including 3, 5, 6 and 7, with the mostprevalent number being 5. The simulation reaches a stable stateby 15, 000 steps. At 1, 500 steps (10% of simulation time) weare able to predict whether the simulation will form four discsor not with an overall accuracy of 83%.

The parallel line dataset contains 508 instances of twovertical parallel line segments and 100 instances of either ”Y-shape” structures or one vertical line, as seen in Figure 5. Thesimulation reaches a stable state by 10, 000 steps. We calculatethe prediction of the final configuration at step 1, 000. Sincethis is an unbalanced dataset, we under-sample the majorityset. We randomly choose 100 out of the 508 two-line-segmentsinstances and merge them with the minority set. Leave-one-outcross validation is then performed on the newly constructedbalanced dataset. We achieve an average overall accuracy of80.9±2.5% over 100 randomly constructed balanced datasetswhen predicting the formation of two vertical lines.

Since the overall line-segment results are produced viaaveraging the results from numerous randomly generated par-titions, we include a standard deviation with the average. The

Step 0 Step 1000 Step 3500 Step 10000 Step 35000

Fig. 2. Shape evolution of the quarter-moon.

other examples have balanced datasets, and therefore leave-one-out cross-validation produces a unique partitioning andno deviations.

VI. CONCLUSION AND FUTURE WORK

Based on the statistical moments of agents’ positions andthe time derivatives of these moments, we are able to predictthe final outcome of a spatial self-organization process at 5%or 10% percent of the simulation time with an overall accuracyof 81% to 91%. Now that we can predict the outcome of thisprocess at an early stage, we are developing algorithms thatwill control or influence the agents’ interactions so that theyconsistently form into one stable configuration.Acknowledgment: The authors would like to thank SantiagoOntanon and Christian Kuehn for many helpful discussions.This work was funded by National Science Foundation grantsCCF-0636323 and IIS-0845415.

REFERENCES

[1] L. Bai, M. Eyiyurekli, and D. Breen, “Automated shape compositionbased on cell biology and distributed genetic programming,” in Proc.Genetic and Evolutionary Computation Conference, 2008, pp. 1179–1186.

[2] L. Bai and D. E. Breen, “Chemotaxis-inspired cellular primitives for self-organizing shape formation,” in Morphogenetic Engineering, R. Doursat,H. Sayama, and O. Michel, Eds. Berlin: Springer, 2012, pp. 209–237.

[3] E. Eisenbach et al., Chemotaxis. London: Imperial College Press, 2004.[4] N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector

Machines and Other Kernel-Based Learning Methods. CambridgeUniversity Press, 2000.

[5] C. Reynolds, “Flocks, herds and schools: A distributed behavioralmodel,” in Computer Graphics (Proc. ACM SIGGRAPH), vol. 21, no. 4,1987, pp. 25–34.

[6] G. Serugendo, A. Karageorgos, O. Rana, and F. Zambonelli, Eds.,Engineering Self-Organising Systems. Berlin: Springer, 2004.

[7] G. Weiss, Ed., Multiagent Systems, 2nd Edition. Cambridge, MA: MITPress, 2013.

[8] I. G. Kevrekidis, C. W. Gear, and G. Hummer, “Equation-free: Thecomputer-aided analysis of complex multiscale systems,” AIChE Jour-nal, vol. 50, no. 7, pp. 1346–1355, 2004.

[9] J. Sudeikat and W. Renz, “A systemic approach to the validation ofself–organizing dynamics within MAS,” in Agent-Oriented SoftwareEngineering IX. Springer, 2009, pp. 31–45.

[10] G. Ferrari-Trecate, A. Buffa, and M. Gati, “Analysis of coordination inmulti-agent systems through partial difference equations,” IEEE Trans.on Automatic Control, vol. 51, no. 6, pp. 1058–1063, 2006.

[11] L. Cardelli, “On process rate semantics,” Theoretical Computer Science,vol. 391, no. 3, pp. 190–215, 2008.

[12] G. Simon, A. Lendasse, M. Cottrell, J.-C. Fort, and M. Verley-senothers, “Time series forecasting: Obtaining long term trends withself-organizing maps,” Pattern Recognition Letters, vol. 26, no. 12, pp.1795–1808, 2005.

[13] H. Hamann, “A reductionist approach to hypothesis-catching for theanalysis of self-organizing decision-making systems,” in Proc. IEEEConference on Self-Adaptive and Self-Organizing Systems, 2013, pp.227–236.

[14] X. Lin, D. Soergel, and G. Marchionini, “A self-organizing semanticmap for information retrieval,” in Proc. ACM Conference on Researchand Development in Information Retrieval, 1991, pp. 262–269.

[15] W.-T. Wong, F. Y. Shih, and J. Liu, “Shape-based image retrieval usingsupport vector machines, fourier descriptors and self-organizing maps,”Information Sciences, vol. 177, no. 8, pp. 1878–1891, 2007.

[16] J. M. Wooldridge, “Applications of generalized method of momentsestimation,” The Journal of Economic Perspectives, vol. 15, no. 4, pp.87–100, 2001.

[17] L. Bai, M. Eyiyurekli, P. Lelkes, and D. Breen, “Self-organized sortingof heterotypic agents via a chemotaxis paradigm,” Science of ComputerProgramming, vol. 78, no. 5, pp. 594–611, 2013.

[18] S. Gilbert, Developmental Biology, 8th ed. Sunderland, MA: SinauerAssociates, Inc., 2006.

[19] J. Davies, Mechanisms of Morphogenesis: The Creation of BiologicalForm. Amsterdam: Elsevier, 2005.

[20] M. Eyiyurekli, P. Manley, P. Lelkes, and D. Breen, “A computationalmodel of chemotaxis-based cell aggregation,” BioSystems, vol. 93, no. 3,pp. 226–239, September 2008.

[21] T. Sekimura, S. Noji, N. Ueno, and P. Maini, Eds., Morphogenesis andPattern Formation in Biological Systems. Tokyo: Springer, 2003.

[22] J. Koza, Genetic Programming: On the Programming of Computers byMeans of Natural Selection. MIT Press, 1992.

[23] S. Wiggins, Ed., Introduction to Applied Nonlinear Dynamical Systemsand Chaos, 2nd Edition. New York: Springer, 2003.

[24] C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vectormachines,” ACM Trans. on Intelligent Systems and Technology, vol. 2,no. 3, pp. 27:1–27:27, 2011, software available at http://www.csie.ntu.edu.tw/∼cjlin/libsvm.

Step 0 Step 200 Step 500 Step 2000 Step 10000

Fig. 3. Shape evolution of the ellipse.

Step 0 Step 500 Step 1500 Step 3500 Step 15000

Fig. 4. Shape evolution of the discs structure.

Step 0 Step 500 Step 1000 Step 5000 Step 10000

Fig. 5. Shape evolution of the line segments structure.