1  

Modeling Higher-Order Dependencies with Persistent Homology

Julio Cesar Santiago B.S. in Mathematics and B.S. in Economics

Junior 16 Duke University

Abstract: High-order dyadic dependencies are not readily observed in global trade networks and as a result, a more frictionless model of international commerce has laid claim to the vast majority of GTN modeling and analysis. We found there was a substantial gap between eigenvectors from 2002:2005 Bimarkov GTN vs. those from 2006:2008. Our main theoretical claim is that the observed gap between eigenvectors indicates the existence of dependencies of a higher degree than those second and third order dependencies captured by the GBME function. By computing the persistent homology of the Bimarkov GTNs from 2004:2008, we claim to have captured a higher- level dependency that has yet to be rigorously studied. The H1 cycles can be topologically described as “loops” within clustered countries. Clusterability is recognized as a third-order dependency. Our findings suggest the behavior of these “loops” (i.e. birth/death time) could possibly be a well-defined higher order dependency by itself. We aim to conduct future research using behavioral clustering methods-such as single track vectors.

 

2  

Acknowledgements I thank Dr. Michael Ward for licensing his software and data, but most importantly, for having me onboard his team as an undergraduate. I thank Dr. Rachel Kranton for taking the time to meet, and discuss her work on strategic interactions. I would also like to thank Nicholas Aldebrando for writing the export function for the adjacency matrices from R to Matlab. I thank Dr. Paul Bendich for his patience and forward guidance throughout the development of this paper. I thank Dr. Mauro Maggioni for suggesting the application of his Diffusion Geometry package and running a Bimarkov chain on the GTN data sets.

 

3  

Outline. This paper is organized as follows. Section 1.1 presents the global trade network

using a standard gravity model equation. Section 1.2 presents the global trade network

using the GBME model. For the sake of clarity, note we will be using the GTN as

computed from Section 1.2 to run the Bimarkov chains and TDA tools. Section 1.3 runs

a Bimarkov chain on the GTNs from 2002:2008. Section 1.4 presents the results from

Section 1.3 using Dr. Maggioni’s Diffusion Geometry package. Section 1.5 computes the

H1 persistence diagram for the GTNs from 2002:2008. Section 1.6 proposes the use of

singular behavior vectors as a means of capturing “loop” formation.

1.1 Global Trade Network (GTN) using Standard Gravity Model The global trade network is defined as a graph of import/export relationships between

countries in a given year. The standard gravity model has been the manner in which

political economists have typically come to understand and model the internal commerce

system. The gravity model of a GTN describes the flow of goods and services between

two locales as proportional to the combined "mass” of the two locales. This approach is a

frictionless model of free trade. Note however, a gravity model can only explain about

fifty percent of the total variation in bilateral international commerce.

In the standard gravity model, direct bilateral trade linkages are regarded as one of the

most strategic interactions two countries can engage in. However, we argue these direct

trade linkages only manage to make sense of about fifty percent of the total variation an

economic shock will have on nation C, that does not engage in a direct trading

relationship with nation A - where the economic shock originated. Trade paths

connecting any pair of non-direct trade partners must then serve as a key component in

figuring out the likelihood of so-called contagion of a macroeconomic crisis across the

GTN. The spread of economic shocks will later feature prominently in our empirical

discussion of the 2002:2008 RWBC GTN data sets.

 

4  

The motivation behind using a gravity model stems from a long-standing assumption that

the GTN does not possess a complex network structure given you can approximate the

distribution of its link weights by a log-normal density model. Log-normality can be

easily explained as the limit outcome of uncorrelated link-weight multiplicative growth

processes. By fitting a gravity equation to the original international bilateral trade data to

account for explanatory variables of link weights, Fagiolo (2009) presented a convincing

argument that dismissal of the GTN’s structure is ill founded. Fagiolo (2009) showed

that, once all gravity equation dependence is removed from the original data, the residual

GTN was characterized by power-law distributed link weights.

Even more striking, Fagiolo (2009) estimated the correlation between original and

residual weights as not statistically significant from zero. The original link weights in

Figure 1 are remarkably heterogeneous and do not exhibit the kind of fat tailed behavior

that is associated with network complexity. Notice however, that the residual GTN is

characterized by power law distributed links that suggests trade flows with inherent

complex behavior. We think this discrepancy is driven in large part by entrenched

similarities between countries that cannot be fully appreciated in Figure 1.

Original GTN Residual GTN

 

5  

The original GTN, where all correlation coefficients were statistically different from

zero, hinted to a trade structure where countries that trade more intensively are also high-

income ones, they are more clustered and central, but tend to trade with relatively less

connected partners. This is an inappropriate configuration for a real global trade network.

In Fagiolo (2009) results, the topological properties of the residual GTN are almost

uncorrelated with their original GTN counterparts - once size, geographically-related and

other determinants of trade have been removed from the data. More interestingly, Fagiolo

(2009) showed that the RWBC was the only topological property from Table 1 with the

same power-law shape in both the residual and original GTNs. All other topological

properties were originally well fitted by lognormal densities. We can infer that each ith-

RWBC nod somewhat reflects the whole structure of the network. It is also important to

point out, the fact that the RWBC is power-law distributed in both original and residual

GTNs suggests that complexity is an intrinsic feature of the global trade network.

1.2 Global Trade Network (GTN) using GBME model1 Equation 1 denotes the standard linear regression model used in the field of quantitative

international relations to model dyadic networks.

Yij = B’Xij + ei,j

Let Y denote an nxn matrix that contains dyadic measurements. It follows then, the Yij

entry is a measurement of the relation from the country i to country j. In directed

                                                                                                               1 See Hoff, Peter. 2003. “Bilinear Mixed Effect Data”. Technical Report no. 430 . Dept. of Statistics. University of Washington. Academic paper can be accessed at http://www.stat.washington.edu/research/reports/2003/tr430.pdf for a

Topological Properties of Gravity Equation Model

Node strength Sum of all link weights of a node Node Average Nearest Neighbor (ANNS) Average strength of a trade partner of that node

Weighted Node-Clustering (WCC) Measured as the relative weighted intensity of trade triangles with that node as one of the vertices.

Random Walk Betweennes Centrality (RWBC)

Accounts for global centrality of a node in the weighted network

 

6  

networks such as the GTN, asymmetric relationships are possible. Let X denote an nxnxr

array, so that Xij is a vector of length r describing characteristics specific to the dyad (i, j).

The errors terms, eij , are typically taken to be independent of the covariates. But in order

to capture higher order dependencies in the dyad, the assumption of error independence

will need to be relaxed. Hoff (2002) incorporated the dependent, dyadic structure of the

data to Equation 1 by relaxing the eij independence assumption. We mined our data sets

using Hoff (2002) version of the GBME function in order to capture higher-order

dependencies in the trade dyads.

eij = ai + bj + yij + zi’j2

The ai + bj terms are the sender and receiver effects induced in bilateral trade interaction

between two countries. These random effects can capture second-order forms of

dependence such as reciprocity and within-actor correlation. The yij + zi’j terms are of

particular importance in making the GBME model effective at accommodating for the

structure of the data. The yij term is the dyadic error independent of the other bilinear and

random effects, used in Ward and Hoff (2005). This error structure is then incorporated

into the linear predictor model in which dyads are conditionally dependent.

Θi,j = B’Xij + ai + bj + yij

The zi’j inner product is often referred to as the “bilinear” or multiplicative interaction

term. Define an unobserved, latent k-dimensional vector zi for each node i in the network.

These vectors can be thought of as representing a position in an unobserved latent

characteristic space. The inner product term uses the latent similarity between countries

to output a global trade network.

                                                                                                               2 ai =sender effects bj= receiver effects yij= dyadic error independent of the other bilinear ad random effects zi

’j = the inner product of zi

and zj k-dimensional vectors for ith, jth actors in the latent space

 

7  

1.3 Running Bimarkov Chain on Global Trade Networks

A spectral graph application involves a trade-off when dealing with time-varying

systems. The trade-off is choosing either to pool all 2002:2008 trade observations into

one single randomized network or run the walk on each network separately. Neither

seems satisfactory given our findings. The first alternative masks potentially interesting

processes that drive structural change on a yearly basis. We pooled all 2002:2008 GTNs

into one single realization as shown in Figure 2. We noticed that when the networks were

pooled, structures might appear different than what they are. As a result, depending on

the research question being asked, the statistical inferences made from pooling a dynamic

network may be quite misleading.

function [T, p] = Bimarkov(K, Options)

Bimarkov computes the bimarkov normalization function p(x) for the nonnegative, symemtric kernel K(x,y) using an iterative

scheme. The function p(x) is the unique function s.t. diag(1./sqrt(p)*K*diag(1./sqrt(p)) is both row and column stochastic. Note that a

bimarkov kernel necessarily has L^2 norm 1.

In:

K = an NxN matrix speciying a nonnegative, symmetric kernel with

nonzero row sums

Options = a structure containing zero or more of the following fields

MaxIters : maximum number of iterations (default = 100)

AbsError : Bimarkov stops when each row sum is within AbsoluteError

of 1.00 (default = 1e-5)

Quiet : when true, all output other than error messages will be

suppressed

Out:

T = bimarkov normalized kernel

p = column vector giving the bimarkov normalization function

(c) Copyright Yale University, 2006, James C Bremer Jr.

(c) Copyright Duke University, 2007, Mauro Maggioni

Random Walk on Randomized GTN pooled from 2002:2008

 

8  

We also modeled each GTN separately and then ran a bimarkov chain for each year. The

second alternative definitely made subtle differences in structure more apparent as shown

in Figure 3. The most troubling limitation with this alternative approach is that estimating

five separate networks ignores the knowledge about previous periods. For example, in

our analysis, we saw a notable change in the GTN structure between 2002:2005 vs.

2006:2008, which looked too extreme given the unobserved ties between the two time

intervals. This topological observation is shown more clearly in Panel 1.

Plotting the eigenvectors for each randomized GTN separately

 

9  Random Walk on GTNs from 2002:2005

Random Walk on GTNs from 2006:2008

 

10  

1.4 Inconclusive Findings with Diffusion Geometry Hoff (2002) GBME model successfully captured second- and third- order dependencies

in the global trade network as presented in Ward and Hoff (2005) and Ward and Dorff

(2012). However, a wider scope is need if we are to study higher order dependencies

beyond that of a 3-node dyadic interaction. The most striking theoretical claim that comes

across all recent network analysis research on international commerce is how little of an

understanding we have of dependent, dyadic structures.

We argue the GBME model is incomplete and renders an incoherent configuration of the

ebb and flow of interactions among actors in the global trade network, as evidenced in

Figure 4. Figure 4 shows the spectral density of the individual column vector bimarkov

normalization function from 2002:2008. Each vector contains the set of all eigenvalues

computed per randomized GTNs from 2002:2008. Our most striking empirical result was

the gap between the eigenvectors from 2002:2005 vs. 2006:2008, which validates our

previous observation of a topological change across two distinct time intervals.

While we know eigenvalues are an extremely important parameter in spectral graph

theory, relatively little is known about the eigenvectors belonging to various eigenvalues

in dynamical systems.

Spectral Density of Eigenvectors for each GTN from 2002:2008

 

11  

1.5 Application of Persistent Homology to Time-Varying Global Trade

Networks

Analyzing the spectral density of the column vector bimarkov normalized function for

each GTNs from 2002:2008 rendered an incomplete, incoherent configuration of the ebb

and flow of strategic interactions among countries in the GTN. If we learned anything

from recent research Hoff (2002); Ward and Hoff (2005); Ward and Dorff (2012), is how

little we understand about the underlying dependent, dyadic structures in GTNs. The

long-standing view would make the case that the GTN is not a complex network structure

since you can approximate the link weights wij using a simple lognormal density model.

We believe this has mislead researchers against studying the structure in GTN data as a

subject of its own right, not just as a means to enhance existing predictive models. Ward

and Hoff (2005) showed global trade networks exhibit tremendous amount of residual

structure that persists over long periods of time. Our first observation was striking in its

own right. Recall from Fagiolo (2009), the RWBC was the only topological property

characterized by the same power-law shape in both the original and residual GTN. Yet

we computed the persistence diagrams of the original GTNs from 2004:2008 and found

these to have interesting homology without the need to run a random walk through the

adjacency matrixes.

It is important to note, how one should go about interpreting the persistence diagram of a

GTN, as shown in Figure 5. We are not interested in how far a point of H1 is from the

diagonal. We are interested in ascertaining whether there is any type of structure or

higher order dependency in the data, regardless of how persistent that feature may be.

Intuitively, we would not expect loops or H1 cycles amongst countries that were not close

to being with. It is reasonable to assume any H1 would form between already clustered

nation states and as a result, would be plotted closed to the diagonal if present in the

GTN.

 

12  

The gap between the column vector bimarkov normalization functions from 2004:2005

vs. 2006-2008 left us with an incomplete statistic from which to draw inferences.

However, look closely at the birth/death times of the H1 cycles present in the GTN from

2002:2008. Two key empirical findings are a persistent and observable pattern of

decreasing lifetime for H1 cycles or “loops” from 2004:2005 and that H1 cycles in GTNs

from 2006:2008 are born much later than those from 2002:2005.

The gap between time-varying eigenvectors as shown in Figure 4, suggested there was a

geometric trade-off. Earlier we were forced to choose between preserving the integrity of

the dyadic structure or running into incomparable time scale models. Our main

theoretical claim is that with persistent homology we avoid encountering such a data

trade-off and can capture higher order dependencies (>3). We argue this provides a

sufficient proof of concept for incorporating the study of a GTN homology into the field

of quantitative international relations.

 

13  

3

                                                                                                               3 See http://www.ctralie.com/Teaching/Math412_F2014_MusicAssignment/TDATools_UsersGuide.pdf for the MATLAB user guide to TDA tools package. We used the rca1dm function to compute the H1 cycles and the plotpersistencediagram function to get the figures shown above.

H1 Persistence Diagrams for each GTN from 2006:2008

H1 Persistence Diagrams for each GTN from 2002:2006

 

14  

1.6 New Higher-Order Dependency: Behavioral Clustering

We claim to have captured a new higher- order dependency that was not previously

observed in the GBME model: the lifetime of an H1 cycle. These H1 cycles can be

topologically described as “loops” in a network graph, as shown in Figure 6.

We present the following argument for classifying the H1 cycle as a higher order (>3)

dependency. Clusterability is modeled as a third order dependency in Ward and Hoff

(2005) GBME model. Our argument is not focused on the number of clusters within a

network but rather the behavior these clusters exhibit through time. 4

                                                                                                               4  Birth times are plotted in orange. Death times are plotted in blue.

Birth/Death time of H1 cycles in the GTNs from 2002:2003

 

15  

                                   

   The gap between the birth time curve (orange) and the death time curve (blue) captures

the lifetime of H1 cycles in a given GTN. Note that GTNs from 2004:2005 have a sudden

decrease in birth/death times of their H1 cycles. However, the key observation rests on the

onset of birth times. GTNs from 2002:2005 have an earlier onset of H1 cycles, while

GTNs from 2006:2008 has late-bloomer H1 cycles. The GMBE model captures the third

Birth/Death time of H1 cycles in the GTNs from 2004:2005

Birth/Death time of H1 cycles in the GTNs from 2006:2008

 

16  

order dependency of clustering between nation states, but it says nothing about the

behavior of these clusters throughout time.

Future Work.

In Munch (2013)5, the concept of a singular behavioral vector was rigorously introduced.

Behavior vectors are simple vectors in a metric space, which capture a particular feature

of a network’s behavior. These vectors can describe anything from the simplest of

behaviors like acceleration or stopping times, to more complex patters like loop

formation.

The looping vector aims to condense one particular behavior characteristic into a small

vector quantity. We propose the use of persistent homology to determine the location of

almost-loops in the bimarkov GTNs from 2002:2008 and the length of a loops’ lifecycle

as a function of time in each network graph. We only claim to have observed a possible

higher order dependency in global trade networks, we do not however posses the

computational tools to measure this new dependency.

                                                                                                               5 See Chapter 5 of Munch, Elizabeth. 2013. “Applying of Persistent Homology to Time Varying Systems”. PhD Dissertation, Duke University for a rigorous introduction to behavioral clustering. Section 5.2 presents the concept of a singular track vector.

 

17  

Bibliography Alon, Noga and Benny Sudakov. 2000. “Bipartite Subgraphs and the Smallest Eigenvalue” Combinatorics, Probability and Computing, vol.9, pp.1-12 Bell, Francis K.; Cvetkovic, Dragos,; Rowlinson, Peter and Slobodan K. Simir. 2008a. “Graphs for which the least eigenvalue is minimal, I” Linear Algebra and its Applications, 429, p.234-241 Bell, Francis K.; Cvetkovic, Dragos,; Rowlinson, Peter and Slobodan K. Simir. 2008b. “Graphs for which the least eigenvalue is minimal, II” Linear Algebra and its Applications, 429, p. 2168-2179 Blume, Larry. 1993. “The statistical mechanics of strategic interaction”, Games and Economic Behavior, 5(3), 387-424 Brandt, Patrick T; Freeman, John R. and Schrodt, Philip. 2011. “Real Time, Time Series Forecasting of Political Conflict”. Conflict Management and Peace Science 28, 1: 41-64 Bramoulle, Yann; Kranton, Rachel and D’Amours, Martin. January 2011. “Strategic Interaction and Networks”. Department of Economics, Duke University. Working Paper. Corbo, Jacomo; Calvo-Armengol, Antoni, and David Parkes. 2007. “The importance of network topology in local contribution games”. In Proc. Of the international workshop on Internet and Network Economics Cunningham, David; Gleditsch, Kristian Skrede and Idean, Salehyan. 2009. “It Takes Two: A Dyadic Analysis of Civil War Duration and Outcome.” Journal of Conflict Resolution 53, 4: 570-597 Desai, Madhav and Vasant Rao. 1994. “A Characterization of the Smallest Eigenvalue of a Graph”. Journal of Graph Theory, vol. 18, no.2, pp. 181-194 D’Orazio, Vito; Yonanime, James E and Schrodt, Philip. 2011. “Predicting Intra-State Conflict Onset: An Event Data Approach using Euclidean and Levenshtein Distance Measures”. Presented at the 69th Annual Meeting of the Midwest Political Science Association, Chicago, IL, USA Edelsbrunner, Herbert; Cohen-Steiner, David and Harer, John. “Stability Persistence Diagram”. Discrete and Computational Geometry 37:103-120 Gabbay, Michael. “Mapping the Factional Structure of the Sunni Insurgency in Iraq”. Counterterrorism Center Sentinel 1,4: (2008) 10-12 Hoff, Peter D; Raftery, Adrian E and Handcock, Mark S. December 2002. “Latent Space Approaches to Social Network Analysis”. Journal of American Statistical Association

 

18  

Hoff, Peter D. July 2, 2003. “Bilinear Mixed Effects Models for Dyadic Data”. Journal of the American Statistical Association. Metternich, Nils; Dorff, Cassy; Gallop, Max; Weschle, Simon and Ward, Michael. March 7, 2013. “Anti-Government Networks in Civil Conflicts: How Network Structures Affect Conflictual Behavior”. American Journal of Political Science. Munch, Elizabeth. 2013. “Applying of Persistent Homology to Time Varying Systems”. PhD Dissertation, Duke University Little, Anna V; Rosasco, Lorenzo and Maggioni, Mauro. 2012. “Multiscale Geometric Methods for Data Sets I: Multiscale SVD, Noise and Curvature”. Computer Science and Artificial Intelligence Laboratory Technical Report Pardalos, Panos and Stephen Vavasis. 1991. “Quadratic Programming with One Negative Eigenvalue is NP-Hard” Journal of Global Optimisation, 1, pp. 15-22 Schrodt, Philip A. 2009. “TABARI: Textual Analysis by Augmented Replacement Instructions, Version 0.7” http:// eventdata.psu.edu/tabari.dir/tabari.manual.0.7.3b3.pdf Schrodt, Philip A and Yilmaz, Omur. 2007. “CAMEO Conflict and Meditation Event Observations Codebook”. http://eventdata.psu.edu/cameo.dir/cCAMEO.CDB.09b5.pdf