Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f...
Transcript of Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f...
![Page 1: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/1.jpg)
Hierarchical Graph Clustering:Quality Metrics & Algorithms
Thomas Bonald
Joint work withBertrand Charpentier, Alexis Galland & Alexandre Hollocou
LTCI Data Science seminarMarch 2019
![Page 2: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/2.jpg)
Motivation
I Clustering is a fundamental problem in data science
I The objective is to group together items that are “similar” toeach other −→ unsupervised learning
Many applications:
I Recommendation
I Anomaly detection
I Visualization
I Storage / processing
I Search engines
I Image segmentation
I NLP
![Page 3: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/3.jpg)
An ill-posed problem
I What is a good clustering?
I How many clusters?
![Page 4: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/4.jpg)
Kleinberg’s impossibility theorem
Viewing clustering as a function f : Rn×d → P({1, . . . , n})
Axioms
1. Scale-invariance: ∀α > 0, f (αx) = f (x)
2. Richness: f surjective
3. Consistency: ∀y � x , f (y) = f (x)
There is no clustering function f satisfying these 3 axioms!Kleinberg, NIPS 2002
In fact, this is possible with 3 replaced by:
3’. Refined consistency: ∀y � x , f (y) = f (x) or |f (y)| 6= |f (x)|Cohen-Addad, Kanade & Mallmann-Trenn, NIPS 2018
![Page 5: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/5.jpg)
Hierarchical clustering
Data Dendrogram
![Page 6: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/6.jpg)
Example in biology
2,035 tumors, 16,634 non-redundant genes
Large Expression Dataset: ExpO (GSE2109)
Data from Expression Oncology Project (http://www.intgen.org)
2035 tumors (various types), with 16634 non-redundant genes (after remappingto the NM subset of RefSeq).
)zoom
Computation time: ⇠11 minutes. Peak memory usage: ⇠1 Gb
9
Wirapati 2009
![Page 7: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/7.jpg)
Hierarchical clustering algorithms
Divisive algorithms
I e.g., through successive k-means
Agglomerative algorithms
I Successive merges of the closest clusters a, b ⊂ {1, . . . , n}Linkage d(a, b)
Single mini∈a,j∈b ||xi − xj ||Complete maxi∈a,j∈b ||xi − xj ||Average 1
|a||b|∑
i∈a,j∈b ||xi − xj ||Ward |a||b|
|a|+|b| ||ga − gb||2
Lance & Williams 1967
I Local search by the nearest-neighbor chainBruynooghe 1977, Benzecri 1982, Murtagh 1983
![Page 8: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/8.jpg)
Graph data
Many datasets can be represented by graphs:
I social networks, transport networks, databases, etc.→ explicit links
I authors-papers, words-documents, consumers-products, etc.→ implicit links
These graphs can be represented by sparse matricesDataset #nodes #edges density
Amazon 335k 925k ≈ 10−5
Wikipedia 12M 378M ≈ 10−6
Twitter 42M 1.5G ≈ 10−6
Usual clustering algorithms do not apply as pairwise distances arenot defined and the number of node pairs is huge!
![Page 9: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/9.jpg)
Questions
1. How to cluster a graph?
2. How to assess the quality of this clustering?
![Page 10: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/10.jpg)
Outline
1. Node pair sampling
2. Flat clustering
3. Hierarchical clustering
4. Quality metric
5. Experiments
![Page 11: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/11.jpg)
Notation
Weighted, undirected graph G of n nodesThe weights represent the strengths of the links
wij =
{weight of edge i , j , if any0 otherwise
wi =∑j
wij w =∑i
wi =∑i ,j
wij
![Page 12: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/12.jpg)
Node pair sampling
p(i , j) =wij
wp(i) =
∑j
p(i , j) =wi
w
![Page 13: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/13.jpg)
Entropy
A simple metric for assessing the complexity of the graph:
H = −∑i ,j
p(i , j) log p(i , j)
Note: This is not what is known as graph entropy...Korner 1973
H ≈ 12 bits
![Page 14: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/14.jpg)
Mutual information
A simple metric for assessing the clustering structure of the graph:
I =∑i ,j
p(i , j) logp(i , j)
p(i)p(j)
Alush, Friedman & Goldberger 2016
I ≈ 4 bits
![Page 15: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/15.jpg)
Outline
1. Node pair sampling
2. Flat clustering
3. Hierarchical clustering
4. Quality metric
5. Experiments
![Page 16: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/16.jpg)
Modularity
Quality of a clustering c : {1, . . . , n} → {1, . . . , k}
M(c) =∑i ,j
(p(i , j)− p(i)p(j)) δc(i),c(j)
Newman & Girvan 2004
![Page 17: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/17.jpg)
Cluster pair sampling
For any clustering C ∈ P({1, . . . , n}):
∀a, b ∈ C , p(a, b) =∑
i∈a,j∈bp(i , j) p(a) =
∑i∈a
p(i)
![Page 18: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/18.jpg)
Modularity at cluster level
Quality of a clustering C ∈ P({1, . . . , n}):
M(C ) =∑c∈C
p(c , c)−∑c∈C
p(c)2
Simpson 1949
![Page 19: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/19.jpg)
Modularity maximization
maxC
M(C )
I NP-hard problem
I The Louvain algorithm, fast and efficientBlondel, Guillaume, Lambiotte & Lefebvre 2008
![Page 20: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/20.jpg)
Clustering of OpenFlights by Louvain
3,097 airports, 18,193 flights
M(C ) ≈ 0.66
![Page 21: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/21.jpg)
Resolution parameter
For some parameter γ > 0:
Mγ(c) =∑i ,j
(p(i , j)− γp(i)p(j)) δc(i),c(j)
Reichardt & Bornholdt 2006
![Page 22: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/22.jpg)
Clustering of OpenFlights by Louvain
3,097 airports, 18,193 flights
γ = 2
![Page 23: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/23.jpg)
Outline
1. Node pair sampling
2. Flat clustering
3. Hierarchical clustering
4. Quality metric
5. Experiments
![Page 24: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/24.jpg)
An agglomerative algorithmWe need a measure of proximity between nodes
σ(i , j) =p(i , j)
p(i)p(j)
Observe that
σ(i , j) =p(i |j)p(i)
=p(j |i)p(j)
![Page 25: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/25.jpg)
The maximum resolution
Mγ(c) =∑i ,j
(p(i , j)− γp(i)p(j)) δc(i),c(j)
γ+ = maxi ,j
p(i , j)
p(i)p(j)
![Page 26: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/26.jpg)
Algorithm
While there are at least 2 nodes:
I find the node pair i , j maximizing σ(i , j)
I merge nodes i , j
I update σ
Sequence of similarities / resolutions σ1 ≥ σ2 ≥ . . . ≥ σn−1
![Page 27: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/27.jpg)
Hierarchical clustering of Openflights
3,097 airports, 18,193 flights
![Page 28: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/28.jpg)
Other hierarchical clustering algorithms
Divisive algorithms
I e.g., through successive bisections
Agglomerative algorithms
I Successive merges of the two closest clusters a, b ⊂ {1, . . . , n}Linkage σ(a, b)
Single maxi∈a,j∈b p(i , j)Average 1
|a||b|p(a, b)
Sampling ratio p(a,b)p(a)p(b)
I Local search by the nearest-neighbor chain
See also Newman 2004, Pons & Latapy 2005, Chang 2011
![Page 29: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/29.jpg)
Outline
1. Node pair sampling
2. Flat clustering
3. Hierarchical clustering
4. Quality metric
5. Experiments
![Page 30: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/30.jpg)
Intuition
I Two nodes sampled from the edges are expected to have acommon ancestor relatively low in the hierarchy
I This corresponds to the smallest cluster of the hierarchycontaining these two nodes
![Page 31: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/31.jpg)
Example on Openflights
![Page 32: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/32.jpg)
Example on Openflights
![Page 33: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/33.jpg)
Example on Openflights
![Page 34: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/34.jpg)
Example on Openflights
![Page 35: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/35.jpg)
Tree sampling
I Let T be any rooted binary tree with leaves {1, . . . , n}I For any node x ∈ T ,
p(x) =∑
i ,j :i∧j=x
p(i , j)
I We denote by c(x) the corresponding cluster
![Page 36: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/36.jpg)
Dasgupta’s cost
I Average cluster size: ∑x∈T
p(x)|c(x)|
Dasgupta 2016Cohen-Addad et. al. 2017
![Page 37: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/37.jpg)
Back to tree sampling
I Let T be any rooted binary tree with leaves {1, . . . , n}I For any node x ∈ T ,
p(x) =∑
i ,j :i∧j=x
p(i , j) q(x) =∑
i ,j :i∧j=x
p(i)p(j)
![Page 38: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/38.jpg)
Tree sampling divergence
I Kullback-Leibler divergence between sampling distributions:
Q(T ) =∑x∈T
p(x) logp(x)
q(x)
![Page 39: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/39.jpg)
Tree sampling divergence
I Kullback-Leibler divergence between sampling distributions:
Q(T ) =∑x∈T
p(x) logp(x)
q(x)
I Interpretable in terms of graph reconstruction!
![Page 40: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/40.jpg)
Graph reconstructionGiven a tree T and the node weights w1, . . . ,wn,what is the best reconstruction of the graph (say G )?
I Build the graph G with weights:
wij ∝ wiwj σ(x)
where σ(x) is some similarity attached to x = i ∧ j
I Apply the loss function:
D(p||p) =∑i ,j
p(i , j) logp(i , j)
p(i , j)
Main result
minp←T
D(p||p) = I − Q(T )
![Page 41: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/41.jpg)
Hierarchical clustering of Openflights
3,097 airports, 18,193 flights
H ≈ 15 bits I ≈ 4 bits Q ≈ 2.6 bits Q =Q
I≈ 0.65
![Page 42: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/42.jpg)
General trees
The tree sampling divergence is applicable to any tree T :
Q(T ) =∑x∈T
p(x) logp(x)
q(x)
![Page 43: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/43.jpg)
General trees
The tree sampling divergence is applicable to any tree T :
Q(T ) =∑x∈T
p(x) logp(x)
q(x)
In particular, it can be used for:
I Flat clustering (trees of height 2)
I Tree compression
![Page 44: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/44.jpg)
Flat clusteringFor any clustering C ∈ P({1, . . . , n}):
Q(C ) =∑c∈C
p(c , c) logp(c, c)
p(c)2
+
(1−
∑c∈C
p(c , c)
)log
1−∑
c p(c , c)
1−∑
c p(c)2
![Page 45: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/45.jpg)
Tree compression of Openflights
3,097 airports, 18,193 flights
Full hierarchy (3097 levels) Compact hierarchy (97 levels)
![Page 46: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/46.jpg)
Local hierarchy: Beijing Capital International Airport
![Page 47: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/47.jpg)
Local hierarchy: Carrasco International Airport
![Page 48: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/48.jpg)
Outline
1. Node pair sampling
2. Flat clustering
3. Hierarchical clustering
4. Quality metric
5. Experiments
![Page 49: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/49.jpg)
Experiments
Remind the two metrics:
I Dasgupta’s cost ∑x∈T
p(x)|c(x)|
I Tree sampling divergence∑x∈T
p(x) logp(x)
q(x)
Comparison of these metrics on two tasks:
1. Tree detection
2. Graph reconstruction
![Page 50: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/50.jpg)
Tree detection
Idea:
I Generate two noisy versions G1,G2 of some graph G
I Compute the corresponding trees T1,T2
I Guess the tree associated with each graph G1,G2
T1 = arg maxT=T1,T2
Q1(T ) T2 = arg maxT=T1,T2
Q2(T )
The score is the fraction of correct answers:
1
2(P(T1 = T1) + P(T2 = T2))
![Page 51: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/51.jpg)
Results
0.00 0.05 0.10 0.15 0.20Noise
0.6
0.7
0.8
0.9
1.0
Clas
sifica
tion
scor
e
TSDDC
Graph G = HSBM16 blocks of size 20, 2 levels of hierarchy
![Page 52: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/52.jpg)
Graph reconstruction
Idea:
I Generate some hierarchical random graph G
I Compute trees with different linkages
I For each tree, reconstruct the graph G
I Compare the quality of the tree and the reconstructionscores
Reconstruction scores:Streaming the edges of G in decreasing order of weights,
I Area-Under-ROC
I Average-Precision-Score
I Average rank of each edge of G
![Page 53: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/53.jpg)
Results
0.2 0.3 0.4 0.5 0.6 0.7 0.8Dasgupta's cost
0.0
0.2
0.4
0.6
0.8
1.0
Reco
nstru
ctio
n sc
ore
aucapsrank
0.2 0.3 0.4 0.5 0.6 0.7 0.8Tree sampling divergence
0.0
0.2
0.4
0.6
0.8
1.0
Reco
nstru
ctio
n sc
ore
aucapsrank
![Page 54: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/54.jpg)
Summary
Viewing graphs as probability measures:
I A novel agglomerative algorithm, based on the linkage:
σ(i , j) =p(i , j)
p(i)p(j)
I A novel quality metric, the Tree Sampling Divergence:∑x∈T
p(x) logp(x)
q(x)
interpretable in terms of graph reconstruction
Ongoing work on:
I TSD for flat clustering
I Fast hierarchical clustering
![Page 55: Hierarchical Graph Clustering: Quality Metrics & Algorithms · There is no clustering function f satisfying these 3 axioms! Kleinberg, NIPS 2002 In fact, this is possible with 3 replaced](https://reader034.fdocuments.us/reader034/viewer/2022050508/5f99b0209b47d3205457d650/html5/thumbnails/55.jpg)
scikit-network
A Python package under development, inspired by scikit-learn:https://github.com/sknetwork-team/scikit-network