Statistical properties of network community structure
description
Transcript of Statistical properties of network community structure
Statistical properties of network community structureJure Leskovec, CMUKevin Lang, Anirban Dasgupta and Michael MahoneyYahoo! Research
Network communities
Communities: Sets of nodes with lots
of connections inside and few to outside (the rest of the network)
Assumption: Networks are
(hierarchically) composed of communities (modules)
Communities, clusters, groups,
modules
Community score (quality) How community like is a
set of nodes? Want a measure that
corresponds to intuition Conductance
(normalized cut):Φ(S) = # edges cut / # edges inside
Small Φ(S) corresponds to more community-like sets of nodes
Community score (quality)
Score: Φ(S) = # edges cut / # edges inside
Community score (quality)
Score: Φ(S) = # edges cut / # edges inside
Bad communit
yΦ=5/7 = 0.7
Community score (quality)
Score: Φ(S) = # edges cut / # edges inside
Better communit
y
Φ=5/7 = 0.7
Bad communit
y
Φ=2/5 = 0.4
Community score (quality)
Score: Φ(S) = # edges cut / # edges inside
Better communit
y
Φ=5/7 = 0.7
Bad communit
y
Φ=2/5 = 0.4
Best communit
yΦ=2/8 = 0.25
Network Community Profile Plot We define:
Network community profile (NCP) plotPlot the score of best community of size k
Search over all subsets of size k and find best: Φ(k=5) = 0.25
NCP plot is intractable to compute Use approximation algorithm
NCP plot: Small Social Network Dolphin social network
Two communities of dolphins
NCP plotNetwork
NCP plot: Zachary’s karate club
Zachary’s university karate club social network During the study club split into 2 The split (squares vs. circles) corresponds to cut B
NCP plotNetwork
NCP plot: Network Science Collaborations between scientists in Networks
NCP plotNetwork
Geometric and Hierarchical graphs
Hierarchical network
Geometric (grid-like) network – Small social
networks– Geometric and– Hierarchical networkhave downward NCP plot
Our work: Large networks Previously researchers examined community
structure of small networks (~100 nodes) We examined more than 70 different large
social and information networks
Large real-world networks look
completely different!
Example of our findings Typical example:
General relativity collaboration network (4,158 nodes, 13,422 edges)
NCP: LiveJournal (N=5M, E=42M)
Better and better
communities
Worse and worse
communities
Best community
has 100 nodes
Com
mun
ity sc
ore
Community size
Explanation: Downward part Whiskers are responsible for
downward slope of NCP plot
Whisker is a set of nodes connected to the network by a single
edge
NCP plot
Largest whisker
Explanation: Upward part Each new edge inside the
community costs moreNCP plot
Φ=2/1 = 2Φ=8/3 = 2.6
Φ=64/11 = 5.8
Each node has twice as many
children
Suggested Network Structure
Network structure: Core-
periphery, jellyfish, octopus
Whiskers are responsible for
good communities
Denser and denser core
of the network
Core contains 60%
node and 80% edges
Caveat: Bag of whiskersWhat if we allow
cuts that give disconnected communities?
Cut all whiskers and compose
communities out of them
Caveat: Bag of whiskersCo
mm
unity
scor
e
Community size
We get better community scores when
composing disconnected sets of
whiskersConnected communitie
sBag of whiskers
Comparison to a rewired network
Rewired network: random
network with same degree distribution
What is a good model?
What is a good model that explains such network structure?
None of the existing models work
Pref. attachment Small World Geometric Pref. Attachment
FlatDown and Flat
Flat and Down
Forest Fire model works Forest Fire: connections spread like a fire
New node joins the network Selects a seed node Connects to some of its neighbors Continue recursively
As community grows it
blends into the core of
the network
Forest Fire NCP plot
rewired
network
Bag of whisk
ers
Conclusion and connections Whiskers:
Largest whisker has ~100 nodes Independent of network size Dunbar number: a person can maintain social
relationship to 150 people Bond vs. identity communites
Core: Newman et al. analyzed 400k node product network▪ Largest community has 50% nodes▪ Community was labeled “miscelaneous”
Conclusion and connections NCP plot is a way to analyze network
community structure Our results agree with previous work on small
networks But large networks are fundamentally
different Large networks have core-periphery structure
Small well isolated communities blend into the core of the networks as they grow