1/52 Overlapping Community Search Graph Data Management Lab, School of Computer Science...

1/52Overlapping Community Search

Graph Data Management Lab, School of Computer Science

GDM@FUDAN

www.gdm.fudan.edu.cnGraph Data Management Lab, School of Computer Science

GDM@FUDAN

www.gdm.fudan.edu.cn

Email: [email protected]

Online Search of Overlapping Communities

Wanyun Cui, Fudan UniversityYanghua Xiao, Fudan University

Haixun Wang, Microsoft Research AsiaYiqi Lu, Fudan University

Wei Wang, Fudan University

Presenter. Wanyun Cui



GDM@FUDAN


GDM@FUDAN



Outline Motivation Model Algorithm Experiments Applications



GDM@FUDAN


GDM@FUDAN



Complex network Complex network is everywhere.

Social Network



GDM@FUDAN


GDM@FUDAN




Internet



GDM@FUDAN


GDM@FUDAN




Protein Network



GDM@FUDAN


GDM@FUDAN




InternetSocial Network Protein Network



GDM@FUDAN


GDM@FUDAN



Community structures Complex network is everywhere. Most real life networks have community

structures.• The graph can be divided into different groups such that

the vertices within each group are closely connected and the vertices between different groups are sparsely connected

InternetSocial Network Protein Network



GDM@FUDAN


GDM@FUDAN



Overlapping community structure Overlapping community: a vertex may belong to

multiple communities



GDM@FUDAN


GDM@FUDAN



Overlapping community structure Overlapping community: a vertex may belong to

multiple communities

C1: small boatC2: meaning of bucketC3: big boatC4: table wares



GDM@FUDAN


GDM@FUDAN



Finding community structures Two possible ways to find the community

structure• OCD: overlapping community detection• OCS: overlapping community search



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS OCD: divides the entire network to find

communities



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS

Disadvantages of OCD• Too costly• Global criterion• Unfriendly to

dynamic graph

Facebook network: over 800 million nodes and 100 billion links

algorithm complexity

Girvan–Newman algorithm

O(|E|3)

LPA Almost linear

LA O(|C||E|+|V|)



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS


dynamic graph

A fixed parameter or criterion is not appropriate for all vertices and queries.• Communities of a student• Communities of Barack

Obama



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS


dynamic graph

Graphs in real life are always evolving over time.

We cannot afford to run OCD very frequently.

OCD loses its freshness and effectiveness



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS


dynamic graph

Usually performed in an offline fashion



GDM@FUDAN


GDM@FUDAN



OCS: problem definition OCS:

• Given graph G, a query vertex v• Return: all communities that v belong to

Given: Return:



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS

Advantages of OCS:• More efficient• Personalized

criterion• Light weight

We just need to find communities within the local neighborhoods of the vertex.

Our OCS solution only needs several milliseconds to find answer



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS


criterion• Friendly to

dynamic graph



GDM@FUDAN


GDM@FUDAN



OCD vs. OCS


criterion• Light weight

A good choice to find communities in an online fashion



GDM@FUDAN


GDM@FUDAN



Applications of OCS

• Friend recommendation on Facebook.

• Semantic expansion.• Infectious disease

control.• Etc.



GDM@FUDAN


GDM@FUDAN



Challenges of OCS

• Modeling• Complexity and

scalability

A community should be dense enough

Overlapping aware

Generality



GDM@FUDAN


GDM@FUDAN



Challenges of OCS

• Modeling• Complexity and

scalability

OCS in the worst case may need to enumerate an exponential number of valid communities.• Computational hard

Approximate approach



GDM@FUDAN


GDM@FUDAN



Outline Introduction Model Algorithm Experiments Applications



GDM@FUDAN


GDM@FUDAN



Model Community structure

awareness

Overlapping awareness

Generality

The inner edges of a community should be dense

Clique as the unit of community

A clique of 6 vertices



GDM@FUDAN


GDM@FUDAN




awareness


Generality

Two k-cliques are adjacent if they share k-1 vertices

A community is a component in the k-clique graph

Original graph Clique graph (k=4)



GDM@FUDAN


GDM@FUDAN




awareness


Generality

Weaken the strict constraint on clique density and clique adjacency

quasi-clique

adjacency



GDM@FUDAN


GDM@FUDAN




awareness


Generality

Weaken the strict constraint on clique density and clique adjacency

quasi-clique

adjacency

It’s ok if a few edges are missing in the clique



GDM@FUDAN


GDM@FUDAN




awareness


Generality

Loose the strict constraint of clique and adjacency

quasi-clique

𝛼 adjacency

If two cliques share at least 𝛼 vertices, they are 𝛼 adjacent.



GDM@FUDAN


GDM@FUDAN




awareness


Generality

Loose the strict constraint of clique and adjacency

quasi-clique

𝛼 adjacency

Original graph Clique graph (=1)



GDM@FUDAN


GDM@FUDAN



(𝛼 ,𝛾 )−𝑂𝐶𝑆 Given graph G, query vertex v, k, , and , find all

connected quasi-clique components containing v.

k=4



GDM@FUDAN


GDM@FUDAN



Alpha-gamma ocs Given graph G, query vertex v, k, , and , find all

connected quasi-clique components containing v.

k=3



GDM@FUDAN


GDM@FUDAN



Parameter selection and k

• In general, larger k leads to larger

• Has an upper bound and a lower bound corresponding

to and k



GDM@FUDAN


GDM@FUDAN






GDM@FUDAN


GDM@FUDAN



Algorithm Exact algorithm

Approximate algorithm



GDM@FUDAN


GDM@FUDAN



Exact Algorithm Example

• k=4, (3,1)-OCS• Query vertex = Bob



GDM@FUDAN


GDM@FUDAN



Exact Algorithm Example


Drawback• exponential enumerations



GDM@FUDAN


GDM@FUDAN



Approximate Algorithm Example


Approximate• the new clique contains at

least one new vertex



GDM@FUDAN


GDM@FUDAN






GDM@FUDAN


GDM@FUDAN



Experiments Setup

Dataset

Intel Core2 2.13GHz

4GB memory

64 bit windows 7



GDM@FUDAN


GDM@FUDAN



Experiments Setup

Dataset

Dataset |V| |E|

WordNet 82676 133445

DBLP 560851 1816613

Google 916427 4322051

Livejournal 4847572 42851237



GDM@FUDAN


GDM@FUDAN



Effectiveness It successfully unveils multiple research interests Example

• Jiawei Han • K=6

Jiawei Han

C1: multimedia data miningC2: stream data miningC3: information network



GDM@FUDAN


GDM@FUDAN



Effectiveness Our model is flexible to support different

parameters. Example

• Jiawei Han • K=9

Jiawei Han



GDM@FUDAN


GDM@FUDAN



Effectiveness

For most vertices, OCS model can find non-trivial results.



GDM@FUDAN


GDM@FUDAN



Performance OCS is more efficient than OCD.

Competitors:• LA

• <Identification of overlapping community structure in complex networks using fuzzy c-means clustering>

• OSLOM

• <Finding statistically significant communities in networks> Amortized time

• (Total time of OCD)/n



GDM@FUDAN


GDM@FUDAN



Performance: influence of parameters For the same k and , a smaller costs more time For the same k and , a smaller costs more time



GDM@FUDAN


GDM@FUDAN



Accuracy of approximate algorithm More than 70% accuracy can be consistently

achieved, in some cases almost 90% accuracy can be achieved



GDM@FUDAN


GDM@FUDAN






GDM@FUDAN


GDM@FUDAN



Diversity-based Social Network Analysis What is the distribution of diversity? Can we find people with really large diversity?



GDM@FUDAN


GDM@FUDAN



Name disambiguation Ambiguous names with a significant number of

entities also have a large number of communities.

Real person’s communities is smaller than these ambiguous names.



GDM@FUDAN


GDM@FUDAN



Contributions Problem definition Model Guide for parameter selection Algorithms Extensive experiments and applications



GDM@FUDAN


GDM@FUDAN



Q&A

Thank you!

1/52 Overlapping Community Search Graph Data Management Lab, School of Computer Science...

Documents

Transcript of 1/52 Overlapping Community Search Graph Data Management Lab, School of Computer Science...