Post on 06-Apr-2018
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
1/32
Thiago Christiano Silva
Liang Zhao
Roberto Alves Gueleri
Institute of Mathematics and Computer Science
University of So Paulo, So Carlos, So Paulo, Brazil
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
2/32
Summary Introduction
Complex Networks
Communities
Competitive Learning
Prior Related Work
Proposed Technique Description of the Technique
Mathematical Analysis of the Model
Time Complexity Analysis of the Model
Parameter Sensitivity Analysis
Computer Simulations Artificial Data Sets
Real-world Data Sets
Conclusions
Future Works
2
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
3/32
Complex Networks Interest shifting away from small to large-scale networks
Ubiquitous in Nature and everyday life
E.g.: WWW, Biological Neural Networks, Social Networks, Food
Webs, Metabolic Networks, Electrical Energy
Complex network representation unifies the structure,
dynamics and functions of a system which it represents
Inherent ability to describe the topological structure of the
original system
Presence of structures known as communities
- S. Fortunato, Community detection in graphs, Physics Reports 486 (3-5) (2010) 75174.
- M. Newman, The structure and function of complex networks, SIAM Review, vol. 45, no. 2, pp. 167256,
2003
- G. Palla, I. Dernyi, I. Farkas, and T. Vicsek, Uncovering the overlapping community structure of complex
networks in nature and society, Nature, vol. 435, pp. 814818, 2005. 3
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
4/32
Communitiesy A group of vertices with high concentrations of edges
within this group, and low concentrations between
different groups
4
- M. Girvan, M.E.J. Newman, Community structure in social and biological networks, Proc. Natl. Acad. Sci. USA 99 (12)
(2002) 78217826
- A. Lancichinetti, S. Fortunato, F. Radicchi, Benchmark graphs for testing community detection algorithms, Phys. Rev. E78 (4) (2008) 046110
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
5/32
Competitive Learningy Observed in nature and in many social systems sharing limited
resources
y Water, food, mates, territory, recognition, etc.
y Important field of Machine Learning
y Widely implemented in neural networks
y Several real-world applications
y Early works include:
y Self-organizing maps (SOM)
y Differential Competitive Learning
y Adaptive Resonance Theory (ART)
5
- T. Kohonen, The self-organizing map, Proceedings of the IEEE, vol. 78, no. 9, pp. 14641480, 1990.
- B. Kosko, Stochastic competitive learning, IEEE Trans. Neural Networks, vol. 2, no. 5, pp. 522529, 1991.
- S. Grossberg, Competitive learning: From interactive activation to adaptive resonance, Cognitive Science,
vol. 11, pp. 2363, 1987.
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
6/32
Prior Related Worky Originally proposed by Quiles et Al. in the unsupervised
learning approach
y Several particles walk in the network and compete with
each other to mark their own territory, while attempting
to reject intruder particles
y Each particle can perform:
y Random Walk
y Deterministic Walk
y Only a procedure of particle competition is introduced
without formal definition
y Only applied to community detection tasks
6
- M. G. Quiles, L. Zhao, R. L. Alonso, and R. A. F. Romero, Particle competition for complex network community
detection, Chaos, vol. 18, no. 3, p. 033107, 2008.
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
7/32
Contributions of the Proposed Techniquey A new type of competitive learning mechanism inspired
by the work in Quiles et Al.
y Unlike the original model, this is applied to the semi-
supervised learning approachy A particle-cooperative mechanism is introduced
y Here, the particle competition is formally represented by
a stochastic dynamical system
y We have applied the model for data classification
y The competitive walking process reaches dynamics
equilibrium when each class is dominated by a single or
team of particles
7
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
8/32
Description of the Techniquey
8
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
9/32
Notationy
9
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
10/32
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
11/32
Particles Movement Policyy
11
Active Term Exhausted
Term
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
12/32
Description of Each Movement Termy
12
RANDOM TERM
y Adventurous Behavior
y Does not take into account
the dominated vertices
PREFERENTIAL TERM
y Defensive Behavior
y Prefers visiting vertices with
high domination levels
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
13/32
13
Description of Each Movement Term
13
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
14/32
Stochastic Dynamical System
y
14
OBS.:One can see that the proposed dynamical system is Markovian, since it
only depends on the present state to completely define the immediate future state
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
15/32
Initial Conditionsy
15
y All particles start with the same energy level given by:
y All particles start in the active mode:
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
16/32
16
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
17/32
Mathematical Analysisy
17
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
18/32
18
2
4
3
113
14
176
5
7
8
9
15
16
18
19
20
10
11
12
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
19/32
Time Complexity Analysisy
19
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
20/32
20- L. Danon, A. Daz-Guilera, J. Duch, and A. Arenas, Comparing community structure identification, J. Stat. Mech., p.P09008, 2005.
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
21/32
Parameter Sensitivity Analysisy
21
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
22/32
Parameter Sensitivity Analysisy
22
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
23/32
23
Computer Simulations
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
24/32
2424
Computer Simulations
Particles are purposefully
inserted into the worstcase scenario at the
beginning (all in one
community)
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
25/32
2525
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
26/32
Real-world data sets (Chapelles Benchmark)
26
- O. Chapelle, B. Schlkopf, and A. Zien, Eds., Semi-supervised Learning, ser. Adaptive computation and machine
learning. Cambridge, MA, USA: MIT Press, 2006.
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
27/32
Some competing techniques
27
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
28/32
28
Only 10 pre-labeled samples
All parameters have been optimized using the MATLAB Genetic
Toolbox
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
29/32
29
Only 100 pre-labeled samples
All parameters have been optimized using the MATLAB Genetic
Toolbox
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
30/32
Large-scale multi-class data sety
30
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
31/32
Conclusionsy We have proposed a semi-supervised technique based on
competitive learning
y A rigorous definition has been provided using a nonlinear
stochastic dynamical system
y A mathematical analysis has been carried out
y A parameter sensitivity analysis has been conducted
y Computer simulations have been performed and
satisfactory results have been obtained
y More importantly, this work is an attempt to provide an
alternative way to the study of competitive learning
31
8/3/2019 Presentation ENIA 2011: Semisupervised Learning in Complex Networks
32/32
Future Worksy Study the overlapping characteristics of the vertices or
sub-graphs in the network
y Tackle the problem of data classification reliability
y Usually, some pre-labeled samples are mislabeled
y We will study this case and will attempt to avoid the
propagation of these false labelsy This is an important issue, since the task of labeling involves
human efforts: hence, susceptible for errors
32