Post on 21-Dec-2015
Linear and Non-Linear ICA-BSS
• I C A -------- Independent Component Analysis • B S S -------- Blind Source Separation
Carlos G. PuntonetDept.of Architecture and Computer TechnologyCircuits and system for information processing groupUniversity of Granada (Spain)
The Problem of “linear” blind separation of p sources:
Original signals: s(t)=[s1(t),....,sp(t)]T
Mixture: e(t)=[e1(t),...,ep(t)]T
Mixture matrix: A(t) pxp
The goal is to estimate A(t) by means of W(t) such that the output vector, s*(t) is:
s(t)W1(t)e(t)
W(t)A(t)PD
e(t)A(t)s(t)
*( ) ( )s t s t
REAL APLICATIONS
BSS is Independent Component Analysis (ICA)
Noise Elimination in general
Speech Processing (Cocktail Party, Noise environment,...)
Sonar, Radar
Sismic waves
Preprocessing recognition
Image Processing
Biomedicine (ECG, EEG, fMRI,...)
Geometric methods I: Digital
* Binary SignalsBinary Signals
S1u = (1,...,0,...,0)t
..................... Si
u = (0,...,1,...,0)t
..................... Sp
u = (0,...,0,...,1)t
The image of a base vector Siu is the vector Aoi, i.e.
the column i of the unknown mixture matrix Ao:
h(Siu) = Aoi
Geometric methods II: Slopes
Fij(t)ei(t).ej(t)1 ej(t)0,i,j{1,...,p}
Fij/sk 0Fijaik/ajk i,j,k{1,...,p}
S(0,0,...,sl,...,0,0)T l{1,...,p}
For input Vectors:
Slope Function:
Extreme values:
aij = min { ei . ej-1 } ej > 0 œi,j{1,..,p}
** Fast method for p=2 signals
** Valid for random or bounded sources
** Slopes are the independent components
** Modifiable for p>2
GENERAL p-DIMMENSIONAL METHOD
11 1 1j j 1p p
i1 1 ij j ip p
p1 1 pj j pp p
... ... a s a s a s... ... ... ... ...
... ... W = a s a s a s... ... ... ... ...
... ... a s a s a s
Obtained matrix W:
p-1 p
i j i j k ki=1 j=i+1
cos , ) > cos ( , ) , k {1,...,p}v v w w v wp-1 p
i=1 j=i+1
= (
For p points verifying minimum value of:
** Geometric method for p signals
** Valid for random or bounded sources
** Slopes are the independent components
** No order statistics
** Probability of obtaining p points close to the hiper- parallelepiped edges ?
Geometric methods III: Speech
- For Linear mixtures
- Unimodal p.d.f.’s (non-uniform’s)
- Detection of max.density points in the mixture space
- Normalization and detection in the sphere radius-unit.
ICA COMPONENTS FROM KURTOSIS:
• If K(e1)>0 and K(e2)>0 Maximum Density Zones
• If K(e1)<0 or K(e2)<0 Border Detection
Separation of Sources usingSimulated Annealing and Competitive Learning
Univ. Regensburg and Univ. Granada
- New adaptive procedure for the linear and “non-linear” separation - Signals with non-uniform, symmetrical probability distributions- Simulated annealing, competitive learning, and geometric methods- Neural network, and multiple linearization in the mixture space- Simplicity and rapid convergence - Validated by speech signals or biomedical data.
Geometric methods IV: Heuristic + Neural networks
e1
e2
w4 (D)
w3 (D)
w2 (D)
w1 (D)
D1
D2
D3
D4
eD (t)
Observation space with n p-spheres (n=4, p=2)
k1<||e(t)||<k 00k{1,...,n}Space Quantization:
d(i,k)||wi(k,t)e(k,t)||i{1,...,2p}k{1,...,n}
wi(k,t1)wi(k,t)(k,t)sgn[e(k,t)wi(k,t)]Ki(t)Ki(t)exp(1(t)||wi(k,t)wi(k,t)||2) ii{1,...,2p}
W ρk
w1 1 ρk... w1 p ρk
wp 1 ρk... wp p ρk
k {1,...,n }
Competitive Learning:
Simulated Annealing:
EEij(t)<(cum22[si(t)sj(t)])2> i,j{1,...,p}
Energy Function:
Fourth-order cumulant :
Wsijk(t)2rij1 i,j{1,...,p}ijk{1,...,n}Wsijk(t)2rij1 i,j{1,...,p}ijk{1,...,n}
Weights generation:
2 2 2 2 222 ( ( ), ( )) ( ) ( ) ( ) ( ) 2 ( ) ( )i j i j i j i jcum s t s t s t s t s t s t s t s t
Simulated Annealing and Competitive Learning
Wijk(t1)Wsijk(t)(t)Wcijk(t)(1(t))ij{1,...,p}k{1,...,n}
1
0
time SA CL
1
2
e (t)
e1
e2
NON-LINEAR: Contour for where the mixture can be considered linear
( ) ( )
( ) ( )
w ( , ) w ( , )( ) , { ,..., } { ,..., }
w ( , ) w ( , )
( ) { (1) (2) ... ( ) ( ( ), ) ( ( ), )} {1...2 }
k
j i k j j ki
j j k j j k
k k
t tt i j p k n
t t
j p d j d m m p m j
W
1
1
1 1
e 1
e 2
x 3 component
Simulation 1: 3 signals
e 1
e 2
W S21
W S12
W S13 / W
S23
e 1
e 2
W21
W12
W13 / W23
s 1
s 2
e 1
e 2
Simulation 2: Non-linear mixture of 2 digital 32-valued signals
e 1
e 2
w4 (4)
w2 (4)
w1(4)
w3 (4)
e1(t)2sgn[x1(t)]x1(t)21.1x1(t)x2(t)e2(t)2sgn[x2(t)]x2(t)21.1x2(t)x1(t)
Wρ(1)1 1.7
1.6 1Wρ (2)
1 0.25
0.22 1
Wρ(3)1 0.2
0.22 1Wρ(4)
1 0.1
0.15 1
Simulation 3: EEG signals
Eye blink -->
Low wave 1 -->
Musc. Spik. -->
Low wave 2 -->
Cardi. Contam. -->
Neural network for the separation
s1
s2
sp
e2
ep
e1
Wp p-1
W12 W
21W
p1
W2p
W1p
1 1 1
1...
s ( 1) e ( , ) W ( ) s ( ) { } { }ki i k ij j
j p
t t t t i p i j k n
GENETIC ALGORITHMS FOR NON LINEAR ICA
)()( tAsFtx
n
jjjiji txgwty
1
))(()(
12
1
)(
k
j
P
kjkjj xgxg
n
j
kj
P
kjkiji xgwy
1
12
1
Genetic Algorithms are one of the most popular stochastic optimisation techniques. Inspired by natural genetics and the biological evolutionary process:
* A scheme for encoding solutions to a problem in the form of a chromosome (chromosomal representation).
* An evaluation function which indicates the fitness of each chromosome relative to the others in the current set of chromosomes (referred to as population).
* An initialisation procedure for the population of chromosomes.
* A set of parameters that provide the initial settings for the algorithm: the population size and probabilities employed by the genetic operators.
*The GA evaluates a given population and generates a new one iteratively, with each successive population referred to as a generation, from genetic operations: reproduction, crossover and mutation.