LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG]...

57

Transcript of LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG]...

Page 1: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

�� � ��� � � �� � � �� � � � � � �� �� � � � �� � �� � � � �� � � � �� � � � � ���� � � � � � � � �� � �� � � � �� � � �� � � �� � �� � � � �� � � � � � �� � � � � � � � � � � � � � � � � � � � �

� �� � � !"# ! "

$ %& '( ) *+ & ,

- . . /10 2 23 3 34 5 678 94 : 7 2<; =1>? 7@ ? > 7

( (A BC D DE

Page 2: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� ��

� � � � �� � � � � � �� �� � �

& � � ! � � " �� � � ��� " � ! �# � �� � ! �# � � � ) � � � � � ! �� � " �� � !# � " � !

� � �� � �� �� � � �� �

� �! � " �� � �� #�$ %� � &� $ #� � $ �' � �( � �� )� ! �

� γ* Ψ* � �( � $ ! ��$ !

� + � ( ( � �� ,� � � � * , � � "� � & �$ �� � " �! �

,- . # � � � ) � � � � � ! �

� /� ( � " "� �� ' �$ # � ��$ ! � �� " �! � ) "� !

� 0 � � �! ( � �1 "� ! � � �$ # � �� �2

& � � ! � � � " �� � � � ! � �3 � ! � � �� � � !# � �� � ! ��� " � ! � ,- . # � � � ) � � � � � ! �

� � � �$ � ! � � "� � �( � $ ! ��$ �� 4 � �� � � 5� $ 6( � � &�� 7 � � " �! � � ��$ �98 � $ � ( � :� $ $ � �� � �� ( � # � � �

� � � �$ � ! ! � � "8 � � � � � � �� %� " � �� � ��$ #� � �! �� ; "� � %� * �$ � * � � � ; �� ! ,< = ( � " � �* # "� ! ! � !

( � � � � � � � �� � � � � � " 3 � � � � �� � � ! � � � �" � � " ! � ! �� � � � �" ! � ! � � " � � 3 �� ! �

� 7$ � � � ) "1 ( � #� $ �� � " � $ ) �� "� & �� ! �� � # �� � � "�

� = * ,< = � � � � �� 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� �

> � %� � "? @ @A C BDCE

Page 3: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � � �� � � � � � � �� � �� �� �

* � " � � � 3 " � � � � �� � � � " � � �� # !

� � �� �� � � " �� $ � ! ! � # �� $ � �� ! � ) 5� �! x ∈ X 6 "� � � ! #� � � & � � �� ! y ∈ Y = {1, . . . , Q}

� � : � � �� 1 ! � ��2 �! �� $ #� �98 � $ #� � � "� � " �� �� �� � (X,Y ) 6 %� "� � � ! �� $ ! X × Y � �! �� � ) � � ! � � %� $ � � $ �

( � ! � � � �� � � � )� ) � " � � � P

� � � ) "1 ( � � "� "� � 5� �$ �� �� (X,Y )� ! � �$ #�$ $ � �

* ! �� � � � � � � � ��� � !

� Dm = ((Xi, Yi))1≤i≤m � m* � # � � $ � � " "�$ #�$ ! � � �� � �� #� � �� ! �� (X,Y ) �$ � � ��� $ �� $ �� !

� G � '� ( � " "� �� ' �$ # � ��$ ! g �� X �� $ ! RQ � F � '� ( � " "� �� � 1 & "� ! �� � � # �! ��$ f �� X �� $ ! Y ⋃ {∗} �

f(x) = argmax1≤k≤Q gk(x)� � f(x) = ∗ � $ #� ! �98 �2 ��� �

* ! ! ���� � �� ! " �� ! � � � �" !

� ` ' �$ # � ��$ �� �� � �� � ` (y, g(x)) = 1l{gy(x)≤maxk 6=y gk(x)} � ` (y, f(x)) = 1l{f(x)6=y} �� , � "� # � ��$ �98 � $ � ' �$ # � ��$ g∗( �$ �( �! � $ � ! � � G "� � �! �� �

R(g) = E [` (Y, g (X))] = P (f(X) 6= Y )

> � %� � "? @ @A � BDCE

Page 4: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � �� � � ���� � � � � � � � � � � �� � � � � � �

B 3� � � � �� � E �A � � � � �� � M � �� �� M �� � � � �� � RQ × {1, . . . , Q} � � � R �� � � � �� �

∀(v, k) ∈ RQ × {1, . . . , Q} , M(v, k) =

1

2

(

vk − maxl 6=k

vl

)

M(v, ·) = max1≤k≤Q M(v, k)

B 3� � � � �� � C � . � " � !# � � � ) � � � � � ! � ! g � " ��� !� !# � � ! (x, y) �

∀(g, x, y) ∈ G × X × Y , M(g, x, y) = M (g(x), y)

B 3� � � � �� � � � % � 3 " � � ! " � ∆ ! � ∆∗ � g = (gk)1≤k≤Q � � � � �� � X � � � RQ

� � � � � � �� ∆g = (∆gk)1≤k≤Q� � X � � � RQ� � � �� � � � � �

∀x ∈ X , ∆g(x) = (M (g(x), k))1≤k≤Q

� � � � � � �� ∆∗g = (∆∗gk)1≤k≤Q� � X � � � RQ� � � �� � � � � �

∀x ∈ X , ∆∗g(x) = (max {∆gk(x),−M (g(x), ·)})1≤k≤Q

> � %� � "? @ @A C BDCE

Page 5: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � �� � � ���� � � � � � � � � � � �� � � � � � �

∆#� � ( � "� #� ∆� � ∆∗ �� $ ! "� ! �2 � � � ! ! ��$ ! %� "� ) "� ! � � � � "� ! �� �2 � � �� � �� � � !

��2 � � R(g) = E[

1l{∆#gY (X)≤0}

]

B 3� � � � �� � C �& � � ! � # � " � ! � �� �� γ ∈ R∗+ � � � �! �� � 6( � � &� γ � g� �� � �� � � � � �

Rγ(g) = E[

1l{∆#gY (X)<γ}

]

=

X×Y

1l{∆#gy(x)<γ}dP (x, y)

�! �� � � ( � �� � �� � 6( � � &� γ �

Rγ,m(g) =1

m

m∑

i=1

1l{∆#gYi(Xi)<γ}

A �# � � � ! � ! �� � � � �� � �� 3 � � � !" � ∆#γ G

� � � γ ∈ R∗+ ! � � � πγ : R → [−γ, γ] "� ' �$ # � ��$ " �$ �� �� � ! � �� � �� � �� $ �� �� �

πγ(t) =! � &$ � (t) · min (|t| , γ)

∆#γ g =

(

∆#γ gk

)

1≤k≤Q, ∆#

γ gk = πγ ◦ ∆#gk, ∆#γ G =

{

∆#γ g : g ∈ G

}

> � %� � "? @ @A � BDCE

Page 6: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � �� � � � � � � � � � � � � ∆#γ G � � �� � � � � � � �� � � �� �

�� �� � � ε� � � � �� � � ε� � � � � �� � �98 � $ ! � � ! * � $ ! � ( ) "� E′ �98 � $ � ! �� #� � ! � � � � *( � �� � ��� � (E, ρ)

B 3� � � � �� � � �+ � # � " ! � � ! �� !" � " ! �

N (ε, E′, ρ) � � � �� � � � � � � � �� � � � � � � � � � � � �� ε� � �� �� � � � � � � �� E′ �� � +∞ �

N (p)(ε, E′, ρ) � � � ε� � � � �� � � � � � �� � � � � � � � � � � � �� � � � � E′ � � � � � � � ! � E′ �

> � %� � "? @ @A � BDCE

Page 7: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� �� � �� � � � � � � � � � � � � � � �� � � � � � � � � � �

�� � � � � � � � � � � � �� � � � � �� � �� � � � � � � � �

� � 3 � " � # !E �& � � ! � � " �� � � � - � � � �� �E � �E � �� �� F� � � � � � � � � � �� � � � � � �� � � �

X � � � � � � � � � �� � � �� � � N(

F , (Xi)1≤i≤n

)

� � � �� � � � � �� � � � ��� � � � � � � � � � � �

� � � � � � � � � � �� � � � � � � (Xi)1≤i≤n � δ ∈ ]0, 1[ �� � � �� � � � � � � �� � �� � � � � � � � � � 1 − δ� �

� �� � � � � � � � � � �� f � F � � �� � � � � � � � � � � � � � �� � � �� � � � � � � �

R(f) ≤ Rm(f) +

1

m

(

ln(

EN(

F , (Xi)1≤i≤2m

))

+ ln

(

4

δ

))

+1

m

ln(

EN(

F , (Xi)1≤i≤2m

))

� ! � "8 �� � � � � � � �� �� F ! � � "8 � # � � $ � � " "�$ (Xi)1≤i≤2m �

> � %� � "? @ @A BDCE

Page 8: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� �� � �� � � � � � � � � � � � � � � �� � � � � � � � � � �

�� � � � � � � � � � � � �� � � � � �� � �� � � � � � RQ

B 3� � � � �� � � ��� � ! �� )# 3 �" � ! dxn � � � � � �� � � � � �� xn = (xi)1≤i≤n ∈ Xn� � �� � ��

�� � � � �� � � � �� � � dxn� � � G � �� � � � �� � � � � � �

∀(g, g′) ∈ G2, dxn(g, g′) = max1≤i≤n

‖g(xi) − g′(xi)‖∞

, � � � N (ε,G, n) = supxn∈Xn N (ε,G, dxn)

� � 3 � " � # !C �& � � ! � � " �� � � � � � �� G �� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � �

RQ � � � � � � � � �� � � � �� � � � � � � � Q� � � � �� � � � � �� � � � �� � � � � � � Γ ∈ R

∗+ � δ ∈ ]0, 1[ �� �

� �� � � � � � � �� � �� � � � � � � � � � 1− δ� � � � � � � � � �� � � � � � � � � � � � γ � � � ]0,Γ]� � � �� � �

� � � � � � � �� g � G � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �

R(g) ≤ Rγ,m(g) +

2

m

(

ln(

2N (p)(

γ/4,∆#γ G, 2m

))

+ ln

(

γδ

))

+1

m

> � %� � "? @ @A E BDCE

Page 9: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� �� � � � �� � � �

B 3� � � � �� � � Ψ ) � �# !� � �� � � ��� !� ) B � � � �� �� � �E � � � � �� �� F� � � � � � � � � � �� � � � � �

�� � � � X � � � � � � � � � ��� � � � � � � {1, . . . , Q} � �� �� Ψ� � � � � � � � � � � � �� � � �� � ψ �

{1, . . . , Q} � � � {−1, 1, ∗}� � � � � � �� � ∗� �� � � � � � � � � �� �� � � � �� �� � �

� � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X � � � �� � �� Ψ* � � " % �� �! � � �� F� � � � � �� � �

� � � � �� � � �� ψn =(

ψ(i))

1≤i≤n

� � � Ψn � � � � � �� � � � � � � � � � � vy � {−1, 1}n� � � � �� � �

� � � �� fy � � � F� � � �� � �� � �

(

ψ(i) ◦ fy(xi))

1≤i≤n= vy

� � Ψ* � �( � $ ! ��$ � F� � � � Ψ� � � � (F)� � � � � � � � � � � �� � �� � �� � � � � � � � � � � � � XΨ� �� � � � �� � � �� F� � � � � � � � � � � � � � �� � �� � � � � � � � � � � � � �� � �� �

& !# � " !E �� � � F � Ψ � � � � � � � � � � � � �� � �� � � � � �� � � � � � � � � � � � � �� �� � �� ��

� �� � � � � �� � � � � � � � � � � � � �� � � � � � � � � � � � � �� � � � � � � � � � � � �� � �� � � � � � �

� � � � � � � � {−1, 1, ∗}� � � � �� � � � � �� � � � � � � � �� �� � � �� � �� � � � � � �� � � �� � � � � �

� � � � � � � �� � �

Ψ� � � � (F) = � � � � � ({(x, ψ) 7→ ψ ◦ f(x) : f ∈ F , ψ ∈ Ψ})

> � %� � "? @ @A � BDCE

Page 10: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � � � � � � � � �� �� � � � � � �� � � � �� � � �

B 3� � � � �� � E � B �# !� � �� � � " � � � � ! � B � � ! � �E �E �+ � � � " � � �� �E �E � � �� �� F� � � � � �

� � � � �� � � � � � �� � � � X � � � � � � � � � {1, . . . , Q} � � � � �( � $ ! ��$ &� � � � � �� � � F�

� � � � � (F)� � � �� Ψ� � � � � �� � F � � � � � � � � � � � �� � � � � � Ψ = {ψk : 1 ≤ k ≤ Q}� �� � � � � �� � � ��

ψk �� � � �� � � � � 1� � � � � � �� � � � � � � � � � k � �� � � � � −1 � � � � � � � � � �� � �� � �

� � � � � �� � � � � �� � �� �� � � � � � � � � � � �� � �� � � � � � � � �� � � �� �� � �� � � � � � � � � �� � ψk� � �

� � � � � �� � � � �� � �� �� � � � � � � � �� � � � �

B 3� � � � �� � � � B �# !� � �� � � ! + � � � " � � �� �+ � � � " � � �� �E �E � � � � �� F� � � � � � � � � � �� �

� � � � �� � � � X � � � � � � � � � {1, . . . , Q} � � � � �( � $ ! ��$ �� 4 � �� � � 5� $ � F� � � � � � (F)� � � ��

Ψ� � � � � �� � F � � � � � � � � � � � �� � � � � � Ψ = {ψk,l : 1 ≤ k 6= l ≤ Q}� �� � � � � �� � � �� ψk,l �� � �

�� � � � � 1� � � � �� �� � � � � � � � � � k� �� � � � � −1� � � � � � �� � � � � � � � � � l� � �� � � � � ∗

� �� � � � � � � � � � � � �

+� � �� $ � � ��$ �� "� � �( � $ ! ��$ &� � � � � �� � ! 8 �$ ! � �� � �� "� ( � �� � �� �� � � #�( � � ! � � ��$ � $ #�$ �� � �� � ! "�

� �� $ � � ��$ �� "� � �( � $ ! ��$ �� 4 � �� � � 5� $ ! 8 �$ ! � �� � $ � �� "� ( � �� � �� �� � � #�( � � ! � � ��$ � $ #�$ �� � � $ �

> � %� � "? @ @A E D BC E

Page 11: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � � � �� � � � �� �� �� � � � � � � � � �� �� � � � �� � � γ

B 3� � � � �� � E D � B �# !� � �� � � � � � ) �� � � � !" �� � � ��� ! � " � � � , �� � � �" ! �E � � C � �� �� G� � � � � �

� � � � �� � � � � X � � � � � � � � � � � �� � � � γ ∈ R∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X

� � � �� � �� γ* � � " % �� �! � � �� G� � � � � �� � � � � � � vb = (bi) ∈ Rn � � � � �� � � � � � � � � � �

vy = (yi) � {−1, 1}n� � � � �� � � � � � �� gy � � � G� � � �� � �� � �

∀i ∈ {1, . . . , n} , yi (gy(xi) − bi) ≥ γ

� � � �( � $ ! ��$ '� �* ! � � � �� � �$ & 6( � � &� γ� � � Pγ � �( � $ ! ��$ � � �� � � � � � G� Pγ� � � � (G)� � � �

� � � � � � � �� � �� � �� � � � � � � � � � � � � X γ� �� � � � �� � � � � G� � � � � � � � � � � � � � �� � �� � � � � � �

� � � � � � �� � �� �

> � %� � "? @ @A E E BC E

Page 12: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

γ� Ψ� �� � � � �� � � �

B 3� � � � �� � E E � γ ) Ψ ) � �# !� � �� � � � � � �� G� � � � � � � � � � �� � � � � � �� � � � X � � � � � �

� � � RQ � �� �� Ψ� � � � � � � � � � � � �� � � �� � ψ � {1, . . . , Q} � � � {−1, 1, ∗}� � � � � � �� � ∗

� �� � � � � � � � � �� �� � �� �� �� � � � � � � γ ∈ R∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} �

X � � � �� � �� γ* Ψ* � � " % �� �! � � Ψ* � � " % �� �! � � %� # � $ � ( � � &� γ � � � � ∆#G� � � � � �� � � � � � � �� � � ��

ψn =(

ψ(i))

1≤i≤n

� � � Ψn � � � � � � vb = (bi) � Rn � �� � � �� � � � � � � � � � � vy = (yi) �

{−1, 1}n� � � � �� � � � � � �� gy � � � G� � � �� � �� � �

∀i ∈ {1, . . . , n} ,

� � yi = 1, ∃k : ψ(i)(k) = 1 ∧ ∆#gy,k(xi) − bi ≥ γ

� � yi = −1, ∃l : ψ(i)(l) = −1 ∧ ∆#gy,l(xi) + bi ≥ γ

� � γ* Ψ* � �( � $ ! ��$ � � � Ψ* � �( � $ ! ��$ 6( � � &� γ� � ∆#G� � � � Ψ� � � � (∆#G, γ)� � � � � �� � � � � ��

� �� � �� � � � � � � � � � � � � X γ� Ψ� �� � � � �� � � �� ∆#G� � � � � � � � � � � � � � �� � �� � � � � � � � � � �

� � �� � �� � � � �� � �� $ � � ��$ ! � � � � � � � 6 #� " "� �� "� � �( � $ ! ��$ '� �* ! � � � �� � �$ & "� � ! �� � Q = 2 �

> � %� � "? @ @A E C BC E

Page 13: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � � � � �� � � � �� � �� � �� � � � � � � γ

B 3� � � � �� � E C � B �# !� � �� � � ! + � � � " � � �� � # � " � ! γ � �� �� G� � � � � � � � � � �� � � � � �

�� � � � X � � � � � � � � � RQ �� � � � γ ∈ R

∗+� � � � � � � � � � � sXn = {xi : 1 ≤ i ≤ n} � X � � � ��

� �� γ* 4 * � � " % �� �! � � � � �� � � � �� � � � � � � � � γ � � �� ∆#G� � � � � �� � � � � � �

I(sXn) = {(i1(xi), i2(xi)) : 1 ≤ i ≤ n}

� n� � � � � � � � � � �� � � �� � � � � � � � � {1, . . . , Q} � � � � � � vb = (bi) � Rn � �� � � �� � � � � � �

� � � � � � � �� vy = (yi) ∈ {−1, 1}n� � � � �� � � � � � �� gy � G� � � �� � �� � �

∀i ∈ {1, . . . , n} ,

� � yi = 1, ∆#gy,i1(xi)(xi) − bi ≥ γ

� � yi = −1, ∆#gy,i2(xi)(xi) + bi ≥ γ

� � � �( � $ ! ��$ �� 4 � �� � � 5� $ 6( � � &� γ � �� � � � � � ∆#G� � � � � � (∆#G, γ)� � � � � � � � � � � �� � �� �

�� � � � � � � � � � � � � X γ� � � �� � � � �� � � � � ∆#G� � � � � �� � � � � � � � �� � �� � � � � � � � � � �

� � �� � �� �

> � %� � "? @ @A E � BC E

Page 14: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� �� � � � � � � � � � � � �� � � � � � � � �

� � � � � � � � � � � � � � �� � � � � � X � � � � RQ

$ !# # !E � � �� G� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � [−MG ,MG ]Q � � � � �

� � � � � � � � � � ε � � � ]0,MG ] � � � � � � � � � � � �� � n� � � �� � �� � � n ≥ � � � � � (∆G, ε/6)� �

� �� �� � � �� �� � � � � � � �

N (p)(ε,∆∗G, n) < 2

(

n Q2(Q− 1)

3MG

ε

⌋2)

l

d log2

enC2Q

2j

3MGε

k

−1”

/d”m

� d = � � � � � (∆G, ε/6) �

+� � � � � %� $ 8 � ! � � "� ! %� "� ) "� ! � "8 � � �� � �� � � ∆∗� ! � � � ( � "� # � �� � "8 � � �� � �� � � ∆ �

> � %� � "? @ @A E C BC E

Page 15: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � &� � � $ � � � � � � "� ! ( � �1 "� ! ( � " � �* # "� ! ! � ! 6 &� � $ �� ( � � &�

� � �� � � � �� � � � � � � �� � � � � � � � � � � �

� � 3 � " � # ! � � � �� G �� � � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � [−MG ,MG ]Q �

� � � � � � � �� � � � �� � � � � � � � Q� � � � �� � � � � �� � � � � � � � �� �� δ ∈ ]0, 1[ �� � � �� � � � � � � �� � ��

� � � � � � � � � 1 − δ� � � � � � � � � �� � � � � � � � � � � � γ � � � ]0,MG ]� � � �� � � � � � � � � � ��

g � G � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �

R(g) ≤ Rγ,m(g)+√

2

m

ln

4

(

2m Q2(Q− 1)

12MG

γ

⌋2)

l

d log2

emQ(Q−1)“

2j

12MGγ

k

−1”

/d”m

+ ln

(

2MG

γδ

)

+

1

m

� d = � � � � � (∆G, γ/24) �

R(g) ≤ Rγ,m(g) + c ln (m)

d

m

� " � ��� � � � �� � E � * � � !" � !� � ! � � " ! � ! �� " !# !� � � � �� "# ! � �lim

m→+∞supP

P

(

supn≥m

supg∈G

(R(g) −Rγ,n(g)) > ε

)

= 0 limm→+∞

supP

P

(

supn≥m

supg∈G

|Rγ(g) −Rγ,n(g)| > ε

)

= 0

> � %� � "? @ @A E � BC E

Page 16: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

�� � � � � �� � � � � � � � � � � � �� � � � � � � �� � � � �

A �# � � � ! � ! �� � � � �� � � � ! � � � !

, � �� $ � κ� $ $ � :� � ! :( � �� � ��� � ! � ( �* � �� $ � � � ! � � �' ! � � X� � (Hκ, 〈·, ·〉Hκ) "� � � , #� � � � ! � �$ �� $ �

, � �� $ � H = (Hκ, 〈·, ·〉Hκ)Q� � H = ((Hκ, 〈·, ·〉Hκ

) + {1})Q

H � '� ( � " "� �� ! ' �$ # � ��$ ! h = (hk)1≤k≤Q �� X �� $ ! RQ �� " "� ! �� � �

∀k ∈ {1, . . . , Q} , hk(·) =

mk∑

i=1

βikκ(xik, ·) + bk

� %� # {xik : 1 ≤ i ≤ mk} ⊂ X (βik)1≤i≤mk∈ R

mk� � bk ∈ R � �$ ! � ��� � "� ! " �( � �� ! �� #� ! ' �$ # � ��$ !

"� � ! �� � "� ! � $ ! � ( ) "� ! {xik : 1 ≤ i ≤ mk} �� % �� $ $ � $ � �� $ ! � ! �� $ ! X� � ! � $ ! �� "� $ � �( � �$ � � � �� �� �

"� $ � :� �

A �# � � � ! � ! �� � � � �� � � " 3 � � � � � � � ! �

, � � ! * � $ ! � ( ) "� #�$ %�2 � �� H � � �� $ � �� � �� ! #�$ �� � �$ �� ! ! � � � $ ! � � ! * � ! �� #� � �$ � �

> � %� � "? @ @A E � BC E

Page 17: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

� � � � � � � � � � � � � �� � � � � � � � � �

� ( � � � ! � � � � � �

� � � �� � �$ � :� � �� = � � #� � κ � " �2 �! �� � $ � ' �$ # � ��$ Φ �� " "� �� � �

∀(x, x′) ∈ X 2, κ(x, x′) = 〈Φ(x),Φ(x′)〉

�� 〈·, ·〉� ! � "� � � � � � � � ! #� "� �� � �� "8 � ! �� #� `2 � , � � � Φ (X ) = {Φ (x) : x ∈ X}

� � � � � � � �� � � � � � �� � "8 � $ �� ! � ! �� #� ! �� � � " )� � � (EΦ(X ), 〈·, ·〉)� $ &� $ �� �! �� � "� ! Φ (X )

=⇒ H ��� � � � �� � #�$ ! � � �� �� #�( ( � � $ � '� ( � " "� �� ' �$ # � ��$ ! � �$ � ! ( � " � � %� � � �� ! ! � � Φ (X )

h(·) = (〈wk, ·〉 + bk)1≤k≤Q

w = (wk)1≤k≤Q ∈ EQΦ(X ) b = (bk)1≤k≤Q ∈ R

Q

+ � "# ! � � " H ! � EQΦ(X )

∥h∥

H=

Q∑

k=1

∥hk

2

Hκ=

Q∑

k=1

〈wk, wk〉 =

Q∑

k=1

‖wk‖2 = ‖w‖

‖w‖∞ = max1≤k≤Q

‖wk‖

> � %� � "? @ @A E BC E

Page 18: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

Q ≥ 3 �� � � � � � � � � � � � � �� � �� � � � �� � � ���� � � � � � � � � �� � � � �

((xi, yi))1≤i≤m ∈ (X × {1, . . . , Q})m � � � � � � � � � �� � �� � � �

` �� �� � � ' �$ # � ��$ �� �� � �� #�$ %�2 � � #�$ ! �� � � �� � � �� � � �� "� ' �$ # � ��$ �� ��� � �� # � � �$ �1 � � �

. ) ,- . � �� � � �� � � � � � " � � �� # ! � ! � " � � " �# # � � �� � �� � !� ! � � � " � � � ! �

� " � � �� # !E

minh∈H

{

m∑

i=1

` �� � � (yi, h(xi)) + λ‖h‖2H

}

� �� �

∑Qk=1 hk = 0

� � 3 � " � # ! � ! " ! � " 3 � !� � � � �� �

� � � �� � 1 ( � � �� ) " � � �� � "8 � � � � � $ � �! ! � &� � "� � �! � "� � ��$ � � � � � ) "1 ( � � � � � % �� $ � 6 �� � � %� � "� ! %� "� � � !

�� ! #� � � # �� $ �! βik �� $ ! �

∀k ∈ {1, . . . , Q} , hk(·) =m∑

i=1

βikκ(xi, ·) + bk

� "� ! %� "� � � ! �� ! ; ) �� �! ; bk! 8 � $ � � � � �! � $ � �� � � � � " � #� � ��$ �� ! #�$ � � � ��$ ! �� � � � $ * 0 � # � � �

> � %� � "? @ @A E E BC E

Page 19: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

�� �� � � � � � � � � � � � � � �� � � �

( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !

� " � � �� # !C � . ) ,- .E � - � � � �� � � � ��� �E � �E ��� ! � �� � � � � � � �� � �E � �E ��� � � �

minh∈H

1

2

Q∑

k=1

‖wk‖2 + C

m∑

i=1

k 6=yi

ξik

� �� �

〈wyi− wk,Φ(xi)〉 + byi

− bk ≥ 1 − ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

> � %� � "? @ @A E � BC E

Page 20: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

�� �� � � � � � � � � � � �� � � � � � �� � � �� � �

. � " � ! � � 3 � # 3 �" � ! �d �� � � � = min

1≤k<l≤Q

{

min

[

mini:yi=k

(hk(xi) − hl(xi)) , minj:yj=l

(hl(xj) − hk(xj))

]}

∀(k, l), 1 ≤ k < l ≤ Q,

d �� � � � ,kl =1

d �� � � �

min

[

mini:yi=k

(hk(xi) − hl(xi) − d �� �� � ) , minj:yj=l

(hl(xj) − hk(xj) − d �� � � � )

]

∀(k, l), 1 ≤ k < l ≤ Q, γkl = d �� �� �

1 + d �� � � � ,kl

‖wk − wl‖

$ � !� !� �" ! � ! � 3 � � � � � � � ! " ! � � ! �# � " � ! � � 3 � # 3 �" � ! �

k<l

‖wk − wl‖2 = Q

Q∑

k=1

‖wk‖2 −∥

Q∑

k=1

wk

2

∧Q∑

k=1

wk = 0 =⇒

Q∑

k=1

‖wk‖2 =d2�� �� �

Q

k<l

(

1 + d �� � � � ,kl

γkl

)2

> � %� � "? @ @A C D BC E

Page 21: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

�� � � � � � � � � � � � � �� � � �

( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !

� " � � �� # ! � � . ) ,- .C � * " �# # !" � , �� � !" �C D DE �

minh∈H

{

1

2

Q∑

k=1

‖wk‖2 + Cm∑

i=1

ξi

}

� �� � 〈wyi− wk,Φ(xi)〉 + δyi,k ≥ 1 − ξi, (1 ≤ i ≤ m), (1 ≤ k ≤ Q)

( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � � � !

, � �� $ � αi. = (αik)1≤k≤Q δyi,. = (δyi,k)1≤k≤Q τi. = (τik)1≤k≤Q = Cδyi,. − αi.� �

τ = (τik)1≤i≤m,1≤k≤Q

� " � � �� # ! C � . ) ,- .C �

minτ

1

2

m∑

i=1

m∑

j=1

τTi. τj.κ(xi, xj) −

m∑

i=1

τTi. δyi,.

� �� �

τik ≤ Cδyi,k, (1 ≤ i ≤ m), (1 ≤ k ≤ Q)

1TQτi. = 0, (1 ≤ i ≤ m)

> � %� � "? @ @A C E BC E

Page 22: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

�� �� � � � � � � � �� � � � � � � � �

( � �� " � �� # ! � � � � � " !� � � � � � � ! ) �� "# � � � �� � � " �# � � !

� " � � �� # ! � � . ) ,- . � � $ ! ! �� �� � �C D D C �

minh∈H

1

2

Q∑

k=1

‖wk‖2 + C

m∑

i=1

k 6=yi

ξik

� �� �

〈wk,Φ(xi)〉 + bk ≤ − 1Q−1 + ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)∑Q

k=1 wk = 0,∑Q

k=1 bk = 0

& 3 � � � � � � ! �� � � � � � �� � ! ��� � �� � �C D D C � � !� � " � � � � " � � ! � � �C D D �

� � �� = * ,< = � ! � "� ! � � "� � �$ � "8 � � � � � $ � �! ! � &� ! � � �� � :� ! * #�$ ! �! �� $ � �

> � %� � "? @ @A C C BC E

Page 23: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

� �� � �� � � � �� � � �

� " � � �� # ! � � . ) ,- . � ! � ! � �� � ! � � � � � �� � � � � � � �� � � � � � "# ! ‖ · ‖∞ �

minh∈H

1

2t2 + C

m∑

i=1

k 6=yi

ξik

� �� �

〈wyi− wk,Φ(xi)〉 + byi

− bk ≥ 1 − ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

ξik ≥ 0, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

‖wk‖ ≤ t, (1 ≤ k ≤ Q)

Mξ =((

δk,l − 1Q

)

δi,j

)

1≤i,j≤m,1≤k,l≤Q

� " � � �� # ! � . ) ,- . � ! $ ! ! � $ �� ! � � � � � � �� �� � � � � " � � � ! � �

minh∈H

{

1

2

Q∑

k=1

‖wk‖2 + CξTMξξ

}

� �� �

〈wk,Φ(xi)〉 + bk ≤ − 1Q−1 + ξik, (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)

∑Qk=1 wk = 0,

∑Qk=1 bk = 0

> � %� � "? @ @A C � BC E

Page 24: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

� �� � � � �� � � � ��� � � � �� � � � � �� � � � � �

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

x_2

x_1

� � � � ? � � #� � � & � � �� ! " �$ �� �� � ( � $ � ! � �� � � ) "� ! �� $ ! R2

> � %� � "? @ @A C C BC E

Page 25: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� ,< = ( � " � �* # "� ! ! � !

� �� � � � �� � � � ��� � � � �� � � � � �� � � � � �

� � � � � � � : ��� � � "� $ ! ! � �� � � �� � � ! � �( � � &� ! � � � #� ! �� "� = * ,< = " �$ �� �� �

> � %� � "? @ @A C � BC E

Page 26: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� � � � � �� � � � �� � �� � �� � � � � � � � � � �� � � �

� � 3 � " � # ! C � � �� H �� � � � � � � � � � � �� � � � � � �� � � � � � � � � � � � � � � Q� � � � �� � � � � � � �

���� � �� �� � � � Φ (X ) � � � � �� � � � � �� �� � � � � � �� ΛΦ(X )� �� � � � � ��� � � � � � EΦ(X )� � �

� � � � w � � � � ‖w‖∞ ≤ Λw � � b = 0 �� �� � � � �� � � � � � � ε ∈ R∗+�

� � � � �(

∆H, ε)

≤ C2Q

(

ΛwΛΦ(X )

ε

)2

+� � � � � %�

� $ 8 � ! � � "� ! %� "� ) "� ! � "8 � � �� � �� � � ∆� ! � � � ( � "� # � �� � "8 � � �� � �� � � ∆∗

� ! 8 � � � � �� � �� � # �� ( � $ � ! � � "� � � �$ # � ��� �� � � #�( � � ! � � ��$ � $ #�$ �� � � $ � � "� � � �$ # � ��� �� ! � �� � �� !

� � � �� " "� "8 � � � " �! � � ��$ �� "� $ � �( � ‖ · ‖∞� �$ �$ #� " "� �� "� $ � �( � ‖ · ‖ �� � � " �! �� �� � "� � �$ � " �! � �� � � �

Q = 2 : Pε* � �( (Hκ) ≤(

ΛwΛΦ(X )

ε

)2

> � %� � "? @ @A C � BC E

Page 27: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� �� � � �� �� � � �� � � �� � � � � � � � � � � � �� � � � �

B 3� � � � �� � E � � . � � !� � ! � ! & � � !# � �� !" � � � � � n ∈ N∗� � � � � A� � � � � �� � � �

� � � � � a = (ai)1≤i≤n � � � � � � � � � Rn � (σi)1≤i≤n� � � �� � � � � � � � � � � � � ( � :� $ $ � ��

� �� ( � # � � � � ! ! � # � �� 6 A� Rn(A)� � � �� � � � � � �

Rn(A) = E supa∈A

1

n

n∑

i=1

σiai

� � 3 � " � # ! � � '� 3 � � � � � 3 � ! � � �� 3 " !� � ! � �� " � 3 ! � � . � B � � "# � � �E �E � � � � � � n ∈ N∗� � � ��

(Ti)1≤i≤n� � � �� � n � � � � � � � � �� � � � �� � � �� � � � � � � � � � � � � � � � � � � � T � �� �� g

� � � � �� � T n � � � R � � � � � � � � �� � � � � �� � � � � � � � � �� � �� � � (ci)1≤i≤n � � � � � � �

∀i ∈ {1, . . . , n} , sup(ti)1≤i≤n∈T n,t′i∈T

|g(t1, . . . , tn) − g(t1, . . . , ti−1, t′i, ti+1, . . . , tn)| ≤ ci.

� �� � � � �� � � � � � � τ ∈ R∗+� �� � � � � � � � �� � � � �� g (T1, . . . , Tn)� � � �� � �� �

P {g (T1, . . . , Tn) − Eg (T1, . . . , Tn) > τ} ≤ e−2τ2

c

P {Eg (T1, . . . , Tn) − g (T1, . . . , Tn) > τ} ≤ e−2τ2

c

� c =∑n

i=1 c2i �

> � %� � "? @ @A C BC E

Page 28: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� �� � � �� �� � � �� � � �� � � � � � � � � � � � �� � � � �

& � � ! � # � " � ! �� � !� �� 3 � � �� � �3 � � � . ) ,- . � ! * " �# # !" ! � , �� � ! "

R(h) = E[

(1 − ∆hY (X))+]

� � 3 � " � # ! � � � �� H �� � � � � � � � � � � �� � � � � � �� � � � � � � � � � � � � � � Q� � � � �� � � � � � � �

���� � �� �� � � � Φ (X ) � � � � �� � � � � �� �� � � � � � �� ΛΦ(X )� �� � � � � ��� � � � � � EΦ(X )� � �

� � � � w � � � � ‖w‖∞ ≤ Λw � � b = 0 � �� �� KH = ΛwΛΦ(X ) + 1 �� � � �� � � � � � � �� � �� � � � �

� � � � � 1 − δ� � � �� � � � � � � � � � �� h � H � � �� � � � � � � � � � � � � � �� � � � �� � � � � � �

R(

h)

≤ Rm

(

h)

+4√m

+4Q(Q− 1)Λw

m

m∑

i=1

κ (Xi, Xi) +KH

ln(

)

2m

R(

h)

≤ Rm

(

h)

+O

(

1

m

)

> � %� � "? @ @A C E BC E

Page 29: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� � � � � � � � �� � � � � �

� � 3 � " � # ! �- � � � �� �E � �E � � � � �� � � � � � � � � �� � �� � � � � �� � �� � � �� �� Lm� � � � ��

� � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � � � � � � γ = 1‖w‖� � � �� � �� � � � �� � � � � �

��� � � � � � � � � �� � �� � � � � � � � � � � �� � � � � �� � � � � Lm �

Lm ≤ D2m

γ2

� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �

� � � �� � � �

> � %� � "? @ @A C � BC E

Page 30: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� � � � � � � � �� � � � � � � �� � � �� �� � � � � � � � � � � � � � �� � � �

d � � = d � � = 1

� � 3 � " � # ! E � � � �� � � � � � � � � � � � � � � � � � �� � � �� � � � � � � � � � � � � � Q

� � � � �� � � � � � � � � �� � � � � �� Lm � � � �� � � � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � �

� �

Lm ≤ K�Q

D2m

k<l

(

1 + dkl

γkl

)2

� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �

� � � �� � � �

* � � � � �� � ! K ��� +� %� "� � � �� K� � ! 8 � ) � �� $ � �� � � �! � "� � ��$ �� � � � &� � ( ( � ! �� � �� � � � �� � ! � $ $ �( ) � � � &� "� �

$ � ( ) � � �� %� # �� � � ! ! � � � � � �

� � � � Q = 2 K� � = 2 � � "� ) � �$ � ! � � � � � � � 6 "� ) � �$ � ;� � : �$ *( � � &� ; ) �* # "� ! ! �

> � %� � "? @ @A � D BC E

Page 31: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �! �� � ! &� � � $ � �! � � ! � "� # � ��$ �� ( � �1 "� � � � � "� ! ,< = ( � " � �* # "� ! ! � !

� � � � � � � � �� � � � � � � �� � � �� �� � � � � � � � �� � � � � � � � �

d � � � = QQ−1

� � 3 � " � # ! � � � � �� � � � � � � � � � � � � � � � �� � � � � � Q� � � � �� � � � � � � � � �� � � �� �� Lm

� � � �� � � � � � � � � � � � � � � �� � � � �� � � � � � � � � � � � � � � �

Lm ≤ D2m

k<l

(

1 + d � � � ,kl

γkl

)2

� Dm � � � � � � � � �� � �� � �� � � � �� �� � � � �� � � � � � � �� � � � � � �� � � � � � � � � � � � �

� � � �� � � �

� � �� ) � �$ � ! � � � � � � � � $ #� � � 6 "� ) � �$ � ) �* # "� ! ! � � � � � Q = 2 �

> � %� � "? @ @A �E BC E

Page 32: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

�� � �� � � � �� � � � � � �� � � � � � � � � � � � � � � � � � � � �

* � � � !� � ! � �� �� � � ! �2 � "� � �� � ��$ ' �$ # � ��$ $ � " "� �� ! �$ ' � �( � � ��$ ! � � � %� $ � $ � �� ! &� � $ � !

� � � &� � ( ( � ! �� ! � �� � $ �� &� �� ! & �$ �( � ! � �� ! ! � �� � "� #�$ $ � �! ! � $ #� �� "� ! �� � # �� � � � � �� ! � � � � � �$ � ! �

� � � � � � % �� ( � ! ! � %� �� ! � ��� � $ #� ! � � � � � � �� � ! � #� � �! ! � $ #� �2 � �$ � $ � �� " "� �� ! )� ! � ! �

�� �� � � �� �� � �� � � � � � � ��� � � �� � � �� � � � � �� �� � � � � � �� �� � ��

? � � � �� �( �$ � � ��$ �2 � �� �( � $ �� "� �� "� ! �� � # �� � � � � � � # � � �� 1 ! "� � � �� � � � "� � ! �� 8 � " "� � ! � � �� " �! � ) "�

=⇒$ � #� ! ! � � � �� �� ! ! � � �98 � $ � � � � � � # � � ) �� # � �( � ��� � 6 � $ � � � � � � # � � � � � � � # � � %�

� � ) "1 ( � #� $ �� � " � $ ) �� "� & �� ��� �( � � �� $ � �98 � ) � � �� �

"8 � ! ! � $ � �� " �� ! &� � $ �� ! �� � ! � ��$ ! � � %� � �� ! � $ �� � � �� ( � $ � �� � �$ $ �� ! ! � ��� � $ � �� " "� !

> � %� � "? @ @A �C BC E

Page 33: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � � � � � � �� � �� � � ��� � �� �� �� �� � � � � � � �� � � � � � � � � � � � � �

• , 3 !� � ! � � �" � � " ! � " �# � �" ! � 1.6 · 106 �3 !� � ! � �� � � ! � �

�� �� �� � �� � �� � � � � � � �� � � � � � �� � � � �4 4 4

• , �" � � " ! � ! �� � � � �" !

�� �� � � � �� � � � � �� � �� � � �� ��� � � � � � ��� � �� �� � � α � � � � � � � � � � �� � � β � � �� �� �� �

• , �" � � " ! � !" � � � �" ! � 2.7 · 104 � �" � � " ! � � B �� � � ! � �

> � %� � "? @ @A � � BC E

Page 34: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � � � � � � � � �� �� � � �� � �� � � � �� � � � � � �� � � �� � � � � � �

� � � �� � � � � �� � � ! � � �3 !� � ! � ! � !x = (xi)−n≤i≤n � %� # �� � � #� �� $ � � $ � � " : ��� � � � �� � #�$ �� $ � �98 � $ � '� $ � �� � �98 � $ � " : ! � �� �� � " "� 2n+ 1 �

κθ,G (x,x′) = exp

(

−n∑

i=−n

θ2i ‖xi − x′i‖2

)

� � � �� � � � � �� � � ! � � � � � � !# !� � �# � � � � � ! �

x = (xi)−n≤i≤n �� " �� � xi =∑22

j=1 θijaj � #� ( ) �$ � �! �$ #�$ %�2 � �

〈xi, x′i〉 = 〈

22∑

j=1

θijaj ,22∑

k=1

θ′ikak〉 =22∑

j=1

22∑

k=1

θijθ′ik〈aj , ak〉

> � %� � "? @ @A � C BC E

Page 35: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � � � � � �� � � � � �� � � � �� � � � � �� � � �� �� �� � � � �� � � � � � G �

� � � � �

� S �( � �� � #� �� ! � ) ! � � �� � ��$ � � � � �� 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � � $ ! �� �� �� � �

E �E � �> � %� � "? @ @A � � BC E

Page 36: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � � � �� � � �� � � � � S � � � � �� � � � � � � � � � �

� A = (aij) ∈ M22,22(R) �� � � � �! � $ �� � ��$ %� # �� � �� " "� �� ! � # � �� ! � ( �$ �! �� $ " � &$ � �� G = AAT �( � �� � #� �� ! � � � � � � �! ! #� "� �� � ! � � � � � �2 �( � � ��$ �� S �� � � $ � ( � �� � #� ! :( � �� � �� �

! � ( �* � �� $ �� � � ! � � � %�

� �� & �$ � " �! � � ��$ �� S �

S = PDP−1 = PDPT

� � ! � � � �� � & �$ � "� � � �! �� � S� ! � ! :( � �� � �� � �AAT = PD+P

T

�� D+� ! � � � � � � �� �� D� $ � � ( � "� �� $ � �� � 0 "� ! %� "� � � ! � � � � � � ! $ � &� � � %� !

�$ � $ � � � � � �

A = P√

D+

> � %� � "? @ @A � � BC E

Page 37: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � � � � � � � � � � � �� � � � � � � � �� � � θ

( � � � � !# !� � � ! � � � � � � * " � � � � �� �� � �� �� � �C D DC �

A(κ, κ′) =〈κ, κ′〉‖κ‖‖κ′‖ =

X 2 κ(x, x′)κ′(x, x′)dPX (x)dPX (x′)

X 2 κ(x, x′)2dPX (x)dPX (x′)√

X 2 κ′(x, x′)2dPX (x)dPX (x′)

( � � � � !# !� � � � � � ) � � � � ! !# � �" � !

ADm(Kθ,G,Kt) =

〈Kθ,G,Kt〉F‖Kθ,G‖F ‖Kt‖F

θ∗ = argmaxθ∈Θ

ADm(Kθ,G,Kt)

> � %� � "? @ @A � BC E

Page 38: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � �1� � � � � � � � �� � � �� �� � � � � � �� � � � � � ���� � � � � � � �� � � ��� � �� �

�� �� � � Q = 3 � � � � � �! � $ �� $ �! � � � �( � �2 �� ! #� � � & � � �� !

! � y = y′, κt(x, x′) = 1

! � y 6= y′, κt(x, x′) = − 1

Q−1

> � %� � "? @ @A �E BC E

Page 39: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� � �� � � �� � � � � � �� � �� θ∗

0

0.2

0.4

0.6

0.8

1

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6

thet

a

position dans la fenetre

"weights.txt" using 1:2

� � � � A � < � # �� � � θ� ) �� $ � �� � � " � &$ � ( � $ �$ � :� � * # � ) "�

� " & � � � �� ( � �98 � � � � � $ � �! ! � &� � �� ! #� $ �� � $ &� � � �� $ � ! �� # � � ! � � �� �

> � %� � "? @ @A � � BC E

Page 40: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � " � #� � ��$ 6 "� � � � � � # � ��$ �� "� ! �� � # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � !

� �� � �� � � � � � � � � � �� � �

< � " � �� � ��$ #� � �! �� 6 � �� ! ! � � � $ � )� ! � �� E D � � � " � � 3 �� ! � #�$ ! � � �� �� ! �� C �E � � " 3 � � � �

� �� �2 �98 � �� $ � � � � � � @ � �! � ��� � $ #� � " � &$ � ( � $ � * � � � � " � " � &$ � ( � $ � * ! � � � ��

= = * ,< = = = * ,< = = = * ,< =

Q3 � � � � �? � � �? � @ �? � � � A � > � > � �

Cα @ � � � @ � � � @ � � � @ � � � @ � � � @ � � >

Cβ @ � � � @ � � A @ � � � @ � � � @ � �? @ � � A

Cc @ � �A @ � � � @ � � � @ � � � @ � � � @ � � �

Sov � � � > � � � � � � � � � � � � � � � @ � � �A

Sovα � � �A � � � > � � � � � � � � � � � � � � � @

Sovβ � � � � � � �A � � � � �? � � �A � � � � � �

Sovc � � � � � � � � � � � � � � �A � � �? � � � �

> � %� � "? @ @A C D BC E

Page 41: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� �$ # "� ! ��$ ! � � ��� � ! ��� # � � %� !

� � � � � � �� � � � � � � � � � � � �� � � �

& � � ! � � � " �� � � � ! � �3 � ! � � �� � � !# � �� � ! ��� " � ! � . ) ,- .

� + � ! γ* Ψ* � �( � $ ! ��$ ! 5� � � $ � � � � � "� ! = * ,< = �� � "� ! = � � "� ( �( � � � "� �� � "� � �( � $ ! ��$

'� �* ! � � � �� � �$ & � � � � "� ! ,< = ) �* # "� ! ! � ! �

� � 8 �( � � � �� $ �! � � � &� 1 ! � � ! �� $ � 6 � � � # �� � � �� $ ! "� ' � �( � "� � ��$ �� "� ) � �$ � < � � "� ( � 5� � � � ��$ �� !

( � ! � � � ! �� #� �� # � � � �

� +� ! � "� # � ��$ �� ( � �1 "� ' � � �$ � � � $ � �� � � � �� �� � # � � � � � � "8 � %� "� � � ��$ �� ! � �! �� � ! &� � � $ � �! � � �� !

) � �$ � ! ! � � "� � �! �� � � ( � �� � �� � ��� � $ � � ! � �� ) " �! ! �$ ! �

� " 3 � � � � �� � � ! � � � �" � � " ! � ! �� � � � �" ! � ! � � " � � 3 �� ! �

� + � ! = * ,< = � � � %� $ � � � � % � �� � �� � � � � " �! �� ! � � # � � � �� ! = #�( ( � � " �( � $ �! �� )� ! � �� !

( � �� � �� �� � � � � � # � ��$ �� "� ! �� # �� � � ! � #�$ �� �� � �� ! � � � � � �$ � ! �

� + � � � %� "� � �� ( � $ � �� ( � �1 "� ! � : ) � � �� ! �$ � � &� � $ � ! : ! �1 ( � ! � �! #� �( �$ � $ �! � � & �$ �� � � �' ! #�$ ! � � �� �

"8 � $ � �� ! � � �$ # � �� "� ! � � � ��$ ! � � � � � ) �� $ �� �� ! � � � &� 1 ! � $ ) �� "� & �� ! �� � # �� � � "� � � � � � # � � %� �

> � %� � "? @ @A CE BC E

Page 42: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � � $ ! � % � �� � "� !

�� � � �� � �� �� � �� � �

> � %� � "? @ @A CC B �

Page 43: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � � $ ! � % � �� � "� !

� �� � � � � � � � � � � � �� � �

$ !# # !C �- � � � �� � * � !" � � !� � � � �E � E � , � !" �E � C � , � ! � � � �E � C � �� � � F�

� � � � � � � � � �� � � � � � �� � � � X � � � � � � � � � �� � � ΠF� � ' �$ # � ��$ �� #� � �! ! � $ #� � � � �

n ∈ N∗ � � � � � � ΠF (n) = supsXn⊂X N (F , sXn) � d� � � �( � $ ! ��$ < � ΠF (d) = 2d �

ΠF (d+ 1) < 2d+1 � �� �� � � �� � � n ≥ d�

ΠF (n) ≤d∑

i=0

Cin <

(en

d

)d

� e � � �� � � � � � �� � �� ��� � � � � � � � � �

> � %� � "? @ @A C � B �

Page 44: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � � $ ! � % � �� � "� !

� �� � � � � � � � � � � � �� � � � � � � � �

� � � � � � � � � � � � � � �� � � � � � X � � � � {1, . . . , Q}

$ !# # ! � � � � � � � !" � $ � � � �E � � � � �� � � F� � � � � � � � � � �� � � � � � �� � � � X �

� � � � � � � � {1, . . . , Q}� ΠF� � � � � �� � � � � �� � � � � d� � � � � � �� � � � � �� � � � � � �� � � �� � �

n ≥ d�

ΠF (n) ≤d∑

i=0

Cin

(

C2Q+1

)i<

(

(Q+ 1)2en

2d

)d

> � %� � "? @ @A C C B �

Page 45: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� � ��� �� � � � � $ ! � % � �� � "� !

� �� � �� � �� � ���� � � � � �� � � � � ��

� � � � � �� �� �� �� � �� �� � X� � � � R

� �� � �� ���� �� ! "#$ %'& ( () *,+ -./ G 012 3546 .7 72 82 3 -19 / . -1: : 0; 01 8 -6 4 . 12 X <= 4 72 0;: 8 4 1 :

[0, 1]> ? - 0; / - 0/ 2 = 4 72 0; 8A@ ε 8 4 1: ]0, 1]2 / / - 0/ 2 = 4 72 0; 2 1 / .B ;2 82 n: 4 / .: 354 .: 4 1 / n ≥ Pε/4

C 8 .6 (G) D -1

8 .: E -: 2 82 7 4F -; 1 2 : 0 .= 4 1 / 2 G

N (ε,G, n) < 2

(

4n

ε2

)d log2(2en/(dε))

-H d = Pε/4

C 8.6 (G)>

IJK LMNO P PQ � R S R)

Page 46: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � �� �� �� � � � � � � � � � � � �� �� �� � � � � � � � � � � �� � � � � � � �� � � � ��

� �� � �� � ��� ���� �� �� � �

���J � �N M ��� �� M L� ! �" � �! � � � �J # L" � M � M! MJ N � $ J L N J � M" � � �M � % LJ �'& M �� �� � F� �! �M" �N � " � �! N J

� M" � � �M � ( �� � N �2 : E 49 2 82 : ); 4 E�*2 : � � F $ GF

+,- � � .� .� �/ �� 0 �� 1 �� / �� �2 3 � � � �

4 � , � � 5� � & 6 �� � .� �� 7 % & ( () *+ -./ ΠGF7 4 3 -19 / . -1 82 9 ; -.: : 4 19 2 82 7 @2 : E 49 2 82 : ); 4 E*2 : 82

F> 87 -;: G

PDm

(

supf∈F

(

R(f) −Rm(f)√

R(f)

)

> ε

)

≤ 4ΠGF (2m) exp

(

−mε2

4

)

IJK LMNO P PQ � 9 S R)

Page 47: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � � � � � �� �� � � �� � �� � �� � � � � � �� �� �� � �

� M � �! X� � � � �J � �� � N � �� � �! (H, 〈·, ·〉H)� � � � �J �� � � M N �� L! � � # � ! M � � �� L X � H ⊂ RX ��

� 7�� � � � � �� � 2 - � � . ��� � � �- �� � % & ( R 6 *

� M! κ� �� # � ! M �� � X ×X� J � � R

∀x ∈ X $ � M! κxN J # � ! M �� � X� J � � R� �� � M � �J L κx : t 7→ κ(x, t)

κ� �! J � ��� N � � � 1 - � 4 0 ;2 E ; - 8 0 .: 4 1 / � � L H �M �! �� � N � " � �! �M�

� � ∀x ∈ X , κx ∈ H

O � ∀x ∈ X , ∀h ∈ H, 〈h, κx〉H = h(x) � E ; - E ; .� / � 82 ;2 E ; - 8 0 .: 4 19 2 �

�- � � / � � �� 2 � � �� .� � � 7�� � � � � �� � 2 - � � .

� M � � � �J � L� � L � � M �J �! �� M �! � $ H� �! J � ��� N � � �2 : E 49 2 82 � .7F 2 ; / < 1 - � 4 0 ;2 E ; - 8 0 .: 4 1 / ���� � � ��

� � � � �J � � � �" �! LM �� � � �� " M �� �� � M � � �M! M # � � �! � � � � �J � � L� � L � � M �J �! � �

IJK LMNO P PQ � ) S R)

Page 48: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

Q = 2� � � � � � � � � � � �� � � � � � � � � � � � �� � � � � � � � � �� � �

((xi, yi))1≤i≤m ∈ (X × {−1, 1})m� 2 1: 2 6 F 72 8A@ 4 E E ;2 1 / .: : 4 )2

h = (h1, h2) = (h1,−h1) $ h(x) = h1(x) = ∆#h1(x) = 12 (〈w1 − w2,Φ (x)〉 + b1 − b2)

` ��� (y, h(x)) =(

1 − yh(x))

+

� 3 -19 / . -1 82 E2 ; / 2 9 * 4 ; 1 .B ;2 �

�� - �� � .2 �� � �� � � � � �� 5� �� � � � � 1 � � � � � .2 �� / � � 0 �� � � � � � � � .2 � � *

� � � �� 5 � � �

minh∈H

{

m∑

i=1

` ���(

yi, h(xi))

+ λ∥

¯h∥

2

}

4 � , � � 5� �� � � � � � ,- �� .� .2 � �

� �! & � L �" � �! J �N M! �� � N �J � � L� �! M � �J %� �N J L � � N �! M �� � � L �N �" � Q � L�K M � �! �! L �K � L N � �K J N � � L �

� � � �� M � �! � βi� J � ��

h(·) =

m∑

i=1

βiκ(xi, ·) + b

�N J K J N � � L� � � � MJ M � � b �� � �� �� � M! �J LJ � �N M J! M �� � � �� M! M � �� � � � & �� � � �� L �

IJK LMNO P PQ � � S R)

Page 49: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � �� � � � � � �� �� � � � � �� � � �

�� 1 � �2 .� � � � �� � � � �� .2 - - � 1 �� 3 � � � � � � .2 �� � � �� �

αik� " � N! M �N M J! � � L� � �J % LJ � %� J � � M � � N J �! LJ M �! � 〈wyi− wk,Φ(xi)〉 + byi

− bk ≥ 1 − ξik

α = (αik)1≤i≤m,1≤k≤Q $ (αiyi)1≤i≤m = 0

� � � �� 5 � � ( � � �� & *

minα

{

1

2αTH � � α− 1T

Qmα

}

: > 9 >

0 ≤ αik ≤ C (1 ≤ i ≤ m), (1 ≤ k 6= yi ≤ Q)∑

i:yi=k

∑Ql=1 αil −

∑mi=1 αik = 0 (1 ≤ k ≤ Q− 1)

H � � =((

δyi,yj− δyi,l − δyj ,k + δk,l

)

κ(xi, xj))

1≤i,j≤m,1≤k,l≤Q

w∗k =

i:yi=k

Q∑

l=1

α∗ilΦ(xi) −

m∑

i=1

α∗ikΦ(xi)

IJK LMNO P PQ � ( S R)

Page 50: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

C_1 / C_2C_2 / C_3

x_1x_2

x_3

�� �� I � � J! � % LM � � � � N M � �J M L�" � �! � � �J LJ �N � �� J � � R3

IJK LMNO P PQ R 6 S R)

Page 51: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

C_1 / C_2C_1 / C_3C_2 / C_3

x_1x_2

x_3

� � � � � P � � � ��� L �N J � � � � �J LJ! � � L � �! K � ! � � L � �� � � L! � � N J � � � ( � N M � �J M L�

IJK LMNO P PQ R& S R)

Page 52: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � � � � � � � � � � � � � �� � � � � � � � � � � � � � � � � �� � � � �� � �� � � � � � �� � � � � � � � � � � �

∆ 6= ∆∗ � � L Q ≥ 3

� �� � �� � ��� ���� �� �� � � 1, � , � �� 2 - ,

� � � LJ! � � L ∆� # � L � M! �! L � �� � M � # L" J! M � � � L �! J �N M L N � N M � � � �! L� � � �J LJ! M � �! J �J M! � �� � � NK � LM �� L � � � �M � %N �! �

�2 � ��- 2 �� � � 1, � , � �� 2 - , �

� � � LJ! � � L ∆∗� �� # � L � M! �J �J � ��� � � M � # L" J! M � � � L J N � N � L � �� � L �� �� L � ��� M" � � �M � ( �

% � � � LJ N M � ��

=⇒J �! � � � L �J N M �� L N J ! LJ � �M! M � � �! L� ∆∗�! ∆J � � MK � J � � � N � " " �� � �J � � L� �& � N J & % � � � LJ N M � �

IJK LMNO P PQ R� S R)

Page 53: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� �� � � � � � � � � � � � � �� � � � � � �� � � � � � � � ��� �� � � � �� �

� , � � 2 .2 �� & � � � � � � �- � � �� .� � �2 � � �� � ��- �� �� � *+ -.2 1 / (E, ρ) 01 2 : E 49 2 E: 2 0 8 -C6 � / ; . � 02

� - 0 (E, ‖ · ‖E) 01 2 : E 49 2 82 � 4 1 49 * �2 / E′ 01 : - 0: C2 1: 2 6 F 72 F -; 1 � 82 E> 87 -;: D E - 0; n ∈ N∗ D72

nC .B 6 2 1 -6 F ;2 8 @2 1 / ; - E .2 82 E′ D εn (E′) D2 : / G

εn (E′) = inf {ε > 0 : N (ε, E′, ρ) ≤ n}

� , � � 2 .2 �� & R � � � � � �- � � �� .� � �2 � � �� � � �, � � . �� � � 2 � , �2 � � � � � � , *+ -.2 1 / (E, ‖ · ‖E)2 /

(F, ‖ · ‖F ) 82 0� 2 : E 49 2 : 82 � 4 1 49 * > + -./ L(E,F )7 @2 : E 49 2 82 � 4 1 49 * 82 / - 0: 72 : - E� ; 4 / 2 0;:

�7 . 1 � 4 . ;2 : F -; 1 � : � 82 (E, ‖ · ‖E) 8 4 1: (F, ‖ · ‖F )6 01 . 82 7 4 1 -; 6 2 G

∀S ∈ L(E,F ), ‖S‖ = supe∈E:‖e‖E=1 ‖S(e)‖F>

εn(S) = εn(S(UE))

IJK LMNO P PQ R S R)

Page 54: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� �� � � � � � � � � � � � � �� � � � � � �� � � � � � � � ��� �� � � � �� �

� , � � 2 .2 �� & 9 ��� �, � � . �� � � �, 0�� � � .2 �� * ? - 0; n ∈ N∗ D: -./ xn ∈ Xn> � @ � � LJ! � � L� � �K J N � J! M �

Sxn: 0; H2 : / 8� � 1 . E 4 ; G

Sxn : H −→ `Qn∞

h = (wk)1≤k≤Q 7→ Sxn

(

h)

= (〈wk,Φ(xi)〉)1≤i≤n, 1≤k≤Q

U ={

h ∈ H : ‖w‖∞ ≤ 1}

� � N M � � � �! L� N (ε,U , n)�! N � � � " � L� �� � � �! L � M �� � Sxn� �! # � L � M �J L N J � L � �M! M � �� MK J �! � �

� � � � �- 2 .2 �� � + -.2 1 / ε ∈ R∗+2 / n ∈ N

∗>

supxn∈Xn

εp(Sxn) ≤ ε =⇒ N (ε,U , n) ≤ p

IJK LMNO P PQ R� S R)

Page 55: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � � � � � � � �� � � �� � � � � � � ��� �� � � � �� �

� � � � � � � � � � � � � � �� � � � � �� � � �� � �� �� �� �� � �

� � � � �- 2 .2 �� � � � �� � � . � � � � �2 %'& ( ( 6 *+ -.2 1 / E2 / F 82 : 2 : E 49 2 : 82 � 4 1 49 * 2 / S ∈ L (E,F )>

+ . S2 : / 82 ; 4 1 ) r D 4 7 -;: E - 0; n ∈ N∗ D

εn(S) ≤ 4‖S‖n−1/r

4 � , � � 5� � & & + -./ H7 4 3546 .7 72 82 : 3 -19 / . -1 : ; � 4 7 .: 4F 72 : E 4 ; 012 � C+ � � < Q9 4 / � ) -; .2 : : - 0:

7 @ * � E -/ * B : 2 � 02 Φ (X )2 : / . 19 7 0: 8 4 1: 7 4F - 072 82 ; 4 � -1 ΛΦ(X )9 2 1 / ; � 2 : 0; 7 @ -; . ) . 1 2 82 EΦ(X ) D � 02 72

= 2 9 / 2 0; w= � ; . �2 ‖w‖∞ ≤ Λw2 / � 02 b ∈ [−β, β]Q> + . 7 4 8.6 2 1: . -1 82 7 @2 : E 49 2 EΦ(X )2 : / � 1 .2 2 /

� ) 4 72 < d D 4 7 -;: D E - 0; / - 0/ γ ∈ R∗+ D

N (p) (γ/4,∆γH, 2m) ≤(

2

γ

+ 1

)Q

·(

64ΛwΛΦ(X )

γ

)Qd

R(h) ≤ Rγ,m(h) +O

(

1

m

)

IJK LMNO P PQ R R S R)

Page 56: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ��� M ��� �� � � L � � �JK M L �N � �

� � � � � � � � �� � � �� � � � � � � ��� �� � � � �� �

� � � � � � � � � � � � � � �� � � � � �� � � �� � �� �� �� � � �� � �

4 � , � � 5� � & � � 4 � , � � 5� �� � �� � � 7� � � �� % � � �� � � . � � � � � 2 %'& ( ( 6 *+ -.2 1 / H 01 2 : E 49 2 82

� .7F 2 ; / 2 / S 01 - E� ; 4 / 2 0; 4 E E 4 ; / 2 1 4 1 / < L (`n1 , H) - 0 L (H, `n∞)> 87 -;: D E - 0; / - 0/ 9 - 0 E72 8 @2 1 / .2 ;:

(k, n)= � ; . � 4 1 / 1 ≤ k ≤ n D -1 4ek(S) ≤ c

(

1

klog2

(

1 +n

k

)

)1/2

‖S‖,

-H 72 � " � L�� � � �! L � M �� �J� M �� � ek(S)2 : / � ) 4 7 < ε2k−1(S)2 / c2 : / 012 9 -1: / 4 1 / 2 01 .= 2 ;: 2 7 72 >

4 � , � � 5� � & + -./ H7 4 3546 .7 72 82 : 3 -19 / . -1 : ; � 4 7 .: 4F 72 : E 4 ; 012 � C+ � � < Q9 4 / � ) -; .2 : : - 0:

7 @ * � E -/ * B : 2 � 02 Φ (X )2 : / . 19 7 0: 8 4 1: 7 4F - 072 82 ; 4 � -1 ΛΦ(X )9 2 1 / ; � 2 : 0; 7 @ -; . ) . 1 2 82 EΦ(X ) D � 02 72

= 2 9 / 2 0; w= � ; . �2 ‖w‖∞ ≤ Λw2 / � 02 b ∈ [−β, β]Q> 87 -;: D E - 0; / - 0/ γ ∈ R

∗+ D

N (p)(γ/4,∆γH, 2m) ≤(

2

γ

+ 1

)Q

· 216cΛwΛΦ(X)

γ

q

2Qm

ln(2)−1

R(h) ≤ Rγ,m(h) +O

(√

1√m

)

IJK LMNO P PQ R 9 S R)

Page 57: LIPN – Laboratoire d'Informatique de Paris Nord...1 * # 6 3 " #! G X [ MG;MG] Q Q 2 ]0;1[ 1 ]0;MG] g G R(g) R;m(g)+ v u u u u t 2 m 0 B @ln 0 B @4 2mQ2(Q 1) 12MG 2! l dlog2 emQ(Q

� ���M ������

�L ���JKML�N ��

�������

����

������ ��� �

�����������������������������

E

H

C

?

<−−−−−−−−−−−−−−−−−−−−−−−−>

P(V in C | S)

...VKPVDNFDWSNYHGKWWEVAKYPNSVEKYGKCGWAE...

...VKPVDNFDWSNYHGKWWEVAKYPNSVEKYGKCGWAE...

NN 2 NN 3NN 1

SIMPA 96SOPMA

post−traitement (Si nécessaire)

Combinaison

P(V en H | S) P(V en E | S)

Réseaux de "filtrage"

M−SVM(GOR IV)

MLRC, M−SVM...

Programmation Dynamique (IHMM)

Contenu de la fenetre glissante S^

IJKLMNOPPQ

R)SR)