ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

40

Transcript of ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

Page 1: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � �� �� � � � � � � � � � � � � �� � � � ��� � � � �

��� � �� ���� � � � ��� � � "! #%$& � $' �� �(

)+* ,� � * - *. . / * � 0 12 ' � 02 / 34 / 0

Page 2: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � �

• � ! � $ 0 / 0. � � 3& 0 � � / * 0 � � �$ * � / ,� � / * 0

• � , � / $ / �� � 2 / � 4 � / 0 / , /�� � � / * 0• ! � �� � * � ! 1 0 ! � � � 0 1 � * 0 � / � � ! 0 �'

•2 ! . & ��� $ / � � � / * 0

• � � � ! 0 1 / � �� , �� ! � 0 1 � � �$ * � / ,� � / * 0 � $ $ * $

Page 3: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � ��� � � � � �

� ��� � � ! / 0 � $ * 1& � ! � ! � ! � $ 0 / 0. �$ * #� ! , � � � ! �$ * #� ! ,

* 3 3& 0 � � / * 0 � � �$ * � / ,� � / * 0 3$ * , � �� $ � ! 1� �� � ! 1 ! 0 !

� ! 4 ! ' / 1 ! � � * 3 � * � � 3& 0 � � / * 0 �� ! , � / $ / �� � ! $ $ * $ � 0 1. ! 0

! $ � � / � � � / * 0 ! $ $ * $ � ! � ! 0 / 0 � $ * 1& � ! � ! � , � / $ / �� � 2 / � 4

� / 0 / , /�� � � / * 0 � � �$ * � � � 0 1 � ! ��� * 4 ! ' $ ! & / $ ! , ! 0 � �

* 0 � �. * $ / � , �& � / 0. / � � ! � �� � * � ! 1 0 ! � � � 0 1 � * 0 � / � � ! 0 �' �

! � ! 0 1 ! � � $ / # ! � 4 ! ' � �. * $ / � ,�� ) / 4 * 0 *� $ ! . & ��� $

/ � � � / * 0 � � � � �� � / � ! � � ! � ! $ ! & / $ ! , ! 0 � ��

��� � � �� �� ��� � � � , / � /� $ / �' � / � #� � / � / 1 ! � � / 0 �$ * #� # / � / �'

� ! * $' �

Page 4: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � �� � � � � �� � � ��� � � � � � � � ��

� ��� � � � � � � �

! � � �& , ! � � �

X

� 0 1

Y

� $ ! ��� * � ! � � * 3 $ � 0 1 * ,� � $ /� #� ! ��

! � $ ! . /� ! 0 � �� � ��� ��� � �� �

S

� * 0 � / � � / 0.`

�� , �� ! � 1 $ � � 0

/ � / � 1 � 3$ * , � ! �$ * #� # / � / �' 1 / � � $ / #& � / * 0X × Y

x , y �

, . . . ,

x`, y`

! � / � � ,� 4 ! 3$ ! & ! 0 �& � ! * 3 � ! � � � � � � � � ��� � �� � � � � ���

� � � ��� � � �� � $ / � � ! 0py|x �

p

x, y

� � p

y|x � · p

x

� � / � � $& � /� � � * 0 * � ! � � � � ! � / ! �

p

x, y

� � � � �� � #& � � ���

�� �� �

Page 5: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � ��� � ��� � � � � � � �X Y

P(x)

P(y|x)

Page 6: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� �� �� � � � �� �� � � � � � ��� � � � � � � � � � � � �

� � � � � �� � � � � � �� � � � � � � � � � � � � � � � � �

) ! #� � / �. * � � * 3 �� � � � � � �� � � � � � � ��� � / � � * & � ! � ! � $ � / 0 / 0.

� ! �

S

� * �� ! � $ 0 � � 3& 0 � � / * 0

fS

� � � � * * 4 � � � � 0 ! �

x

� � �& !

xnew

� 0 1 � $ ! 1 / � � � � ! � � � * � /� � ! 1� � �& ! * 3y

ypred

� fS

xnew

� 3

y

/ � � $ ! � �� � � �& ! 1 $ � 0 1 * ,� � $ /� #� ! � � ! � � ! � � � � � � � � � �

� 3

y

�� 4 ! � � � �& ! � 3$ * , � 0 & 0 * $ 1 ! $ ! 1 0 / � ! � ! �� � ! � � !

� � � �� � � � �� � � � � �� � � � � � 0 ��� * ���� � � �� � � ! $ 0 �� � � � / �� � / * 0

� $ * #� ! , �� � ! � � � /. 0 * 0 ! �� � � � �

y

� � �& ! * 3 � � � 0 1 � ! * � ! $

���� � � �

y

� � �& ! * 3 − � �

Page 7: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � �� � � � � � �

� 0 * $ 1 ! $ � * , ! � �& $ ! . * * 1 0 ! � � * 3 * & $ 3& 0 � � / * 0 � � ! 0 ! ! 1 �

� � � �� � � � � �

V � � 0. ! 0 ! $ � � � � ! � ! �

V

f

x

, y∗

� 1 ! 0 * � ! � !

� $ / � ! � ! �� ' � ! 0 � ! � ! !x

� 0 1. & ! � � � � � � ! � � � * � /� � ! 1

y

� � �& ! / �

f

x

� � ! 0 / � / � � � �& � � �'

y∗�

Page 8: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � �� � � � � � � � � � � � � �� � � �

* $ $ ! . $ ! � � / * 0 � � ! , * � � � * , , * 0 � * � � 3& 0 � � / * 0 / � � & � $ !

� * � � * $ � � � * � �

V

f

x

, y

� �

f

x

� − y � �

! � * & � 1 � � � * & � ! � ! � # � * �& � ! � � �& ! � * $ � � � * � �

V

f

x

�, y

� � |f

x

� − y|

�� � 0 / 4� � , * $ ! . ! 0 ! $ � �ε

/ 0 � ! 0 � / � /� ! � * � � 3& 0 � � / * 0 / �

V

f

x

, y

� � |f

x

� − y| − ε ���

Page 9: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � �� � � � � � � � � � � � � � � � � � � � �

��� � � ��� � � � �� � � � � � � � � � ��� � � � � �� ��� � � ��� � � � � � �� � ��� ��� � � � ��

V

f

x

, y

� � � �−yf �x

� �

��� � � � � � � � � � � � � � � � � ��� � �� �� � � � � � � ! �� � � � � � � � � �� " � � � � � � � � �

# � � � �� � ��� � �� � � � � �� $ � #� �% � ��� � ��� � � � � � � � � � � � � �

V

f

x�, y

� � � − y · f �

x

� �'&

Page 10: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � �� � ��� � � � � � � � � � � � � � � � ��� � � � � � �

� ��� � � � !� � � � � �

f � � � � � � !� � � � � �

V � � � � � # �� � � � � � � � � �� � � � � � � � � �

P� � � �

X

� � �

Y � � � � ��� � � � ��� �� � �� � � � �� � �� � !f

� ��

I

f

∫V

f

x

, y

�x, y

� � � �� � ��� � � ��� �� � �� � � �� �� � � � � � � � � � # �� � � � � � � � � � � � �

! �� � � � �� � � � � � � � � � � � � � ��� �� � � � �dµ

� � �� � �

dP

���

�� � � � � � � % � � � � � % �

I

f

� � � � � � � � � ��� "� � � � � � � � � � � � % � �

P �

� ��� � � � !� � � � � �

f � � � � � � !� � � � � �V � � � � � � � � ��� ��� " � � �

S

� � � �� � �� "

� !

`

� � � � # � ��� �� � � � � �� �� � �� � ��� �� � � � � �� � �� � �� � � � � �� % � � !

f

���

IS

f

`

∑V

f

xi

, yi

Page 11: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � �� ��� � � � � �

) ! � � � � � � � � � �� � �� H / � � ! � �� � ! * 3 3& 0 � � / * 0 � � � �

� ! � � � * � * & $ � �. * $ / � , � * � ! � $ � � � � / � * 3 � ! 0 � * � ! 0 � / �

$ ! � � ! � � � * � ! � , * & 0 � * 3 1� �� � � � / ��� #� ! �

Page 12: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � ��� � � � ��� � � � � � ��� � � � �

� /� ! 0 � � $ � / 0 / 0. � ! �

S

� 0 1 � 3& 0 � � / * 0 � �� � ! H� ! , � / $ / �� � $ / � 4

, / 0 / , /�� � � / * 0 �� � 0 / 4 � 0 1 � � 3& 0 � � / * 0fS

� � � , / 0 / , / � ! �

� ! ! , � / $ / �� � $ / � 4 *� ! $ � � � 3& 0 � � / * 0 �f ∈ H

fS

� � $. , / 0f∈H

IS�

f�

� � $. , / 0f∈H

�`

∑̀

i � V

f

xi

, yi

* $ 0 * � � � ! � $ ! � � �& , / 0. � ! ! � / � � ! 0 � ! * 3 �& � � 3& 0 �

� / * 0 � �

Page 13: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � ��� � � � � � � � �� ��� � � � � �� � �

� � � � � ��� � � � ��� � � � � � ��� � � � �

* $ � ! � * �& � / * 0 * 3 �2 � � * # ! & � ! 3& � / 0 � ! � * 0 � ! � � * 3

� ! � $ 0 / 0. � � ! � * �& � / * 0 ,& � � # !• � � * 0 � / � � ! 0 � �

• / � � � � * ,& � � ! � / � �� # ! & 0 / & ! � 0 1 # ! � � �� #� ! � � ! � ��

� * � ! 1 0 ! � � � �

Page 14: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � ��� � � � � � � � �

��� � � � � ��� � , ! � 0 � � � � � ! 1 / �! $ ! 0 � !IS

fS

� − I �

fS

� ,& � �

. * � * � ! $ * � � � ! 0& , # ! $ * 3 � $ � / 0 / 0. ! �� , �� ! � / 0 � $ ! � � ! ��

� � � / �

` → ∞� � 0 * � ! $ � * $ 1 �� � ! � $ � / 0 / 0. ! $ $ * $ 3 * $ � !

�2 � � * �& � / * 0 ,& � � � * 0� ! $. ! � * � ! ! � � ! � � ! 1 ! $ $ * $ � 0 1

� & � # ! � � �$ * �' � 3 * $ / �� � � ! $ � / � ! � ! � * �& � / * 0 � * & � 1 0 * �

# ! � �$ ! 1 / � � /� ! � �

Page 15: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � ��� � � �� � � ��� � � � � � � � � � � � ��� � � � � � � �� � �

� � � � � � � � � � � �� � � �

x

f(x)

Page 16: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � ��� ��� � � � � � �� � � � � � � � � � � �

x

f(x)

Page 17: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � ��� �� � � ��� � � � � � � �

x

f(x)

Page 18: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � �� � �� � � � � � � � � � � � � � � � � � �

� � � � � � � � � � � � �� � � � � � � � � � � � � � � �

� � � � � � � � � � � � �� � � � �

x

f(x)

Page 19: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � ��� � � � � � � � � �� � � � � � � � ��

� � � ! � / � � � ! ! ��� � ! $ � �$ * � ! $ � * / � ! * 3 � ! ' � * � ! � / � � �� � !

H ! 0 �& $ ! � � * 0 � / � � ! 0 �' * 3 �2 � � ) ! 4 ! ' � $ * � ! $ �' / � � !

& � � �$ * � ! $ �' � ! �� ' � � � � ���� � � * 3 3& 0 � � / * 0 � F / � �

& 0 / 3 * $ , � � /� ! 0 4 * �� 0 � ! � � / & � � � ���� � � � �� 3 * $ � � �

ε >

� � 0 1

3 * $ � � �

µ�� -

µ

( � / ,

`→∞

�& �

f∈F|I �f

� − IS

f

� | > ε

)

� �

! � / � � # ! ! � �� * $ / 0. � / � 1 ! 0 / � / * 0 � 0 1 ! & /� � � ! 0 � 1 ! 0 /

� / * 0 � � / 0 1 ! �� / � / 0 � � � �� � � 3 H / � � & � � ���� � �� � / � 1 / $ ! � ��'

/ , �� / ! � � � �

IS�

f�−I �

f

� 3 * $ ! � ! $' 3& 0 � � / * 0

f

/ 0 H. ! � � � ,� � �

� �

`→∞�

Page 20: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

�� ��� � � � � �� � � � �

�2 � / � / 0 . ! 0 ! $ � � � 0 / � �� � * � ! 1 �$ * #� ! , � � � �� 0 # ! ,� 1 !

� ! � � � * � ! 1 #' � 0 � � � $ * �$ /� � ! � * / � ! * 3 H�

� � � ! � / � � � ! ! ��� � ! $ � � ! � �� � * � ! 1 0 ! � � / � ,� / 0 �' & � ! 1 � * , ! � 0

� ��� � �� � � * 3 � ! � * �& � / * 0 fS

1 ! � ! 0 1 � � * 0 � / 0& * & ��' * 0 � !

� $ � / 0 / 0. � ! �

S� � 0 �� $ � / �& � � $ � � � 0. / 0. * 0 ! * 3 � ! � $ � / 0 / 0.

� * / 0 � � � * & � 1 � �! � � � ! � � � 0 1 � ! � � � ! � * �& � / * 0 � �

`

. * ! � � *

/ 0 0 / �' �

Page 21: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � �� � � � � � � � � � �� ��� � � � � �

�� ��� � � � � � � � � �

� �$ * #� ! , / � �� � �� � �� � / 3 / � � � * �& � / * 0

• ! � / � � �

• / � & 0 / & !

• 1 ! � ! 0 1 � � * 0 � / 0& * & ��' * 0 � ! 1� �� ! �. � / � / � � ��� � � � �

� �$ * #� ! , / � � � �� � �� � / 3 / � / � 0 * � � ! � �� � * � ! 1 �

Page 22: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � ��� � � �� � � ��� � � � � � � � � � � � ��� � � � �

� � � � � �� � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 23: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � � � � � � � � � � �� � � � � � �� � � � � � �

� � � � � � � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 24: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � � � �� � � � � � � � �� � � � � � �� � � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 25: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � � � � � � � � � � �� � � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 26: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � �� � � ��� � � �� �� � �� � � � � � � � �

� � � � � � � � � � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 27: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � � � � � � � � � �� � � � � � � � � � � � � � �

� � � � � � � � � � � � � �� � � � � � �

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Page 28: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � �� � � � ��� � � � �

) ! #� � / � / 1 ! � * 3 $ ! . & ��� $ /�� � � / * 0 / � � * $ ! � � * $ ! � ! � �� � * � ! 1 0 ! � �

* 3 �2 � #' � * 0 � � $ � / 0 / 0. � ! ' � * � ! � / � � �� � ! H� � 0 / 0 1 /

$ ! � � � � ' � * 1 * � * / � � * & � ! ) / 4 * 0 *� $ ! . & ��� $ /�� � � / * 0 � / �

/ � 0 * � �2 � � �

Page 29: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � �� � � � ��� � � � �

�2 � 0 1 � � ! 3& 0 � � / * 0 / 0 H � / � , / 0 / , /�� ! �

`

∑̀

i � V

f

xi

, yi

� / � / 0. ! 0 ! $ � � � 3 * $ � $ # / � $ � $' ' � * � ! � / � � �� � ! H� / � �� ���

� � � � �� � 0 � � ! � 1 � � ! , / 0 / , /�� ! *� ! $ � ! ' � * � ! � / � � �� � ! H�3 * $ � � ! 1 � * � / � /� ! �� $ � , ! � ! $λ� � ! $ ! . & ��� $ / � ! 1 3& 0 � � / * 0� �

�`

∑̀

i � Vf

xi

, yi

� �

λ‖f‖

K ,

� �

� ! $ ! ‖f‖ �

K

/ � � ! 0 * $ , / 0 HK � � ! 2 ! � $ * 1& � / 0. � ! $ 0 ! �

� / � # ! $ � � �� � ! 2 � � � � � 1 ! 0 ! 1 #' � ! 4 ! $ 0 ! �

K �

Page 30: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � �� � � � ��� � � � �

� � � ! � / � � � ! ! / 0 3& �& $ ! ���� � � ! �

• ) / 4 * 0 *� $ ! . & ��� $ /�� � � / * 0 ! 0 �& $ ! � � ! � �� � * � ! 1 0 ! � � ! . ! �

/ � � ! 0 � ! � & 0 / & ! 0 ! � � � 0 1 ! � � ! � /� � �' � ��� � �� � � / 0 � � ! $'

� � $ * 0. 3 * $ , � * 3 � ! � * �& � / * 0

• ) / 4 * 0 *� $ ! . & ��� $ / � � � / * 0 ! 0 �& $ ! � � * 0 � / � � ! 0 �'

• ) / 4 * 0 *� $ ! . & ��� $ /�� � � / * 0 / � �� * � ! �' $ ! ��� � ! 1 � * � #& � 1 / 3

3! $ ! 0 � 3$ * , � �� � 0 *� $ ! . & � � $ /�� � � / * 0 � ! . �2 � * 0 � '

� * � ! � / � � �� � ! H � / � / � � #� � � / 0 � 2 � � � �

Page 31: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � �• � 0 � ! 0 ! � � ���� � � � ! � / � � / 0 � $ * 1& � ! 2 � � � � ! ' � / � � # !

� ! ' � * � ! � / � � �� � ! � � ! � / � � � * $ 4 � / � �

• ! � / � � � � � * 1 ! $ /� ! � ! � * �& � / * 0 * 3 ) / 4 * 0 *� $ ! . & ��� $

/ � � � / * 0 �

Page 32: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

��� � � � � � � � � � � � � � � � ���

� � � � � � � �

��� � � � � � � � � � � � � �

� 0 � 1 1 / � / * 0 � * � ! ' � * � ! � / � � �� � ! H� � ! � �� � ! � ! � � � * �

* & $ � �. * $ / � , � � * � ! � $ � � � ! 1 ! 0 ! � � �

) ! �� � �� � �� � �� T / � � � �� � ! * 3 3& 0 � � / * 0 �� � * � ! 0 � �$ / * $ /

/ 0 � 0' . /� ! 0 �$ * #� ! , � � � � / �� � �& , ! 1 � * � * 0 �� / 0 � ! � � $& ! �

3& 0 � � / * 0 � � � , / 0 / , /�� ! � � ! $ / � 4 � � 3 � ! 0 � T / � � * � ! 0 � * # !

� � � 3& 0 � � / * 0 � / 0

L � � * $ � � � 1 / �! $ ! 0 � /� #� ! 3& 0 � � / * 0 ��

Page 33: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � � � � � � � � � � � � � � � � �� � � � � � � � � � � � �

� ! �

fH

# ! � ! 3& 0 � � / * 0 / 0 H � / � � ! � ,� � � ! � � � $& ! $ / � 4 �

! 1 ! 0 ! � ! �� �� ��� � � � � � * # !

I

fS� − I �

fH

� � � ! 1 / 33! $ ! 0 � ! / 0 � $& ! $ / � 4 # ! ��� ! ! 0 � ! # ! � � 3& 0 � � / * 0 / 0 H � 0 1

� ! 3& 0 � � / * 0 / 0 H � ! � � �& � � �' 0 1 � ) / � / � � � � � ! �� '

# ! �� & � ! * & $ 0 / � ! �� , �� ! 1 * ! � 0 * �. /� ! & � ! 0 * & . / 0 3 * $

,� � / * 0 � * � * * � ! � * � ! � # ! � � � 3& 0 � � / * 0 / 0 H� ! � 1 � / 4 !

� / � � * # ! � ,� � � �

� ,� / 0 � * � / � * 3 � / � � * & $ � ! / � � # * & 0 1 / 0. � � ! �� , �� ! ! $

$ * $ � 1 ! � ! $ , / 0 / 0. � * 0 1 / � / * 0 �& 0 1 ! $ � / � � ! �� 0 � �� � ! � � �

I

fS

� − I �

fH

� � / � � # ! � ,� � � � / � /. � $ * #� # / � / �' � �

� � � � / , �� ! $& � ! � � ! ! � � ! � � � � � / 3 H / � � � ! � �� # ! � � ! 1 � �

� ! 0 � � �`

. ! � � ��� $. ! � ! �� , �� ! ! $ $ * $ � / � � # ! � * , ! � ,� � � �

Page 34: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

��� � � � � � � � � � � � � � �

� ! �

f � # ! � ! 3& 0 � � / * 0 / 0 T � / � � ! � ,� � � ! � � � $& ! $ / � 4 �

! 1 ! 0 ! � ! � � � � � � �� � � � � � � � � * # !I

fH

�−I �

f � � � � !

1 / �! $ ! 0 � ! / 0 � $& ! $ / � 4 # ! �� ! ! 0 � ! # ! � � 3& 0 � � / * 0 / 0 H � 0 1

� ! # ! � � 3& 0 � � / * 0 / 0 T � ) / � / � � � � � ! �� ' # ! �� & � ! H / �

� ,� � � ! $ � � 0 T � ! � 1 � / 4 ! � / � ! $ $ * $ � * # ! � ,� � � � * * �

! � / � � 3 * �& � � ! � � * 0 � ! � � �$ * � / ,� � / * 0 ! $ $ * $ / 0 � � � �� � #& �

� ! � / � � ! � �� * $ ! / ��

� � � � / , �� ! $& � ! � � ! ! � � ! � � � � � � � H. $ * � � # /. . ! $ � � !

� � �$ * � / ,� � / * 0 ! $ $ * $ . ! � � � ,� � � ! $ � � 3 T ⊆ H � � / � / � �

� / �& � � / * 0 �� � � ! 1 � � � �� � � ��� � � � � � � � � � �� � � ! � � �$ * � / ,� � / * 0

! $ $ * $ / � � ! $ * �

Page 35: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � �� � ��� � � � � � � � �

! 1 ! 0 ! � ! �� � � � � � ��� � � � � � � � � � * # !I

�fS

� − I

f � � �

� ! 1 / �! $ ! 0 � ! / 0 � $& ! $ / � 4 # ! ��� ! ! 0 � ! 3& 0 � � / * 0 � ! � � �& � � �'

0 1 � 0 1 � ! # ! � � 3& 0 � � / * 0 / 0 T � ! � 1 $ ! � � �' � / 4 ! � / � � * # !

� ,� � � �) ! . ! 0 ! $ � � / � � � / * 0 ! $ $ * $ / � � ! �& , * 3 � ! �� , �� ! ! $ $ * $ � 0 1

� ! � � � $ * � / ,� � / * 0 ! $ $ * $ I

fS

� − I �

f � � � I

fS� − I �

fH

� � �

I

fH

� − I �

f � � �

� 3 � ! �� 0 ,� 4 ! # * � � ! � � � $ * � / ,� � / * 0 � 0 1 � ! �� , �� !

! $ $ * $ � ,� � � � � ! . ! 0 ! $ � � /�� � � / * 0 ! $ $ * $ � / � � # ! � ,� � � � ) ! $ ! / �

� � $ � 1 ! * � # ! ��� ! ! 0 � ! � � � $ * � / ,� � / * 0 ! $ $ * $ � 0 1 � ! �� , �� !

! $ $ * $ � � �

Page 36: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � ��� � � � � � � � � � � � � � � � � �� � �

� � � * & � 1 � � $ ! � 1' # ! / 0 �& / � /� ! �' �� ! � $ � � � ,� 4 / 0. H # /.

,� 4 ! � � ! � � �$ * � / ,� � / * 0 ! $ $ * $ � ,� � � � ) / � / , �� / ! � � � � � !

�� 0 ! � � � ,� 4 ! � ! . ! 0 ! $ � � /�� � � / * 0 ! $ $ * $ � ,� � � #' ,� 4 / 0.

H # /. �� 0 � ! * � ! $ � 0 1 � � ! � / � � � * � � � � ,� 4 / 0. H � ,� � � � / � �

,� 4 ! � ! �� , �� ! ! $ $ * $ � ,� � � � � 0 �� $ � / �& � � $ � / 3 H / � � & � �

���� � �� � ! �� , �� ! ! $ $ * $ � / � � . * � * � ! $ * � �

` → ∞� #& � * �

& / � 4 �' / �. * ! � � * � ! $ * 1 ! � ! 0 1 � 1 / $ ! � ��' * 0 � ! � � / � ! � * 3 H�) / � / , �� / ! � � � � � ! � � 0 � � * 4 ! ! � H � � � ,� � � � � � * � � / #� ! � & $ � ! $ , * $ ! � T / � � ! � 3 ,� ' * $ ,� ' 0 * � # ! � & � � ���� � �� �

� 1 ! � � �' � � ! � * & � 1 � / 4 ! � * 0 1 � ! * � � / ,� � � $ � 1 ! * � # ! ��� ! ! 0

� ! � ! � * 0 � / � � / 0. $ ! & / $ ! , ! 0 � ��

Page 37: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

� � � � �� � ��� � � � � � � � � � � � � � � � � � � � �� �

! 1 ! 0 ! � ! . ! 0 ! $ � � /�� � � / * 0 ! $ $ * $ � * # !

I

fS

� − I �

f � � � � 0

� ! � / � ! $ � �& $ ! � � ! � $& ! $ / � 4 * 3 � ! 3& 0 � � / * 0 � ! 0 1 � I �

fS

/ � � * , ! � / , ! � �� � � ! 1 � ! . ! 0 ! $ � � /�� � � / * 0 ! $ $ * $ � � 0 � ! �� � !

� ! $ !

I

f � � � � � � ! ��� * � � �$ * � � ! � � $ ! ! & /� � � ! 0 ��

Page 38: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi

l1

l2

l1 < l2

l1, l2 = number of data

Page 39: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi
Page 40: ocw.mit.edu · l1 l2 l1 < l2 l1, l2 = number of data. Title: 02-head.dvi