lec15 · 2017. 10. 24. · Title: lec15 Created Date: 10/24/2017 1:43:49 AM

Department of Computer ScienceCSCI 5622: Machine Learning

Chenhao TanLecture 15: VC Dimension

Slides adapted from Jordan Boyd-Graber, Chris Ketelsen

1

Logistics

• Project feedback• Peer feedback• Instructor feedback

• HW3 scores• Midterm

2

Learning objectives

• Learning about the VC dimension for infinite dimensional H

• See some examples of computing the VC dimension

• See some generalization bounds in terms of VC dimension

3

Assumptions and Definitions

4

Generalization Bound of Finite H

5

Generalization Bound of Infinite H

6

Generalization Bound of Infinite H

7

Complexity Measures - Counting Bits

8

Complexity Measures - Counting Bits

9

Complexity Measures – Growth function

10


11


12


13


14


15


16


17


18


19


20


21

Complexity Measures – VC dimension

22


23


24


25


26


27


28


29


30


31


32


33


34


35


36


37


38


39


40


41


42


44


45


46

Wrap-up

• VC dimension is a convenient measure of the complexity of infinite hypothesis classes

• Nice generalization bounds that depend on the VC dimension

47

lec15 · 2017. 10. 24. · Title: lec15 Created Date: 10/24/2017 1:43:49 AM

Documents

Transcript of lec15 · 2017. 10. 24. · Title: lec15 Created Date: 10/24/2017 1:43:49 AM