UNIFORM CONVERGENCE MAY BE UNABLE TO EXPLAIN GENERALIZATION IN DEEP LEARNINGVAISHNAVH NAGARAJAN1 ZICO KOLTER1,2
1Computer Science Department, Carnegie Mellon University 2Bosch Center for Artificial Intelligence, Pittsburgh
THE HIGH LEVEL MESSAGE
THE GENERALIZATION PUZZLE & UNIFORM CONVERGENCE (U.C)
PROGRESS SO FAR
Our 1st finding: GENERALIZATION BOUNDS ↑ WITH TRAINING SET SIZE
Key element in proof: TIGHTEST UNIFORM CONVERGENCE
CONCLUSIONS AND FUTURE WORK
REFERENCES
PROOF INTUITION
Our 2nd finding: PROVABLE FAILURE OF UNIFORM CONVERGENCE
FAILURE OF U.C. IN A HYPERSPHERE EXAMPLE
Top Related