Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer...

38
Neural Networks Professor Michael C. Mozer CSCI 3202

Transcript of Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer...

Page 1: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus

Neural Networks

Professor Michael C. MozerCSCI 3202

Page 2: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 3: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 4: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 5: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 6: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 7: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 8: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 9: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 10: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 11: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 12: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 13: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 14: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 15: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 16: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 17: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 18: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 19: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 20: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 21: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 22: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 23: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 24: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 25: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 26: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 27: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 28: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 29: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 30: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus

Exponential number of hidden units is bad— large network— poor generalization

With domain knowledge we could pick an appropriate hidden representation.

e.g., perceptron scheme

AlternativeLearn hidden representation

ProblemWhere does training signal come from?Teacher specifies target outputs, not target hiddens.

Page 31: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 32: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 33: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 34: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 35: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 36: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 37: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus
Page 38: Neural Networks Professor Michael C. Mozer CSCI 3202 · Neural Networks Professor Michael C. Mozer CSCI 3202. Exponential number of hidden units is bad ... DICK TRACY NEURAL- B catvus