Non-linear hypotheses
description
Transcript of Non-linear hypotheses
Introduction to Programming
Neural Networks: RepresentationNon-linear hypotheses
Machine Learning1Non-linear Classification
x1x2size
# bedrooms
# floors
age
Andrew Ng
You see this:
But the camera sees this:What is this?Andrew NgComputer Vision: Car detectionTesting:
What is this? Not a carCars
Andrew Ng
Learning Algorithmpixel 1pixel 2pixel 1pixel 2Raw image
CarsNon-Cars
Andrew Ng
pixel 1pixel 2Raw imageCarsNon-CarsLearning Algorithmpixel 1pixel 2Andrew Ng
pixel 1pixel 2Raw imageCarsNon-Cars50 x 50 pixel images 2500 pixels (7500 if RGB)
pixel 1 intensitypixel 2 intensitypixel 2500 intensityQuadratic features ( ): 3 million features
Learning Algorithmpixel 1pixel 2
Andrew NgNeural Networks: RepresentationNeurons and the brain
Machine Learning9Neural NetworksOrigins: Algorithms that try to mimic the brain.Was very widely used in 80s and early 90s; popularity diminished in late 90s.Recent resurgence: State-of-the-art technique for many applicationsAndrew Ng[Roe et al., 1992]
Auditory cortex learns to see
Auditory Cortex
The one learning algorithm hypothesis[Roe et al., 1992]Andrew Ng
Somatosensory cortex learns to see
Somatosensory Cortex
The one learning algorithm hypothesis[Metin & Frost, 1989]Andrew NgSeeing with your tongue
Human echolocation (sonar)
Haptic belt: Direction senseImplanting a 3rd eyeSensor representations in the brain[BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]Andrew NgNeural Networks: Representation
Machine LearningModelrepresentation I15
Neuron in the brainAndrew Ng
Neurons in the brain[Credit: US National Institutes of Health, National Institute on Aging]Andrew Ng
Neuron model: Logistic unit
Sigmoid (logistic) activation function.
Andrew NgNeural NetworkLayer 3
Layer 1Layer 2Andrew NgNeural Network
activation of unit in layer
matrix of weights controlling function mapping from layer to layer
If network has units in layer , units in layer , thenwill be of dimension .
Andrew NgNeural Networks: RepresentationModel representation II
Machine Learning22Add .Forward propagation: Vectorized implementation
Andrew NgLayer 3
Layer 1Layer 2Neural Network learning its own featuresAndrew NgLayer 3
Layer 1Layer 2Other network architecturesLayer 4Andrew NgNeural Networks: RepresentationExamples and intuitions I
Machine Learning27Non-linear classification example: XOR/XNOR , are binary (0 or 1).
x1x2x1x2
Andrew NgSimple example: AND
00011011
1.0
Andrew NgExample: OR function
00011011
-102020Andrew NgNeural Networks: RepresentationExamples and intuitions II
Machine Learning32
Negation:
01
Andrew NgPutting it together:
00011011
-30202010-20-20-102020Andrew NgNeural Network intuitionLayer 3
Layer 1Layer 2Layer 4Andrew NgHandwritten digit classification
[Courtesy of Yann LeCun]Andrew NgHandwritten digit classification[Courtesy of Yann LeCun]
Andrew NgAndrew NgNeural Networks: RepresentationMulti-class classification
Machine LearningAndrew Ng39Multiple output units: One-vs-all.
PedestrianCarMotorcycleTruck
Want , , , etc.
when pedestrian when carwhen motorcycleAndrew NgMultiple output units: One-vs-all.
Want , , , etc.
when pedestrian when carwhen motorcycleTraining set: one of , , ,
pedestrian carmotorcycle truckAndrew NgAndrew Ng