Non-linear hypotheses

42
Neural Networks: Representati on Non- linear hypothese s Machine Learning

description

Non-linear hypotheses. Neural Networks: Representation. Machine Learning. Non-linear Classification. x 2. x 1. size. # bedrooms. # floors. age. But the camera sees this:. What is this?. You see this: . Computer Vision: Car detection. Not a car. Cars. Testing : What is this? . - PowerPoint PPT Presentation

Transcript of Non-linear hypotheses

Introduction to Programming

Neural Networks: RepresentationNon-linear hypotheses

Machine Learning1Non-linear Classification

x1x2size

# bedrooms

# floors

age

Andrew Ng

You see this:

But the camera sees this:What is this?Andrew NgComputer Vision: Car detectionTesting:

What is this? Not a carCars

Andrew Ng

Learning Algorithmpixel 1pixel 2pixel 1pixel 2Raw image

CarsNon-Cars

Andrew Ng

pixel 1pixel 2Raw imageCarsNon-CarsLearning Algorithmpixel 1pixel 2Andrew Ng

pixel 1pixel 2Raw imageCarsNon-Cars50 x 50 pixel images 2500 pixels (7500 if RGB)

pixel 1 intensitypixel 2 intensitypixel 2500 intensityQuadratic features ( ): 3 million features

Learning Algorithmpixel 1pixel 2

Andrew NgNeural Networks: RepresentationNeurons and the brain

Machine Learning9Neural NetworksOrigins: Algorithms that try to mimic the brain.Was very widely used in 80s and early 90s; popularity diminished in late 90s.Recent resurgence: State-of-the-art technique for many applicationsAndrew Ng[Roe et al., 1992]

Auditory cortex learns to see

Auditory Cortex

The one learning algorithm hypothesis[Roe et al., 1992]Andrew Ng

Somatosensory cortex learns to see

Somatosensory Cortex

The one learning algorithm hypothesis[Metin & Frost, 1989]Andrew NgSeeing with your tongue

Human echolocation (sonar)

Haptic belt: Direction senseImplanting a 3rd eyeSensor representations in the brain[BrainPort; Welsh & Blasch, 1997; Nagel et al., 2005; Constantine-Paton & Law, 2009]Andrew NgNeural Networks: Representation

Machine LearningModelrepresentation I15

Neuron in the brainAndrew Ng

Neurons in the brain[Credit: US National Institutes of Health, National Institute on Aging]Andrew Ng

Neuron model: Logistic unit

Sigmoid (logistic) activation function.

Andrew NgNeural NetworkLayer 3

Layer 1Layer 2Andrew NgNeural Network

activation of unit in layer

matrix of weights controlling function mapping from layer to layer

If network has units in layer , units in layer , thenwill be of dimension .

Andrew NgNeural Networks: RepresentationModel representation II

Machine Learning22Add .Forward propagation: Vectorized implementation

Andrew NgLayer 3

Layer 1Layer 2Neural Network learning its own featuresAndrew NgLayer 3

Layer 1Layer 2Other network architecturesLayer 4Andrew NgNeural Networks: RepresentationExamples and intuitions I

Machine Learning27Non-linear classification example: XOR/XNOR , are binary (0 or 1).

x1x2x1x2

Andrew NgSimple example: AND

00011011

1.0

Andrew NgExample: OR function

00011011

-102020Andrew NgNeural Networks: RepresentationExamples and intuitions II

Machine Learning32

Negation:

01

Andrew NgPutting it together:

00011011

-30202010-20-20-102020Andrew NgNeural Network intuitionLayer 3

Layer 1Layer 2Layer 4Andrew NgHandwritten digit classification

[Courtesy of Yann LeCun]Andrew NgHandwritten digit classification[Courtesy of Yann LeCun]

Andrew NgAndrew NgNeural Networks: RepresentationMulti-class classification

Machine LearningAndrew Ng39Multiple output units: One-vs-all.

PedestrianCarMotorcycleTruck

Want , , , etc.

when pedestrian when carwhen motorcycleAndrew NgMultiple output units: One-vs-all.

Want , , , etc.

when pedestrian when carwhen motorcycleTraining set: one of , , ,

pedestrian carmotorcycle truckAndrew NgAndrew Ng