KERAS/TENSORFLOW

François FayardBAYNCORE

November 29, 2017PARIS

2

Deep LearningThe perceptron

The Mark I Perceptron MachineCornell Aeronautical Laboratory

(1957)

3

Classifying points in the plane

P = (x1, x2)

4

The Perceptron

The point P = (x1, x2) will be classified as:

• blue if u⍺(x1, x2) > 0

• red if u⍺(x1, x2) < 0

5

A Perceptron is a linear classifier

P = (x1, x2)

6

The loss function for a classifier and a given point

For any given point P = (x1, x2), we define the loss L(⍺, P) by:

7

The loss function for a classifier

• The loss for a classifier u⍺ is defined as

• We seek for the classifier with the smallest loss

8

Stochastic Gradient Descent

• To minimize L:

• We start with a random value ⍺

• We compute the gradient of L analytically (backpropagation) using only a random subset of the images (mini batch)

• We update ⍺ by

where η is the learning rate

• We iterate the process until convergence

9

Introducing Keras

• Keras is an API specification for building Deep Learning models across platforms

Keras API specification

TensorFlow-Keras …Theano-Keras

10

Installing Keras & TensorFlow

11

Installing Keras is easy with Intel

Intel Confidential

12

Training a neural network is easy with Keras

Intel Confidential

13

Linear classifiers don’t always do the job

P = (x1, x2)

14

Hidden LAYERSOur first hidden layer

15

Our first hidden layer

16

The activation function

17

Training a neural network is easy with Keras

18

Deep learning does the job

P = (x1, x2)

19

MNIST

20

Reading digits from MNIST

21

Deep Learning at Bell Labs: Late 80s

Yann Le CunDirector of AI Research

Facebook

22

Defining the neural network

23

Defining the neural network

24

The learning phase

25

Benchmarks: Intel TensorFlow

26

The benchmark: AlexNetWon the ImageNet Challenge in 2012. Topology:• 5 convolutional layers• 3 fully connected layers• 60 million parameters, 650 000 neurons

27

Google TensorFlow performance on AlexNet

0

50

100

150

200

250

300

350

400

Original

Imag

es p

er s

econ

d

Training - Google Inference - Google

28

Running vTune on Google TensorFlow

29

Google/Intel TensorFlow performance on AlexNet

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Original

Num

ber o

f im

ages

per

sec

ond

Inference - Google Inference - Intel

0

200

400

600

800

1000

1200

1400

1600

1800

2000

Original

Num

ber o

f im

ages

per

sec

ond

Training - Google Training - Intel

30

Running vTune on Intel TensorFlow

31

Take Home Messages

• Starting Deep Learning is easy with Keras

• Keras is better seen as an API. It can be used with different frameworks.

• Use Intel Optimized Python Distribution for better performance

Questions

32