APPG AI

DEEP LEARNING DEMYSTIFIED

2

ABOUT ME

- Deep Learning Solution Architect @ NVIDIA - Supporting delivery of AI / Deep Learning solutions

- 10 years expereince deliverying Machine Learning of all scale (from embedded, mobile to Big Data)

- My past experience:

- Capgemini: https://goo.gl/MzgGbq

- Jaguar Land Rover Research: https://goo.gl/ar7LuU

Adam Grzywaczewski – adamg@nvidia.com

3

NEURAL NETWORKS

4

NEURAL NETWORKS ARE NOT NEWAnd are disapintingly simple as an algorithm

5

NEURAL NETWORKS ARE NOT NEWAnd have historically never worked

6

WHY NOW?

7

LIFE AFTER MOORE’S LAW40 Years of Microprocessor Trend Data

1980 1990 2000 2010 2020

102

103

104

105

106

107

Single-threaded perf

1.5X per year

1.1X per yearTransistors

(thousands)

GPU-Computing perf

1.5X per year 1000X

By 2025

8

2016 – Baidu Deep Speech 2Superhuman Voice Recognition

2015 – Microsoft ResNetSuperhuman Image Recognition

2017 – Google Neural Machine TranslationNear Human Language Translation

100 ExaFLOPS8700 Million Parameters

20 ExaFLOPS300 Million Parameters

7 ExaFLOPS60 Million Parameters

For both compute and data

NEURAL NETWORK ARE GREEDY

9

100 EXFLOPS = 2 YEARS ON A DUAL SOCKET CPU SERVER

10

ONCE A FAILED MACHINE LEARNING EXPERIMENT

11

WHAT IS POSSIBLE TODAY?Overwhelming majority of our day to day activities

Based on a presentation from Andrew Ng

“If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.”

Andrew Ng , Founder of Google Brain

12

EMPIRICAL EVIDENCE

13

14

MAKING IMPOSSIBLE PROBLEMS EXPENSIVE

Hestness, J., Narang, S., Ardalani, N., Diamos, G., Jun, H., Kianinejad, H., ... & Zhou, Y. (2017). Deep Learning Scaling is Predictable, Empirically. arXiv preprint arXiv:1712.00409.

With enough data we can find an approximation of every problem

• Translation

• Language Models

• Character Language Models

• Image Classification

• Attention Speech Models

15

MORE THAN JUSTIFIABLE EXCITEMENT

16

FUNDAMENTAL CHANGE TO THE ECONOMYImpact

17

AI IS THE NEW ELECTRICITY

“Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the

next several years, …”

Andrew Ng , Founder of Google Brain

Affecting every aspect of our lives

18

CALL TO ACTION

19

ENSURE VISIBILITY

It will affect all aspects of our lives so all departments

(they need skills to make unbiased decisions)

Entire UK Government needs to understand the technology

20

SUPPORT EDUCATION

It is possible to teach logic, software development AI and robotics in primary schools

(more important than ever)

At all levels

21

SUPPORT RESEARCH

Just a single research department of a large internet company has 10x more research infrastructure than the largest UK AI cluster.

There is no escaping from large datasets and as a consequence large compute.

Need to compete with large US and Chinese organisation

22

LAB OVERVIEW

23

DEEP LEARNING INSTITUTEDLI Mission

Helping people solve challenging problems using AI and deep learning.

• Developers, data scientists and engineers

• Self-driving cars, healthcare and robotics

• Training, optimizing, and deploying deep neural networks

24

HOW IT WORKS

25

HOW IT WORKS

26

HOW IT WORKS

27

HOW IT WORKS

28

LAUNCHING THE LAB

29

NAVIGATING TO QWIKLABS

1. Navigate to: https://nvlabs.qwiklab.com

1. Login or create a new account

30

ACCESSING LAB ENVIRONMENT

3. Select the event specific In-Session Class in the upper left

3. Click the “Image Classification with DIGITS” Class from the list

31

LAUNCHING THE LAB ENVIRONMENT

5. Click on the Select button to launch the lab environment

• After a short wait, lab Connection information will be shown

• Please ask Lab Assistants for help!

32

LAUNCHING THE LAB ENVIRONMENT

6. Click on the Start Lab button

33

LAUNCHING THE LAB ENVIRONMENT

You should see that the lab environment is “launching” towards the upper-right corner

34

CONNECTING TO THE LAB ENVIRONMENT

7. Click on “here” to access your lab environment / Jupyter notebook

35

CONNECTING TO THE LAB ENVIRONMENT

You should see your “Image Classification with DIGITS” Jupyternotebook

36

JUPYTER NOTEBOOK

1. Place your cursor in the code

2. Click the “run cell” button

2. Confirm you receive the same result

37

STARTING DIGITS

Instruction in Jupyter notebook will link you to DIGITS

38

ACCESSING DIGITS

• Will be prompted to enter a username to access DIGITS

• Can enter any username

• Use lower case letters