SXSW
-
Upload
christopher-friel -
Category
Documents
-
view
67 -
download
0
Transcript of SXSW
![Page 1: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/1.jpg)
hello, world
EKR
Everything you need to know to survive the
upcoming robot apocalypse!
What is Deep Learning?
![Page 2: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/2.jpg)
Whether we are based on carbon or on silicon makes no fundamental difference; we should each be treated with appropriate respect.
Arthur C. Clarke CBE FRAS 2010: Odyssey Two
![Page 3: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/3.jpg)
With artificial intelligence we are summoning the demon
Elon Musk CEO & CTO, SpaceX CEO, Tesla Motors Chairman, Solar City
![Page 4: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/4.jpg)
The development of full artificial intelligence could spell the end of the human race
Stephen Hawking CH CBE FRS FRSA Director of Research, Center for Theoretical Cosmology, Cambridge
![Page 5: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/5.jpg)
hello, world
EKR
With great power comes great
responsibility. How does this stuff work?
That’s Scary!
![Page 6: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/6.jpg)
1+1 = ?A. 0 B. 1 C. 2 D. None of the Above
![Page 7: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/7.jpg)
1+1 = ?A. 0 B. 1 C. 2 D. None of the Above
![Page 8: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/8.jpg)
2*2 = ?A. 2 B. 22 C. 4 D. A and B
![Page 9: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/9.jpg)
2*2 = ?A. 2 B. 22 C. 4 D. A and B
![Page 10: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/10.jpg)
Where is the maximum of this function?
A.
B.
C.
D.
x
y
![Page 11: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/11.jpg)
Where is the maximum of this function?
A.
B.
C.
D.
x
y
![Page 12: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/12.jpg)
hello, world
EKR
How does such a simple thing get to
such a level as “summoning the
demon?”
That’s Just Math!
![Page 13: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/13.jpg)
Going deeper with ConvolutionsGoogle used a new variant of convolutional neural network called “Inception” for classification, and for detection the R-CNN [5] was used. The results and the approach that Google’s team took are summarized here [2, 3]. Google’s team was able to train a much smaller neural network and obtained much better results compared to results obtained with convolutional neural networks in the previous year’s challenges.
1: Computers can recognize objects
![Page 14: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/14.jpg)
Show and Tell: A Neural Image Caption GeneratorAutomatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image.
2: Computers can write image captions
![Page 15: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/15.jpg)
Human-level control through deep reinforcement learningWe tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.
3: Computers can play video games
![Page 16: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/16.jpg)
4: Computers can evolve better models
Mar I/O: Evolving Neural Networks through Augmenting TopologiesNeural networks can be combined with other machine learning techniques to solve complex problems like model selection. Mar I/O is a program made of neural networks and genetic algorithms that kicks butt at Super Mario World.
![Page 17: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/17.jpg)
5: Computers can dream
Twitch Deep Dream VisualizationA convolutional neural network trained to recognize objects in images can be run backwards to dream. Over the spring of 2015, this dreaming neural network was streamed online.
![Page 18: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/18.jpg)
hello, world
EKR
Let’s figure out how to design and use this
stuff?
That’s impressive.
![Page 19: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/19.jpg)
x00
x01
x02
y00Σ φw00
w01
w02
![Page 20: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/20.jpg)
x00
x01
x02
yΣ φ
Input Layer
w
w
w
![Page 21: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/21.jpg)
x
x
x
yΣ φ
Weighted Sum
w00
w01
w02
![Page 22: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/22.jpg)
x
x
x
yΣ φ
Transfer Function
w
w
w
![Page 23: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/23.jpg)
x
x
x
y00Σ φ
Output Layer
w
w
w
![Page 24: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/24.jpg)
Neuron Objective: Choose w00, w01, w02
so that is y00 close to y. Minimize the “approximation error.” By doing that, you’ve learned to reproduce the original function.
y00 = φ(w00 * x00 + w01 * x01 + w02 * x02)
![Page 25: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/25.jpg)
Simple Example: Draw a line through these points by choosing an appropriate value for w.
y = w * x
x
y y y y
A. B. C. D.y = 0.25 * x y = 0.50 * x y = 1.00 * x y = 3.00 * x
![Page 26: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/26.jpg)
x
y y y y
A. B. C. D.y y y = 1.00 * x y
Simple Example: Draw a line through these points by choosing an appropriate value for w.
y = w * x
![Page 27: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/27.jpg)
x
y y y y
A. B. C. D.y y y = 1.00 * x y
Option C. minimizes the classification error of the example. This value of w “learns” how to separate green dots from blue ones!
![Page 28: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/28.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
What about more complex functions? Use more neurons!
![Page 29: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/29.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 30: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/30.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 31: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/31.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 32: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/32.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 33: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/33.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 34: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/34.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 35: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/35.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 36: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/36.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 37: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/37.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 38: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/38.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 39: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/39.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 40: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/40.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Feed-forward Pass
![Page 41: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/41.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 42: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/42.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 43: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/43.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 44: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/44.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 45: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/45.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 46: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/46.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 47: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/47.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 48: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/48.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 49: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/49.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 50: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/50.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 51: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/51.jpg)
x00
x02
x01
Σ φ y00
y01
Σ φ
Σ φ
Σ φ
Σ φ
Σ φ
Backpropagation Pass
![Page 52: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/52.jpg)
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φφ
φ
φφ
φφ
φφ
Big networks can become very complex and hard to train, many layers deep with billions of parameters.
That’s where the term deep learning comes from.
![Page 53: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/53.jpg)
CNNConvolutional
Neural Network
RNNRecurrent Neural
Network
L-STMLong Short-Term
Memory
Q
DQNDeep Q-Network
C
NTM
MR
W
Neural Turing Machine
![Page 54: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/54.jpg)
hello, world
EKR
Why is this so important now if the
idea has been around for decades?
I understand!
![Page 55: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/55.jpg)
Recall Moore’s Law
1970 1975 1980 1985 1990 1995 2000 2005 2010 20151,000
10,000
100,000
1,000,000
10,000,000
100,000,000
1,000,000,000
10,000,000,000
4004
8088
Pentium
Pentium 4
Pascal
80386
Pentium 3
KeplerMaxwell
TeslaNahalem
Core i7
![Page 56: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/56.jpg)
• 2880 CUDA Cores
• 7.1 Billion Transistors
• 15 SMX units
• > 1 TFLOP FP64
• 1.5M L2 Cache
• 384-bit DDR5
GK110 / GTX Titan X, 980, …
![Page 57: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/57.jpg)
hello, world
EKR
Exponentially faster computers have made
zombie algorithms from the 1950s
apocalpyse-capable! What do we do?
Thanks Gordon!
![Page 58: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/58.jpg)
Make Autonomous Weapons?
Autonomous weapons select and engage targets without human intervention. They might include, for example, armed quadcopters that can search for and eliminate people meeting certain pre-defined criteria, but do not include cruise missiles or remotely piloted drones for which humans make all targeting decisions. Artificial Intelligence (AI) technology has reached a point where the deployment of such systems is — practically if not legally — feasible within years, not decades, and the stakes are high: autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear arms.
In summary, we believe that AI has great potential to benefit humanity in many ways, and that the goal of the field should be to do so. Starting a military AI arms race is a bad idea, and should be prevented by a ban on offensive autonomous weapons beyond meaningful human control.
Stephen Hawking Director of research at the Department of Applied Mathematics and Theoretical Physics at Cambridge, 2012 Fundamental Physics Prize laureate for his work on quantum gravity Elon Musk SpaceX, Tesla, Solar City Steve Wozniak, Apple Inc., Co-founder, member of IEEE CS Jaan Tallinn co-founder of Skype, CSER and FLI Frank Wilczek MIT, Professor of Physics, Nobel Laureate for his work on the strong nuclear force Max Tegmark MIT, Professor of Physics, co-founder of FLI Daniel C. Dennett, Tufts University, Professor, Co-Director, Center for Cognitive Studies, member of AAAI Noam Chomsky MIT, Institute Professor emeritus, inductee in IEEE Intelligent Systems Hall of Fame, Franklin medalist in Computer and Cognitive Science Barbara Simons IBM Research (retired), Past President ACM, ACM Fellow, AAAS Fellow Stephen Goose Director of Human Rights Watch's Arms Division Anthony Aguirre, UCSC, Professor of Physics, co-founder of FLI Lisa Randall, Harvard, Professor of Physics Martin Rees Co-founder of CSER and Astrophysicist
Stuart Russell Berkeley, Professor of Computer Science, director of the Center for Intelligent Systems, and co-author of the standard textbook “Artificial Intelligence: a Modern Approach" Nils J. Nilsson, Department of Computer Science, Stanford University, Kumagai Professor of Engineering, Emeritus, past president of AAAI Barbara J. Grosz Harvard University, Higgins Professor of Natural Sciences, former president AAAI, former chair of IJCAI Board of Trustees Tom Mitchell CMU, past president of AAAI, Fredkin University Professor and Head of the Machine Learning Department Eric Horvitz, Microsoft Research, Managing director, Microsoft Research, past president of AAAI, co-chair of AAAI Presidential Panel on Long-term AI Futures, member of ACM, IEEE CIS Martha E. Pollack University of Michigan, Provost, Professor of Computer Science & Professor of Information, past president of AAAI, Fellow of AAAS, ACM & AAAI Henry Kautz, University of Rochester, Professor of Computer Science, past president of AAAI, member of ACM Demis Hassabis, Google DeepMind, CEO Yann LeCun, New York University & Facebook AI Research, Professor of Computer Science & Director of AI Research Oren Etzioni, Allen Institute for AI, CEO, member of AAAI, ACM Peter Norvig, Google, Research Director, member of AAAI, ACM Geoffrey Hinton University of Toronto and Google, Emeritus Professor, AAAI Fellow Yoshua Bengio, Université de Montréal, Professor
![Page 59: SXSW](https://reader034.fdocuments.us/reader034/viewer/2022042821/55d1f958bb61eb721e8b472b/html5/thumbnails/59.jpg)
Engage with deep learning, it’s powerful; please don’t make evil killer robots. #ekr
Chris Friel [email protected]
@cfriel