Deep Learning with Coherent Nanophotonic Circuits
Yichen Shen, Nicholas Harris, Dirk Englund, Marin Soljacic
Massachusetts Institute of Technology
@ Berkeley, Oct. 2017
11/8/2017 1Deep Learning with Coherent Nanophotonic Circuits
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits2
Neuromorphic Computing
Biological Neural Networks Artificial Neural Networks
Artificial Neural Networks (ANN)
11/8/2017 3Deep Learning with Coherent
Nanophotonic Circuits
Breakthroughs in deep learning: • Natural Language
Processing (NLP)• Game Playing (Go, Atari)• Autonomous Vehicles• Control• Ad Placement• Researches (drug discovery,
material study)• Etc.
Basic Algorithm of ANN
Deep Learning with Coherent
Nanophotonic Circuits411/8/2017
Matrix Multiplication:
Nonlinear Activation:
…
… …
…
1
2
9
Hardware and Data Enable Deep Learning
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits5
The Need for SpeedMore Data Bigger Models More need for Computation
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits6
But Moore’s Law is no-longer providing more computation…
The Market:
On clouds:Millions of high power AI processors ($10,000 each) in data centers by 2020
On premise:Billions of compact AI processors needed due to the rise of autonomouse driving, AR and IoT.
Von Neumann ASIC/FPGA Optical Processing
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits7
Optical AI Computing
In Deep Learning
Key Operation is dense M x V
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits8
ajbi
Wij= x
In Optics, Matrix Multiplication
is very common & (usually)
consumes no energy !
Convolution / FFT
Matrix
Multiplication
Programmable Nanophotonic Processors
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits9
100µm
S U ( 4 ) C o r e D M M C
a
b
c
Phase Shifter
Waveguide
Transm
issio
n
Voltage2
O p t i c a l I n t e r f e r e n c e U n i t ( O I U )
MZI
Detectors
Input Modes
φi✓i
J. Mower et al, Physical Reviews A, 92, 032322 (2015)Carolan, Jacques, et al. "Universal linear optics." Science 349.6249 (2015): 711-716.
ANN does NOT require high resolution
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits10
Sze et al, arXiv:1703.09039 (2017)
Deep Learning Inference is “Passive”
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits11
Once the Optical Neural Network is trained, no
need to update the weights frequently…
Deep Learning is very parallelizable
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits12
Multiple wavelengths can be used to
simultaneously execute batch of data
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits13
Coherent Optical Neural Networks (ONN)
X1
X2
X3
X4
h1(1)
h2(1)
h3(1)
h4(1)
Z (1) = W0X
h1(i)
h2(i)
h3(i)
h4(i)
h1(n)
h2(n)
h3(n)
h4(n)
Y1
Y2
Y3
Y4
h( i ) = f (Z ( i ) ) Y = Wn h(n )
Input LayerHidden Layers
Output Layer
Layer i Layer n
Optical Input Optical Output
Layer 1X Y
a
b
c
x i n xou t
Waveguide
0
0
Optical Interference Unit Optical Nonlinearity Unit
0
0
d
S(n)V (n) U (n)
Vowel X
M (n)M (1) = U (1)S(1)V (1) M (2) M (n− 1)
fNL()fNL()
Photonic Integrated Circuit
fNL
Optical Vowel Recognition
(4d 4 classes)
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits14
11/8/2017Deep Learning with Coherent Nanophotonic Circuits 15
Y. Shen and N. Harris et al, Nature Photonics, 11, 441-446 (2017)
da
bLaser OIU Detectors Computer
U1S1V1
Tra
nsm
issio
nOIU1
OIU2 CPU OIU
3OIU
4
U1S1V1fSA ( Iin)
Input Output
Instance
Insta
nce
60µm
Optical Interference Unit
SU(4) Core DMMC
Dq Df
c
V2Dq
18µm
Directional Coupler
Phase ShifterSimulated Optical Nonlinearity
Experimental Result
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits16
Simulation Result: 165/180=91.7%Experiment Result: 138/180=76.7%
VowelIdentified
VowelIdentified
Vo
wel
Sp
ok
en
Vo
wel
Sp
ok
en
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits17
Fully Connected Neural Networks
Recurrent Neural Networks Convolutional Neural Networks
Recurrent Neural Networks
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits18
Commonly used for Speech Recognition and Language Processing
Convolution Neural Networks
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits19
Scott Skirlo and Yichen Shen et al, Manuscript in Preparation
Optical Convolutional Neural Network
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits20
Scott Skirlo and Yichen Shen et al, Manuscript in Preparation
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits21
Unified Buffer (localStorage SRAM)
ADC arrays
*modified block diagram from TPU architecture
Speed and Energy Efficiency Comparison with Electrical ANN
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits22
NVIDIA TITAN X ONN (with thermal PS)
Architecture Von Neumann Neuromorphic
Power Consumption 1 kW 1-2 kW
Operation Speed 10 TFLOP 10,000 TFLOP
Y. Shen and N. Harris et al, Nature Photonics 11, 441 (2017)
11/8/2017Deep Learning with Coherent Nanophotonic Circuits 23
Some History on Optical Neural Networks
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits24
Acknowledgement
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits25
Nicholas HarrisPhD, EECSMIT
Scott SkirloPhD, Physics MIT
Prof. Marin SoljacicPhysics, MIT
Prof. Dirk EnglundEECS, MIT
Li JingPhD, Physics MIT
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits26
Optical Convolutional Neural Network
Scott Skirlo and Yichen Shen et al, Manuscript in Preparation
Nonlinearity
11/8/2017Deep Learning with Coherent
Nanophotonic Circuits27
A. Selden, British Journal of Applied Physics 18 , 743 (1967)M. Soljacic, Physical Review E 66, 055601 (2002)Z. Cheng et al, IEEE Journal of Selected Topics in Quantum Electronics 20.1 (2014): 43-48.
Nonlinear Photonic System
Iin Iout=f(Iin)
Saturable Absorption Photodiode
For Deep Learning, the constraint on nonlinearity is weak
Top Related