ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez...
-
Upload
kylee-shapley -
Category
Documents
-
view
214 -
download
0
Transcript of ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez...
![Page 1: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/1.jpg)
ISTC-EC @ Cornell
Accelerating Belief Propagation in Hardware
Skand Hurkat and José MartínezComputer Systems Laboratory
Cornell Universityhttp://www.csl.cornell.edu/
![Page 2: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/2.jpg)
ISTC-EC @ Cornell
The Cornell Team
• Prof. José Martínez (PI), Prof. Rajit Manohar@ Computer Systems Lab
• Prof. Tsuhan Chen@ Advanced Multimedia Processing Lab
• MS/Ph.D. students– Yuan Tian, MS ’13– Skand Hurkat– Xiaodong Wang
![Page 3: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/3.jpg)
ISTC-EC @ Cornell
The Cornell Graph
![Page 4: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/4.jpg)
ISTC-EC @ Cornell
The Cornell Project
• Provide hardware accelerators for belief propagation algorithms on embedded SoCs (retail/car/home/mobile)– High speed– Very low power– Self-optimizing– Highly programmable
BP Accelerator within SoC
Graph Inference Algorithm
Result
![Page 5: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/5.jpg)
ISTC-EC @ Cornell
What is belief propagation?
Belief propagation is a message passing algorithm for performing inference on graphical models, such as Bayesian networks or Markov Random Fields
![Page 6: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/6.jpg)
ISTC-EC @ Cornell
What is belief propagation?
• Labelling problem• Energy as a measure of convergence• Minimize energy (MAP label estimation)• Exact results for trees– Converges in exactly two iterations
• Approximate results for graphs with loops– Yields “good” results in practice• Minimum over large neighbourhoods• Close to optimal solution
![Page 7: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/7.jpg)
ISTC-EC @ Cornell
Not all “that” alien to embedded
𝑠0
𝑠11
𝑠12
𝑠13
𝑠21
𝑠22
𝑠23
𝑠31
𝑠32
𝑠33
𝑠41
𝑠4 2
𝑠4 3
𝑠5𝑠0 𝑠1 𝑠2 𝑠3 𝑠4 𝑠5
Remember the Viterbi algorithm?• Used extensively in digital communications
![Page 8: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/8.jpg)
ISTC-EC @ Cornell
What does this mean?
• Every mobile device uses Viterbi decoders– Error correction codes (eg: turbo codes)– Mitigating inter-symbol interference (ISI)
• Increasing number of mobile applications involve belief propagation– More general belief propagation accelerators can
greatly improve user experience with mobile devices
![Page 9: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/9.jpg)
ISTC-EC @ Cornell
Target markets
Retail/Car/Home/Mobile• Image processing
– De-noising– Segmentation– Object detection– Gesture recognition
• Handwriting recognition– Improved recognition through
context identification
• Speech recognition– Hidden Markov models are
key to speech recognition
Servers• Data mining tasks
– Part-of-speech tagging– Information retrieval– “Knowledge graph” like
applications
• Machine learning based tasks– Constructive machine learning– Recommendation systems
• Scientific computing– Protein structure inference
![Page 10: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/10.jpg)
ISTC-EC @ Cornell
Hardware accelerator for BP
BP Accelerator within SoC
Graph Inference Algorithm
Result
![Page 11: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/11.jpg)
ISTC-EC @ Cornell
Work done so far
Software• General purpose MRF inference
library– Support for arbitrary graphs– Floating point math– Parallel techniques for faster
inference
• Library optimized for grid graphs– Optimized data structures– Template can use any data type– Multiple inference techniques
optimized for early vision– Stereo matching in 200 ms
Hardware• High level synthesis of message
update unit– Vivado HLS (C-to-gates) tool used to
synthesize message update unit on ZedBoard
– ∼2x improvement in inference speed on CPU+FPGA compared to CPU-only inference
– Fixed point math
• GraphGen collaboration– On-going work– Stereo matching task mapped to
multiple platforms– 10x speedup on GPU w.r.t. CPU only
implementation
![Page 12: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/12.jpg)
ISTC-EC @ Cornell
Work done so far
Software• General purpose MRF inference
library– Support for arbitrary graphs– Floating point math– Parallel techniques for faster
inference
• Library optimized for grid graphs– Optimized data structures– Template can use any data type– Multiple inference techniques
optimized for early vision– Stereo matching in 200 ms
Hardware• High level synthesis of message
update unit– Vivado HLS (C-to-gates) tool used to
synthesize message update unit on ZedBoard
– ∼2x improvement in inference speed on CPU+FPGA compared to CPU-only inference
– Fixed point math
• GraphGen collaboration– On-going work– Stereo matching task mapped to
multiple platforms– 10x speedup on GPU w.r.t. CPU only
implementation
![Page 13: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/13.jpg)
ISTC-EC @ Cornell
Work done so far
Software• General purpose MRF inference
library– Support for arbitrary graphs– Floating point math– Parallel techniques for faster
inference
• Library optimized for grid graphs– Optimized data structures– Template can use any data type– Multiple inference techniques
optimized for early vision– Stereo matching in 200 ms
Hardware• High level synthesis of message
update unit– Vivado HLS (C-to-gates) tool used to
synthesize message update unit on ZedBoard
– ∼2x improvement in inference speed on CPU+FPGA compared to CPU-only inference
– Fixed point math
• GraphGen collaboration– On-going work– Stereo matching task mapped to
multiple platforms– 10x speedup on GPU w.r.t. CPU only
implementation
![Page 14: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/14.jpg)
ISTC-EC @ Cornell
Hierarchical belief propagation
![Page 15: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/15.jpg)
ISTC-EC @ Cornell
Results – Stereo Matching
BP-S
BP-M (lo
gspac
e)
Hierarch
ical
Residual
hierarch
ical
440000
450000
460000
470000
480000
02000000400000060000008000000100000001200000014000000
Comparing inference algorithms on “Tsukuba” benchmark
UpdatesEnergy
Energy
Updates
![Page 16: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/16.jpg)
ISTC-EC @ Cornell
Work done so far
Software• General purpose MRF inference
library– Support for arbitrary graphs– Floating point math– Parallel techniques for faster
inference
• Library optimized for grid graphs– Optimized data structures– Template can use any data type– Multiple inference techniques
optimized for early vision– Stereo matching in 200 ms
Hardware• High level synthesis of message
update unit– Vivado HLS (C-to-gates) tool used to
synthesize message update unit on ZedBoard
– ∼2x improvement in inference speed on CPU+FPGA compared to CPU-only inference
– Fixed point math
• GraphGen collaboration– On-going work– Stereo matching task mapped to
multiple platforms– 10x speedup on GPU w.r.t. CPU only
implementation
![Page 17: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/17.jpg)
ISTC-EC @ Cornell
Work done so far
Software• General purpose MRF inference
library– Support for arbitrary graphs– Floating point math– Parallel techniques for faster
inference
• Library optimized for grid graphs– Optimized data structures– Template can use any data type– Multiple inference techniques
optimized for early vision– Stereo matching in 200 ms
Hardware• High level synthesis of message
update unit– Vivado HLS (C-to-gates) tool used to
synthesize message update unit on ZedBoard
– ∼2x improvement in inference speed on CPU+FPGA compared to CPU-only inference
– Fixed point math
• GraphGen collaboration– On-going work– Stereo matching task mapped to
multiple platforms– 10x speedup on GPU w.r.t. CPU only
implementation
![Page 18: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/18.jpg)
ISTC-EC @ Cornell
GraphGen synthesis of BP-M
• BP-M update (logspace messages) implemented using GraphGen (Intel/CMU/UW)
• GPU implementation 10x faster than CPU based implementation
• On-going work on FPGA based implementation and on implementing hierarchical update
![Page 19: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/19.jpg)
ISTC-EC @ Cornell
Cornell Publications (2013 only)
• 3x Comp. Vision & Pattern Recognition (CVPR)• 3x Asynchronous VLSI (ASYNC)• 2x Intl. Symp. Computer Architecture (ISCA)• 1x Intl. Conf. Image Processing (ICIP)
• 1x ASPLOS (w/ GraphGen folks, under review)
![Page 20: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/20.jpg)
ISTC-EC @ Cornell
Year 3 Plans
• GraphGen extensions for BP applications– Multiple inference techniques
• Extraction of “BP ISA”– Ops on arbitrary graphs– Efficient representation
• Amplification work on UAV ensembles– Self-optimizing, collaborative SoCs
• One-day “graph” workshop with GraphGen+UIUC
![Page 21: ISTC-EC @ Cornell Accelerating Belief Propagation in Hardware Skand Hurkat and José Martínez Computer Systems Laboratory Cornell University](https://reader035.fdocuments.us/reader035/viewer/2022070308/551c3b1e550346a3488b480b/html5/thumbnails/21.jpg)
ISTC-EC @ Cornell
Accelerating Belief Propagation in Hardware
Skand Hurkat and José MartínezComputer Systems Laboratory
Cornell Universityhttp://www.csl.cornell.edu/