Communication-Minimizing 2D Convolution in GPU Registers

1

Communication-Minimizing 2D Convolution in GPU Registers

Forrest N. Iandola David SheffieldMichael AndersonP. Mangpo Phothilimthana Kurt Keutzer

University of California, Berkeley

[email protected]

2Forrest Iandola [email protected]

Overview

• Convolution is a recurring computational pattern in a broad range of computer vision applications

• Memory communication is the bottleneck for convolution on modern GPUs

• How to minimize memory communication overhead in convolution– Texture cache– Loop blocking

• Up to 4.5x speedup over existing GPU implementations from NVIDIA, OpenCV, and others

3

Why focus on convolution?• Berkeley ParLab project identified 15 recurring

computational patterns in computer vision

Forrest Iandola [email protected]

• Small filters (2x2 – 7x7)• Feature extraction • Sliding-window object

detection

• If we want fast computer vision, we need fast convolution

CVPR 2007 – 2011

object recognition track

15 C

ompu

ter V

isio

n P

atte

rns


What limits the performance of convolution?

• Roofline model [1] divides a program’s execution time into two parts:– Computational cost (GFLOPS/s)– Communication cost (GB/s) – memory traffic, I/O, etc.

• No program can outperform the hardware bound on computation or communication

[1] S. Williams, A. Waterman, D. Patterson. Roofline: An Insightful Visual Performance Model for Floating Point Programs and Multicore Architectures. Communications of the ACM, 2009

5


Roofline Model of computational performance


Fast

Slow

Memory Bounded Computation Bounded



• Convolution on NVIDIA GPUs:– Communication between the GPU’s off-chip DRAM and

on-chip caches is the bottleneck– This doesn’t include communication between the CPU

and GPU, though this can also be an issue• If we want fast computer vision, we need fast

convolution.• If we want fast convolution on GPUs, we need

to optimize memory communication.

7

Exploiting the GPU Memory Architecture

GPU Global Memory (DRAM)

Texture Cache

Registers L1 Cache / Shared Memory

893 GB/s

123 GB/s

Memory per GPU Multiprocessor

129 Gtexels/s

CPU DRAM NVIDIA GTX6808 GB/s

L2 Cache

Optimization 1:

Use the Texture Cache

Threads

8

Data Reuse with Loop Blocking

Typical Implementation: no data reuse at the register level


9 input pixels

1 output pixel


Data Reuse with Loop Blocking

Our approach: reuse data by doing more work per thread

Optimization 2:

Block the image in registers

9 input pixels

1 output pixel

16 input pixels

4 output pixels

4 inputs per output

Typical Implementation: no data reuse at the register level

11

Comparison with Related Work

NVIDIA GTX680(Kepler)

Inverse roofline model

12



With texture cacheand blocking (ours)

13



14



15



16



17


4.5x speedup NVIDIA

GTX680(Kepler)


Are we done?

• Are we done optimizing memory communication?• I think so. We achieved the memory bandwidth

bound for small filters.

• Future work: optimize computation some more!

19

Conclusions

• If we want fast computer vision, we need fast convolution.

• If we want fast convolution on GPUs, we need to optimize memory communication.

• Up to 4.5x faster than existing GPU languages and libraries

• Download our code! https://github.com/forresti/convolution– Use/modify it for your

language/library/application


https://github.com/forresti/convolution



Communication-Minimizing 2D Convolution in GPU Registers

Documents

Transcript of Communication-Minimizing 2D Convolution in GPU Registers