Rachel MillerResearch Computing Lab

CUDA is a programming language that uses the Graphical Processing Unit (GPU)

Allows calculations to be performed in parallel, giving significant speedup

Used with C programs

http://members.tripod.com/~Michael_Art/Animal_Fun/Baracuda.gif

GPUs are designed to make high speed parallel calculations for displaying graphics, such as games

Use available resources! Over 100 million GPUs are already deployed

30-100x Speed-up over other microprocessors for some applications

GPUs have lots of small Arithmetic Logic Units (ALUs), compared to a few larger ones on the CPU

This allows for many parallel computations, like calculating a color for each pixel on the screen

Image from NVIDIA CUDA Programming Guide

GPUs run one kernel (a group of work) at a time

Each kernel has blocks, which are independent groups of ALUs

Each block is comprised of threads, which are the level of computation

The threads in each block typically work together to compute a value

Host

Kernel 1

Kernel 2

Device

Grid 1

Block(0, 0)

Block(1, 0)

Block(2, 0)

Block(0, 1)

Block(1, 1)

Block(2, 1)

Grid 2

Block (1, 1)

Thread(0, 1)

Thread(1, 1)

Thread(2, 1)

Thread(3, 1)

Thread(4, 1)

Thread(0, 2)

Thread(1, 2)

Thread(2, 2)

Thread(3, 2)

Thread(4, 2)

Thread(0, 0)

Thread(1, 0)

Thread(2, 0)

Thread(3, 0)

Thread(4, 0)

Image from NVIDA

Threads within the same block can share memory

In CUDA, sending information from the CPU to the GPU is often the most expensive part of the calculation

For each thread, local memory is fastest, followed by shared memory; global, constant and texture memory are all slowest

(Device) Grid

ConstantMemory

TextureMemory

GlobalMemory

Block (0, 0)

Shared Memory

LocalMemory

Thread (0, 0)

Registers

LocalMemory

Thread (1, 0)

Registers

Block (1, 0)

Shared Memory

LocalMemory

Thread (0, 0)

Registers

LocalMemory

Thread (1, 0)

Registers

Host

Image from NVIDA

Each thread “knows” the x and y coordinates of the block it is in, and the coordinates of where it is in the block

These positions can be used to compute a unique thread ID for each thread

The computational work done will depend on the value of this thread ID

Example: the thread ID corresponds to a group of matrix elements

All threads in a block will run in parallel IF they are all following the same code; its important to eliminate logical branches, to keep all threads running at the same time

Threads can only reference local memory and shared memory, so any needed information should be put into shared memory

CUDA applications should run parallel operations on lots of data, and be processing intensive

Examples: Molecular Dynamics Simulation Video/Audio Encoding, Manipulation 3D Imaging and Visualization Matrix Operations

These collisions of thousands of tiny balls runs real time on a desktop computer! (And looks better there, too.)

Watch a better version athttp://www.youtube.com/watch?v=RqduA7myZok

Over 170 premade CUDA tools exist, and would be useful building blocks for applications Areas include Imaging, Video & Audio

Processing, Molecular Dynamics, Signal Processing

CUDA can also help an existing application meet its need for speed Process huge datasets faster Can achieve close to real time data processing

Nvidia (the makers of CUDA) created a MATLAB plug-in for accelerating standard MATLAB 2D FFTs

CUDA has a graphics toolbox for MATLAB

More MATLAB plug-ins to come!