Leveraging Microsoft Azure’s GPU N-Series for Compute and Visualization

Karan Batta, Program Manager, Microsoft Azure

Alexey Kamenev, Software Engineer, Microsoft Research

S6839

Agenda

Azure

HPC in the Cloud

Technology/Architecture

CNTK Overview & Demo

Vision

GPU based capabilities in cloud infrastructure

High end performance

Not “Swiss-army knife” approach

Deliver and empower developer scenarios

Achieve true “HPC in the Cloud”

Critical workloads

HPC in the Cloud

APP

exe

exe

exe

exe

Workflow

Rendering Algorithm

Executable

Azure

GPU VMs

Upload data

Submit job

Split job/ setup execution pipeline

Manage

GPU Visualization

Analytics Dynamic Modelling Virtual Desktops

{REST

AP

I}

Return results

Outputs

Where?

Finance

• FX Options

• Risk Management

• Hedge Fund Management

Manufacturing & Oil/Gas

• Automotive design

• Reservoir modelling

• Manipulation of models & parts

Media

• Streaming games/video

• Transcoding

• Social media analysis

Rendering

• VFX/Ray-Tracing rendering

• CAD applications

• Simulations

Technology

DDA (Discrete Device Assignment)

Introduced in Windows Server 2016

Pass-through PCIe devices

Allows for close to bare-metal performance

Architecture

Applications

GPU Provisioning

Host OS

Client OS

Hardware

• Custom Applications

• Data and Applications from the Azure Marketplace

• Bring your own Image

• Azure VM Marketplace Images

• Hyper-V

• DDA

• NVIDIA M60 GPU (Viz SKU)

• NVIDIA K80 GPU (Compute SKU)

Visualization VMs

NV6 NV12 NV24

Cores 6

(E5-2690v3) 12

(E5-2690v3) 24

(E5-2690v3)

GPU 1 x M60 GPU (1/2

Physical Card) 2 x M60 GPU (1 Physical Card)

4 x M60 GPU (2 Physical Cards)

Memory 56 GB 112 GB 224 GB

Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD

Network Azure Network Azure Network Azure Network

Compute VMs

NC6 NC12 NC24 NC24r

Cores 6

(E5-2690v3) 12

(E5-2690v3) 24

(E5-2690v3) 24

(E5-2690v3)

GPU 1 x K80 GPU (1/2

Physical Card) 2 x K80 GPU (1 Physical Card)

4 x K80 GPU (2 Physical Cards)

4 x K80 GPU (2 Physical Cards)

Memory 56 GB 112 GB 224 GB 224 GB

Disk ~380 GB SSD ~680 GB SSD ~1.5 TB SSD ~1.5 TB SSD

Network Azure Network Azure Network Azure Network Azure Network +

RDMA (RoCE)

CNTK Alexey Kamenev

Senior Software Engineer

Microsoft Research

CNTK Overview

• A deep learning tool that balances • Efficiency: Can train production systems as fast as possible

• Performance: Can achieve state-of-the-art performance on benchmark tasks and production systems

• Flexibility: Can support various tasks such as speech, image, and text, and can try out new ideas quickly

• Inspiration: Legos • Each brick is very simple and performs a specific function

• Create arbitrary objects by combining many bricks

• CNTK enables the creation of existing and novel models by combining simple functions in arbitrary ways.

• Historical facts: • Created by Microsoft Speech researchers (Dong Yu et al.) 4 years ago

• Was quickly extended to handle other workloads (image/text)

• Open-sourced (CodePlex) in early 2015

• Moved to GitHub in Jan 2016

Resources

• “Deep Learning in Microsoft with CNTK” – Alexey Kamenev, Microsoft – Hall 3 – 4.30pm

• CNTK (Deep-Learning toolkit) • https://github.com/Microsoft/CNTK

• DDA (Direct Device Assignment) • http://blogs.technet.com/b/virtualization/archive/2015/11/23/discrete-

device-assignment-gpus.aspx

• NVIDIA announcement • http://nvidianews.nvidia.com/news/nvidia-gpus-to-accelerate-microsoft-

azure

Thank you!