With many contributors:A. Agarwal, E. Akchurin, E. Barsoum, C. Basoglu, G. Chen, S. Cyphers, W. Darling, J. Droppo, A. Eversole, B. Guenter, M. Hillebrand, X. Huang, Z. Huang, R. Hoens, V. Ivanov, A. Kamenev, N. Karampatziakis, P. Kranen, O. Kuchaiev, W. Manousek, C. Marschner, A. May, B. Mitra, O. Nano, G. Navarro, A. Orlov, S. Pathak, M. Radmilac, A. Reznichenko, P. Parthasarathi, B. Peng, A. Reznichenko, W. Richert, F. Seide, M. Seltzer, M. Slaney, A. Stolcke, T. Will, H. Wang, W. Xiong, K. Yao, D. Yu, C. Zhang, Y. Zhang, G. Zweig

Objectives

• Describe the Microsoft Cognitive Toolkit (CNTK)• Comparative evaluation with similar technologies• Outline potential applications• Show Jupyter Python demos

Assumptions

• Intermediate knowledge of programming• Intermediate knowledge of data science or machine learning

• Stay Calm: You may fill in gaps later

What is Cognitive Toolkit (CNTK)

The Microsoft Cognitive Toolkit (CNTK)

• Microsoft’s open-source deep-learning toolkit• ease of use: what, not how• fast• flexible

• Supported on both Linux and Windows• Fully open development framework

• CNTK is production-ready: State-of-the-art accuracy, efficient, and scales to multi-GPU/multi-server.

• CNTK expresses (nearly) arbitrary neural networks by composing simple building blocks into complex computational networks, supporting relevant network types and applications.

The Microsoft Cognitive Toolkit (CNTK)

Microsoft’s historicspeech breakthrough • Microsoft 2016 research system for

conversational speech recognition

• 5.9% word-error rate

• enabled by CNTK’s multi-server scalability

[W. Xiong, J. Droppo, X. Huang, F. Seide, M. Seltzer, A. Stolcke,D. Yu, G. Zweig: “Achieving Human Parity in ConversationalSpeech Recognition,” https://arxiv.org/abs/1610.05256]

Microsoft’s historicspeech breakthrough • Microsoft 2017 research system for

conversational speech recognition

• 5.1% word-error rate

• “We introduced an additional CNN-BLSTM (convolutional neural network combined with bidirectional long-short-term memory) model for improved acoustic modeling."

• Additionally, Microsoft enlisted its suite of AI technologies called the Cognitive Toolkit (CNTK) in its record-setting bid.

Comparative Evaluation

28.225.8

16.4

11.7

7.3 6.7

3.60.0

5.0

10.0

15.0

20.0

25.0

30.0

ILSVRC 2010NEC America

ILSVRC 2011Xerox

ILSVRC 2012AlexNet

ILSVRC 2013Clarifi

ILSVRC 2014VGG

ILSVRC 2014GoogleNet

ILSVRC 2015ResNet

ImageNet Classification top-5 error (%)

Microsoft had all 5 entries being the 1-st places in 2015: ImageNet classification, ImageNet localization, ImageNet detection, COCO detection, and COCO segmentation

CNTK Benchmarked by HKBU (September 2016)

• CNTK’s LSTM performance is 5-10x faster than the other toolkits.• For convolution (image tasks), CNTK is comparable, but note the

authors were using CNTK 1.7.2, and current CNTK 2.0 beta 10 is over 30% faster than 1.7.2.

• For all networks, CTNK's performance was superior to TensorFlow performance.

Benchmarking on a single server by HKBU – September 2016

“CNTK is production-ready: State-of-the-art accuracy, efficient, and scales to multi-GPU/multi-server.”

FCN-8 AlexNet ResNet-50 LSTM-64

CNTK 0.037 0.040 (0.054) 0.207 (0.245) 0.122Caffe 0.038 0.026 (0.033) 0.307 (-) -TensorFlow 0.063 - (0.058) - (0.346) 0.144Torch 0.048 0.033 (0.038) 0.188 (0.215) 0.194

G980

Frames Processing

Source: https://github.com/Microsoft/CNTK/blob/master/Documentation/Documents/PerformanceChart.png

• The figure compares processing speed (frames processed per second) of CNTK to that of four other well-known toolkits.

• The configuration uses a fully connected 4-layer neural network (see our benchmark scripts) and an effective mini batch size (8192).

• All results were obtained on the same hardware with the respective latest public software versions as of Dec 3, 2015.

Benchmarking CNTK on Keras: is it Better at Deep Learning than TensorFlow?• Deciding the better Keras framework is not as

clear cut as expected. • Accuracy is mostly identical between the two

frameworks. CNTK is faster at LSTMs/MLPs, TensorFlow is faster at CNNs/Embeddings

• When networks implement both, it’s a tie.

http://minimaxir.com/2017/06/keras-cntk/

Potential Applications

Deep learning at Microsoft

• Microsoft Cognitive Services• Skype Translator• Cortana• Bing• Bing Ads • Augmented Reality• Microsoft Research

Intelligent Kiosk Demo

Intelligent Kiosk

• https://github.com/Microsoft/Cognitive-Samples-IntelligentKiosk

CNTK Code

Jupyter

http://jupyter.org/

Github

• Equifax may have given away my birth date, my social security number, and my credit cards

• But they’re not getting into my Github

• Github: https://github.com/Microsoft/CNTK

• Wiki: https://github.com/Microsoft/CNTK/wiki

• Issues: https://github.com/Microsoft/CNTK/issues

Microsoft Cognitive Toolkit

Microsoft Cognitive ToolkitExampleshttps://github.com/Microsoft/CNTK

Microsoft MML Spark

Jupyter Python Demos

Apache Spark

• SPARK2017

https://www.wintellectnow.com/Home/Instructor?instructorId=Mark_Tabladillo

Summary of CNTK

• Microsoft’s open-source deep-learning toolkit • Supported on both Linux and Windows• Fully open development framework

Abstract

The Microsoft Cognitive Toolkit (CNTK) is a unified deep-learning toolkit that describes neural networks as a series of computational steps via a directed graph. In this directed graph, leaf nodes represent input values or network parameters, while other nodes represent matrix operations upon their inputs. The objectives of this presentation is to 1) describe what CNTK is, 2) present a comparative evaluation with similar technologies, 3) outline potential applications, and 4) demonstrate the technology with Jupyter Python examples.