Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท...
Transcript of Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท...
![Page 1: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/1.jpg)
![Page 2: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/2.jpg)
![Page 3: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/3.jpg)
Capabilities, Costs โ๐ท๐๐๐๐๐๐๐๐๐๐/๐พ๐๐๐
$
![Page 4: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/4.jpg)
![Page 5: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/5.jpg)
ASICsFPGAs
Source: Bob Broderson, Berkeley Wireless group
![Page 6: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/6.jpg)
Xeon CPU NIC
![Page 7: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/7.jpg)
Xeon CPU NICSearch Acc. (FPGA)
Search Acc. (ASIC)
Wasted Power,
Holds back SW
Xeon CPU NICSearch Acc. v2 (FPGA)
NICXeon CPU Math Accelerator
Wasted Power,One more thing that
can break
![Page 8: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/8.jpg)
![Page 9: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/9.jpg)
http://www.globalfoundationservices.com/posts/2014/january/27/microsoft-contributes-cloud-server-specification-to-open-compute-project.aspx
![Page 10: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/10.jpg)
โข Two 8-core Xeon 2.1 GHz CPUs
โข 64 GB DRAM
โข 4 HDDs, 2 SSDs
โข No cable attachments to server
![Page 11: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/11.jpg)
Stratix V
8GB DDR3
PCIe Gen3 x8
![Page 12: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/12.jpg)
Data Center Server (1U, ยฝ width)
![Page 13: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/13.jpg)
FPGA FPGA FPGA FPGAWeb Search Pipeline
![Page 14: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/14.jpg)
FPGA FPGA FPGA FPGA
Math Acceleration
Service Comp.
Vision
Service
Physics
Engine
Web Search Pipeline
![Page 15: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/15.jpg)
West SLIII
East SLIII
South SLIII
North SLIII
x8 PCIe Core
DMA Engine
ConfigFlash (RSU)
DDR3 Core 1DDR3 Core 0
JTAG
LEDs
Temp Sensors
Application
Shell
I2C
xcvrreconfig
2 2 2 2
4256 Mb
QSPI ConfigFlash
4 GB DDR3-1333 ECC SO-DIMM
4 GB DDR3-1333 ECC SO-DIMM
Host CPU
72 72
Role
8
Inter-FPGA RouterSEU
![Page 16: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/16.jpg)
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
SaaS 1
SaaS 2
SaaS
48
SaaS 3
Ranking-as-a-Service (RaaS)
- Compute scores for how relevant each selected
document is for the search query
- Sort the scores and return the results
Selection-as-a-Service (SaaS)
- Find all docs that contain query terms,
- Filter and select candidate documents for
ranking
Selection as a Service (SaaS)
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
IFM 1
IFM 2
IFM 44
IFM 3
RaaS 1
RaaS 2
RaaS
48
RaaS 3
Ranking as a Service (RaaS)
Query
Selected
Documents10 blue links
![Page 17: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/17.jpg)
Query: โFPGA Configurationโ
NumberOfOccurrences_0 = 7 NumberOfOccurrences_1 = 4 NumberOfTuples_0_1 = 1
{Query, Document}
~4K Dynamic
Features
~2K Synthetic
Features
L2 Score
Document
Score
![Page 18: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/18.jpg)
FFE #1 =(2*NumberOfOccurrences_0 + NumberOfOccurrences_1)
(2 * NumberOfTuples_0_1)
NumberOfTuples_0_1 = 1NumberOfOccurrences_1 = 4NumberOfOccurrences_0 = 7
FFE #1 = 9
{Query, Document}
~4K Dynamic
Features
~2K Synthetic
Features
L2 Score
Document
Score
![Page 19: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/19.jpg)
PCIe
Distribution latchesControl/Data
Tokens
Compressed
Document
Feature
Gathering
Network
Free Form
Expression
(FFE)
Stream
Preprocessing
FSM
โข 196 feature families
โข 54 state machines
โข 2.6K dynamic features extracted in
less than 4us (~600us in SW)
![Page 20: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/20.jpg)
Core 0 Core 1 Core 2
Core 3 Core 4 Core 5
ComplexFST
Ou
tpu
t
Cluster
0
![Page 21: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/21.jpg)
FFE: Free-Form
Expressions
FE: Feature Extraction FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
FPGA 5
FPGA 6
FPGA 7
Server
Server
Server
Server
Server
Server
Server
Server
Document
Scoring
Request
8-Stage Pipeline
Compute
Score
Route to
Head
Return
Score
RaaS ServersDocument
Score
Document
Scoring
Request
Compute
Score
Route to
Head
Return
Score
![Page 22: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/22.jpg)
FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
FPGA 5
FPGA 6
FPGA 7
8-Stage Pipeline
FPGA 5
FPGA 6
FPGA 0
FPGA 1
FPGA 2
FPGA 3
FPGA 4
8-Stage Pipeline
FPGA 2
![Page 23: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/23.jpg)
![Page 24: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/24.jpg)
1,632 Servers with FPGAs Running Bing Page Ranking Service (~30,000 lines of C++)
![Page 25: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/25.jpg)
![Page 26: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/26.jpg)
But when will an FPGA handle my Bing Search?
![Page 27: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/27.jpg)
โข Bing is going into production with FPGAs
![Page 28: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/28.jpg)
Huge thanks to our partners at
Top Row: Eric Peterson, Scott Hauck,
Aaron Smith, Jan Gray, Adrian M.
Caulfield, Phillip Yi Xiao, Michael
Haselman, Doug Burger
Bottom Row: Joo-Young Kim, Stephen
Heil, Derek Chiou, Sitaram Lanka,
Andrew Putnam, Eric S. Chung,
Not Pictured: Kypros Constantinides,
John Demme, Hadi Esmaeilzadeh,
Jeremy Fowers, Gopi Prashanth Gopal,
Amir Hormati, James Larus, Simon
Pope, Jason Thong
![Page 29: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/29.jpg)
Microsoft Privacy Policy statement applies to all information collected. Read at research.microsoft.com
Save the planet and return your name badge before you
leave (on Tuesday)
![Page 30: Capabilities, Costs ๐ท ๐ ๐ ๐ ๐ /๐พ๐๐๐ย ยท 2018-01-04ย ยท Ranking-as-a-Service (RaaS) - Compute scores for how relevant each selected document is for the search](https://reader036.fdocuments.us/reader036/viewer/2022062402/5f6ba806676aab161039b561/html5/thumbnails/30.jpg)