Gen-Z Technology: Enabling Memory Centric … Gen-Z is Media Agnostic & Composable 14 Media...
of 22
/22
Embed Size (px)
Transcript of Gen-Z Technology: Enabling Memory Centric … Gen-Z is Media Agnostic & Composable 14 Media...
genzconsortium.org
Gen-Z Technology: Enabling Memory Centric
Architecture
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 2
Why Gen-Z?
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 3
Why Gen-Z?
genzconsortium.org© Gen-Z Consortium 2017 4
Why Gen-Z?
genzconsortium.org
Big Data
AI
Machine Learning
Deep Learning
BI
Businesses’ Need to ‘Monetize’ Data
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 5
Why Gen-Z?
genzconsortium.org6© Gen-Z Consortium 2017genzconsortium.org
Challenge: Multiple Memory Types Need Support
Processor1 ns
SRAM5 ns
DRAM100 ns
Flash200,000 ns
Hard Drive10,000,000 ns Tape
40,000,000,000 ns
Block OperationsMemory Operations
• Byte Addressability
• High Bandwidth/Low Latency
• Simple & Efficient Load/Store operations
• Data Protection methods (RAID, EC)
• Device management: Hotplug-ability
• Rich Data Services (ie deduplication, replication, thin provisioning, etc)
Storage Class Memory (SCM)
genzconsortium.org7© Gen-Z Consortium 2017genzconsortium.org
0
500
1000
1500
2000
2500
3000
3500
4000
4500
2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
# Processor Pins & DDR Channels
3CH
4CH
6CH
8CH
I1366 pins
I2011 pins
Courtesy: Ideas International
I3647 pins
I~ 4200 pins
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
2012(8)
2013(12)
2014(18)
2016(22)
2017(28)
2019(64)
GB/
s
Memory Bandwidth per Core
(Projected)
(~60)
Pin limitations lead to memory channel limitations that leads to memory bandwidth limitations.
Challenge: Memory Bandwidth Per Core Is Shrinking
genzconsortium.org
Power, Cooling, and Physical Limitations• CPU power consumption is increasing• DIMM power consumption is increasing• Number of memory channels is slowly increasing• Physical size of CPU is increasing• Physical size of DIMM is increasing• Speed of Memory Channel is increasing• Number of Cores in each Socket is increasing• Dataset Memory Requirements are increasing
CPU1 CPU2
Challenge: We Are Out Of Real Estate & Resources
genzconsortium.org © Gen-Z Consortium 2017 8
genzconsortium.org9
Storage NIC
x16
GPUCPU
DDR4
x16
HBM64 GB/s
Memory Bandwidth: 100 GB/s
HBM Memory Bandwidth:732+ GB/s
SAS Ethernet
0.0
1.0
2.0
3.0
4.0
5.0
6.0
2012(8)
2013(12)
2014(18)
2016(22)
2017(28)
2019(64)
GB/
s
PCIe Bandwidth per core
(Projected)
x16
• Cores have limited bandwidth as the number of cores increases
• Bottleneck for heterogeneous compute environment
PCIe
Challenge: IO Bandwidth Per Core is Shrinking
(~60)(~60)
genzconsortium.org © Gen-Z Consortium 2017
genzconsortium.org10© Gen-Z Consortium 2017genzconsortium.org
I/O
Communication at the speed of memory
What is a Memory Semantic Fabric?• A communication protocol that speaks the
same language the CPU speaks: load/store, put/get, and read/write
• Connectivity that extends beyond the server to the rack
Accelerators
FPGA GPU
CPUs
SoC SoC GPUFPGA
MemoryMemory
Memory Semantic Fabric
Memory Memory
Pooled MemoryNetwork Storage
What Can Address These Challenges?
A Memory Semantic Fabric!
genzconsortium.org11© Gen-Z Consortium 2017genzconsortium.org
Gen-Z: A New Data Access Technology
Accelerators
FPGA GPU
CPUs
SoC SoC GPUFPGA
Memory MemoryMemory
I/O
Pooled Memory Network Storage
Direct Attach, Switched, or Fabric TopologyGen-Z
Packetized Memory Semantic Protocol
High Bandwidth, Low Latency
Simplify HW/SW Boundary
Compatible & Economical
Universal democratized communication
Memory
Abstracted Hardware Interface
genzconsortium.org
Gen-Z Allows Memory Innovation
Processor
DRAM
DRAM
DRAM
DRAM
Memory Bus
Media Module
4-8 Memory Channels17-25 GB/s / Channel
288pins / DIMMSynchronous Interface
Med
iaCo
ntro
ller
12genzconsortium.org © Gen-Z Consortium 2017
genzconsortium.org
Gen-Z Allows Memory Innovation
Processor
DRAM
DRAM
DRAM
DRAM
Media Module
Gen
-Z L
ogic
Gen-Z MemorySemantic Fabric
Med
iaCo
ntro
ller
Gen
-Z L
ogic
2-8 High-speed Serial LinksLow Latency, High performance
Split Memory ControllerAsynchronous Interface
Processor is media agnostic13genzconsortium.org © Gen-Z Consortium 2017
genzconsortium.org
Gen-Z is Media Agnostic & Composable
14
Med
iaCo
ntro
ller
Gen
-Z L
ogic
Processor
Gen
-Z L
ogic
Media Module
Gen-Z Fabric
SCM
SCM
SCM
SCM
Storage Class Memory
Processor
Gen
-Z L
ogic
Media Module
Gen-Z Fabric GPU or
FPGA
Accelerator
Med
iaCo
ntro
ller
ProcessorMedia Module
SCM
SCM
SCM
SCM
Media Module
GPU or
FPGA
Gen
-Z S
wit
ch
Med
iaCo
ntro
ller
DRAM
DRAM
DRAM
DRAM
Media Module
Processor
CPU
CPU
Med
iaCo
ntro
ller
Media
Media
Media
Media
Media Module
Multiple resources enabled by Universal Interconnect
genzconsortium.org © Gen-Z Consortium 2017
genzconsortium.org
ServerServerServer
Gen-Z Creates More Real Estate• Gen-Z can be used to connect to devices inside and outside the server enabling a composable system
• Gen-Z’s pin efficient interface and ability to multiplex with PCIe pins allows for an overall increase in memory bandwidth
Processor
On Package Memory
DRAM, NVM, SCM
Memory Channels
Local Memory
Accelerators Gen-Z Fabric Switch
Rack-Level Pooled Memory
Pooled Accelerators
genzconsortium.org © Gen-Z Consortium 2017 15
genzconsortium.org
Fully Composable Through Disaggregation Using Gen-Z
genzconsortium.org © Gen-Z Consortium 2017 16
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 17
Example Gen-Z System with Gen-Z & PCIe® I/O
• Gen-Z systems may contain a mix of Gen-Z I/O components and PCI Express® I/O devices• PCIe worked immediately with unmodified OSs by appearing to SW as conventional PCI devices & bridges• Gen-Z is using a similar strategy via the concept of logical PCI devices (LPDs)
Host System
PCIe Root ComplexGen-Z Interface Logic
PCIe Switch
PCIe I/O Device
PCIe I/O Device
...
PCIe I/O Device
Gen-Z LPD Component
Gen-Z Switch
Gen-Z LPD Component
Gen-Z LPD Component
...
Gen-Z Switch
Gen-Z LPD Component
Gen-Z LPD Component
...
PhysicalConfiguration
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 18
I/O Component Sharing• With suitable Gen-Z fabric management SW,
a single Gen-Z I/O component can be shared among multiple Gen-Z systems
• Component sharing offers significant availability benefits, e.g., system failover for continued operation and avoidance of stranded resources
• Each Gen-Z system running an unmodified OS only “sees” LPDs assigned to it by Gen-Z fabric management SW
• Each Gen-Z system’s BDF space for LPDs is orthogonal to the BDF space in other systems
• A Gen-Z I/O component designed to be shared by multiple systems must ensure that the virtual device instances do not interfere with each other
• Gen-Z can deliver the major benefits of PCIe MR-IOV* with less cost & complexity
*MR-IOV: Multi-Root I/O Virtualization and Sharing
genzconsortium.org
More Gen-Z Benefits
genzconsortium.org © Gen-Z Consortium 2017 20
• Gen-Z’s 112GT/s Connector The connector supports vertical, horizontal, right angle and cabled
solutions Same connectors for memory, accelerators, I/O, etc. Gen-Z defined the connector and is donating it to SFF
• Leverage High Volume Industry PHYs Derivative of the IEEE 802.3 PHY PCIe PHY Lower development and product costs
• Unmodified OS Support Minimize cost & effort to adopt Gen-Z solutions
• Enables Right Sized Solutions Enables the right amount of compute, memory, storage,
acceleration, and networking to be applied to each workload
DDR25-50GB/s
Gen464GB/s
Gen-Z 4C100-448GB/s
Gen-Z 2C50-224GB/s
Gen-Z 1C25-112GB/s
genzconsortium.org
Open With Broad Industry Support
genzconsortium.org © Gen-Z Consortium 2017 21
Consortium Members
• Alpha Data• AMD• Amphenol• ARM• Avery Design
Systems• Broadcom• Cadence• Cavium• Cray• Dell EMC• Everspin• FIT• HP Enterprise• Huawei
• IBM• IDT• IntelliProp• Jabil• Lenovo• Lotes• Luxshare-ICT• Mellanox• Mentor Graphics• Micron• Microsemi• Molex• NetApp• Nokia• Numascale
• PLDA Group• Red Hat• Samsung• Seagate• SK hynix• Smart Modular• Spin Transfer
Technologies• TE• Tyco Electronics• VMware• Western Digital• Xilinx• Yadro
genzconsortium.org
Gen-Z Summary
• Scalable system interconnect and protocol
• Optimized for memory-semantic communications
• Breaks processor-memory innovation dependency
• Opportunity to simplify software overhead and complexity
• Unmodified OS support
• Common modular connector and mechanical form factors
• Visit the Gen-Z booth, #739, for more information and see the demonstrations
• Attend “Chat with the Experts” tonight to speak with Mike Krause about Gen-Z at the Fabrics Table
• Visit us at genzconsortium.org
genzconsortium.org © Gen-Z Consortium 2017 22
genzconsortium.orggenzconsortium.org © Gen-Z Consortium 2017 23
