Freitas Storage Class Memory
-
Upload
nitroboozter -
Category
Documents
-
view
227 -
download
0
description
Transcript of Freitas Storage Class Memory
IBM Almaden Research Center
© 2010 IBM Corporation
Storage Class Memory: Technology, Systems and Applications
Rich Freitas
Nonvolatile Memory Seminar
Hot Chips Conference
August 22, 2010
Memorial Auditorium, Stanford University
IBM Almaden Research center
© 2010 IBM Corporation2 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Agenda
Technology
–Disks
–Flash
–Phase Change Memory
Systems
–Memory Systems
–Storage Systems
Applications
IBM Almaden Research center
© 2010 IBM Corporation3 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Definition of Storage Class Memory SCM A new class of data storage/memory devices
–many technologies compete to be the ‘best’ SCM
SCM features:
–Non-volatile
–Short Access times (~ DRAM like )
–Low cost per bit (more DISK like – by 2020)
–Solid state, no moving parts
SCM blurs the distinction between
–MEMORY (fast, expensive, volatile ) and
–STORAGE (slow, cheap, non-volatile)
IBM Almaden Research center
© 2010 IBM Corporation4 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Mobile Desktop X Datacenter
System Targets for SCM
Billions!
Megacenters
IBM Almaden Research center
© 2010 IBM Corporation5 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
History of HDD is based on Areal Density Growth
IBM Almaden Research center
© 2010 IBM Corporation6 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
IBM Almaden Research center
© 2010 IBM Corporation7 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
-6.5%
IBM Almaden Research center
© 2010 IBM Corporation8 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
IBM Almaden Research center
© 2010 IBM Corporation9 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Many device technologies considered for SCM
Phase Change RAM
– most promising now (scaling)
Magnetic RAM
– used today, but poor scaling and a space hog
Magnetic Racetrack
– basic research, but very promising long term
Ferroelectric RAM
– used today, but poor scaleability
Solid Electrolyte and resistive RAM (Memristor)
– early development, promising
Organic, nano particle and polymeric RAM
– many different devices in this class, unlikely
Improved FLASH
– still slow and poor write endurance
bistable material plus
on-off switch
Generic SCM Array
IBM Almaden Research center
© 2010 IBM Corporation10 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
IBM
Infineon
Freescale
Philips
STMicro
HP
NVE
Honeywell
Toshiba
NEC
Sony
Fujitsu
Renesas
Samsung
Hynix
TSMC
4Mb C-RAM (Product)0.25um 3.3V512Mb PRAM (Prototype)
0.1um 1.8V4Mb MRAM (Product)
0.18um 3.3V
Trap Storage
Saifun NROM
Tower
Spansion
Infineon
Macronix
Samsung
Toshiba
Spansion
Macronix
NEC
Nano-x’tal
Freescale
Matsushita
FLASH
ExtensionFRAM MRAM PCRAM RRAM
Solid
Electrolyte
Polymer/
OrganicRamtron
Fujitsu
STMicro
TI
Toshiba
Infineon
Samsung
NEC
Hitachi
Rohm
HP
Cypress
Matsushita
Oki
Hynix
Celis
Fujitsu
Seiko Epson
Ovonyx
BAE
Intel
STMicro
Samsung
Elpida
IBMMacronix
Infineon
Hitachi
Philips
IBM
Sharp
Unity
Spansion
Samsung
Axon
Infineon
Spansion
Samsung
TFE
MEC
Zettacore
Roltronics
Nanolayer
Emerging Memory Technologies
64Mb FRAM (Prototype)0.13um 3.3V
IBM Almaden Research center
© 2010 IBM Corporation11 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Research interest Papers presented at• Symposium on VLSI Technology• IEDM (Int. Electron Devices Meeting)
Flash
nanocrystal Flash
SONOS Flash
FeRAM
MRAM
PCram
RRAM
solid electrolyte
organic
Year
0
10
20
30
40
50
60
Nu
mb
er o
f p
ap
ers
2001 2002 2003 2004 2005 2006 2007 2008
IBM Almaden Research center
© 2010 IBM Corporation12 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
What is Flash?
• Based on MOS transistor
• Transistor gate is redesigned
• Charge is placed or removed
near the “gate”
• The threshold voltage Vth of the
transistor is shifted by the
presence of this charge
• The threshold Voltage shift
detection enables non-volatile
memory function.
source drain
gate
source drain
control gate
source drain
e- e- e- e- e-
control gate
oxide
Floating
Gate e- e-
Flash
Memory
“0”
Flash
Memory
“1”
Floating
Gate
IBM Almaden Research center
© 2010 IBM Corporation13 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
NAND
Cell Size 4 F2
(2 F2 virtual x 2-bit MLC)
Read Access Time 20-50 us
Read 15-25 MB/s
Write 5-8MB/sec
Erase 2ms
Start Up Time 50-100 us
Market Size (2007) $14.2B
Applications Multimedia
Feeds and Speeds for typical NAND Flash
IBM Almaden Research center
© 2010 IBM Corporation14 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Representative NAND Flash Device
Interface: one or two bytes wide
–Transition to ONFI for some vendors
Data accessed in pages
– 2112, 4224 or 8448 Bytes
Data erased in blocks
–Block = 64 - 128 Pages
Power circuits
–Charge Pumps
–Clock drivers
–Etc.
B
U
F
PagePagePagePage
PagePagePagePage
PagePagePagePage
Blo
ck 0
Blo
ck 1
Blo
ck 6
3
B
U
F
C
A
C
H
E
Power
Circuits
ONFI Open NAND Flash Interface
IBM Almaden Research center
© 2010 IBM Corporation15 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Representative Flash SSD Classes
usb commercia l enterprise
0
50
100
150
200
250
Rd BW MB/s
Wt BW MB/s
usb commercial enterprise
10
100
1000
10000
100000
Rd IOPS
Wt IOPS
IBM Almaden Research center
© 2010 IBM Corporation16 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
History of Phase-change memory
[Neale:2001]
• late 1960’s – Ovshinsky shows reversible electrical switching in disordered
semiconductors• early 1970’s – much research on mechanisms, but everything was too slow!
Ele
ctri
cal
con
duct
ivit
y
Crystalline phaseLow resistance
High reflectivity
Amorphous phaseHigh resistance
Low reflectivity
[Wuttig:2007]
IBM Almaden Research center
© 2010 IBM Corporation17 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Phase-change
RAM
Access device
(transistor, diode)
PCRAM
“programmable
resistor”
Bit-line
Word-line
temperature
time
Tmelt
Tcryst
“RESET” pulse
“SET” pulse
VoltagePotential headache:
High power/current
affects scaling!
Potential headache: If crystallization is slow
affects performance!
IBM Almaden Research center
© 2010 IBM Corporation18 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
…go to 3-D with
L layers
Paths to ultra-high density memory
demonstrated(at IEDM 2007)
…store M bits/cell
with 2M multiple levels
demonstrated(at IEDM 2005)
…add N 1-D sub-lithographic “fins”
(N2 with 2-D)
2F 2F
startingfrom standard
4F2 …
IBM Almaden Research center
© 2010 IBM Corporation19 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Industry SCM activities SCM research in IBM
Intel/ST-Microelectronics
spun out Numonyx (FLASH &
PCM)
Samsung, Numonyx sample
PCM chips
Over 30 companies work on
SCM
–including all major IT players
Samsung 512 Mbit PCM chip
IBM sub-litho PCM Alverstone PCM
IBM Almaden Research center
© 2010 IBM Corporation20 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
MRAM alternatives a 3-D shift register
Magnetic Racetrack Memory
• Data stored as pattern of magnetic domains in long
nanowire or “racetrack” of magnetic material.
• Current pulses move domains along racetrack
•Use deep trench to get many (10-100) bits per 4F2
IBM trench
DRAMMagnetic Race Track Memory
S. Parkin (IBM), US patents 6,834,005
(2004) & 6,898,132 (2005)
IBM Almaden Research center
© 2010 IBM Corporation21 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Magnetic Racetrack Memory
• Need deep trench with notches to “pin”
domains
• Need sensitive sensors to “read” presence of
domains
• Must insure a moderate current pulse moves
every domain one and only one notch
• Basic physics of current-induced domain
motion being investigated
Promise (10-100 bits/F2) is enormous…
but we’re still working on our basic understanding
of the physical phenomena…
IBM Almaden Research center
© 2010 IBM Corporation22 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Memory/Storage Stack Latency Problem
CPU operations (1ns)
Get data from L2 cache (10ns)
Access DISK (5ms)
Get data from TAPE (40s)
Storage
Memory Get data from DRAM or PCM (60ns)
Tim
e in
ns
102
108
103
104
105
106
107
109
1010
101
Access FLASH (20 us)
SCM
Access PCM (100 – 1000 ns)
Second
Century
Hum
an S
cale
Day
Month
Year
Hour
IBM Almaden Research center
© 2010 IBM Corporation23 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Architecture
Memory
Controller
DRAM
SCMI/O
Controller
SCM
SCM
Disk
Storage
Controller
CPUInternal
External
Synchronous•Hardware managed•Low overhead•Processor waits•Fast SCM, Not Flash•Cached or pooled memory
Asynchronous•Software managed•High overhead•Processor doesn’t wait•Switch processes•Flash and slow SCM•Paging or storage
IBM Almaden Research center
© 2010 IBM Corporation24 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
SCM in a large System
CPU RAM DISK
CPU SCM
TAPE
RA
M
CPU RAM DISK TAPEFLASH
SSD2008
1980
2015
ArchivalActive StorageMemoryLogic
TAPE...DISK
IBM Almaden Research center
© 2010 IBM Corporation25 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
CPU & Memory System Conceptual Alternatives
CPU VMPMC
DRAM
DIMMX86, PowerPC,
SPARC…
SCMC
SCM
SCM
Card
SCM
Controller
IBM Almaden Research center
© 2010 IBM Corporation26 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
1990 1995 2000 2005 2010 2015 2020
$0.01
$0.10
$1.00
$10.00
$100.00
$1,000.00
$10,000.00
$100,000.00
DRAM
DRAM-pr
FLASH
FLASH-pr
MRAM
SRAM
SCM
Input from the device cost crystal ball
IBM Almaden Research center
© 2010 IBM Corporation27 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
SCM: Generic Storage Design
1
2
n
CPU
DRAM
SCM
Control
SCM SCM SCM
SCM SCM SCM
SCM SCM SCM
MEMORY
IBM Almaden Research center
© 2010 IBM Corporation28 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
IBM Almaden Research center
© 2010 IBM Corporation29 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Challenges with SCM Asymmetric performance
– Flash: writes much slower than reads
– Not as pronounced in other
technologies
Program/erase cycle
–Issue for flash
–Most are write-in-place
Data retention and Non-volatility
–It’s all relative
–Use case dependent
Bad blocks
–Devices are shipped with bad blocks
–Blocks wear out, etc.
The “fly in the ointment” is write endurance
– In many SCM technologies writes are
cumulatively destructive
– For Flash it is the program/erase
cycle
–Current commercial flash varieties
• Single level cell (SLC) 105
• Multi-level cell (MLC) 104
–Coping strategy --> wear leveling
–Typically hidden from applications by
infrastructure
IBM Almaden Research center
© 2010 IBM Corporation30 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Write and/or read endurance and life-time of SCM devices
• In DRAM and disks (magnetic) there is no known wear out mechanism• In flash and many SCM technologies there are known wear out mechanisms
Tlife = Endurance • Fill-Time
Fill-Time: time to write a memory unit (what’s a data unit?)
• Simple wear leveling each write is done to a new (empty) location
DRAM Disk 256GB Flash 8 GB SCM
Endurance >1016 >1011 105 104 108
Wear leveled N N N Y Y
Memory unit 1 B 512 B 128 KB 256 GB 8 GB
Data unit 1 B 512 B 128 KB 128 KB 128 B
Fill Time 100 ns 4 ms 2 ms 4000 s 500 s
Life Time >31 yrs >12 yrs <4 min >12 yrs >190 yrs
IBM Almaden Research center
© 2010 IBM Corporation31 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Operating systems
–Extend state information kept about memory pages
–New mechanisms to manage new resource
–Enhanced to provide hints to other layers of software
–Potential for direct involvement in managing caches and pools
Middle ware and applications evolutionary
–Improved performance impact immediate – full exploitation will occur gradually
–Little near term demand for non-volatility
–Cost improvements will drive memory size
–Memory size will drive larger and more complex data structures.
–Reload time on a crash will be exacerbated
–User’s need for non-volatility, persistence, etc. will be driven by these effects – blurring of memory and storage
SCM impact on software
IBM Almaden Research center
© 2010 IBM Corporation32 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
SAN
32
IBM QuickSilver Project SSD proof of concept
Ultra-fast storage performance without managing 1000’s of disks.
–Demonstrated performance of over 1 million IOPS using 40 SSDs.
–Reduced $/IOPS, significantly lower than traditional disk storage farm.
–Reduced floor space per IOPS
– Improved energy efficiency for high performance workloads.
–Reduced number of storage elements to manage
Quick
Silver
SAN connected hosts
4 x 4Gbps
FC ports per node
Quick
Silver
SAN: Storage Area Network
SVC: San Volume Controller
SVC
IBM Almaden Research center
© 2010 IBM Corporation33 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Main Memory:
Storage:
Applications:
DRAM – Disk – Tape
–Cost & power constrained
–Paging not used
–Only one type of memory: volatile
–Active data on disk
–Inactive data on tape
–SANs in heavy use
–Compute centric
–Focus on hiding disk latency
Shift in Systems and Applications
DRAM – SCM – Disk – Tape
–Much larger memory space for same power and cost
–Paging viable
–Memory pools: different speeds, some persistent
–Fast boot and hibernate
Active data on SCM
–Inactive data on disk/tape
–DAS ??
–Data centric comes to fore
–Focus on efficient memory use and exploiting persistence
–Fast, persistent metadata
IBM Almaden Research center
© 2010 IBM Corporation34 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
0
1 0 0 0 0
2 0 0 0 0
3 0 0 0 0
4 0 0 0 0
5 0 0 0 0
6 0 0 0 0
0 : 0 0 2 : 0 0 4 : 0 0 6 : 0 0 8 : 0 0 1 0 : 0 0 1 2 : 0 0 1 4 : 0 0 1 6 : 0 0 1 8 : 0
T i m
Thro
ughp
ut (I
O/s) 230%
IOPSImprovement
Performance increase of 230% by automatic movement
of 3% of the application's data to SSD
IBM Almaden Research center
© 2010 IBM Corporation35 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
SCM Design Triangle
(Write) Endurance Cost/bit
Speed
Memory-type
Uses
Power!
Storage-type
Uses
IBM Almaden Research center
© 2010 IBM Corporation36 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
Storage Class Memory is a new class of data
storage/memory technology many technologies
are competing to be the ‘best’ SCM
SCM blurs the distinction between memory and
storage
SCM will impact the design of computer systems
and applications
Flash, which has may SCM characteristics, is
available now and various SCMs are in the wings.
EasyTier like software will foster exploitation of
Flash and SCM
Summary
IBM Almaden Research center
© 2010 IBM Corporation37 Storage Class Memory, Technology, Systems and Applications
Rich Freitas, IBM Research
Hot Chips 8/2010
References
FAST2010 Tutorial
–T2: Freitas and Chiu, Solid State Storage: Technology,
Design and Applications
–http://www.usenix.org/events/fast10/tutorials
IBM Journal of Research and Development
–Special issue on storage
–http://www.research.ibm.com/journal/rd52-45.html
–Four papers related to SCM
Questions?