Window Server 2019 Hyper-V Persistent...
Transcript of Window Server 2019 Hyper-V Persistent...
Window Server 2019 Hyper-V&
Persistent Memory
Didier Van HoyeTechnical Architect & StrategistMicrosoft Cloud & Datacenter Management MVPVeeam VanguardPublic & private speaker
Twitter @workinghardinitOne of the Hyper-V Amigos http://hyper-v-amigos.nethttps://blog.workinghardinit.work
Storage at the speed of memory!!!!!
What Is Persistent Memory?
Didier Van Hoye - WorkingHardInIT
➢ Persistent memory (PMEM) = a solid-state high-performance byte-addressable memory device that resides on the memory bus
➢ PMEM has DRAM-like access to data➔ almost the same speed & latency of DRAM but persistent like NAND flash
Persistent Memory … different types
➢Non-Volatile DIMM (NVDIMM)
➢Storage Class Memory (SCM)
… but don’t get caught up in the semantics too much
… lots to happen in this space
Didier Van Hoye - WorkingHardInIT
Persistent Memory … differences!
Non-Volatile DIMM (NVDIMM-N)➢ Combines DRAM & NAND to deliver persistent storage in a
DIMM slot. The DRAM is the storage but data is persisted to NAND at shutdown, crash or power loss. At boot the data is made available again in the DRAM which increased boot times. This requires BIOS configuration & OS support or the modules are seen as normal memory module. Provides the fastest & lowest latency, in small capacity modules (16-32-64GB).
Didier Van Hoye - WorkingHardInIT
Persistent Memory … differences!
Storage Class Memory (SCM)➢ Extends the capacity of a “regular” DIMM by using NAND
as a capacity store and DRAM as the cache for active data. This allows for high DIMM capacity servers at reduced cost, SCM feel and act like regular memory DIMMS so they don’t require special BIOS or Operating system support (*ahum). Less fast due to higher latency, but still very good but with higher capacities (512GB right now).
Didier Van Hoye - WorkingHardInIT
Examples
➢NVDIMM-N
➢3D-Xpoint (Intel Optane)
Both can act as PMEM (storage) or “Normal” Memory … but there are subtle differences …Take away: use as non-persistent memory or as persistent memory (storage: persistent, for DAX or Block address)
Didier Van Hoye - WorkingHardInIT
NVDIMM variants N, F, P
NVDIMM-PWork in Progress(wait for DDR5?)
NVDIMM-NStandard
Available today
NVDIMM-FVendors Specific
• Memory mapped DRAM, Flash is not mapped, used ony for persistence• Access: byte or block mode to DRAM• DIMM like latency: 10s of nanoseconds• Capacity like DIMM: 8-32 GB now, maybe 64-128GB later on• Battery backup for backup / persisting of data
• Memory is system mapped FLASH, DRAM is not system mapped but a cache• Access: block mode to FLASH/NAND• FLASH like latency: 10s of microseconds• Capacity like FLASH: 128GB to X TB
• Both FLASH and DRAM are memory mapped• Access: block mode to FLASH (like NVDIMM-F) & byte or block to DRAM (like NVDIMM-N)• FLASH NVME like latency: 100’s of nanoseconds• Capacity like FLASH NVMe: 128GB to X TB
Didier Van Hoye - WorkingHardInIT
NVDIMM-N
Didier Van Hoye - WorkingHardInIT
NVDIMM-N
Didier Van Hoye - WorkingHardInIT
NVDIMM-N
NVDIMM battery is dead or set as read-only (BIOS)?
Didier Van Hoye - WorkingHardInIT
NAND NAND NAND NANDNANDNANDController
DDR interface (JEDEC DDR4 for now)
NVDIMM-F (vendor specific)
Difference with SCM => no DRAM cache on the moduleBasically NAND on DDR4 / Slow – slowness hidden by regular RAM caching (that part is not persistent!)
Didier Van Hoye - WorkingHardInIT
NAND NAND NANDController
DDR interface (JEDEC DDR4 with extensions, and in DDR 5 time frame)
NVDIMM-P Combines N and F
DRAM DRAM DRAM DRAM DRAM DRAM
NAND NAND NAND
Didier Van Hoye - WorkingHardInIT
No battery requiredRAM is cache not storage
Intel® OPTANE™ DC PERSISTENT MEMORY
Didier Van Hoye - WorkingHardInIT
Cascade Lake
FLASH FLASH FLASH
Controller
DDR interface (JEDEC DDR4 for now)
Storage Class Memory
DRAM DRAM DRAM
FLASH = High performance NAND or 3D-Xpoint (Intel Optane DC)
No battery requiredRAM is cache not storage
FLASH FLASH FLASH
DRAM DRAM DRAM
Didier Van Hoye - WorkingHardInIT
NVDIMM / Storage Class Memory
Didier Van Hoye - WorkingHardInIT
Semantics: when is SCM really SCM and why wouldNVDIMM-P not be SCMI do not so see anything that prevents NVDIMM-P from being SCM is NAND type is fast enough
NVDIMM-P
SCM
PMEM <> Intel® OPTANE™ PX4800X
Didier Van Hoye - WorkingHardInIT
Opportunity zone for PMEM/SCMPMEM/SCM100 ns – 1000 ns
SSD100-200 us
DRAM100 ns
L1-L3 cache1-40 ns
HDD1-10 ms
NVMe10-20 us
Pri
ce/C
apac
ity
Latency (1 million delta in latency)
Break the storage latency/speed barrier …
Didier Van Hoye - WorkingHardInIT
Exponential data & analytics growth …
Didier Van Hoye - WorkingHardInIT
© 2016 Micron Technology, Inc,
It’s about value optimization!
A modern cloud OS & hardware
Didier Van Hoye - WorkingHardInIT
Didier Van Hoye - WorkingHardInIT
BIOS Persistent Memory Configuration
Didier Van Hoye - WorkingHardInIT
Get-PmemUnusedRegion | New-PmenDisk -AtomicityType None
Using Persistent Memory on Windows
Didier Van Hoye - WorkingHardInIT
Persistent Memory Interleaving
Get-PmemUnusedRegion | New-PmenDisk -AtomicityType None
Consume PMEM Storage: Host
Initialize-Disk 6 –PartitionStyle GPT
New-Partition -DiskNumber 6 -UseMaximumSize -driveletter P | Format-Volume
-NewFileSystemLabel PMEM01 -IsDAX $True
Didier Van Hoye - WorkingHardInIT
Consume PMEM Storage: VM
New-VHD P:\DEMOVM-PMDisk01.vhdpmem -Fixed -SizeBytes 30GB
Some things to take notice off:
1. Generation 2 virtual machines only.2. The .vhdpmem extension specifies the virtual
disk is a persistent memory disk3. You have to create it on an NTFS DAX volume4. For now, only fixed size persistent memory
virtual disks are supported
Use a normal virtual machine with a VHDX OS disk and potentially one or more data virtual disk (VHDX).
Didier Van Hoye - WorkingHardInIT
Consume PMEM Storage: VM (2/2) Add-VMPmemController PMEMVM
Add-VMHardDiskDrive -VMName PMEMVM PMEM -ControllerLocation 1 -Path P:\DEMOVM-
PMDisk01.vhdpmem
New-Partition -DiskNumber 1 -UseMaximumSize -driveletter P | Format-Volume -
NewFileSystemLabel VDISKVMPMEM01 -FileSystem NTFS -AllocationUnitSize 65536
Get-Disk | Where Bustype -eq SCM
Inside the virtual machine after startup:
Didier Van Hoye - WorkingHardInIT
Optimal workloads …
Didier Van Hoye - WorkingHardInIT
4K24 workers100 Read100 Sequential
Less optimal workloads …
Didier Van Hoye - WorkingHardInIT
8K24 workers70/30 Read/Write70/30 Sequential/Random
Diskspd.exe for the win ;-)
8K24 workers70/30 Read/WriteRandom IO
diskspd.exe -b8K -d20 -L -F24 -W5 -Sh -r -w30 P:\iobw.tst Q:\iobw.tst
Didier Van Hoye - WorkingHardInIT
Hyper-V PMEM Support Info (1/2)
Didier Van Hoye - WorkingHardInIT
• Only with Generation 2 VMs only (Windows & Linux)• New virtual disk type: VHDPMEM➢Convert-VHD converts between VHDPMEM & VHDX➢Select Atomicity at creation time (none or BTT)➢VHDPMEM are read/write mountable on host (SCSI)
• one vPMEM device = one VDHPMEM file
Hyper-V PMEM Support Info (2/2)
Didier Van Hoye - WorkingHardInIT
• DAX/BTT is supported in guest• Use large pages automatically (2MB)• No support for:➢ Live migration➢ Checkpoints➢ Backup➢ Save/Restore
PMEM used as storage (tiers)➢ ReFS➔ so no DAX, only block access➢ Leverage PMEM individually or aggregated via interleaving for higher
capacity and performance
PMEM usage in S2D
PMEM used as cache ➢ No need to worry about no support for ReFS for DAX or
overhead of block level access➢ Interleaving recommend (high number of cache devices
not practical/possible)
Didier Van Hoye - WorkingHardInIT
PMEM usage in S2D
Didier Van Hoye - WorkingHardInIT
In both cases (Cache/ Storage):➢ You treat the PMEM disk as a normal
fallible disk➢ It is S2D that will provide protection
against disk failures with known mechanisms
PMEM usage in S2D (2/2)
Didier Van Hoye - WorkingHardInIT
Some observations (1/2)
Didier Van Hoye - WorkingHardInIT
Some observations (2/2)
Didier Van Hoye - WorkingHardInIT