Windows 8 dddd (beekelaar)
-
Upload
hypervnu -
Category
Technology
-
view
1.415 -
download
2
Transcript of Windows 8 dddd (beekelaar)
![Page 1: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/1.jpg)
Windows 8 Disk Deduplication Deep Dive
Ronald Beekelaar Virsoft Solutions
Schiphol, 19 jan 2012
![Page 2: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/2.jpg)
Introductions • Presenter
– MVP Security – MVP Virtual Machine Technology – E-mail: [email protected]
• Work
– Security consultancy – Virtualization consultancy – Create many VM-based labs and demos – Software to optimize, manage and run VM – Maintain four datacenters world-wide
• Running Hyper-V labs for customers (MOC, training and demo purposes)
![Page 3: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/3.jpg)
Objectives
• Discuss one interesting new aspect of Windows 8: Disk Deduplication
![Page 4: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/4.jpg)
What is Disk Deduplication ?
• Goal:
– Use less storage space
• Method:
– Ensure that identical content in multiple (large) files is only stored once
• Is block-based, post-process, transparant solution
![Page 5: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/5.jpg)
Standard deduplication modes
• "Source" – Prevent transferring data, if duplicate
• Used by Remote Differential Compression
• "Inline" – Perform deduplication when data is written
• Used by NTFS file compression • Write process is slowed down
• "Post-Process" (or "Background")
– Perform deduplication later, in background, when idle • Used by Windows 8 Data Deduplication
![Page 6: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/6.jpg)
Other methods to save disk space
• SIS (single-instance-store) in Win2000 – Is file-based, not block-based
• NTFS file compression – Is inline, not post-process – Much more CPU intensive
• NTFS hard links – Is not transparent – Is file-based, not block-based
![Page 7: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/7.jpg)
NTFS Hard Links
• Multiple file entries pointing to same data
• Manage – Create: mklink /h link.ext target.ext
– List: fsutil hardlink list file.ext
• Is not transparent – Edit one hardlink file, also changes other files
• Windows uses thousands of hard links (!) – Good reason not to touch C:\Windows\winsxs
![Page 8: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/8.jpg)
Windows 8 dedup architecture
• Is file-system filter driver
– Coordinates between file entry, regular storage and 'chunk' storage
• Dedup service (ddpsvc) runs jobs to deduplicate files
![Page 9: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/9.jpg)
How does Windows 8 dedup work?
• Dedup service recognizes common 'chunks' in files, and places those in Chunk Store – In System Volume Information folder
• Dedup filter driver ensures that applications read correct file content
• File "size" (= content length) does not change in Explorer – Explorer reports "size-on-disk" as 4 KB
![Page 10: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/10.jpg)
How does Windows 8 dedup work?
![Page 11: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/11.jpg)
Windows 8 dedup details
• Dedup works per volume – Also works on portable disks – Dedup does NOT work on C: (Windows) volume
• Chunk size is 32-128 KB (average 80 KB) • By default
– Chunks are compressed in chunk store • Avoids re-compressing compressed files (zip, etc)
– Dedup service ignores files < 64 KB – Dedup service ignores files changed in last 30 days – Dedup service ignores NTFS encrypted files
![Page 12: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/12.jpg)
Savings?
• Depends on file content of course
• Microsoft reported averages:
– General: 50-60% savings
• Documents: 30-50% saving
• Application library: 70-80% savings
• VHD library: 80-95% savings
![Page 13: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/13.jpg)
Performance?
• Write has no direct performance hit – Dedup operations are done post-process
• Read has a ~3% performance hit (if not in cache) – Due to more disk head operations
– Compare with disk fragmentation
• Windows caching is dedup-aware (!) – Dedup improves caching efficience
![Page 14: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/14.jpg)
Reliable? • My opinion: Yes - 100%
• Data is check-summed
– Means: invalid data is detected
• Operations are crash consistent – Means: can interrupt/crash operation at any time without losing
data
• Data is self-describing – Means: it can be read without external data
• Popular 'chunks' (>100x) are stored multiple times – Means: avoids creating IO hotspots on disk
January 20, 2012 NIC 2012
![Page 15: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/15.jpg)
How to enable Windows 8 dedup?
• Install Data Deduplication role service • Start Data Duplication Service (ddpsvc) • Powershell
– import-module Deduplication – help dedup
– enable-dedupvolume D: – set-dedupvolume D: -minimumfileagedays 0
• Default is 30 days
– start-dedupjob D: -type Optimization • Use Unoptimization to undo
– get-dedupjob – get-dedupstatus – get-dedupmetadata
![Page 16: Windows 8 dddd (beekelaar)](https://reader034.fdocuments.us/reader034/viewer/2022052315/5571fd61d8b42ab0668b520e/html5/thumbnails/16.jpg)
Questions ?
• Thanks for your attention