Solid State Drive Technology

Me:

• Matt Simmons

• Network & Virtualization @ NEU-CCIS

• Blog: Standalone SysAdmin

• Email: msimmons@ccs.neu.edu

• Twitter: @standaloneSA

• This Slideshow: slideshare.net/standaloneSA

Spinning Disks !

Key Topics

Voltron Force Assemble!

Variables Affecting Spinning Disk IO Rate

Platter Rotational SpeedSeek Speed

Data DensityController Cache

Logical Block Addressing

• First introduced as an abstraction layer

• Replaced CHS addressing

• Address Space is Linear (block 0 - n)

• Size of address space depends on the standard at time of manufacture.

Disk Geometry

Solid State Drive Technology

NOR Flash

• Reads and writes are atomic single-bit

• Expensive

• Small specific use cases

NAND Flash

• Reads are based on pages or “read blocks” (4k)

• Writes are based on “erasure blocks”

• Cheap (and getting cheaper)

• Broad use cases

The Magic

Insulating Barrier

Pure Silicon

Doped silicon capable of holding an electrical charge

Quantum Tunneling

Doped Silicon

Single Layer Cell (SLC)

Multi-Layer Cell (MLC)

Triple-Layer Cell (TLC)

Gradual Destruction

Energy increases with cell layers

Multiple cells need multiple writes

Barrier accumulates electrons

Electrical potential difference of barrier and cells disappears

Difficulty Going ForwardTLC

000 100

001 101

010 110

011 111

SLC

0

1

MLC

00

01

10

11

4LC

0000 0100 1000 1100

0001 0101 1001 1101

0010 0110 1010 1110

0011 0111 1011 1111

Read / Write Profiles

• Logical addresses abstracted from LBA

• That abstraction can cause complexity

• No seek time

• Reads are generally very fast

• Writes are comparatively slow

Density

• 3-Dimensional

• Charge levels

• Size of cells

• “Dot Pitch” (Cells Per Inch)

• 5nm, 3nm, 2nm

• Varies with “level” count

SLC / ESLC• Low Density, Quick (25µs

Read / 200-300µ Write)

• Write endurance near 100,000 cycles

MLC / EMLC• Reasonably High Density, Slower

(50µs Read / 600-900µs Write)

• Write endurance near 3,000 cycles

TLC• Very High Density, Even Slower

(75µs Read / 900-1350µs Write)

• Write endurance ~ 1,000 cycles

Write Amplification and Garbage Collection

Block Sizes

• Read Block

• 4k (aka “page”)

• Erasure Block

• (Large) multiple of 4k

• aka “block”256KB erasure

block size

Write Amplification

Written Data

Empty Cell

Write Amplification

Written Data

Empty Cell

New Data

Old Data

Write Amplification

Written Data

Empty Cell

Old Data

New Data

Write Amplification

Written Data

Empty Cell

New data written over old cell

Garbage Collection

Garbage Collection

Garbage Collection

Garbage Collection

Garbage Collection

SSD Performance Overview

• Depends on

• Number of flash chips in use

• Number of busses from the processor

• Performance of controller CPU

• Contention

• Bus speed

• Number of erasure blocks used

• Number of previous writes to flash cells

• Chips

• IO Busses

• CPU Cores

Causes of Contention

• Legitimate use

• Garbage collection

• Legitimate (but latent) useage

• IO Blender!

(Bender Blender: http://bit.ly/10vc7Sf)

Solid State in Practice

Solid State Form FactorsRemovable Media Drives

PCI

Parts of an SSD

ControllerMain

Processor

I/O Bus Lanes

RAM Cache

Battery / SuperCapacitor

Interface

Flash Chips

Flash Controllers

• Flash Translation Layer (FTL)

• Stripe Writes

• Interpret bus instructions

• Wear Leveling

• Garbage Collection

Flash Translation LayerLBA (0...n blocks)

F L A S H C H I P S

SSD Aspects & Concerns

Longevity

• Primarily determined by the class of flash

• (e)SLC, (e)MLC, TLC

• Related to wear-leveling

• Under-reported capacity

• Short-stroking improves lifetime (not speed)

Partition Alignment

• Performance and longevity

• As big (or bigger) issue than it was in spinning disks

• Native 4k read blocks

• Far larger erasure blocks

• larger than is practical for block-size matching

TRIM/Discard

• As a command, TRIM refers to ATA-8 spec

• SCSI equivalent is UNMAP, but both are often referred to as TRIM.

• Does not immediately delete unused blocks

• Allows for GC

Linux TRIM Support

• EXT4 / XFS / JFS / BTRFS - Native using ‘discard’ option

• Consider NOOP or Deadline IO scheduler

• Userland:

• fstrim (part of util-linux) for R/W vols

• zerofree for R/O vols

OSX Trim Support

• Comes by default on factory-installed SSDs

• Trim-Enabler

• http://www.groths.org/trim-enabler/

ZFS and SSDs

• ZFS Intent Log (ZIL)

• Adaptive Replacement Cache (ARC)

• arc_summary can help you decide

Windows• Native TRIM

• Win 7, Server 2012+

• ATA TRIM

• Server 2012 R2

• Manufacturer Utilities Only

• < Win 7, Server 2012

Monitor Health w/ S.M.A.R.T.

• S.M.A.R.T. information

• vendor-specific

• Includes flash erase count

• smartctl on Linux and Mac

• Dozens of tools on Windows (check wiki)

Forensics

(http://bit.ly/fast11-wei-paper)

...Our results lead to three conclusions: !First, built-in commands are effective, but manufacturers sometimes implement them incorrectly. !Second, overwriting the entire visible address space of an SSD twice is usually, but not always, sufficient to sanitize the drive. !Third, none of the existing hard drive-oriented techniques for individual file sanitization are effective on SSDs

Reliably Erasing Data From Flash-Based Solid State Drives Michael Wei∗, Laura M. Grupp∗, Frederick E. Spada†, Steven Swanson∗

∗Department of Computer Science and Engineering, University of California, San Diego †Center for Magnetic Recording and Research, University of California, San Diego

Questions?