YAFFS: Yet Another Flash File System 2008-01-18 Speaker : Yunjung Yoo.
-
Upload
brett-laurence-merritt -
Category
Documents
-
view
215 -
download
2
Transcript of YAFFS: Yet Another Flash File System 2008-01-18 Speaker : Yunjung Yoo.
YAFFS: Yet Another Flash File System
2008-01-18Speaker : Yunjung Yoo
Contents• Introduction• History• Design• Terminology• Layout – NAND / RAM• Log-structured File System (1 ~ 5)• Mount / Scan• Read / Write• Read Syscall Flowchart (1 ~ 2)• Garbage Collection• ECC• YAFFS2• OOB Data• Checkpointing• Conclusion
Introduction
• Charles Manning, Aleph One• The first file system designed for NAND• YAFFS1 : 512 byte pages + 16 byte spare areas• YAFFS2 : 2048 byte pages + 64 byte spare areas
• Background– SmartMedia + FAT : low memory footprint / not robust– NOR + JFFS : based on JFS / long boot-time, RAM overhead
History
• Decided to create ’YAFFS’ - Dec 2001• Working on NAND emulator - March 2002• Working on real NAND (Linux) - May 2002• WinCE version - Aug 2002• ucLinux use - Sept 2002• Linux rootfs - Nov 2002• pSOS version - Feb 2003• Shipping commercially - Early 2003• Linux 2.6 supported - Aug 2004• YAFFS2 - Dec 2004• Checkpointing - May 2006
Terminology
• Flash-defined– Page - 2k flash page (512 byte YAFFS1)– Block - Erasable set of pages (typically 64 on 2K NAND)
• YAFFS-defined– Chunk - YAFFS tracking unit.– Usually == page.
Design• 각 파일은 id 를 갖는다 . – inode, id 가 0 이면 유효하지 않은 파일임• 파일은 chunk(= page) 로 나뉘어 저장된다 . (2K / 512B)• Chunk 는 0 과 그 이상의 숫자를 갖는다 . 0 은 그 파일의 헤더이다 .• 객체를 나타낸 자료구조 :
– yaffs_ObjectHeader(NAND), yaffs_Object(RAM)• 플래시의 각 페이지는 file id : chunk # 로 구분한다 .• Chunk 에 대한 정보는 Tags 로 OOB 영역에 저장된다 .• 파일의 내용을 변경할 경우 , 관련된 chunk 는 새로운 데이터와 같은
태그로 새로운 위치에 쓰여진다 .• 이때 , 이전의 페이지와 새로운 페이지를 구분하기 위해 write 가
일어날 때마다 2-bit serial#(YAFFS1) / sequence(YAFFS2) 가 순차적으로 증가한다 . 이것은 crash-recovery 할 때 분별할 수 있게 한다 .
• “Discarded” page 가 꽉 찬 block 은 GC 대상이 된다 .• “Deleted” object 는 “ Unlinked dir” 로 옮겨져 삭제된다 .
Layout | NANDPage Spare
Chunk Tag
ObjectHeader ChunkId == 0
Data ChunkId > 0
<YAFFS1 : Page Layout>
Byte# Usage
0~511 Data. (File data or header)
512~515 Tags
516 Data Status
517 Block Status
518~519 Tags
520~522 ECC
523~524 Tags
525~527 ECC
<YAFFS1: Tags bit usage>
Bit# Usage
18 File id
2 2-bit serial number
20 Page-id(chunk id)
10 Counter (the # of bytes used in the page
12 ECC on tags
2 Unused
Layout | RAM
Header(/)
Header(file1)
Data(file1)
Data(file1)
Header(dir1)
Header(file2)
Data(file2)
<NAND Flash>
Object(file2)
Object(file1)
Object(dir1)
Tnode(file1)
Object(/)
Tnode(file2)
<Main Memory>
Child
Sibling
Parent
Log-structured FS(1)
• Supposed the flash with 4 pages per block• First, Create the file
• Then, Write a few chunks
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Live Object header for this files
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Live Object header for this files
0 1 100 1 Live First chunk of data
0 2 100 2 Live Second chunk of data
0 3 100 3 Live Third chunk of data
Log-structured FS(2)
• Close the file >> Write a new object header for the file
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Del Obsoleted object header
0 1 100 1 Live First chunk of data
0 2 100 2 Live Second chunk of data
0 3 100 3 Live Third chunk of data
1 0 100 0 Live New object header
Log-structured FS(3)
• Open the file for read/write and overwrite part of the first chunk in the file and close the file. The replaced data and object header chunks become deleted.
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Del Obsoleted object header
0 1 100 1 Del Obsoleted First chunk of data
0 2 100 2 Live Second chunk of data
0 3 100 3 Live Third chunk of data
1 0 100 0 Del Obsoleted object header
1 1 100 1 Live New first chunk of data
1 2 100 0 Live New object header
Log-structured FS(4)
• Now resize the file to zero by opening the file with O_TRUNC and closing the file. This writes a new object header with length 0 and marks the data chunks deleted.
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Del Obsoleted object header
0 1 100 1 Del Obsoleted First chunk of data
0 2 100 2 Del Obsoleted 2nd chunk of data
0 3 100 3 Del Obsoleted 3rd chunk of data
1 0 100 0 Del Obsoleted object header
1 1 100 1 Del Obsoleted first chunk of data
1 2 100 0 Del Obsoleted object header
1 3 100 0 Live New object header
Log-structured FS(5)
• Block#1 is dirty block!!• Rename the file, then we need new object header
Block Chunk ObjID ChunkID Delete?
Comment
0 0 100 0 Del Obsoleted object header
0 1 100 1 Del Obsoleted First chunk of data
0 2 100 2 Del Obsoleted 2nd chunk of data
0 3 100 3 Del Obsoleted 3rd chunk of data
1 0 100 0 Del Obsoleted object header
1 1 100 1 Del Obsoleted first chunk of data
1 2 100 0 Del Obsoleted object header
1 3 100 0 Del Obsoleted object header
2 0 100 0 Live New object header
Mount / Scan
• Mount 시 Spare 만 읽는다 .• (chunkId == 0) 인 경우 , Object Header 가 저장된 Page
이므로 , 이를 읽어서 Object 정보를 주 메모리에 동적으로 생성 .
• YAFFS2 의 경우 저장된 checkpoint 를 읽어들인다 .
ObjectHeader
ChunkID=0
File data
ChunkID=1
Object Header
ChunkID=0
....
....
Page1 Page2 Page 32
Read / Write
• Write– Page 단위로 기록한다 .– Page 보다 작거나 파일의 마지막 단편 부분은 정책상 Cache
된다 .
• Read– Cache 에 읽고자 하는 데이터를 찾는다 .– 없는 경우는 실제 Flash 로부터 읽어온다 .
• Read/Write 성능 향상을 위해 ChunkCache 라는 간단한 자료구조 사용– Read/write 회수 감소 , Flash memory 수명 증가 .– Response Time 감소 .
Read Syscall Flowchart(1)
sys_read
generic_file_read
generic_file_directIO do_generic_file_read
file_read_actoryaffs_readpage
VFSLayer
File System SpecificLayer
no yes
no yes
캐시의 내용을 사용자 영역에 복사
페이지 캐시 해시테이블에요청된 페이지가 존재하는가 ?
페이지 캐시를 사용하는가 ?
System Call handlingLayer
Read Syscall Flowchart(2)
yaffs_readpageVFS InterfaceLayer
CoreLayer
MTD InterfaceLayer
yaffs_readpage_unlock
yaffs_readpage_nolock
yaffs_ReadDataFromFile
yaffs_ReadChunkDataFromObject
yaffs_ReadChunkWithTagsFromNAND
nandmtd2_ReadChunkWithTagsFromNAND
Garbage Collection
• 3 blocks reserved for GC (384K)• If no deleted blocks, GC dirtiest• Soft Background deletion:
– Delete/Resize large files can take up to 0.5s– Incorporated with GC– Spread over several writes
• GC is deterministic - does one block for each write (default)
ECC
• Needs Error Correction Codes for reliable use• ECC on Tags and data• 22bits per 256 bytes, 1-bit correction, 2-bit detection• Lots of options:
– Hardware or software– YAFFS or MTD– New MTD, old MTD or YAFFS/Smartmedia positioning
• Make sure bootloader, OS and FS generation all match!• Can be disabled - not recommended!
YAFFS2
• Designed for new hardware:– >=1k page size– No re-writing– Can support Toshiba/Sandisk MLC parts
• Main difference is ‘discarded’ status tracking• ECC done by driver (MTD in Linux case)• Extended Tags (Extra metadata to improve performance)
• RAM footprint 25-50% less• faster (write 1-3x, read 1-2x, delete 4-34x, GC 2-7x)
OOB Data
• YAFFS1:– Derived from Smartmedia, (e.g byte 5 is bad block marker)– 16 bytes: 7 tags, 2 status, 6 ECC– YAFFS/Smartmedia or JFFS2 format ECC
• YAFFS2:– 64 bytes available in 2k page– MTD-determined layout– MTD or hardware does ECC– Tags normally 28 bytes (16 data, 12ecc)
Checkpointing• RAM structures saved on flash at unmount (10 blocks)• Structures re-read, avoiding boot scan• Invalidated by any write• Lazy Loading also reduces mount time.
Conclusion
• Very simple • NAND-friendly • Relatively frugal with resources (especially RAM) • Boot quickly • Journaling (robustness)