LFS Design Document
Transcript of LFS Design Document
1
Implementation of Log Structured File System
Report by Abhinav Jha, Nithya K and Sushanth Kumar Reddy
This document contains the design of the log structured file system programming assignment. The
document will cover in the overall design of the system, features implemented for the phase 1 and the
features planned for the phase 2.
Overview
We have designed a layered structure for the log structured file system. The following layers are present
in the system.
Disk Layer: Does all the operations to write to the virtual disk
Log Layer: This layer treats the disk as a log and will do log read and log write operations.
Inode Layer: Abstracts the internal representation of the file. All operations needed on a file are
provided here and corresponding inodes are also modified.
Vnode layer: This layer is responsible for the file and directory abstraction. It is the external interface to
the LFS
Fuse: fuse layer intercepts messages from FUSE, operates on the LFS through the vnode layer and
returns expected values
Detailed design
Basic Log structure
The virtual disk created by mklfs has the following format. The log is divided into segments. Each
segment is of fixed size.
Disk
Log
Inode
Vnode
Fuse
2
superblock
Data
segment
Data
segment
Data
segment
Data
segment
Data
segment
Data
segment
Log is divided into fixed sized segments.
Super Block
The first segment of the log contains the super block which will have the system Meta data information.
The super block will contain
Checkpoint 1 Checkpoint 2
Super block structure
System Meta data contains all the system information like segment size, disk size, block size etc. We
have 2 checkpoints.
Each checkpoint has the following information
• Last good segment points to the last good segment in the disk of threaded segments.
• Inode version table contains the list of all inodes and their version. This is an optimization that
will help us avoid the need to scan through the inodes while doing a segment clean.
• Segment usage table helps us keep track of the usage of segments and help us find the next free
segment.
• Ifile inode contains the inode of the ifile.
Data Block: The data block contains the following information.
Segment summary
table
Data blocks
Prev
segment
Next
segment
System Meta data
Last good segment
Inode version table
Segment usage table
Ifile inode
3
The initial block contains the segment header which contains the segment summary table. The segment
summary table will contain information about the blocks that the segment contains. For each block it
will store inode number that the block is part of, version (currently unused), offset of the block in the
file. The segment header also contains pointers to next and previous segments.
Log layer data structures
Log layer treats the disk as collected of segments. All the operations done by log are in segments. The
log layer maintains three important data structures.
• Tail segment: Tail segment is always in memory where all the log writes are done.
• Segment read cache: This will contain few segments that are frequently accessed. When a log
read happens, the segment is checked in segment reach cache and then tail segment and if it’s
not present the segment is read from the disk. The replacement policy used is LRU.
• Check pointing cache: This contains the segment summary tables of the segments that are
recently written. The size of the check pointing cache is set by the check pointing interval. Once
the number of segments written crossed the checkpointing interval, checkpointing is triggered.
Check pointing will make sure segment usage table, inode version table, ifile inode and last good
segment number is update. In case of LFS release, checkpointing will also write the tail segment
to the disk.
Inode Layer data structures
The following data structures are used in the inode layer.
• Inode read cache: Frequently used inodes are cached in the inode read cache. This helps
prevent making frequent log read calls. The replacement policy used in LRU.
• Inode write cache: This will contain the inode that is most recently modified. Once the inode is
written to the log it is also copied to the read cache, so that read cache will contain the most
recent copy of the inode.
• Ifile Inode: The ifile inode is kept in memory. Whenever the ifile is changes, the ifile inode has to
be updated. This ifile is frequently written to the log layer which also maintains a copy of ifile
which will eventually be written to disk at the time of checkpointing.
• List of free inodes: A list of free inodes are maintained which the Vnode layer will make use of
while creating a file. It is currently unused in the phase 1 of the project.
Vnode layer data structures
The following data structure is used in Vnode layer.
• Path to inode number mapping: This will contain a list of recently referred file paths and their
inode number. This will reduce the number of calls made to the directory file. This is currently
disabled for the phase 1 of the project.
4
Feature list
Feature Phase 1 Phase 2
Mklfs Complete implementation done
in phase 1.
Log layer
Checkpoint Complete implementation done
in phase 1.
Segment
Cleaner
Segment cleaner utility functions
are planned for the phase 2.
Roll forward Roll forward utility functions are
planned for the phase 2.
Others Tail segment free blocks reusing
to be implemented in the phase
2.
Inode Layer
Direct block
pointers
Complete implementation is
done in phase 1.
Indirect block
points
Partial implementation is done. Finish remaining parts of the
implementation
Roll forward Roll forward utility functions are
planned for the phase 2.
Vnode Layer
Directory
structure
Currently only 1 directory i.e.
root implemented. The file
names are inode number.
Full directory structure
implementation planned for
phase 2.
Links Complete implementation
planned for phase 2.
Fuse Layer Functions
implemented
• lfs_getattr
• lfs_open
• lfs_read
• lfs_unlink
• lfs_write
• lfs_opendir
• lfs_readdir
• lfs_destroy
• lfs_create
• lfs_truncate.
• lfs_readlink
• lfs_mkdir
• lfs_rmdir
• lfs_release
• lfs_releasedir
• lfs_init
• lfs_statfs
NOTE: The current implementation for phase 1 just supports direct block pointers. So since the
directory file can contain only N no of blocks (N is direct block pointers). So the number of files that
you can create is limited by the no of direct block points. For example if N = 4, then root can contain
only 4 blocks. Each block can contain entries for 4 files, so for phase 1 max no of files that can be
created is 15-16.