Chapter 11: File-System...
Transcript of Chapter 11: File-System...
Chapter 11: File-System Implementation
Operating Systems CS 33211 2
Chapter 11: File System Implementation
� File-System Structure
� File-System Implementation
� Directory Implementation
� Allocation Methods
� Free-Space Management
� Efficiency and Performance
� Recovery
� Log-Structured File Systems
� NFS
Operating Systems CS 33211 3
Objectives
� To describe the details of implementing local
file systems and directory structures
� To discuss block allocation and free-block
algorithms and trade-offs
Operating Systems CS 33211 4
File-System Structure
� File structure� Logical storage unit
� Collection of related information
� File system resides on secondary storage (disks). � Allows read, write and modify block of data from disk� Sequence or Random access to any given block of information
� I/O transfers between disk and memory in blocks of typically 512bytes (efficiency)
� File system organized into layers� Efficiency
� Avoids duplication of code
� File System maintains file structure via File control block: storage structure consisting of information about a file� Ownership
� Permissions
� Locations of file contents …
Operating Systems CS 33211 5
The file system provides efficient and convenient
access to information on the disk
� File System Design Challenges:
1. File system UI
� File Attributes, Operations
� Directory structure
2. Mapping logical file system to physical storage (disk)
� Requires efficient algorithms and data structures
File system Design Challenges
Operating Systems CS 33211 6
File System
Layered Design
� In Layered Architecture� Higher levels make use of features in the lower
level
� I/O Control � Uses Device drivers to transfer info between
memory and disk
� Device driver
� Translates commands to hardware instructions for use by controller
� Basic file system:� Issues generic commands to read/write physical
blocks on disk
� File Organization� Translates logical block addresses to physical
block addresses
� Logical File System� Manages directory structure
� Metadata information
� Maintains file structure via File Control Block (FCB)
Operating Systems CS 33211 7
A Typical File Control Block
Operating Systems CS 33211 8
Creating a new file
1. Your application program calls the logical file system
2. Logical file system creates a new FCB (allocates one from free FCBs)
3. Systems reads appropriate directory into memory
� Creates new file name
� Creates new FCB
4. Writes back onto disk
Now the new file may be used for I/O operations
Operating Systems CS 33211 9
In-Memory File System Structures
� Necessary file system structures provided by the
OS.
� Figure 12-3(a) refers to opening a file.
� Figure 12-3(b) refers to reading a file.
Operating Systems CS 33211 10
In-Memory File System Structures
� The open( ) call first searches “system-wide-open file table” to see if file is in use
� If file is in use:� An entry is created in “per-process
open-file table”. Entry points to system-wide open-file-table
� Else:� Search directory structure for fie name
� Copy file’s FCB to system-wide open file table
� An “entry” is created in “per-process open-file table”. Entry points to system-wide open-file-table
� Open( ) call returns a pointer to the entry in the per-process open-file table
All I/O on the file are performed via pointer
Operating Systems CS 33211 11
Virtual File Systems
� Virtual File Systems (VFS) provide an object-oriented way to simplify, organize and modularize File-system implementation
� VFS provides a uniform system call interface (the API) to be used for different types of file systems.
� The API is to the VFS interface, rather than any specific type of file system:
� Using data structures and procedures
OS concurrently supports multiple type of file-systems via VFS interface
Operating Systems CS 33211 12
Schematic View of Virtual File System
� Layer 1: File System Interfaces handles:
� System calls: open(), read(),
write(), close()
� Layer 2: Virtual File System
� Separates generic operations from
implementation details
� API provides a uniform interface to different types of file system implementations
� Layer 3: Supported file system types
Operating Systems CS 33211 13
Directory Implementation
� Linear list simple to program but time-consuming to execute� Maintain a list of file names with pointers to the data blocks.� To create a new file:
� Search dir (or list) to be sure file does not exist� Add new at end of list
� To delete a file:� Search list for file then delete name from file
� Disadvantage: Linear search of directory is time consuming
� Hash Table – linear list with hash data structure.� Linear list stores directory entries
� Harsh table takes a value computed from file name and returns a pointer to file name in the list� decreases directory search time
� collisions – situations where two file names hash to the same location
� Disadvantage: Fixed size
Directory management algorithms can affect efficiency of file-systems
Operating Systems CS 33211 14
Allocation Methods
� An allocation method refers to how disk
blocks are allocated for files:
� Contiguous allocation
� Linked allocation
� Indexed allocation
Operating Systems CS 33211 15
Contiguous Allocation
� Each file occupies a set of contiguous blocks on the disk
� Simple – only starting location (block #) and length (number of
blocks) are required
� Random access
� Wasteful of space (dynamic storage-allocation problem)
� Files cannot grow
Operating Systems CS 33211 16
Contiguous Allocation
� Mapping from logical to physical
LA/512
Q
R
Block to be accessed = ! + starting address
Displacement into block = R
Operating Systems CS 33211 17
Contiguous Allocation of Disk Space
Operating Systems CS 33211 18
Extent-Based Systems
� Many newer file systems (I.e. Veritas File
System) use a modified contiguous allocation
scheme
� Extent-based file systems allocate disk
blocks in extents
� An extent is a contiguous block of disks
� Extents are allocated for file allocation
� A file consists of one or more extents.
Operating Systems CS 33211 19
Linked Allocation
� Each file is a linked list of disk blocks: blocks
may be scattered anywhere on the disk.
pointerblock =
Operating Systems CS 33211 20
Linked Allocation (Cont.)
� Simple – need only starting address
� Free-space management system – no
waste of space
� No random access
� Mapping
Block to be accessed is the Qth block in the linked chain of
blocks representing the file.
Displacement into block = R + 1
LA/511
Q
R
Operating Systems CS 33211 21
Linked Allocation
Operating Systems CS 33211 22
Indexed Allocation
� Brings all pointers together into the index
block.
� Logical view.
index table
Operating Systems CS 33211 23
Example of Indexed Allocation
Operating Systems CS 33211 24
Linked Allocation
File-Allocation Table (FAT)
� FAT resides at beginning of
each disk partition
� Each FAT index represents
block number of the file
� Last block has eof value
� Directory entry contains the
block number of first block
� Each FAT entry contains the
next block number of the file
MS-Dos and OS2 Operating Systems
Operating Systems CS 33211 25
Indexed AllocationSmall Files (max size 256K words; block size 512 words)
� uses directory entry table
� Uses Index table, size normally one disk block (512)
� Provides random access
� Provides dynamic access without external fragmentation,
but have overhead of index block.
LA/512
Q
RQ = displacement into index table
R = displacement into block
Recall, index allocation:
For small files: Mapping from logical to physical:
Requires only 1 index table
Operating Systems CS 33211 26
Indexed AllocationLarge Files (Unbounded Length; block size 512 words)
� Linked scheme – Link several blocks of index table (no limit on size).
LA / (512 x 511)
Q1
R1
Q1 = block of index table
R1 is used as follows:
R1 / 512
Q2
R2
Q2 = displacement into block of index table
R2 displacement into block of file:
Last word is pointer to another index block
For Large Files:
Operating Systems CS 33211 27
Indexed Allocation
Two-Level Index
� Two-level index (maximum file size is 5123)
LA / (512 x 512)
Q1
R1
Q1 = displacement into outer-index
R1 is used as follows:
R1 / 512
Q2
R2
Q2 = displacement into block of index table
R2 displacement into block of file:
Operating Systems CS 33211 28
Indexed Allocation
Multi-level Index
� Directory entry table points to
the start of the outer index
table
� Outer-index table points to a
set of second level index
blocks
� Second level block may
also point to another set of
third level index blocks
� Third level points to fourth
…
� Entry in index table contains
direct access block address
M
outer-index
index table file
Operating Systems CS 33211 29
Combined Scheme:
UNIX File Systems (4K bytes per block)
� Inode contains N pointers of the index
block (E.g., N =12)
� First nine pointers point to direct
blocks (small files ~ 4*9 KB)
� The 10th pointer points to an
indirect block (an index block)
� Index block contains address
of blocks that contain data
blocks
� The 11th pointer points to double
indirect block (an index block)
� The double indirect block
contains?
� The 12th pointer points to triple
indirect block
Operating Systems CS 33211 30
Free-Space Management
Bit Vector
� To create new files, we need to
re-use free space from deleted
files
� How do you keep track of free
space on the disk?
� Maintain a free-space list
� Space not allocated to
files or directory
� Implemented as Bit
Vector
…
0 1 2 n-1
bit[i] =
67
8 0 ⇒ block[i] free
1 ⇒ block[i] occupied
Block number calculation
(number of bits per word) *
(number of 0-value words) +
offset of first 1 bit
Block number calc applies to
First non-zero word
n-Bit vector:
Operating Systems CS 33211 31
Free-Space Management
n-Bit Vector Storage
� n-Bit vector requires extra space
� Example:
block size = 212 bytes
disk size = 230 bytes (1 gigabyte)
n = 230/212 = 218 bits (or 32K bytes)
� Requires 32K bytes to store bit maps
� Easy to get contiguous files
Operating Systems CS 33211 32
Free-Space Management
Linked Free Space List on Disk
� Basic idea is to link all free disk blocks
� Requires a pointer to first free block
� Pointer stored on disk
and cache
� Each block contains
a pointer to next free disk block
� Linked list (free list)
� Cannot get contiguous
space easily
� No waste of space
Operating Systems CS 33211 33
Efficiency and Performance
� Efficiency dependent on:
� Disk allocation and directory algorithms
� Linked allocation Vs Index allocation
� Types of data kept in file’s directory entry
� Pointer size limits the length of file
� 16-bit pointer � 216 (64 KB)
� 32-bit pointer � 232 (4 GB) � Requires more disk space to store index tables
� Performance
� Disk cache – separate section of main memory for frequently used Disk blocks
� Free-behind and read-ahead – techniques to optimize sequential access
Operating Systems CS 33211 34
Performance
Page Cache
� A page cache caches pages rather than disk
blocks using virtual memory techniques
� Memory-mapped I/O uses a page cache
� Routine I/O through the file system uses the
buffer (disk) cache
� This leads to the following figure
Operating Systems CS 33211 35
Performance
I/O Without a Unified Buffer Cache
Operating Systems CS 33211 36
Performance
Unified Buffer Cache
� A unified buffer cache uses the same page
cache to cache both memory-mapped pages
and ordinary file system I/O
Operating Systems CS 33211 37
Performance
I/O Using a Unified Buffer Cache