Silberschatz, Galvin and Gagne 2002 12.1 Operating System Concepts Chapter 12: File System...

Silberschatz, Galvin and Gagne 200212.1Operating System Concepts

Chapter 12: File System Implementation

File System Structure File System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems - omit for now NFS - omit for now

Annotated for use with Silberschatz’s book used during CS350, Fall 03

Instructor's annotation in blue Last updated 12/4/03+


File-System Structure

File structure Logical storage unit Collection of related information

File system resides on secondary storage (disks). Design problem:

How should the file system look to the user … attributes, operations, directory structure

Creating algorithms and data structures to map the logical file system to physical secondary storage device.

File system organized into layers. File control block (FCB) – storage structure consisting of information

about a file – stored on disk, copied to memory for use.


Manages file attribute/control infoManages directory structure: symbolic file names to logical block#,Uses File Control Block (FCB)

Logical block# in physical block# out, (knows location of file and allocation scheme used)

Generic cmds to dev drivers using disk addresses:cylinder, track, sector

Device drivers: interfaces to hardware/controllers, gets generic commands.

Hardware controllers

Layered File System


A Typical File Control Block

Location on disk


Overview On-disk structures

Boot control block – 1st block of partition - used for booting if this is a bootable partition. May also have information on another partition which may be where booting may start (the Master Boot Record, MBR).

Partition control block- attributes of partition, FCB pointers, number blocks in partition, free block count, free block pointers, etc.

Directory structure The FCB’s - (inode in UNIX)

In-Memory structures In memory copy of partition table Cache of recently accesses directories System-wide open file table copy of FCBs for all open files +more Per-process open file table (pointers to system-wide open file table) –

accessed via file descriptor returned from open call – reduces search time


In-Memory File System StructuresThe following example (fig. 12.3) illustrates the necessary file system structures provided by the operating systemsSee Sect. 12.2.1 for scenario description

opening a file

reading a file


Scenario for file operationssee p. 415 of text

Creating a file Allocate a new FCB Read appropriate directory into memory, and add the new file

name and FCB to it Write it back to disk

Opening a file Directory structure searched – parts cached un memory FCB copied to System-wide-open file table File descriptor returned which indexes into the per-process-

open file table. Reading a file

Using file descriptor, access the per processes open file table Which in turn gets the FCB from system wide open file table FCB has information needed to access the data in the file


Directory Implementation

Must search director for a required file

Linear list of file names with pointer to the data blocks. simple to program time-consuming to execute

Hash Table – linear list with hash data structure. decreases directory search time collisions – situations where two file names hash to the

same location fixed size


File system allocation(new slide)

Many files of variable size stored on the same disk

Main problem is how to allocate disk blocks for these files so that disk space is utilized effectively and files can be accessed quickly.

Three major allocations methods: contiguous Linked indexed


Contiguous Allocation

Each file occupies a set of contiguous blocks on the disk.

Minimal head movement – high performance(minimal head movement) - used by IBM VM system

Simple – only starting location (block #) and length (number of blocks) are required.

Random access – for file stating at block b, a block in that file at i blocks into the file would be at block b+i

Good for sequential access also Wasteful of space– external fragmentation – same problem as in dynamic

contiguous memory management (ch 9) – with same solutions.

Files cannot grow – if we over-allocate then internal fragmentation. Difficult to estimate how much space is needed for file

Modified contiguous allocation – give “extents” to make file grow – just postpones the original problem


Contiguous Allocation of Disk Space

Start pointes to 1st block In file

Length is the number of blocks in file


Extent-Based Systems

Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme.

Extent-based file systems allocate disk blocks in extents.

An extent is a contiguous block of disks. Extents are allocated for file allocation. A file consists of one or more extents.

–solves growth problem, but just postpones the original problem- what should be the size of the extent: too large – internal fragmentation, too small: external fragmentation.


Linked Allocation Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

Pointer is to next block – file is treated like a linked list

Sequentially follow pointers to access a given block

pointerblock =


Linked Allocation (Cont.)

Simple – need only starting address Free-space management system – no waste of space No random access

File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2. – Combines linked with indexed (see later)

Incorrect block location calculations removed


Linked Allocation (Cont.)

Solves external, internal (minimal) fragmentation problem, and growth problem – only limit is free space pool(as paging did in memory management, but no random access)

Only sequential access –direct access support poor (must be “simulated” )

Access is slow and variable because of having to read all pointers in blocks before the one desired.

Each block has wasted space for pointer – minimized by clustering blocks – but this may result in increased internal fragmentation

Reliability a problem: lost or damaged pointers


Linked Allocation


Indexed Allocation Brings all pointers together into the index block. One “logical” index block per file … in general may have multiple linked index blocks – see

later. Logical view.

index table

Solves internal, external fragmentation, and growth problem In addition allows direct/random accessIn simplest form, size of index block determines number of blocks in fileIf index block too large, then wasted space for small files.If index block not large enough then need to link index blocks together – see later.


Example of Indexed Allocation


Indexed Allocation (Cont.)

Need index table (ie., block) Random access Dynamic access without external fragmentation, but have

overhead of index block. Index block permanently stored on disk, but may be

cached in memory – to further improve disk performce. If cached, we have the old problem of keeping the disk updated – especially if we have a crash.

Incorrect block location calculations removed


Indexed Allocation – Mapping (Cont.)(new slide)

Problem is when the file so large that there are not enough entries in a single index block.

Want to map from logical to physical in a file of unbounded length Use a scheme in which the index blocks are themselves linked:

Link blocks of index table (no limit on size).

The last entry of a linked block of an index table is a pointer to the next index table

Now no logical limit on the size of a file Multilevel index block is another approach to this problem – see

below: Can combine linked index block idea with multi-level: UNIX

I-node scheme –see below


Indexed Allocation – Mapping (Cont.) Multi-level Index blocks - See text

outer-index

index table file


File-Allocation Table Used for windows/OS/2 PC systems Combines linked and indexed concepts

===> See Instructor’s notes: “Comments on File System Implementation” For

detailed description – available on CS350 website


Combined Scheme: UNIX (4K bytes per block)See text

The UNIX Inode

Performance tradeoff favors small files


Free-Space Management See instructors notes: “Comments on File System

Implementation” from the web site. Bit vector (n blocks)

…

0 1 2 n-1

bit[i] = 0 block[i] free

1 block[i] occupied

A bit for every block on disk – offset into map is block number

Block number calculation (moving from left to right):

(number of bits per word) *(number of 0-value words) +offset of first 1 bit in 1st nonzero wordNote:“number of 0-value words” = # of leading all zero words


Free-Space Management (Cont.) Bit map requires extra space. Example:

block size = 212 bytes - 4K page (common)disk size = 230 bytes (1 gigabyte) - on the small siden = 230/212 = 218 bits (or 32K bytes)a 1.3 GB with 512byte page has bit map of 332KB - stresses memory if cached But easy to get contiguous files

Linked list (free list) Link together all free disk blocks. Keep pointer to 1st free block in a special location & cache it Cannot get contiguous space easily No waste of space

Grouping – modification of free-list approach – an indexed approach 1st block in partition (an index block) contains pointers to n free data blocks Last block in the above n free blocks is another index block pointing to another n free blocks

with the last of these being another pointer block, etc. Similar the the linked index block approach for file allocation. Advantage: the addresses of a large number of free blocks can be found quickly

Counting Keep track of groups of contiguous blocks. Just need address of first block in group & number of free contiguous blocks following – similar

to extents Shorter index list


Free-Space Management (Cont.)

Need to protect: Pointer to free list Bit map

Must be kept on disk Copy in memory and disk may differ. Cannot allow for block[i] to have a situation where bit[i] =

1 in memory and bit[i] = 0 on disk. Solution - keep memory copy & disk copy in synch:

Set bit[i] = 1 in disk. Allocate block[i] Set bit[i] = 1 in memory


Linked Free Space List on DiskFree (unused) blocks physicallyLinked together.

Pointer to 1st block may be cachedin Memory

The FAT method has free spaceManagement built into the access scheme.

The text unjustifiably criticizes thismethod as inefficient due to the need for transverse the list. We should never have to transverse thelist. As long as we have a pointer tothe first block in the list, we need only to access the first block: Takethe first block if you need a new block, or add a free block the head of the list.


Efficiency and Performance Efficiency dependent on:

disk allocation and directory algorithms types of data kept in file’s directory entry - maintaining “statistics” in

the directory - see p. 390? UNIX: vary cluster size as file grows (minimizes internal

fragmentation) UNIX: Pre-allocates inodes on a disk to distribute this structure,

wastes some space but improves performance - see below

Performance disk cache – separate section of main memory for frequently used blocks free-behind and read-ahead – techniques to optimize sequential access improve PC performance by dedicating section of memory as virtual disk,

or RAM disk. History: was frequently done on PCs when all you had was a floppy.

UNIX: distribute inodes across partition - keep data blocks near files inode - minimize seeks


Various Disk-Caching Locations

Track buffer: page 434

• Note the difference between a RAM disk and disk cache:• Contents of RAM disk under user control - same as a real disk• Contents of disk cache under control of OS

• cache disk referenced blocks on assumption that they will be needed again (temporal locality)• Use I/O protocol for disk cache

Disk Controller


Page Cache A page cache caches pages rather than disk blocks using virtual

memory techniques. More efficient that caching disk blocks (disk cache) - higher

performance Use memory management protocol for page cache – uses

memory calls or instructions.

Memory-mapped I/O uses a page cache.

Routine I/O (I/O reads and writes) through the file system uses the buffer (disk) cache.

Problem: what if you want to do memory mapped “paged” caching if the underlying architecture uses ordinary I/O?

This leads to the following figures.


I/O Without a Unified Buffer Cache

Fig. 12.11p. 435

File I/O interfaceUse read/write calls

Because the virtualMemory cannot Interface with theBuffer cache, the Data in buffer cacheIs copied to pageCache – double Caching - inefficient

Virtual MemoryInterface: use memory references

Page caching mappedonto buffer caching

pagesDiskblocks


I/O Using a Unified Buffer CacheA unified buffer cache uses the same page cache to cache both memory-mapped pages and ordinary file system I/O:

page cache?

Only a single Buffer needed – More efficient.


Recovery

“lazy” disk update may be used - can leave the disk in an inconsistent state if a crash ocurrs before a disk update is made.

Consistency checking – compares data in directory structure with data blocks on disk, and tries to fix inconsistencies - a scandisk function?

Use system programs to back up data from disk to another storage device (floppy disk, magnetic tape).

Recover lost file or disk by restoring data from backup Ex. The Master Boot Record is backed up - better to

resore an obsolete version than none at all.


Log Structured File Systems <<omit>>

Log structured (or journaling) file systems record each update to the file system as a transaction.

All transactions are written to a log. A transaction is considered committed once it is written to the log. However, the file system may not yet be updated.

The transactions in the log are asynchronously written to the file system. When the file system is modified, the transaction is removed from the log.

If the file system crashes, all remaining transactions in the log must still be performed.


Network File System - NFS: Overview NFS views a set of interconnected machines as

having independent file system. Goal: allow sharing of file systems omong machines

on the network in a transparent manner. Sharing is not mandatory – it is upon request of a machine.

Sharing is based on a client/server relationship. Sharing is allowed between any pair of machines

raher than only with a dedicated server. The file mounting protocol is used to make a remote

directory accessible in a transparent manner.


The Sun Network File System (NFS)Omit NFS slides for now

An implementation and a specification of a software system for accessing remote files across LANs (or WANs).

The implementation is part of the Solaris and SunOS operating systems running on Sun workstations using an unreliable datagram protocol (UDP/IP protocol and Ethernet.


NFS (Cont.) Omit NFS slides for now

Interconnected workstations viewed as a set of independent machines with independent file systems, which allows sharing among these file systems in a transparent manner. A remote directory is mounted over a local file system

directory. The mounted directory looks like an integral subtree of the local file system, replacing the subtree descending from the local directory.

Specification of the remote directory for the mount operation is nontransparent; the host name of the remote directory has to be provided. Files in the remote directory can then be accessed in a transparent manner.

Subject to access-rights accreditation, potentially any file system (or directory within a file system), can be mounted remotely on top of any local directory.


NFS (Cont.) Omit NFS slides for now

NFS is designed to operate in a heterogeneous environment of different machines, operating systems, and network architectures; the NFS specifications independent of these media.

This independence is achieved through the use of RPC primitives built on top of an External Data Representation (XDR) protocol used between two implementation-independent interfaces.

The NFS specification distinguishes between the services provided by a mount mechanism and the actual remote-file-access services.


Three Independent File Systems Omit NFS slides for now


Mounting in NFS Omit NFS slides for now

Mounts Cascading mounts


NFS Mount Protocol Omit NFS slides for now

Establishes initial logical connection between server and client.

Mount operation includes name of remote directory to be mounted and name of server machine storing it. Mount request is mapped to corresponding RPC and forwarded

to mount server running on server machine. Export list – specifies local file systems that server exports for

mounting, along with names of machines that are permitted to mount them.

Following a mount request that conforms to its export list, the server returns a file handle—a key for further accesses.

File handle – a file-system identifier, and an inode number to identify the mounted directory within the exported file system.

The mount operation changes only the user’s view and does not affect the server side.


NFS Protocol Omit NFS slides for now

Provides a set of remote procedure calls for remote file operations. The procedures support the following operations: searching for a file within a directory reading a set of directory entries manipulating links and directories accessing file attributes reading and writing files

NFS servers are stateless; each request has to provide a full set of arguments.

Modified data must be committed to the server’s disk before results are returned to the client (lose advantages of caching).

The NFS protocol does not provide concurrency-control mechanisms.


Three Major Layers of NFS Architecture Omit NFS slides for now

UNIX file-system interface (based on the open, read, write, and close calls, and file descriptors).

Virtual File System (VFS) layer – distinguishes local files from remote ones, and local files are further distinguished according to their file-system types. The VFS activates file-system-specific operations to handle

local requests according to their file-system types. Calls the NFS protocol procedures for remote requests.

NFS service layer – bottom layer of the architecture; implements the NFS protocol.


Schematic View of NFS Architecture Omit NFS slides for now


NFS Path-Name Translation Omit NFS slides for now

Performed by breaking the path into component names and performing a separate NFS lookup call for every pair of component name and directory vnode.

To make lookup faster, a directory name lookup cache on the client’s side holds the vnodes for remote directory names.


NFS Remote Operations Omit NFS slides for now

Nearly one-to-one correspondence between regular UNIX system calls and the NFS protocol RPCs (except opening and closing files).

NFS adheres to the remote-service paradigm, but employs buffering and caching techniques for the sake of performance.

File-blocks cache – when a file is opened, the kernel checks with the remote server whether to fetch or revalidate the cached attributes. Cached file blocks are used only if the corresponding cached attributes are up to date.

File-attribute cache – the attribute cache is updated whenever new attributes arrive from the server.

Clients do not free delayed-write blocks until the server confirms that the data have been written to disk.

Silberschatz, Galvin and Gagne 2002 12.1 Operating System Concepts Chapter 12: File System...

Documents

Transcript of Silberschatz, Galvin and Gagne 2002 12.1 Operating System Concepts Chapter 12: File System...