© Original by Donald Acton; Changes by George Tsiknis Unix File System The Structure of the Unix...
-
Upload
nancy-tabb -
Category
Documents
-
view
214 -
download
0
Transcript of © Original by Donald Acton; Changes by George Tsiknis Unix File System The Structure of the Unix...
© Original by Donald Acton; Changes by George Tsiknis
Unix File System
The Structure of the Unix file system is not covered in the text
Unit 13 2
Learning Objectives We have seen the physical characteristics of a disk
and how data is transferred between disk and memory
Now we’ll see how OS organizes the data on the disk. You should be able to:draw a picture of the structure of a file in terms of inodes
and data blocks given its sizecompare the structure of two files given their sizes list the actions performed when a file is created, modified
or deletedestimate the number of read/write operations OS has to
perform in order to read/write data from/to a file identify caching as a way to improve I/O performance
Unit 13 3
The File System
Works directly with the view of the disk provided by the controller
Provides a device independent view of a disk
Further levels of abstraction on file system are what the application sees
Application
Unix I/O
File System
Disk Drive
File System
Layering
Unit 13 4
Purpose of a file system
Provides an abstraction so that applications don’t access disk directly
Hides complexity of different drive types Manages the device Provides a structure for organizing the data Provides protection
Unit 13 5
File System Properties
Persistent – survive reboots Correct – reflect the operations performed Robust – must be able to recover after a crash Efficient – must make good use of disk space
and must be fast
Unit 13 6
Unix File System Semantics
A file is simply a sequence of m bytesB0, B1, .... , Bk , .... , Bm-1
Everything is a fileAll I/O devices are represented as filesExamples:
– /dev/ad0s4e (/var disk partition)– /dev/ttyd0 (terminal)– /dev/rmt0 (tape drive)
Even the running kernel is represented as a file:– /dev/kmem
Unit 13 7
Unix File Types
Regular files: Binary/text files they are all the same
Directory file It’s still a file but it contains names and locations of other files
Special files Block (disks), character (terminal) devices
Interprocess communication Named pipes Sockets
Unit 13 8
Blocks – the basic unit
A file is a sequence of bytes stored in a set of fixed sized blocks
Block sizes are multiples of the sector size (e.g. 512, 1024, 4096, 8192)
The sectors comprising a particular block are contiguous All blocks in a particular file system are the same size With respect to a file, the smallest amount of data the
file system transfers to or from a disk is a block Blocks are the virtualization of a disk’s logical blocks
Unit 13 9
File System Primary Services
FS Must be able to:Keep track of which blocks are free
Determine which blocks belong to a particular file
Maintain administrative data like file permissions,
creation and modified times, etc.
Find the list of free blocks, root directory, and some
“other stuff” when the system is started
Unit 13 10
The Primary Pieces
Super block – helps locate everything
Inode – maintains the file’s administrative data,
tracks the file’s data blocks
Data blocks – multipurpose blocks used for File’s data blocks
Indirect blocks
Unit 13 11
File System Data Structures
Inode list – list of free inodes (inodes are just
numbers)
Inode map – given an inode can locate the disk
block the inode is insometimes this is just a formula
Free block list
Superblock
Unit 13 12
Single Indirect Block
UNIX InodeType/mode Link count
File Size
UID GID
Various modification
access and creation
times
Direct Data Block 0
Direct Data Block 1
Direct Blocks 2 - 9
Single Indirect Block
Double Indirect Block
Additional bookkeeping
information
Data Block
Data Block
0
……127
Data Block
Data Block
0
……127
0
……127
Data Block
Double Indirect Block
0
……127
0
……127
0
……127
0
……127
0
……127
Data Block
Direct Data Block 1
Direct Data Block 0Data Block
0
……127
Triple Indirect BlockTriple Indirect Block
The indirect block is exactly
the size of a block
Data Block
Unit 13 13
(Some) Inode Parts
Type/Mode Type (regular file, directory, device, etc) Permissions
– Group, user, other - 3 bits each (e.g. rwxr-x—x, rw-r-----)
Link count How many directory entries point to this file
UID User ID number – identifies owner of file
GID Group ID number – identifies group of file
Unit 13 14
How big is the largest file that could be created in the original Unix file system?
Compute the number of blocks in the file
Direct Blocks 10 blocksSingle Indirect 128 “Double Indirect 128
2 “
Triple Indirect 1283
“
Total 2,113,674 blocksx 512
1,082,201,088 bytes
Unit 13 15
Block Address Ranges
Direct 0 to 5,119 (10 x 512 – 1)
Single indirect 5120 to 70,655 (5,120 + 128 x 512 – 1)
Double indirect 70,656 to 8,459,263
(70,656 + 1282 x 512 - 1) Triple indirect
8,459,264 to 1,082,201,087(8,459,264 + 1283 x 512 – 1)
Unit 13 16
Locate byte 1,000,000,033
Convert to blocks1,000,000,033 / 512 = 1,953,125The remainder of 33 is offset into the blockBased on result determine if direct access, single,
double, or triple indirectThis case is triple indirect Determine which of the 1283 blocks it is
– 1,953,125 – (10 + 128 + 1282) = 1,936,603
Unit 13 17
Continuing the hunt
1,936,603 0x1D8CDB Observe that 128 is 27 so if we look at the above
in binary and then group 7 bits at a time we can quickly get the indexes to the needed indirect blocks
1 D 8 C D B
0001 1101 1000 1100 1101 1011
Unit 13 18
FreeBSD 6.0 dinode.hstruct ufs1_dinode {
u_int16_t di_mode; /* 0: IFMT, permissions; see below. */
int16_t di_nlink; /* 2: File link count. */
union {
u_int16_t oldids[2]; /* 4: Ffs: old user and group ids. */
} di_u;
u_int64_t di_size; /* 8: File byte count. */
int32_t di_atime; /* 16: Last access time. */
int32_t di_atimensec; /* 20: Last access time. */
int32_t di_mtime; /* 24: Last modified time. */
int32_t di_mtimensec; /* 28: Last modified time. */
int32_t di_ctime; /* 32: Last inode change time. */
int32_t di_ctimensec; /* 36: Last inode change time. */
ufs1_daddr_t di_db[NDADDR]; /* 40: Direct disk blocks. */
ufs1_daddr_t di_ib[NIADDR]; /* 88: Indirect disk blocks. */
u_int32_t di_flags; /* 100: Status flags (chflags). */
int32_t di_blocks; /* 104: Blocks actually held. */
int32_t di_gen; /* 108: Generation number. */
u_int32_t di_uid; /* 112: File owner. */
u_int32_t di_gid; /* 116: File group. */
int32_t di_spare[2]; /* 120: Reserved; currently unused */
};
Unit 13 19
Superblock
Located in first available sector after boot sector
Maintains global file system information such as: Location of inode map and free inode list
Location of the inode that corresponds to the root directory
Location of list of free blocks
Block size being used
Sector size (anything except 512 is unusual)
Dirty flag
Loss of the super block is a catastrophe
For security, it may be copied to other well known
locations
Unit 13 20
Installing a Disk
As shipped the disk does not contain a file system (old disk drives didn’t even contain tracks and sectors)
When disk is installed must: Lay down tracks and sectors (nowadays done by manufacturer) Determine if all the sectors are good (nowadays done by
manufacturer) Partition the “disk” into logical functional regions (optional)
Create a file system in each partition (win: format , Unix: newfs ) Put down a superblock Create an inode map and free inode list Build the list of free blocks
Unit 13 21
Metadata
Data about data File metadata:
Basically information in inodeDescribes owner of data, file size, etc
File system metadata:Superblock informationBlock size, inode map, block lists, etc
Unit 13 22
Directory File
A directory is just a file that contains information to locate other files and directories
A directory entry has a structure like the following
typedef struct dirent32 {
ino32_t d_ino; /* "inode number" of entry */
off32_t d_off; /* offset of disk directory
entry */
uint16_t d_reclen; /* length of this record */
char d_name[1];/* name of file */
} dirent32_t;From: FreeBSD 5.2
Unit 13 23
Superblock + Inodes + data blocks and directories
Inode Map
Free Blocks
Inode of ‘/’
SuperblockInode Inode Inode Inode
Inode Map
Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Inode
Block
usr
Directory
Inode
Data
Block
Data
Block
Indirect
Data
Block
Data
BlockData
BlockData
Block
Data
Block
Unit 13 24
How it might look on disk
Spindle
Unit 13 25
Creating a file
When a file is created it is emptyLocate a free inode Fill in inode fieldsRemove inode from free list
Locate directory to add file to Add a new directory entry which includes
– Filename– Inode of file
Unit 13 26
Creating a File
Free Inodes
Inode Map
Free Blocks
…
Superblock
Inode Inode Inode Inode
Inode Map
Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Some Directory
•
••
FileA
FileC
NewFile
Second Directory
•
••
Link Count = 1Link Count = 2
NewFile2
Data
Block
Unit 13 27
Steps in Writing a File Block
If data goes into an existing blockMake changes to in memory copy, write changed
block
OtherwiseLocate a free block, write dataAdd free block to inode or indirect blockMay have to get a block to use as indirect block first
Unit 13 28
Writing a File Block
Free Inodes
Inode Map
Free Blocks
…
Superblock
Inode Inode Inode Inode
Inode Map
Data
Block Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Some Directory
•
••
FileA
FileC
NewFile
Indirect
Block
Unit 13 29
Steps in Deleting a File
Directory entry is removed and directory updated on disk
Link count on file being removed decremented If link count 0
Return disk blocks to free listReturn inode to free inode list
Unit 13 30
Deleting a File
Free Inodes
Inode Map
Free Blocks
…
Superblock
Inode Inode Inode Inode
Inode Map
Data
Block Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Some Directory
•
••
FileA
FileC
NewFile
Indirect
Block
Link Count = 1Link Count = 0
Unit 13 31
Order of Writing Meta Data
If data not written in proper order file system could be inconsistent if system crashes
Examples: Block claimed by two inodes
– (block added to inode before old inode updated)
Inode referenced from two directories when not a link – Inode freed and then immediately reused but directory of initial
reference not update
Unit 13 32
Meta Data Update Requirements
Indirect blocks and inodes written synchronously upon deallocation
Directory updates consisting of:Deleting a fileAdding a fileChanging (renaming) a file are all done synchronously
Causes significant file system slowdown
Unit 13 33
Ordering Operations (BAD)
Free Inodes
Inode Map
Free Blocks
…
Superblock
Inode Inode Inode Inode
Inode Map
Data
Block Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Some Directory
•
••
FileA
FileC
NewFile
Indirect
Block
Unit 13 34
Ordering Operations (Good)
Free Inodes
Inode Map
Free Blocks
…
Superblock
Inode Inode Inode Inode
Inode Map
Data
Block Data
Block
Data
Block
Data
Block
Data
Block
Data
Block
Some Directory
•
••
FileA
FileC
NewFile
Indirect
Block
Unit 13 35
Actions to Read File Data
To open a file and read data the system must:Read the inode mapRead the inode for the fileRead 0 – 3 indirect blocksRead the data block
With this approach 3 – 6 reads are needed
Unit 13 36
Actions to Write File Data
To open a file and write data the system must:Read the inode mapRead the inode for the fileRead 0 – 3 indirect blocks
Allocate and insert 0 – 3 indirect blocks Read and or allocate the data block, change it,
write it back
Unit 13 37
The Case for Caches
Assume IDE disk with 16KB blocks Transfer time for 16KB ≈16x10-5ms Average access time 10ms (really about 12.5ms)
Time to read 16KB of data 10ms * 3(6) = 30(60)ms
Effective transfer rate 16KB/30(60)ms = 533 KB/s (267KB/s)
Manufacturer claimed transfer rate 150MB/s (SATA)
Unit 13 38
Speeding things up
Obvious problem for all the disk accesses Idea
Save (i.e. cache) copies of important data to avoid going to disk
Could cache:– Inode map– Inodes– Even individual disk blocks (indirect blocks would be an
especially good candidate)
Unit 13 39
Where are caches used?
Adapted from: Computer Systems: A Programmer’s Perspective
Registers
On-chip L1cache (SRAM)
Main memory(DRAM)
Local secondary storage(local disks)
Larger, slower,
and cheaper (per byte)storagedevices
Remote secondary storage(distributed file systems, Web servers)
Local disks hold files retrieved from disks on remote network servers.
Main memory holds disk blocks retrieved from local disks.
Off-chip L2cache (SRAM)
L1 cache holds cache lines retrieved from the L2 cache.
CPU registers hold words retrieved from cache memory.
L2 cache holds cache lines retrieved from memory.
L0:
L1:
L2:
L3:
L4:
L5:
Smaller,faster,and
costlier(per byte)storage devices
Unit 13 40
The Cost of Caching
It requires storage, that storage could be used for something else
If changes are made to the cached copy then the copy and entity being cached are inconsistent
Keeping things consistent can be a complex task especially if there are multiple cached copies
Unit 13 41
File System - Summary
Components of a Unix file system: superblock, inodes, directories, indirect blocks, data blocks
The inode does not contain the file’s name A file can be associated with several different names When the inode’s reference count goes to 0 the
resources used by the file can be reclaimed All the interesting information about a file (e.g. owner,
permissions etc.) is contained in the inode To read/write a file block, need to access multiple blocks Caching can speedup access, but it is expensive to
maintain consistency of cached data