Unit-I Foundation of Unix Operating Systems. Contents: Kernel OS Booting Process Buffer Management...
-
Upload
giles-oliver -
Category
Documents
-
view
222 -
download
0
Transcript of Unit-I Foundation of Unix Operating Systems. Contents: Kernel OS Booting Process Buffer Management...
Unit-I
Foundation of Unix Operating Systems
Contents:• Kernel• OS Booting Process• Buffer Management in UNIX• Buffer Cache• Internal File Representation• File Management• System Calls for file system• Access Methods• Free Space Management• Disk Management
UNIX System Overview :
System Hardware
The Kernel
Shell
User
Kernel:
APPLICATION
KERNEL
HARDWARE
• Core and most important part of the OS
• Manages the i/p ,o/p from software and translates them into data processing instructions for CPU and other components of computer.
• Monolithic Kernel
• Micro Kernel
• ExoKernel
Kernel Architecture (UNIX) :
Library
hardware
File Subsystem
character block
Hardware control
Buffer Cache
system call interface
Device driver
Inter process communication
Scheduler
Memory Management
Process Control Subsystem
User programUser level
kernel level
User level
kernel level
Kernel Functions :
Monolithic Kernel :
User Mode
Kernel Mode
Micro Kernel :
User Mode
Kernel Mode
Hybrid Kernel :
User Mode
Kernel Mode
ExoKernel :
GRUB-I
• GRand Unified Bootloader
• Boot loader from GNU project
• Select one of installed OS i.e. kernel configuration
• Loads on startup and allows boot time changes
• Allows network booting, highly portable(FAT/NTFS)
• Can be loaded from external devices.
• GRUB-I or Legacy GRUB
• Cent OS versions below Ubuntu 9.10
Working of GRUB :
• On booting BIOS transfers controls to boot device like CD, hard
disk etc.
• First sector is MBR 512 bytes:-
• 446 bytes for primary bootloader
• 64 bytes for partition information
• 2 boot signature
• MBR loads boot sector to active partition.
• GRUB replaces MBR with its own code.
Working of GRUB :
• STAGE 1: Located in MBR points to stage 2 memory area as
MBR can’t contain all the code.
• STAGE 2: Points to configuration file. Can be located anywhere
with sufficient memory space.
• STAGE 1.5: Used only if the boot information is small enough
to fit in the area immediately after the MBR.
GRUB-II• Latest Version of GNU GRUB.• Replaced GRUB since Ubuntu 9.10 and Fedora 14.• Complete newly written.• Provides flexibility and performance enhancement.• Rescue mode / recovery mode• Custom menus• Themes• Graphical boot menu support• Boot LiveCD ISO images directly from hard drive• New configuration file structure
Kernel Architecture (UNIX) :
Library
hardware
File Subsystem
character block
Hardware control
Buffer Cache
system call interface
Device driver
Inter process communication
Scheduler
Memory Management
Process Control Subsystem
User programUser level
kernel level
User level
kernel level
Buffer Management :• When a process wants to access data from a file, the kernel
brings the data into main memory, alters it and then request to save in the file system.
• To decrease the response time and increase throughput, the kernel minimizes the frequency of disk access by keeping a pool of internal data buffer called buffer cache.
• Buffer cache contains the data in recently used disk blocks.• When reading data from disk, the kernel attempts to read
from buffer cache.• If data is already in the buffer cache, the kernel does not need
to read from disk.• If data is not in the buffer cache, the kernel reads the data
from disk and cache it.
Buffer Cache :• A buffer consists of two parts
• a memory array – contains data from the disk• buffer header – Identifies the buffer
• Buffer is in memory copy of disk block• disk block : buffer = 1 : 1
Buffer Header
device num
block num
status
ptr to next buf on hash queue
ptr to previous buf on hash queue
ptr to next buf on free list
ptr to previous buf on free list
ptr to data area
Buffer Header Status Bits :
• #define BH_Uptodate /*1 if the buffer contains valid data*/
• #define BH_Dirty /*1 if the buffer is dirty*/
• #define BH_Lock /*1 if the buffer is locked*/
• #define BH_Req /*0 if the buffer has been invalidated*/
• #define BH_Protected /*1 if the buffer is protected*/
Structures of the buffer pool :• Buffer pool according to LRU• The kernel maintains a free list of buffer
• doubly linked circular list of buffer• take a buffer from the head of the free list.• When returning a buffer, attaches the buffer to the tail.
free list head
buf 1 buf 2 buf n
Forward ptrs
Back ptrs
Free list of Buffers
Structures of the buffer pool :• When the kernel accesses a disk block
• Search as (device num,block num) rather than entire buffer pool• separate queue (doubly linked circular list)• hashed as a function of the device and block num• Every disk block exists on one and only on hash queue and only
once on the queue
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
Buffers on the Hash Queues
Scenarios for retrieval of a buffer :• Kernel determine the logical device no. and block no.• The algorithms for reading and writing disk blocks use the algorithm getblk• The kernel finds the block on its hash queue
• The buffer is free.• The buffer is currently busy.
• The kernel cannot find the block on the hash queue• The kernel allocates a buffer from the free list.• In attempting to allocate a buffer from the free list, finds a
buffer on the free list that has been marked “delayed write”.• The free list of buffers is empty.
Retrieval of a Buffer:1st Scenario (a)• The kernel finds the block on the hash queue and its buffer is free
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 4
24
Retrieval of a Buffer:1st Scenario (b)
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
freelist header
Remove block 4 from free list
25
Retrieval of a Buffer: 2nd Scenario (a)• The kernel cannot find the block on the hash queue, so it allocates a buffer
from free list
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 18: Not in cache
26
Retrieval of a Buffer: 2nd Scenario (b)
Hash queue headers
18
4
517
105098
9935
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
freelist header
Remove 1st block from free list: Assign to 18
27
Retrieval of a Buffer: 3rd Scenario (a)• The kernel cannot find the block on the hash queue, and finds delayed
write buffers on hash queue
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 18, Delayed write blocks on free list
delay
delay
28
Retrieval of a Buffer: 3rd Scenario (b)
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
18
writing
writing
(b) Writing Blocks 3, 5, Reassign 4 to 18Figure 3.8
29
Retrieval of a Buffer: 4th Scenario• The kernel cannot find the buffer on the hash queue, and the free list is
empty
28blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
4 64
5 97 17
10 50 98
99 35 3
Search for block 18, free list empty
30
Retrieval of a Buffer: 5th Scenario• Kernel finds the buffer on hash queue, but it is currently busy
4
517
105098
9935 3
28 64
97
blkno0 mod 4
blkno1 mod 4
blkno2 mod 4
blkno3 mod 4
Hash queue headers
freelist header
Search for block 99, block busy
busy
31
Algorithm: GetBlock• GetBlock (file_system_no,block_no)
• while (buffer not found)• if (buffer in hash queue)
• if (buffer busy) { //5sleep (event buffer becomes free)Continue}
• mark buffer busy //1• remove buffer from free list• return buffer
• else• if (there is no buffer on free list){ //4
sleep (event any buffer becomes free)Continue}
• remove buffer from free list• if (buffer marked as delayed write){ //3
asynchronous white buffer to diskContinue}
• remove buffer from hash queue //2• put buffer onto hash queue• return buffer
32
Algorithm: ReleaseBlock• ReleaseBlock (locked buffer)
• wakeup all process event, waiting for any buffer to become free• wakeup all process event, waiting for this buffer to become free• raise processor execution level to block interrupt• if (buffer content valid and buffer not old)
• enqueue buffer at the end of free list• else
• enqueue buffer at the beginning of free list• lower processor execution level to allow interrupt• unlock (buffer)
33
Reading and Writing disk blocks• To read block ahead
• The kernel checks if the block is in the cache or not.• If the block in not in the cache, it invokes the disk driver to read the
block.• The the process goes to sleep awaiting the event that the I/O is
complete.• The disk controller interrupts the processor when the I/O is complete• The disk interrupt handler awakens the sleeping processes• The content of disk blocks are now in the buffer• When the process no longer need the buffer, it releases it so that other
processes can access it
34
Reading and Writing disk blocks
• To write a disk block• The kernel informs the disk driver that it has a buffer whose contents
should be output.• The disk driver schedules the block for I/O.• If the write is synchronous, the calling process goes the sleep awaiting
I/O completion and releases the buffer when awakens.• If the write is asynchronous, the kernel starts the disk write. The kernel
release the buffer when the I/O completes.
Internal Representation Of File:
Lower Level File system Algorithms
namei
alloc free ialloc ifree
iget iput bmap
buffer allocation algorithms
getblk brelse bread breada bwrite
Fig. File System Algorithms
File System Algorithms :
• The algorithm iget returns a previously identified inode.
• Algorithm iput release the inode.
• The algorithm bmap sets kernel parameters for accessing a file.
• The algorithm namei converts a user level path name to an inode
using algorithms iget,iput,and bmap.
• Algorithms alloc and free allocate and free disk blocks for files
• Algorithms ialloc and ifree assigns and free inodes for files.
Inode :• contains the information necessary for a process to access a
file• exits in a static form on disk and the kernel reads them into an
in-core inode• consists of - file owner identifier - file type - file access permissions - file access times - number of links to the file - table of contents for the disk address of data in a file - file size
Inodes :• in-core copy of the inode contains - status of the in-core inode - logical device number of file system - inode number - pointers to other in-core inodes - reference count
Algorithm iget :1. The kernel finds the inode in inode cache and it is on inode free list remove from free list
increment inode reference count2. The kernel cannot find the inode in inode cache so it allocates a
inode from inode free list remove new inode from free list reset inode number and file system
remove inode from old hash queue, place on new one read inode from disk(algorithm bread)
initialize inode 3. The kernel cannot find the inode in inode cache but finds the free
list empty error
4. The kernel finds the inode in inode cache but it was locked sleep until inode becomes unlocked
iget (inode_no) //getIncoreInode
• while (not done)• if (inode in inode cache)
• if (inode locked)• sleep(event inode becomes unlocked)• continue
• if (inode on inode free list)• remove from free list• return locked inode
• if (no inode on free list)…. return error• remove new inode from free list• set inode number• remove inode from old hash queue and place on new one• read inode from disk• set reference count 1• return locked inode
Algorithm iput :- The kernel locks the inode if it has not been already locked- The kernel decrements inode reference count- The kernel checks if reference count is 0 or not- If the reference count is 0 and the number of links to the file is 0, then the kernel releases disk blocks for file(algorithm free), free the inode(algorithm ifree) • If the file was accessed or the inode was changed or the file was changed , then the kernel updates the disk inode • The kernel puts the inode on free list - If the reference count is not 0, the kernel releases the inode lock
iput (inode_no) //releaseIncoreInode
• lock inode if not locked• decrement inode reference count• if (reference count==0)
• if (inode link==0)• free disk block• set file type to 0• free inode
• if (file accessed or inode changed or file changed)• update disk inode
• put inode on free list• Release inode lock
Structure of a regular file :
direct0
direct1
direct2
direct3
direct4
direct5
direct6
direct7
direct8
direct9
single indirect
double indirect
triple indirect
Inode Data Blocks
Fig. Direct and Indirect Blocks in Inode
Structure of a regular file :
Assuming System V UNIX : Assume that a logical on the file system holds 1K bytes and that a block
number is addressable by a 32 bit integer, then a block can hold up to 256 block numbers
10 direct blocks with 1K bytes each=10K bytes 1 indirect block with 256 direct blocks= 1K*256=256K bytes
1 double indirect block with 256 indirect blocks=256K*256=64M bytes
1 triple indirect block with 256 double indirect blocks=64M*256=16G bytes
Directories :
Fig. A UNIX directory tree
Directories :• A directory is a file whose data is a sequence of entries, each
consisting of an inode number and the name of a file contained in the directory.
• Path name is a null terminated character string divided by slash (“/”)• Directory layout for /etc
Byte Offset in Directory Inode Number File Name
0 83 .
16 2 ..
32 1798 init
48 1276 fsck
. . Mknod
. . ……
224 0 crash
Path conversion to an inode :
• if (path name starts with root)• working inode= root inode
• else• working inode= current directory inode
• while (there is more path name)• read next component from input• read directory content• if (component matches an entry in directory)
• get inode number for matched component• release working inode• working inode=inode of matched component
• else• return no inode
• return (working inode)
Super block :• File System
• consists of - the size of the file system - the number of free blocks in the file system - a list of free blocks available on the file system - the index of the next free block in the free block list - the size of the inode list - the number of free inodes in the file system - a list of free inodes in the file system - the index of the next free inode in the free inode list - lock fields for the free block and free inode lists - a flag indicating that the super block has been modified
boot block super block Inode list data blocks
Inode assignment to a new file :
free inodes 83 48 empty
18 19 20 array1
Super Block Free Inode List
index
free inodes 83 empty
18 19 20 array2
Super Block Free Inode List
index
Assigning Free Inode from Middle of List
Inode assignment to a new file :
Assigning Free Inode – Super Block List Empty
470 empty
array1
Super Block Free Inode List
index
0
535 free inodes 476 475 471 array2Super Block Free Inode List
048 49 50
Inode assignment to a new file :
• algorithm ialloc : assigns a disk inode to a newly created file
-super block is unlocked 1.There are inodes in super block inode list and inode is free
get inode number from super block inode listget inode (iget)initialize inodewrite inode to diskdecrement file system free inode count
2. There are inodes in super block inode list but inode is not freeget inode number from super block inode listget inode (iget)write inode to diskrelease inode (iput)
Inode assignment to a new file : 3. Inode list in super block is empty
lock super block get remembered inode for free inode search
search disk for free inode until super block full orno more free inodes(bread and brelse)
unlock super blocksuper block becomes freeif no free inodes found on disk , stop otherwise, set remembered inode for next free inode
search - If super block is locked, sleep
Inode assignment to a new file :• Locked inode illoc()
• while (not done)• If (super block locked)
• Sleep (event super block becomes free)• Continue
• If (inode list in super block empty)• Lock super block• Get remember inode for free inode search• Search until super block full or no more free inode• Unlock super block and wake up (event super block free)\• If no free inode found on disk return error• Set remebered inode for next free inode search
• Get inode number from super block inode list• Get inode• Write inode to disk• Decrement free inode count• Return inode
Freeing inode :
• ifree(inode_no)• Increment free inode count• If super block locked return• If (inode list full) //at super block
• if (inode number <remembered inode)• Set remembered inode as input inode
• Else• Store inode number in inode list
• return
Freeing inode :
535 476 475 471
free inodesremembered inode
Original Super Block List of Free Inodes
index
Free Inode 499
499 476 475 471
free inodesremembered inode index
Free Inode 601
499 476 475 471
free inodesremembered inode index
Allocation of disk blocks :
109 106 103 100 …………………………..
211 208 205 202 ………………… 112
109
310 307 304 301 ………………… 214211
409 406 403 400 ………………… 313310
linked list of free disk block number
Allocation of disk blocks :
109 …………………………………………………………
211 208 205 202 …………………………….. 112
109 949 …………………………………………………..
211 208 205 202 ………………………………. 112
super block list
original configuration
109
109
After freeing block number 949
Allocation of disk blocks :
211 208 205 202 ……………………………… 112
344 341 338 335 ………………………………. 243
After assigning block number(109)
replenish super block free list
211
109 ………………………………………………………..
211 208 205 202 ………………………………. 112
109
After assigning block number(949)
System Calls :
• System calls: The mechanism used by an application program to request service from the operating system.
• This allows the OS to perform restricted actions such as accessing hardware devices or the memory management unit.
read (fd, buffer, nbytes) :
Some System Calls :
File Management :
Fig. A Typical File Control Block
File Allocation Methods :• An allocation method refers to how disk blocks are allocated
for files:
• Contiguous allocation
• Linked allocation
• Indexed allocation
Contiguous allocation :
Linked allocation :
Indexed allocation :
Access Methods :• Sequential Access
read nextwrite next resetno read after last write
(rewrite)• Direct Access
read nwrite nposition to n
read nextwrite next
rewrite nn = relative block number
Free Space Management :• Bit vector (n blocks)
…
0 1 2 n-1
bit[i] = 1 block[i] free
0 block[i] occupied
• Relatively simple
Free Space Management :• Linked list
• Hard to find contiguous space easily• But no waste of space
• Grouping • Store addresses of n free blocks in the first block• Last of these addresses is to a block that contains addresses
of another n free blocks• So many free blocks can be found at one time
• Counting• Clusters of contiguous free blocks recorded together• Keep list of first block address, count of contiguous free
ones
Disk Management :• Low-level formatting, or physical formatting — Dividing a disk into sectors
that the disk controller can read and write• Each sector can hold header information, plus data, plus error correction code
(ECC)• Usually 512 bytes of data but can be selectable
• To use a disk to hold files, the operating system still needs to record its own data structures on the disk• Partition• Logical formatting • To increase efficiency most file systems group blocks into clusters
• Disk I/O done in blocks• File I/O done in clusters
• Boot block• bootstrap stored in ROM
• Bad Blocks• Sector Sparing• Sector Slipping
Swap Space :• Swap-space — Virtual memory uses disk space as an extension of main
memory.• Swap-space Use :
• Hold the process image• Swap-space Location :
• Using normal file system• Separate Raw partition
Fig. Data Structures for Swapping on Linux Systems