Transcript of Operating Systems 3 File Systems Dr John Cowell phones off (please) 1CSCI1412-OS-3.
- Slide 1
- Operating Systems 3 File Systems Dr John Cowell phones off
(please) 1CSCI1412-OS-3
- Slide 2
- Overview Low level file systems partitions, sectors, clusters
formatting FATs what is a FAT and how a file is stored
fragmentation and defragmentation how a file is located High level
file systems drives, directories, files file attributes and
permissions sharing information CSCI1412-OS-32
- Slide 3
- 3
- Slide 4
- Low-level File Management Physically, a hard disk consists of
platters, surfaces, cylinders, heads and sectors Logically, a hard
disk consists of partitions or drives ( C:, D:, E:, etc.),
directories and files It is the operating system that carries out
the translation between the underlying physical organisation and
representation of data and the logical layer visible to the user
different partitions (drives) may even have different translations
that are entirely transparent to the user CSCI1412-OS-34
- Slide 5
- File System Layers In order to provide a convenient and
efficient translation, an operating system generally implements a
number of intermediate layers to insulate logical structure from
device dependencies CSCI1412-OS-35 application programs logical
file system files, directories, etc. BIOS Basic Input Output System
IDE controller integrated drive electronics (hard disk drive)
controller hard disk
- Slide 6
- Partitions A partition is a collection of adjacent cylinders on
the hard drive that are grouped together each partition can have
its own operating system and so its own logical file system the
partitions are defined (position, size) in a table there are only
four entries allowed in the partition table MS-DOS/Win uses a
logical structure referred to as FAT this is specific to MS-DOS/Win
and is not generic used for floppy drives and flash memory other
operating systems have their own file systems XP, Vista: NTFS;
Linux: ext2 these arent compatible: partition may be unreadable
CSCI1412-OS-36
- Slide 7
- Formatting An analogy imagine a large area of tarmac it would
be chaos if drivers tried to park anywhere! now imagine that spaces
are painted with white lines the area of tarmac has now become a
car park Formatting is the process of laying down the file system
structures on the partition each partition needs to be formatted
separately two file allocation tables are written one root
directory entry is written all sectors are tested to see if they
are bad & marked CSCI1412-OS-37
- Slide 8
- Surface Organisation The very first sector on a hard drive is
the primary boot sector which also contains the partition table
CSCI1412-OS-38
- Slide 9
- Sectors and Clusters Remember that the surface of the disk is
divided into blocks of bytes called sectors on current PCs 1 sector
= 512 bytes Since hard disks have become so much larger there are
so many sectors that is inefficient to deal with sectors
individually sectors are grouped into blocks, known as clusters A
file is stored in a whole number of clusters any space left over in
the final cluster is wasted sometimes called slack space
CSCI1412-OS-39
- Slide 10
- Directory Entries After the two FATs on a partition, the
operating system puts the root directory (\) a directory consists
of a number of directory entries Each directory entry consists of
the file name, extension, other information and the cluster number
of the initial data stored in the file CSCI1412-OS-310 The
directory entry holds the cluster number of the first cluster in
which the file is stored file name 8 ext 31 attribute flags
reserved 10 time 2 date 2 start 2 size 4
- Slide 11
- File Allocation Tables The file allocation table (FAT) is a
database that holds the status of every cluster on the partition
because it is so important two copies of FAT are stored if it is
corrupted all data on the partition can be lost! The directory
entry holds the start cluster of a file the FAT entry indexed by
that cluster number contains the number of the next cluster storing
the files data there is a special value to signify end of file (
FFF8 16 ) Special values signify free blocks and bad blocks free
blocks are signified by 0000 16 bad blocks are signified by FFF7 16
CSCI1412-OS-311
- Slide 12
- FAT I llustration Suppose the clusters are 4K in size and we
want to store a file, \FRED.DOC , which is 6 Kb long
CSCI1412-OS-312 0123 4567 891011 12131415 16171819 20212223
24252627 28293031 root directory namestart command.com2
autoexec.bat3 config.sys4 fred.doc8 bill.doc9 jack.doc10 0xFFFD
0xFFF8 0 0 0 0 0 0xFFF7 0 0 0 0 1 2 3 4 5 6 7 8 9 10 FAT 0 0 0 0 0
11 12 13 14 15 0 0 0 0 0 0 0 0 0 0 0 16 17 18 19 20 21 22 23 24 25
26 0 0 0 0 0 27 28 29 30 31 0xFFF8 12 0xFFF8 16 1724250xFFF8 26
2714 13 18 22210xFFF8
- Slide 13
- Cluster Contents More details on the cluster contents
CSCI1412-OS-313 0xFFFD 0xFFF8 0 0 0 0 0 0xFFF7 0 0 0 0 1 2 3 4 5 6
7 8 9 10 FAT 0 0 0 0 0 11 12 13 14 15 0 0 0 0 0 0 0 0 0 0 0 16 17
18 19 20 21 22 23 24 25 26 0 0 0 0 0 27 28 29 30 31 0xFFF8 12
0xFFF8 16 17 24 25 0xFFF8 2627 14 13 18 22 21 0xFFF8 root directory
namestart command.com2 autoexec.bat3 config.sys4 fred.doc8
bill.doc9 jack.doc10 cluster contents FAT-1FAT-2 root directory (\)
unused FRED.DOC (0..4095) free bad BILL.DOC (0..4095) JACK.DOC
(0..4095) FRED.DOC (4095..6143) free 0 1 2 3 4 5 6 7 8 9 10 11
12
- Slide 14
- Windows 9X File System Windows 9X uses a minor variation of the
MS-DOS FAT file system structure, called VFAT (which developed into
FAT32) it is still a linked-list allocation using a FAT index
Directory entries have been expanded to allow file names of up to
256 characters The FAT table can be selected as 16 bit or 32 bit
16-bit FAT gives 65536 entries the file system is compatible with
MS-DOS 64K entries 32K clusters 2Gb maximum partition size 32-bit
FAT gives 4 10 9 entries extremely large maximum partition size!
CSCI1412-OS-314
- Slide 15
- NTFS The (V)FAT structure has serious shortcomings large disk
sizes are only possible using large clusters the FAT offers poor
performance and is non-robust (key file data such as time, date,
size are kept in the directory entry) there is no file access
protection mechanism Windows XP and Vista use an entirely different
system full access protection security large capacity allocation
scheme internationalisation using long Unicode file names robust
fault-tolerance with transaction logging CSCI1412-OS-315
- Slide 16
- NTFS NT File System was developed for Windows NT Implemented as
the file system used by Windows 2000 and XP Uses 64-bit addresses
theoretically supporting disk partitions of up to 2 64 bytes
capacity Supports 255 character case sensitive file names in
Unicode but the case sensitivity is not supported by the Win32 API!
Paths are limited to 32767 bytes CSCI1412-OS-316
- Slide 17
- File System Structure NTFS partition is organised as linear
sequence of blocks block size can be 512 bytes to 64 Kb actual
block size is a compromise between large blocks (more efficient
data transfer) and small blocks (less file fragmentation) Most
important part of the file structure is the Master File Table (MFT)
each row (record) is 1 Kb long Each record contains description of
a single file or directory, including the files attributes the
addresses of the data blocks actually containing the files data
Data block address list may require more than one record in the MFT
if the file is especially large in that case, the base record
contains the addresses of the overflow MFT records Size of the MFT
is also flexible may grow to a maximum of 248 records
CSCI1412-OS-317
- Slide 18
- Master File Table Any file system needs to store information
about itself, such as blocks used, volume name, etc This metadata
is stored in a series of files file names start with a $ NTFS uses
first 16 records for this The address of the MFT itself is stored
in the boot block of the disk partition when the partition is
created allows the MFT to be placed anywhere on the disk to avoid
bad blocks 16First user file 15(Reserved for future use)
14(Reserved for future use) 13(Reserved for future use) 12(Reserved
for future use) 11$ExtendExtensions: quotas, etc 10$UpcaseCase
conversion table 9$SecureSecurity descriptors for all files
8$BadClusList of bad blocks 7$BootBootstrap loader 6$BitmapBitmap
of blocks used 5$Root directory 4$AttrDefAttribute definitions
3$VolumeVolume file 2$LogFileLog file for recovery 1$MftMirrMirror
copy of MFT 0$MftMaster File Table CSCI1412-OS-318
- Slide 19
- MFT Records Each MFT record consists of a series of paired
values Each pair of values comprises attribute header, value
several different types of these such as file name, attribute list,
and data Value(s) associated with the data attribute contain either
addresses of data blocks, or for a small file, the data itself an
immediate file CSCI1412-OS-319
- Slide 20
- How NTFS Works The OS will attempt always to place data into
contiguous blocks but this is not always possible The diagram
details a file of nine blocks in size This example shows the data
blocks grouped into three runs of blocks This is a short file, ie
the data blocks will not fit into the MFT record, but all the
information about the file will The header shows that the offset of
the first block is 0 ie, how far from the start of the file is the
first data block The second header value gives the number of the
first block that is not part of the file effectively, the length of
the file in blocks After the header come the pairs of values
pointing directly to the used blocks first value gives the block
number second value gives the number of contiguous blocks
CSCI1412-OS-320
- Slide 21
- I ntegrity Checking If the computer crashes or is switched off
in the middle of updating the file system(s) it can be left in a
state of confusion or corruption the two FATs may not contain the
same information the file clusters may not reflect the FAT list the
initial directory entry may not point to the file Some operating
systems (e.g. UNIX ) automatically check the file system integrity
on start-up MS-DOS/Win requires you to run SCANDISK NTFS uses
journaling logs file system updates that are about to happen checks
afterwards to see that they did happen undoes any partially
completed updates CSCI1412-OS-321
- Slide 22
- Fragmentation Initially all the free space is in one big block
clusters are allocated in sequence to files as needed
CSCI1412-OS-322 FRED.DOCBILL.DOCJACK.DOC FREE Suppose BILL.DOC is
deleted and the clusters freed FRED.DOCJACK.DOC FREE
FRED.DOCJOHN.DOCJACK.DOC Then a longer file, JOHN.DOC, is stored on
disk JOHN.DOC the file JOHN.DOC is said to be fragmented after lots
of deletions / additions a disk can be very fragmented a badly
fragmented disk will be much slower to read and write
- Slide 23
- Defragmentation Defragmentation is the name given to the
process of rearranging the clusters so that all files are held in
adjacent clusters and all free space is at the end CSCI1412-OS-323
0123 4567 891011 12131415 16171819 20212223 24252627 28293031 0123
4567 891011 12131415 16171819 20212223 24252627 28293031 the whole
disk is read and the clusters are moved a defragmented disk will
improve performance this process can be very time consuming a
defragmenting tool is included in later versions of Windows
- Slide 24
- Finding Files When the system request a file from disk
directory is read to see if file exists directory entry has address
of first cluster of file disk drive heads are moved to correct
track OS waits for correct sector to arrive under disk head sector
is read & data transferred to disk cache buffer when buffer
full, contents transferred to RAM next sector is read, until all
sectors in cluster have been accessed FAT re-accessed to find next
cluster process repeated until end-of-file is found in FAT
CSCI1412-OS-324
- Slide 25
- 25CSCI1412-OS-3
- Slide 26
- High Level File Systems In addition to dealing with low-level
issues such as partition tables, FATs, clusters and sectors the
operating system deals with higher-level issues of file management
directory structures file attributes security Most operating
systems also have some concept of file types or special files
directories are identified by attribute in directory entry other
file types in Windows are identified by extension e.g. EXE, COM,
BAT, DOC, XLS, TTF, DLL CSCI1412-OS-326
- Slide 27
- Directories and Folders Directories, or folders as they are
also known, are used to organise collections of files in a typical
modern computer, even single-user PC, there can be 100,000s of
files if they were kept in a single, flat, file structure it would
be extremely time consuming to find any particular file using
directories users can group together files applications, system
files, user files documents, spreadsheet, presentations A directory
is a special file type in which the OS stores the names of other
files or other directories directories within directories are
sub-directories CSCI1412-OS-327
- Slide 28
- Typical Single User File System Note: the separation of user
files applications operating system Allows backups of work easy
installation, reinstallation and removal of applications operating
system upgrades CSCI1412-OS-328
- Slide 29
- Drives In Windows systems different physical devices and file
systems are assigned different drive labels A: Floppy disk C: First
hard disk (partition) D: Second hard disk (partition) E: CD-ROM
this organisation is visible to the user and has big drawbacks if
the physical organisation needs alteration In Unix there is one
logical file system structure which may consist of many physical
devices the devices may even be distributed across a network
CSCI1412-OS-329
- Slide 30
- File Attributes The operating system keeps track of certain
file properties or attributes date / time of creation & last
modification in Windows R : read-only, H : hidden, S : system A :
archive (has the file changed since last backup) in Unix file
owner, group; file permissions In MS-DOS/Win the file extension
also has meaning to the operating system EXE : executable, BAT :
batch command script CSCI1412-OS-330
- Slide 31
- File Permissions In both Windows XP/Vista and UNIX the concept
of MS- DOS/Win file attributes has been extended to a more
comprehensive implementation of file permissions the directory
entry for each file stores the user name of its owner (the creator)
and their default group there are three sets of file permissions
flags user (owner)read ( r ), write ( w ) and execute ( x )
permissions groupread ( r ), write ( w ) and execute ( x )
permissions otherread ( r ), write ( w ) and execute ( x )
permissions CSCI1412-OS-331
- Slide 32
- Personal & Shared I nformation - 1 Using these file
permissions it is possible to restrict other users access to your
files the user flags apply to the owner of the file e.g. if there
is no user write permission, not even the owner can write to the
file (i.e. the file is read-only) the group flags apply to other
users who belong to the same group as the group of the file e.g.
other members of group staff may be granted permission to read
documents belonging to jcowell (staff) jcowell could write to a
file owned by group cci with group write permissions (as jcowell is
also a member of group cci) CSCI1412-OS-332
- Slide 33
- Personal & Shared I nformation - 2 Using these file
permissions it is possible to restrict other users access to your
files, cont. the other flags apply to all other users who are not
even in the same group if others have read & write access, then
any other user can read or write to the files In UNIX the ( x )
flags indicate either an ordinary file is executable (like.EXE in
MS-DOS) a directory is searchable by the appropriate users e.g. if
jcowell makes a directory rwx,r-x,--- then group staff members can
read it and enter into it but other users cannot
CSCI1412-OS-333
- Slide 34
- Summary Low level file systems partitions, sectors, clusters
formatting FATs what is a FAT and how a file is stored
fragmentation and defragmentation how a file is located High level
file systems drives, directories, files file attributes and
permissions sharing information CSCI1412-OS-334