Operating Systems 3 File Systems Dr John Cowell phones off (please) 1CSCI1412-OS-3.

Operating Systems 3 File Systems Dr John Cowell phones off (please) 1CSCI1412-OS-3

Overview Low level file systems partitions, sectors, clusters formatting FATs what is a FAT and how a file is stored fragmentation and defragmentation how a file is located High level file systems drives, directories, files file attributes and permissions sharing information CSCI1412-OS-32

Low-level File Management Physically, a hard disk consists of platters, surfaces, cylinders, heads and sectors Logically, a hard disk consists of partitions or drives ( C:, D:, E:, etc.), directories and files It is the operating system that carries out the translation between the underlying physical organisation and representation of data and the logical layer visible to the user different partitions (drives) may even have different translations that are entirely transparent to the user CSCI1412-OS-34

File System Layers In order to provide a convenient and efficient translation, an operating system generally implements a number of intermediate layers to insulate logical structure from device dependencies CSCI1412-OS-35 application programs logical file system files, directories, etc. BIOS Basic Input Output System IDE controller integrated drive electronics (hard disk drive) controller hard disk

Partitions A partition is a collection of adjacent cylinders on the hard drive that are grouped together each partition can have its own operating system and so its own logical file system the partitions are defined (position, size) in a table there are only four entries allowed in the partition table MS-DOS/Win uses a logical structure referred to as FAT this is specific to MS-DOS/Win and is not generic used for floppy drives and flash memory other operating systems have their own file systems XP, Vista: NTFS; Linux: ext2 these arent compatible: partition may be unreadable CSCI1412-OS-36

Formatting An analogy imagine a large area of tarmac it would be chaos if drivers tried to park anywhere! now imagine that spaces are painted with white lines the area of tarmac has now become a car park Formatting is the process of laying down the file system structures on the partition each partition needs to be formatted separately two file allocation tables are written one root directory entry is written all sectors are tested to see if they are bad & marked CSCI1412-OS-37

Surface Organisation The very first sector on a hard drive is the primary boot sector which also contains the partition table CSCI1412-OS-38

Sectors and Clusters Remember that the surface of the disk is divided into blocks of bytes called sectors on current PCs 1 sector = 512 bytes Since hard disks have become so much larger there are so many sectors that is inefficient to deal with sectors individually sectors are grouped into blocks, known as clusters A file is stored in a whole number of clusters any space left over in the final cluster is wasted sometimes called slack space CSCI1412-OS-39

Directory Entries After the two FATs on a partition, the operating system puts the root directory (\) a directory consists of a number of directory entries Each directory entry consists of the file name, extension, other information and the cluster number of the initial data stored in the file CSCI1412-OS-310 The directory entry holds the cluster number of the first cluster in which the file is stored file name 8 ext 31 attribute flags reserved 10 time 2 date 2 start 2 size 4

File Allocation Tables The file allocation table (FAT) is a database that holds the status of every cluster on the partition because it is so important two copies of FAT are stored if it is corrupted all data on the partition can be lost! The directory entry holds the start cluster of a file the FAT entry indexed by that cluster number contains the number of the next cluster storing the files data there is a special value to signify end of file ( FFF8 16 ) Special values signify free blocks and bad blocks free blocks are signified by 0000 16 bad blocks are signified by FFF7 16 CSCI1412-OS-311

FAT I llustration Suppose the clusters are 4K in size and we want to store a file, \FRED.DOC , which is 6 Kb long CSCI1412-OS-312 0123 4567 891011 12131415 16171819 20212223 24252627 28293031 root directory namestart command.com2 autoexec.bat3 config.sys4 fred.doc8 bill.doc9 jack.doc10 0xFFFD 0xFFF8 0 0 0 0 0 0xFFF7 0 0 0 0 1 2 3 4 5 6 7 8 9 10 FAT 0 0 0 0 0 11 12 13 14 15 0 0 0 0 0 0 0 0 0 0 0 16 17 18 19 20 21 22 23 24 25 26 0 0 0 0 0 27 28 29 30 31 0xFFF8 12 0xFFF8 16 1724250xFFF8 26 2714 13 18 22210xFFF8

Cluster Contents More details on the cluster contents CSCI1412-OS-313 0xFFFD 0xFFF8 0 0 0 0 0 0xFFF7 0 0 0 0 1 2 3 4 5 6 7 8 9 10 FAT 0 0 0 0 0 11 12 13 14 15 0 0 0 0 0 0 0 0 0 0 0 16 17 18 19 20 21 22 23 24 25 26 0 0 0 0 0 27 28 29 30 31 0xFFF8 12 0xFFF8 16 17 24 25 0xFFF8 2627 14 13 18 22 21 0xFFF8 root directory namestart command.com2 autoexec.bat3 config.sys4 fred.doc8 bill.doc9 jack.doc10 cluster contents FAT-1FAT-2 root directory (\) unused FRED.DOC (0..4095) free bad BILL.DOC (0..4095) JACK.DOC (0..4095) FRED.DOC (4095..6143) free 0 1 2 3 4 5 6 7 8 9 10 11 12

Windows 9X File System Windows 9X uses a minor variation of the MS-DOS FAT file system structure, called VFAT (which developed into FAT32) it is still a linked-list allocation using a FAT index Directory entries have been expanded to allow file names of up to 256 characters The FAT table can be selected as 16 bit or 32 bit 16-bit FAT gives 65536 entries the file system is compatible with MS-DOS 64K entries 32K clusters 2Gb maximum partition size 32-bit FAT gives 4 10 9 entries extremely large maximum partition size! CSCI1412-OS-314

NTFS The (V)FAT structure has serious shortcomings large disk sizes are only possible using large clusters the FAT offers poor performance and is non-robust (key file data such as time, date, size are kept in the directory entry) there is no file access protection mechanism Windows XP and Vista use an entirely different system full access protection security large capacity allocation scheme internationalisation using long Unicode file names robust fault-tolerance with transaction logging CSCI1412-OS-315

NTFS NT File System was developed for Windows NT Implemented as the file system used by Windows 2000 and XP Uses 64-bit addresses theoretically supporting disk partitions of up to 2 64 bytes capacity Supports 255 character case sensitive file names in Unicode but the case sensitivity is not supported by the Win32 API! Paths are limited to 32767 bytes CSCI1412-OS-316

File System Structure NTFS partition is organised as linear sequence of blocks block size can be 512 bytes to 64 Kb actual block size is a compromise between large blocks (more efficient data transfer) and small blocks (less file fragmentation) Most important part of the file structure is the Master File Table (MFT) each row (record) is 1 Kb long Each record contains description of a single file or directory, including the files attributes the addresses of the data blocks actually containing the files data Data block address list may require more than one record in the MFT if the file is especially large in that case, the base record contains the addresses of the overflow MFT records Size of the MFT is also flexible may grow to a maximum of 248 records CSCI1412-OS-317

Master File Table Any file system needs to store information about itself, such as blocks used, volume name, etc This metadata is stored in a series of files file names start with a $ NTFS uses first 16 records for this The address of the MFT itself is stored in the boot block of the disk partition when the partition is created allows the MFT to be placed anywhere on the disk to avoid bad blocks 16First user file 15(Reserved for future use) 14(Reserved for future use) 13(Reserved for future use) 12(Reserved for future use) 11$ExtendExtensions: quotas, etc 10$UpcaseCase conversion table 9$SecureSecurity descriptors for all files 8$BadClusList of bad blocks 7$BootBootstrap loader 6$BitmapBitmap of blocks used 5$Root directory 4$AttrDefAttribute definitions 3$VolumeVolume file 2$LogFileLog file for recovery 1$MftMirrMirror copy of MFT 0$MftMaster File Table CSCI1412-OS-318

MFT Records Each MFT record consists of a series of paired values Each pair of values comprises attribute header, value several different types of these such as file name, attribute list, and data Value(s) associated with the data attribute contain either addresses of data blocks, or for a small file, the data itself an immediate file CSCI1412-OS-319

How NTFS Works The OS will attempt always to place data into contiguous blocks but this is not always possible The diagram details a file of nine blocks in size This example shows the data blocks grouped into three runs of blocks This is a short file, ie the data blocks will not fit into the MFT record, but all the information about the file will The header shows that the offset of the first block is 0 ie, how far from the start of the file is the first data block The second header value gives the number of the first block that is not part of the file effectively, the length of the file in blocks After the header come the pairs of values pointing directly to the used blocks first value gives the block number second value gives the number of contiguous blocks CSCI1412-OS-320

I ntegrity Checking If the computer crashes or is switched off in the middle of updating the file system(s) it can be left in a state of confusion or corruption the two FATs may not contain the same information the file clusters may not reflect the FAT list the initial directory entry may not point to the file Some operating systems (e.g. UNIX ) automatically check the file system integrity on start-up MS-DOS/Win requires you to run SCANDISK NTFS uses journaling logs file system updates that are about to happen checks afterwards to see that they did happen undoes any partially completed updates CSCI1412-OS-321

Fragmentation Initially all the free space is in one big block clusters are allocated in sequence to files as needed CSCI1412-OS-322 FRED.DOCBILL.DOCJACK.DOC FREE Suppose BILL.DOC is deleted and the clusters freed FRED.DOCJACK.DOC FREE FRED.DOCJOHN.DOCJACK.DOC Then a longer file, JOHN.DOC, is stored on disk JOHN.DOC the file JOHN.DOC is said to be fragmented after lots of deletions / additions a disk can be very fragmented a badly fragmented disk will be much slower to read and write

Defragmentation Defragmentation is the name given to the process of rearranging the clusters so that all files are held in adjacent clusters and all free space is at the end CSCI1412-OS-323 0123 4567 891011 12131415 16171819 20212223 24252627 28293031 0123 4567 891011 12131415 16171819 20212223 24252627 28293031 the whole disk is read and the clusters are moved a defragmented disk will improve performance this process can be very time consuming a defragmenting tool is included in later versions of Windows

Finding Files When the system request a file from disk directory is read to see if file exists directory entry has address of first cluster of file disk drive heads are moved to correct track OS waits for correct sector to arrive under disk head sector is read & data transferred to disk cache buffer when buffer full, contents transferred to RAM next sector is read, until all sectors in cluster have been accessed FAT re-accessed to find next cluster process repeated until end-of-file is found in FAT CSCI1412-OS-324

25CSCI1412-OS-3

High Level File Systems In addition to dealing with low-level issues such as partition tables, FATs, clusters and sectors the operating system deals with higher-level issues of file management directory structures file attributes security Most operating systems also have some concept of file types or special files directories are identified by attribute in directory entry other file types in Windows are identified by extension e.g. EXE, COM, BAT, DOC, XLS, TTF, DLL CSCI1412-OS-326

Directories and Folders Directories, or folders as they are also known, are used to organise collections of files in a typical modern computer, even single-user PC, there can be 100,000s of files if they were kept in a single, flat, file structure it would be extremely time consuming to find any particular file using directories users can group together files applications, system files, user files documents, spreadsheet, presentations A directory is a special file type in which the OS stores the names of other files or other directories directories within directories are sub-directories CSCI1412-OS-327

Typical Single User File System Note: the separation of user files applications operating system Allows backups of work easy installation, reinstallation and removal of applications operating system upgrades CSCI1412-OS-328

Drives In Windows systems different physical devices and file systems are assigned different drive labels A: Floppy disk C: First hard disk (partition) D: Second hard disk (partition) E: CD-ROM this organisation is visible to the user and has big drawbacks if the physical organisation needs alteration In Unix there is one logical file system structure which may consist of many physical devices the devices may even be distributed across a network CSCI1412-OS-329

File Attributes The operating system keeps track of certain file properties or attributes date / time of creation & last modification in Windows R : read-only, H : hidden, S : system A : archive (has the file changed since last backup) in Unix file owner, group; file permissions In MS-DOS/Win the file extension also has meaning to the operating system EXE : executable, BAT : batch command script CSCI1412-OS-330

File Permissions In both Windows XP/Vista and UNIX the concept of MS- DOS/Win file attributes has been extended to a more comprehensive implementation of file permissions the directory entry for each file stores the user name of its owner (the creator) and their default group there are three sets of file permissions flags user (owner)read ( r ), write ( w ) and execute ( x ) permissions groupread ( r ), write ( w ) and execute ( x ) permissions otherread ( r ), write ( w ) and execute ( x ) permissions CSCI1412-OS-331

Personal & Shared I nformation - 1 Using these file permissions it is possible to restrict other users access to your files the user flags apply to the owner of the file e.g. if there is no user write permission, not even the owner can write to the file (i.e. the file is read-only) the group flags apply to other users who belong to the same group as the group of the file e.g. other members of group staff may be granted permission to read documents belonging to jcowell (staff) jcowell could write to a file owned by group cci with group write permissions (as jcowell is also a member of group cci) CSCI1412-OS-332

Personal & Shared I nformation - 2 Using these file permissions it is possible to restrict other users access to your files, cont. the other flags apply to all other users who are not even in the same group if others have read & write access, then any other user can read or write to the files In UNIX the ( x ) flags indicate either an ordinary file is executable (like.EXE in MS-DOS) a directory is searchable by the appropriate users e.g. if jcowell makes a directory rwx,r-x,--- then group staff members can read it and enter into it but other users cannot CSCI1412-OS-333

Summary Low level file systems partitions, sectors, clusters formatting FATs what is a FAT and how a file is stored fragmentation and defragmentation how a file is located High level file systems drives, directories, files file attributes and permissions sharing information CSCI1412-OS-334

Operating Systems 3 File Systems Dr John Cowell phones off (please) 1CSCI1412-OS-3.

Documents

Transcript of Operating Systems 3 File Systems Dr John Cowell phones off (please) 1CSCI1412-OS-3.