Ch12 system administration

Disks and File Systems II

Chapter 12

Chapter Goals

• Understand how to create a file system.

• Understand how to make a file system available.

• Troubleshooting disks and file systems.

• Repairing file systems.

• Tuning file system performance.

Working with the newfs Command• On most UNIX systems, the default type of file system

created by newfs is a ufs file system. – Two notable exceptions are Linux, which defaults to an

EXT2 file system, and Irix, which defaults to an XFS file system.

• The syntax for invoking the newfs command:

# newfs [mkfs options] /dev/rdsk/c#t#d#s#– Invoking newfs with the -N option causes newfs to

display the basic parameters of the file system without actually creating it.

How newfs Connects to mkfs• As previously mentioned, the newfs command is a front end

to mkfs. – The newfs command invokes mkfs to per form the actual

creation of the file system on the target device. – The mkfs command requires a long list of parameters.

TIP: Use of the -v option flag with newfs will allow the administrator to see which parameters it passes to mkfs.

• Many file system parameters may be altered by using an option flag to newfs instead of manually invoking mkfs.

Mounting File Systems• File systems can be mounted via several methods:

– manual mounting by invoking the /etc/mount command, – mounting the file system at boot time via the /etc/[v]fstab file, – or mounting via Volume Manager.

• General Procedure for Mounting a File System– The general procedure for mounting a file system is as follows.– 1. Format the disk drive.– 2. Partition the disk drive.– 3. newfs the partition.– 4. Create a mount point for the file system.– 5. Determine which method will be used to mount the file system

(manual mount, [v]fstab mount, Automounter, or Volume Manager).– 6. Mount the file system.

Mounting File Systems• Mounting via the fstab

– The [v]fstab file is used to mount file systems at boot time. • The “v” is bracketed, as some versions of UNIX call this file

/etc/fstab (primarily BSD UNIX), other versions of UNIX call this file /etc/vfstab (primarily System V UNIX), and a few versions of UNIX call this file /etc/checktab (HP/UX).

– This file is read by the /etc/mountall command when it is run as part of the system boot sequence.

– A quick way to add items to [v]fstab is to use the - p option to the mount command.

– It is also possible to add new file systems by editing the [v]fstab file and entering the required information manually.

– The [v]fstab file format may contain minor modifications on different variants of UNIX,.

• A tab or spaces separate the fields of this file. • The following is a typical Solaris [v]fstab file.

Mounting File Systems#device device mount FS fsck mount mount

#to mount to fsck point type pass at boot options

/proc - /proc proc - no -

fd - /dev/fd fd - no -

swap - /tmp tmpfs - yes -

/dev/dsk/c0t3d0s0 /dev/rdsk/c0t3d0s0 / ufs 1 no -

/dev/dsk/c0t3d0s6 /dev/rdsk/c0t3d0s6 /usr ufs 2 no -

/dev/dsk/c0t3d0s5 /dev/rdsk/c0t3d0s5 /opt ufs 5 yes -

/dev/dsk/c0t3d0s1 - - swap - no -

/dev/dsk/c1t3d0s1 - - swap - no -

File System Check– If the fsck pass field contains a minus sign () or a zero (0), no file system

integrity checking is performed.

– A file system with an fsck pass of one (1) indicates that the file system is to be checked sequentially (in the order it is listed in the [v]fstab file).

– Note that the / (root) file system is always checked first.

– File systems with an fsck pass number greater than one (1) are checked in parallel (simultaneously).

– For systems with a large number of disks spanning multiple disk controllers or disk busses, parallel file system checking is generally faster than sequential checking.

– For efficiency, use fsck on file systems of similar size on different disks simultaneously.

• Once the appropriate information has been entered in the [v]fstab file, you can mount all file systems by invoking the following command.

# /etc/mount -a

Mounting File Systems• The Volume Manager provides users with a means of

mounting/unmounting removable media without granting system users root privileges typically required to mount/unmount a file system.

• Under Linux, you can add the “user” option to the list of options for a file system. This allows any user to mount/unmount the file system (an operation that usually requires root access). – For example, the following fstab entry would allow any user to mount the

/jaz file system located on the removable disk /dev/sda1. The nosuid option disallows execution of suid programs from this medium.

/dev/sda1 /jaz ext2 defaults,user,exec,nosuid,noauto 0 0 • usermount is a graphical tool developed by [email protected].

– The usermount tool is available for several UNIX distributions that will allow users to manage removable media, such as floppy disks or zip disks.

– When the tool is invoked, it scans /etc/fstab for all file systems that have been configured to allow users to mount and unmount them.

– The file system can be mounted or unmounted by pressing the toggle button labeled Mount. usermount is available at http://www.cwi.nl/~mdejonge/software/usermount/.

Creating and Mounting• Identifying Mounted File Systems

– Connecting a file system to the tree structure is called “mounting” the file system. – UNIX provides several ways for the administrator to identify which file systems are

currently mounted on the system. Three of the most common methods are:• invoke /etc/mount command invoked with no arguments, • Invoke the df command, and • examination of the /etc/mnttab file

– The df Command– Invoking the df command with no argument results in the display of information for

all mounted file systems. glenn% df/ (/dev/dsk/c0t3d0s0):11690 blocks 9292 files/usr (/dev/dsk/c0t3d0s6):786776 blocks 322309 files/tmp (swap): 218816 blocks 3288 files/opt (/dev/dsk/c0t3d0s5):91236 blocks 73801 files

Creating and Mounting• The /etc/mnttab File• Another way to determine which file systems are mounted is by

examination of the /etc/mnttab file. – This file is created and maintained by the /etc/mount command. – Every time a file system is mounted, an entry is added to the mnttab file.

When a file system is unmounted, the entry is removed from the mnttab file.

# cat /etc/mnttab/dev/dsk/c0t3d0s0 / ufs rw,suid,dev=800018 869622929/dev/dsk/c0t3d0s6 /usr ufs rw,suid,dev=80001e 869622929/proc /proc proc rw,suid,dev=2740000 869622929fd /dev/fd fd rw,suid,dev=27c0000 869622929/dev/dsk/c0t1d0s6 /var ufs rw,suid,dev=80000e 869622929swap /tmp tmpfs dev=0 869622932

Creating and Mounting• The mount Command• The mount command allows the administrator to view which file systems

are mounted, as well as providing a method of mounting file systems. When invoked without arguments, the mount command lists mounted file systems by their mount points, showing the device mounted at each mount point, the mount options used, and the time the file system was mounted.

glenn% /etc/mount/ on /dev/dsk/c0t3d0s0 read/write/setuid on Sat Apr 1 1:23:45 2000/usr on /dev/dsk/c0t3d0s6 read/write/setuid on Sat Apr 1 1:23:45 2000/proc on /proc read/write/setuid on Sat Apr 1 1:23:45 2000/dev/fd on fd read/write/setuid on Sat Apr 1 1:23:45 2000/tmp on swap read/write on Sat Apr 1 1:23:45 2000/opt on /dev/dsk/c0t3d0s5 setuid/read/write on Sat Apr 1 1:23:45 2000

Creating and Mounting• Unmounting a File System• The complement to mounting a file system is to unmount it using the

/etc/umount command. – Exercise great care when unmounting file systems! – Some file systems are required for proper system operation. – Other file systems may be unmounted while allowing the system to

continue to operate, but the system will not perform as expected. – The syntax for the umount command follows.

# umount mount_point or

# umount device

Creating and Mounting• Volume Manager• Adding entries to the Solaris vfstab file works well for hard disks but is

not suitable for removable media such as floppies and CD-ROMs. • These devices tend to be mounted and unmounted much more

frequently than hard disks, and the user performing this “mount” operation may not have the root privileges required to mount a normal file system.

• To handle such situations, Solaris uses the Volume Manager.

Creating and Mounting• Configuring Volume Manager• Changing the actions taken and devices under control of Volume

Manager is a simple matter of modifying the /etc/vold.conf file. ## Volume Daemon Configuration file## Database to use (must be first)db db_mem.so# Labels supportedlabel dos label_dos.so floppylabel cdrom label_cdrom.so cdromlabel sun label_sun.so floppy# Devices to useuse cdrom drive /dev/dsk/c0t6 dev_cdrom.so cdrom0use floppy drive /dev/diskette dev_floppy.so floppy0# Actionsinsert /vol*/dev/diskette[0-9]/* user=root /usr/sbin/rmmountinsert /vol*/dev/dsk/* user=root /usr/sbin/rmmounteject /vol*/dev/diskette[0-9]/* user=root /usr/sbin/rmmounteject /vol*/dev/dsk/* user=root /usr/sbin/rmmountnotify /vol*/rdsk/* group=tty /usr/lib/vold/volmissing -c# List of file system types unsafe to ejectunsafe ufs hsfs pcfs

Creating and Mounting• The rmmount.conf Configuration File• The /usr/sbin/rmmount command has its own configuration file named

/etc/rmmount.conf. – Although not often modified, this file allows the specification of additional

actions to occur when a disk is mounted. – A common use of this feature is to allow CD-ROMs mounted by Volume

Manager to be automatically shared, or made accessible to other workstations on the network via NFS.

– To accomplish this, a share line is added to the bottom of the /etc/rmmount.conf file as follows.

share cdrom*• This line would share any CD-ROM mounted by Volume Manager

without any restrictions. To control access the administrator can add options to the share line in a form similar to the share command.

Creating and Mounting• Mounting Non-ufs File Systems with Volume Manager• Volume Manager is also able to handle file system types other than ufs. • For instance, inserting an MS-DOS-formatted floppy and using File

Manager or the volcheck command results in the disk being mounted under the /floppy directory on a mount point that bears the floppy volume name.

• Starting and Stopping Volume Manager• UNIX makes it possible to disable Volume Manager and work directly

with the CD-ROM and/or floppy drive. • To disable Volume Manager, invoke the /etc/init.d/volmgt script with the

stop option. • To restart Volume Manager, invoke the /etc/init.d/volmgt script with the

start option.

File System Repair• Most versions of UNIX, by default, provide very

similar file system capabilities. – The default file system does not provide any

extraordinary safeguards against file loss due to events such as power failure or system crashes.

– These file systems rely on their understanding that everything will get written to disk, and nothing bad will happen.

– In the event that something bad does occur, a few utilities are provided to clean up the resulting mess.

File System Repair

• Older versions of UNIX used to have three utilities which were used to check the consistency of various portions of the file system structures:– icheck – this utility was used to check the inode structure to ensure

that the proper blocks were used, and unused.– dcheck – this utility was used to check that all of the data blocks for

a given inode were properly linked.– ncheck – this utility generates a list of filenames and inodes given a

file system name. It was used to scan the filesystem, and list all of the filenames, and inodes in that file system.

File System Repair

• Current versions of UNIX have a tool that combined the functionality of these three utilities into a single (automated) tool:– fsck – File system check utility

File System Repair

• There are two categories of file system damage:– Recoverable File System Damage

• This type of damage can be repaired using the fsck utility.

– Generally caused by improper shutdown, or system crashes.

– File system data, or meta data has not been stored on the disk, and the file system state is inconsistent.

– Fsck can generally fix these problems using information stored in alternate superblocks.

– Any orphaned files/directories are placed in the lost+found directory at the top of the file system.

» Since the file name is not stored in the inode, these files are stored using the inode number for the file name.

File System Repair

• Restoring Files Versus Repairing a File System

–Sometimes the file system damage is severe enough that it would take fsck several passes to repair the damage.

– In such cases, it might be more productive to newfs the file system, and reload the contents from backups.

» MAKE SURE THAT YOU HAVE A GOOD BACKUP OF THE DATA BEFORE MAKING THE DECISION TO NEWFS AND RELOAD!!!

File System Repair

– Unrecoverable File System Damage• Some types of disk corruption are not

repairable using fsck.–Superblock unreadable.

–Head crash

–Other hardware failure

File System Repair• Journaling File Systems

– The benefit of using a journaled file system is that the system is more immune to data loss the nature of which is the result of a power failure or system crash.

– Large file systems often suffer significant data loss when a system crashes.

– Journaling file systems provide significant improvement in (reduced) reboot time, and data retention on large file systems.

– A drawback of using a journaled file system is that you have to reserve disk space for the journal. On most systems, the journal is a small meta-partition on the disk.

Windows File System Check• Windows also provides a file system check

program.– Some versions of Windows refer to this program as

scandisk, while other versions refer to it as checkdsk.• Performs the same functions as fsck under UNIX.

– The Windows chkdsk utility hides many details from the administrator.

• The simple way to check a Windows disk for errors is to right-click on the disk in the My Computer window, and select the Properties item.

• Under the Properties item, select the Tools tab. You will be given the following three choices of tasks you can perform on the disk.

Windows File System Check

• Error-checking: Allows you two options: the ability to scan the surface of the disk and attempt to repair bad blocks, and the ability to automatically repair file system errors as they are found. – Orphans are placed in the top level of the file system and given

names like “FILE0000, DIR0001, …”

• Backup: Allows you to run a file system backup or restore operation.

• Defragmentation: Allows you to clean up file system storage blocks used to store fragments of files.

Recognizing Failing Disks• Soft Errors

– These are typically bad spot appearing on the disk surfaces. This is expected.

– Beware of large numbers of bad spots appearing over a short period.

Recognizing Failing Disks• Soft errors can usually be repaired using the format utility.

– Make sure that you have a good backup of all data before attempting this!

– Use non-destructive scans whenever possible. Use read then write back mode if you must do write testing.

Recognizing Failing Disks

• Hard Errors– Hard errors are typically caused by head crashes, cabling

problems, or electronics failures

– WARNING: Whenever a fatal disk error is reported, the administrator should be very concerned about the integrity of the files on the disk drive. Steps must be taken to rectify the problem immediately, or data may be destroyed.

Repairing Disk Errors• Perform a full backup of the failing disk, or better yet, the

entire system.• Reboot the system to determine if the failure is due to

corrupted system information.• If the block number of a bad spot is known, use the system’s

format utility to map out the spot on the drive.• If there are several bad spots on the drive, use the surface

verification portion of the format utility to perform nondestructive surface analysis of the drive to search for and optionally repair bad spots.

• Replace the drive with a known good drive. Copy all data to the new drive. Note that this option can be time consuming and expensive.

Improving Disk Subsystem Performance

• File System Optimization– Optimize file systems across multiple spindles and controllers.

– Optimize file system structures for the type of files to be stored on the file system.

• The default newfs settings are good for general purpose file systems. But they are also very inefficient for certain types of applications.

• File systems containing primarily large files require fewer inodes. This leads to a lot of lost space.

• File systems containing primarily small files require more inodes. This leads to a problem where there is space left on the device, but no file system structures are available to address the unused space!


• File system tuning can provide only limited performance improvements.

• Performance tuning is critical to the first type (active files), and probably not very important for the archive server.

• In most instances, disk farms are set up to provide high-speed access to important data. Typical applications include database systems, medical images, and other space-intensive information services.

• In a few instances, the disk farm is used for long-term archival storage. Because the information on these systems is not accessed very often, high performance is not always the driving criterion in file system tuning.

• Although tunefs is capable of changing many characteristics of the disk subsystem, it is not a do-all/save-all I/O sub system fix-it tool.


• The tunefs command allows you to “tune” file system parameters as you build a file system (more/less inodes, more/less free space, …).

• Tuning for Special Disks and Disk Controllers– By default, newfs uses the information in the disk label to calculate

several factors related to the ability of the disk, controller, and CPU to read or write information. As the disk rotates, each block in the track moves past the disk heads. If the controller and CPU are fast enough, blocks may be written or read in the order they come under the heads.

– Slower controller/CPU combinations must skip one or more blocks between read/writes to keep up with the I/O demands of the system. The newfs and tunefs commands allow the rotational interleave aspect of the file system to be adjusted by calculating the required time for a single block read to be processed.

– Some disk controllers are capable of writing or reading multiple blocks in a single operation due to high-speed buffer memory located in the controller.


• Tuning for Small and Large Files

– Applications that consistently read and write very small (or very large) files can often benefit from file system tuning.

• Large files are often split between cylinder groups. A cylinder group is a collection of cylinders used by the disk I/O routines in the kernel to improve disk performance by grouping a file’s data blocks close together on the disk.

• When a large file is split over two or more cylinder groups, the file’s data blocks are spread across the disk. Consequently, extra seek time is required.

• Adjusting the maximum number of blocks that can be allocated to a single file within a cylinder group may help reduce this problem.


• Tuning for Small and Large Files• It is also possible to adjust basic allocation block and

fragment sizes on the file system. – Larger allocation block and fragment sizes favor large

files by reducing the time required for file allocation at the expense of reduced space efficiency.

– If an application stores small files (exclusively), a smaller allocation block will improve speed, and a smaller fragment size will improve space utilization efficiency by avoiding the allocation of blocks that are much larger than the data to be stored.


• Tuning for Storage Space Efficiency Versus Speed– The disk storage routines in the UNIX kernel have

two strategies available for disk allocation: time and space.

• Time efficiency refers to the time required to allocate space and write files. Optimizing time is wasteful of space because transfers often result in gaps being created in lieu of long, continuous disk writes.

• Space efficiency refers to the efficient use of scattered blocks on the disk. Optimizing space wastes time because a file is allocated to blocks scattered around the disk, and the disk heads must move more frequently to read or write a file.

Summary

• Understanding how file systems are created, mounted, checked, and tuned enables the administrator to effectively manage corporate disk space.

• More importantly, understanding how to mount, create, and unmount file systems on an active system can result in minimization of down-time for corporate users.

• The sysadmin should understand typical failure modes, and how to repair these problems.

• Performance tuning is also a big part of the system administrator’s duties.

Ch12 system administration

Technology

Transcript of Ch12 system administration