Unix Programming Unix Programming EnvironmentEnvironment
Part 3-3 File Systems in Unix
Prepared by Xu Zhenya( [email protected] )
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
AgendaAgenda 1. Hierarchical Architecture & Tree-structured Name Space1. Hierarchical Architecture & Tree-structured Name Space
File, Directory, Hard Link, Mountable File SystemFile, Directory, Hard Link, Mountable File System
2. Access Control ( Permission )2. Access Control ( Permission )
3. Implementation ( layout )3. Implementation ( layout )
4. File, Directory and inode4. File, Directory and inode
5. Special Files5. Special Files
Symbol LinkSymbol Link
device files, PIPE & socket, etcdevice files, PIPE & socket, etc
6. System Data Files and Information6. System Data Files and Information
7. Commands Summary7. Commands Summary
Textbook: Chapter2, Chapter1( 1.2&1.3 )Textbook: Chapter2, Chapter1( 1.2&1.3 )
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Tree-structured Name SpaceTree-structured Name Space
• using “Slash”: •/usr/bin
• Physical Disk, Logical Disk, File System
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File ( 1 ) - TypesFile ( 1 ) - Types Everything visible to the user in UNIX systems( System Resource ) can be represented as a Everything visible to the user in UNIX systems( System Resource ) can be represented as a
file in the file system – even file in the file system – even including the processes and the network connectionsincluding the processes and the network connections. . UNIX stores information in byte-oriented files. UNIX stores information in byte-oriented files. You can view the different types of files with ls -l You can view the different types of files with ls -l
ls -ld /home /dev/null /etc/passwd ls -ld /home /dev/null /etc/passwd ls -l /dev/hda1 ls -l /dev/hda1 ls -l /etc/X11/X ls -l /etc/X11/X
File Type Meaning Example
- a normal file /etc/passwd
d a directory /
l symbolic link /dev/modem
b block device file /dev/hda
c character device file /dev/tty1
p FIFO or named pipe
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File ( 2 ) - Objects in Unix File SystemsFile ( 2 ) - Objects in Unix File Systems
+GetObjectName()+SetObjectName()+RenameObject()+GetObjectFileLength()+GetObjectLinkCount()+GetObjectInode()+GetObjectType()+GetAttributes()+SetAttributes()+UpdateObjectHeader()+DoGenericObjectDeletion()+Unlink()+Dump()
Object
+ReadDataFromFile()+WriteDataFromFile()+ResizeFile()+UnlinkFile()+FlushFile()+CheckFileSanity()
Regular File
+FindObjectByName()+AddObjectToDirectory()+RemoveObjectFromDirectory()
Directory Special Files SymLink HardLink
1
*
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Types of normal files Types of normal files
All files under a UNIX just contain bytes of data. All files under a UNIX just contain bytes of data.
However, the magic number specifies a particular byte (or However, the magic number specifies a particular byte (or two/three) to be tested. Magic numbers, tests and file types two/three) to be tested. Magic numbers, tests and file types are described in the magic file, /usr/share/magic ( Linux )are described in the magic file, /usr/share/magic ( Linux )
You can determine the file type using the You can determine the file type using the filefile command. command.
ExampleExample$ file Image43.gif disks.jpg todo two.html$ file Image43.gif disks.jpg todo two.html
$ file /bin/ls$ file /bin/ls
$ file /arm/system/rootfs/gdbserver$ file /arm/system/rootfs/gdbserver
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Attributes File Attributes Information UNIX stores about files includes / a file’s metadataInformation UNIX stores about files includes / a file’s metadata
location of the datalocation of the data: Includes the device and also pointers to data blocks : Includes the device and also pointers to data blocks
FilenameFilename:Up to 255 characters, should avoid using * ? $ ' " \ -. Can't use / :Up to 255 characters, should avoid using * ? $ ' " \ -. Can't use /
user and groupuser and group who own the file who own the file :: Stored as UID and GID Stored as UID and GID
who can do what with the file (who can do what with the file (permissionspermissions):): Explained on the coming slides Explained on the coming slides
how big the file ishow big the file is :: Size in bytes. Linux filesystem (ext2) supports file sizes up to 4Tb (terra Size in bytes. Linux filesystem (ext2) supports file sizes up to 4Tb (terra bytes). bytes).
when it was last modified, accessed or changed( inode ) when it was last modified, accessed or changed( inode )
how many how many linkslinks to it: Also discussed below. to it: Also discussed below.
Stored on an Stored on an i-nodei-node. . Attributes can be viewed using: Attributes can be viewed using: ls -l ls -l stat: a system call to fetch a file’s attributesstat: a system call to fetch a file’s attributes
ExampleExample: $ ls -ld / /dev: $ ls -ld / /dev What type of files are / and /dev? What type of files are / and /dev?
How come /dev is bigger than /?How come /dev is bigger than /?
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
A typical file system in UnixA typical file system in Unix
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
A typical file system in UnixA typical file system in Unix
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Systems in Modern Unix SystemsFile Systems in Modern Unix Systems
VFS
XXX-FS
BUFFERCACHE
open read mount sync
file
inode super block
getblk() bread() wait_on_buffer()
XXX_open()
XXX_read()XXX_fsync()
XXX_lookup()
XXX_unlink()XXX_mkdir()
XXX_create() XXX_write_super()
XXX_delete_inode()
XXX_read_inode()
dcache
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
filesystem meta-data access filesystem data access
superblock
bitmapblockgroup
directoryentry
(name)cache
inodecache
buffer cache
write read mmap
page cache
I/O I/O
File Systems in Modern Unix SystemsFile Systems in Modern Unix Systems
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Protection File Protection UNIX must protect files, restrict access to certain files. Examples:UNIX must protect files, restrict access to certain files. Examples:
/etc/passwd: written only by the root user./etc/passwd: written only by the root user.
high score file for a game high score file for a game
UNIX achieve this by UNIX achieve this by
specifying three valid file operationsspecifying three valid file operationsRead, write and execute Read, write and execute
User, user groupUser, user group
/etc/passwd, /etc/group, /etc/shadow/etc/passwd, /etc/group, /etc/shadow
the super user: rootthe super user: root
Process, user, & user group: loginProcess, user, & user group: login
dividing users into three groupsdividing users into three groups user - person who owns the file user - person who owns the file group - group who owns the file group - group who owns the file other - everybody else other - everybody else
allow the owner to specify valid operations for each fileallow the owner to specify valid operations for each file
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Operations and Permission (1)File Operations and Permission (1)
Operation Effect on a file Effect on a directory
read read the contents of the file find out what files are in the directory, e.g. ls
write delete or add something from/to the file
be able to create or remove a file from the directory
execute be able to run a file/program be able to search a file within a directory
Notes: (read, write and execute) operate slightly differently on files and directories
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Operations and Permission (2)File Operations and Permission (2)
File Permissions Description
/ drwxr-xr-x
file type => directoryuser/owner (root) => read, write and executegroup (root) => read and executeother => read and execute
/etc/passwd -rw-r--r--
file type => normal fileuser/owner (root) => read and writegroup (root) => readother => read
/home/upe/teaching/ drwxrwxrwx
file type => directoryuser/owner => read, write and executegroup => read, write and executeother => read, write and execute
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
setuid/setgid and sticky bitsetuid/setgid and sticky bit Looking at “Looking at “/etc/passwd” or “/etc/shadow” file /etc/passwd” or “/etc/shadow” file
-r-------- 1 root root 718 Jul 24 19:32 /etc/shadow-r-------- 1 root root 718 Jul 24 19:32 /etc/shadow
setuid / setgid bitsetuid / setgid bit Whether a process can write to a file is based on their Whether a process can write to a file is based on their effective uid/gideffective uid/gid (not the (not the real real
uid/giduid/gid). ).
setuid and setgid bits change the effective uid and gid of a process setuid and setgid bits change the effective uid and gid of a process
Used when you want to provide restricted access: => May lead to Security? Used when you want to provide restricted access: => May lead to Security?
modified by any user with the setuid “modified by any user with the setuid “passwdpasswd” command” command -r-sr-xr-x 1 root bin 15613 Apr 28 1998 /usr/bin/passwd-r-sr-xr-x 1 root bin 15613 Apr 28 1998 /usr/bin/passwd
““Sticky” BitSticky” Bit A very long time ago, Unix ran on machines with 64K memory: swap areaA very long time ago, Unix ran on machines with 64K memory: swap area
Some UNIX systems use these bits to change the behavior of directories. Some UNIX systems use these bits to change the behavior of directories.
For example to set the mode of the /tmp on a system so any user can create or For example to set the mode of the /tmp on a system so any user can create or delete her own files, but cannot delete other's file.delete her own files, but cannot delete other's file.
drwxrwxrwdrwxrwxrwtt 11 root root 24576 Oct 13 17:34 /tmp 11 root root 24576 Oct 13 17:34 /tmp
You must set the sticky bit as root. You must set the sticky bit as root.
On older systems setting the SGID bit resulted the same behavior.On older systems setting the SGID bit resulted the same behavior.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Controlling Terminal
Process Group
Current
Session
foreground
Background
Process
Parent1
1..n
+BelongTo
1
1..n
Leader
UserGroup
User
euid
11
Supplementary
ruid
saved_euid
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Numeric and Symbolic Permissions Numeric and Symbolic Permissions UNIX actually stores permissions as UNIX actually stores permissions as numbersnumbers. But humans generally . But humans generally
don't do numbers well. The nice commands (like ls stat) change them to don't do numbers well. The nice commands (like ls stat) change them to symbolic.symbolic.
SymbolSymbol OperationOperation Numeric equivalent Numeric equivalent
rr ReadRead 44
ww WriteWrite 22
xx ExecuteExecute 11
ss Setuid or setgidSetuid or setgid 4000 setuid 2000 setgid 4000 setuid 2000 setgid
tt Sticky bitSticky bit 10001000
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Permissions (2)Permissions (2)
Conversion from symbolic to numeric Conversion from symbolic to numeric
split symbols into three user groupssplit symbols into three user groups user - rwx user - rwx group - r-- group - r-- other - r-x other - r-x
replace symbols with numeric equivalent and addreplace symbols with numeric equivalent and add user - rwx = 4 + 2 + 1 = 7 user - rwx = 4 + 2 + 1 = 7 group - r-- = 4 group - r-- = 4 other - r-x = 4 + 1 = 5 other - r-x = 4 + 1 = 5
bring them together to form the numeric permissionsbring them together to form the numeric permissions rwxr--r-x = 745 rwxr--r-x = 745
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Permissions (3)Permissions (3)
Changing file permissions Changing file permissions
chmodchmod: Change the file permissions for a file. Only the owner of a file : Change the file permissions for a file. Only the owner of a file and the root can use it. and the root can use it.
chgrpchgrp: Change the group owner of a file. You can only change it to a : Change the group owner of a file. You can only change it to a group you belong to. group you belong to.
chownchown: Change the user owner of a file. Only root can use this. : Change the user owner of a file. Only root can use this.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
chmod commandchmod command chmod [-Rfv] operation fileschmod [-Rfv] operation files
-R : Optional, tells the command to recursively descend directories and change permissions as it -R : Optional, tells the command to recursively descend directories and change permissions as it goes. goes.
Operation : Operation : Specifies how to change the permissions of the file. Can use symbolic or numeric Specifies how to change the permissions of the file. Can use symbolic or numeric permissions permissions
Files Files : The list of files to change file permissions of.: The list of files to change file permissions of.
Types of operation can be Types of operation can be Numeric: Permissions specifed using numeric formNumeric: Permissions specifed using numeric form
chmod 770 my.filechmod 770 my.file
Symbolic: Symbolic: chmod [-Rfv] [augo] [+-=] [rwxst] file [file ...]chmod [-Rfv] [augo] [+-=] [rwxst] file [file ...]
Who : Who : Specify which category of user to changeSpecify which category of user to change u - user, g - group, o - others, a - all u - user, g - group, o - others, a - all
op : op : How to change permissionsHow to change permissions +: add, -: subtract, = : set +: add, -: subtract, = : set
Permission : Permission : Symbolic permissions.Symbolic permissions.
Examples Examples chmod u+rwx temp.dat chmod u+rwx temp.dat chmod go-rwx temp.dat chmod go-rwx temp.dat chmod -R a-rwx /etc chmod -R a-rwx /etc chmod 770 temp.datchmod 770 temp.dat
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
umask commandumask command umask works doing the bitwise AND with the bitwise complement of the umask. Bits that are set in
the umask correspond to permissions that are not assigned to newly created files. The explanation of various values:
0 All permissions granted
1 no execute
2 no write
3 no write or execute
4 no read
5 no read or execute
6 no read or write
7 no permissions at all
The most common umask values are: 002 others: no write
022 group + others: no write
027 group: no write, others: none
077 group + others: none
0666 & ~022 = 0644 = rw-r--r--: open( 0666 ) and umask( 022 ), the default setting extern int open (__const char *__file, int __oflag, ...); extern int creat (__const char *__file, __mode_t __mode);
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Hard LinkHard Link
UNIX allows you to provide one file with UNIX allows you to provide one file with more than one more than one namename. Hard links are a way to create a new filename which . Hard links are a way to create a new filename which points to an existing i-node. points to an existing i-node.
A directory is simple a file which contains a list of <filename, inode> A directory is simple a file which contains a list of <filename, inode> pairs. pairs.
Create using the “Create using the “lnln” command ” command
Notes: Notes:
Hard links only work :Hard links only work :
when the files are on the same device/partition when the files are on the same device/partition
on a file system which supports links on a file system which supports links
Any operation performed on the data in link is performed on Any operation performed on the data in link is performed on the original file ( Data Block ). the original file ( Data Block ).
Don’tDon’t make a loop on the file system. => Difficult to fix this question. make a loop on the file system. => Difficult to fix this question.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Soft/Symbolic LinksSoft/Symbolic Links
Differences from hard links include: Differences from hard links include:
created using the -s option for the ln command created using the -s option for the ln command
$ ln -s the_file symbolic_link$ ln -s the_file symbolic_link
can cross partitions and devices can cross partitions and devices
doesn't link by inode doesn't link by inode
can't create a symbolic link to a file that doesn't exist? can't create a symbolic link to a file that doesn't exist?
file permissions are not usedfile permissions are not used
Notes: Notes:
chmodchmod operations performed on a hard link are reflected on both the operations performed on a hard link are reflected on both the hard link file and the file it is linked to. hard link file and the file it is linked to.
chmodchmod operations on soft links are reflected on the original file but not operations on soft links are reflected on the original file but not on the soft link - the soft link will always have full file permissions on the soft link - the soft link will always have full file permissions (lrwxrwxrwx).(lrwxrwxrwx).
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
find command (1)find command (1)
Format: Format:
find [ path-list ] [ expression ] find [ path-list ] [ expression ]
path-listpath-list : : Optional. Is a list of files/directories in which find should Optional. Is a list of files/directories in which find should search. find is recursive. search. find is recursive.
Expression Expression : Optional. Specifies what is being searched for and what : Optional. Specifies what is being searched for and what to do with it. to do with it.
A find expression can contain the following components A find expression can contain the following components
Options : Modify find's operation Options : Modify find's operation
Tests : Specify what we are looking for Tests : Specify what we are looking for
-name “pattern”, -user “name”, -size “1234”, etc-name “pattern”, -user “name”, -size “1234”, etc
Actions : Specify what we're going to do with what we find Actions : Specify what we're going to do with what we find
Operators : Used to combine expressions Operators : Used to combine expressions
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Action Effect
-exec command ; Execute a command. String {} is replaced by the current filename.
-fls file like -ls but write to file
-fprint file Print the full filename into file
-fprint0/-print0
print filename followed by a null character
-ok command ; Same as -exec but ask the user if it is ok before executing the command
-print Print the name of the file followed by a newline
-printf format Allows the use of C-like printf formats
-ls list the current file in ls -dils format
find command (2)find command (2)
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
find command (3)find command (3) Examples: Examples:
$ find . -user upe$ find . -user upe
$ find / -name \*.html$ find / -name \*.htmlthe * must be quoted so that the shell doesn't interpret it. the * must be quoted so that the shell doesn't interpret it.
$ find /home -size +2500k -mtime -7$ find /home -size +2500k -mtime -7files are greater than 2500 kilobytes in size and their data have been modified files are greater than 2500 kilobytes in size and their data have been modified in the last seven days. in the last seven days.
$ find . -exec grep hello \{\} \; $ find . -exec grep hello \{\} \;
Search all the files under the local directory for the word hello. Search all the files under the local directory for the word hello.
$ find / -name \*.bak -ok rm \{\} \;$ find / -name \*.bak -ok rm \{\} \;
ask the user if it is ok to remove themask the user if it is ok to remove them
Convert all .c & .h from the DOS format into the UNIX : 012, 015Convert all .c & .h from the DOS format into the UNIX : 012, 015
$ find . -name "*.[ch]" -print -exec sh -c "tr -d '\015' < {} > {}.$$; mv -f {}.$$ {}" \;
$ dos2unix
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
find command (4)find command (4)
Notes: Notes:
[for Win32 Cygwin][for Win32 Cygwin] The syntax of the directories a little different, The syntax of the directories a little different, because Windows has drive letters and Unix does not. In Cygwin Bash because Windows has drive letters and Unix does not. In Cygwin Bash the drive letter is expressed as /cygdrive/c. To find all "tmp" files from the drive letter is expressed as /cygdrive/c. To find all "tmp" files from directory C:/temp down:directory C:/temp down:
$ mount$ mount
$ find /cygdrive/c/temp/ -name "*tmp*" -print $ find /cygdrive/c/temp/ -name "*tmp*" -print
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Device Files (1)Device Files (1) DeviceDevice - generic name of system component that OS has to "talk" to. - generic name of system component that OS has to "talk" to.
Physical devicesPhysical devices - eg. hard disks, serial devices, CDROMs, sound card. - eg. hard disks, serial devices, CDROMs, sound card.
Logical devicesLogical devices - eg. virtual terminals, memory, kernel, network ports, - eg. virtual terminals, memory, kernel, network ports, loopback disk devices(use a regular file as a block deviceloopback disk devices(use a regular file as a block device ) ). .
Device filesDevice files - allow programs to interact with devices via OS kernel. - allow programs to interact with devices via OS kernel.
- /dev: the location of device files- /dev: the location of device files
- not real files - not real files
- do not contain data - do not contain data
- entry point into kernel or - entry point into kernel or device driversdevice drivers. .
Device driversDevice drivers - are coded routines used for interacting with devices - are coded routines used for interacting with devices - " - "go betweengo between" for low level hardware and kernel/user interface. " for low level hardware and kernel/user interface. - compiled into kernel or dynamically loaded in memory. - compiled into kernel or dynamically loaded in memory.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Device Files (2)Device Files (2) Major and Minor device number Major and Minor device number
used by kernel to communicate with devices used by kernel to communicate with devices
kernel maintains list of its available device drivers: device switch tablekernel maintains list of its available device drivers: device switch table
major number tells kernel which device driver major number tells kernel which device driver
minor device number determine which physical device minor device number determine which physical device
$ ls -al /dev/hda /dev/hdb$ ls -al /dev/hda /dev/hdbbrw-rw---- 1 root disk 3, 0 Apr 28 1995 /dev/hdabrw-rw---- 1 root disk 3, 0 Apr 28 1995 /dev/hdabrw-rw---- 1 root disk 3, 64 Apr 28 1995 /dev/hdb brw-rw---- 1 root disk 3, 64 Apr 28 1995 /dev/hdb
major number 3 controls both hda & hdb major number 3 controls both hda & hdb
hda - minor device number of 0 hda - minor device number of 0
hdb - minor device number of 64 hdb - minor device number of 64
Creating device files: Creating device files:
mknod /dev/ttyS9 c 4 240mknod /dev/ttyS9 c 4 240
See how to implement a simple device driver.See how to implement a simple device driver.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Device Files (3)Device Files (3)
Demo using “tty” devices on Linux / SolarisDemo using “tty” devices on Linux / Solaris
# ls –l `tty`# ls –l `tty`
# chmod a+w `tty`# chmod a+w `tty`
$ echo “hi, root!” > `tty`$ echo “hi, root!” > `tty`
Notes:Notes:
Pseudo Terminals - Non-Physical terminals Pseudo Terminals - Non-Physical terminals
master pseudo-terminals & slave pseudo-terminalsmaster pseudo-terminals & slave pseudo-terminals
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
File Systems OperationsFile Systems Operations Disks, Partitions and file systemsDisks, Partitions and file systems
fdisk /dev/sda fdisk /dev/sda
cat /proc/filesystem # on Linuxcat /proc/filesystem # on Linux mkfsmkfs - make file system creates I-Nodes and data blocks - make file system creates I-Nodes and data blocks mountmount - attach a file system to part of the directory hierarchy - attach a file system to part of the directory hierarchy
$ mount –t dos /dev/hda1 /mnt/dos$ mount –t dos /dev/hda1 /mnt/dos $ umount /dev/fd0 $ umount /dev/fd0 * /etc/fstab, mount* /etc/fstab, mount
fsckfsck – check the integrity of a file system – check the integrity of a file system Booting timeBooting time
Manual checking Manual checking
Misc: Misc: Mtools: Mtools: utilities provide a convenient way of accessing DOS-formatted floppies utilities provide a convenient way of accessing DOS-formatted floppies
without having to mount and unmount filesystems. without having to mount and unmount filesystems.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Example ( 1 )Example ( 1 )
# create a file whose size is 8Mb# create a file whose size is 8Mb
dd if=/dev/zero of=/tmp/file bs=1M count=8dd if=/dev/zero of=/tmp/file bs=1M count=8
# dump a harddisk’s MBR: dd if=/dev/hda …# dump a harddisk’s MBR: dd if=/dev/hda …
losetup /dev/loop0 /tmp/filelosetup /dev/loop0 /tmp/file
# losetup -e des /dev/loop0 /file# losetup -e des /dev/loop0 /file
mkfs -t ext2 /dev/loop0mkfs -t ext2 /dev/loop0
mount -t ext2 /dev/loop0 /mntmount -t ext2 /dev/loop0 /mnt
# a quick method: mount –o loop /tmp/file /mnt# a quick method: mount –o loop /tmp/file /mnt
umount /dev/loop0umount /dev/loop0
losetup -d /dev/loop0losetup -d /dev/loop0
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Example ( 2 )Example ( 2 )
See how to implement a simple file system.See how to implement a simple file system.
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Important Files and DirectoryImportant Files and Directory /etc/passwd, /etc/shadow, /etc/group/etc/passwd, /etc/shadow, /etc/group
/var/log/*/var/log/*
/etc/rc[1-6].*: /etc/inittab/etc/rc[1-6].*: /etc/inittabDirectories Contains
/bin binaries required when in single user mode
/boot Boot configuration files for the OS loader and the kernel image
/dev device files
/etc System configuration files and scripts
/home User home directories
/lib Main OS shared libraries and kernel modules
/lost+found Storage directory for "recovered" files
/mnt temporary point to connect devices to
/proc pseudo directory structure containing information about the kernel, currently running processes and resource allocation
/root Home directory of the root (super) user
/sbin System administration binaries and tools
/tmp Storage of temporary files, sticky bit
/usr Contains almost everything else including libraries, binaries, applications and packages
/var Variable data, usually machine specific. Including spool directories for mail and news. Also system logs and messages
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
SummarySummary Files under UNIX Files under UNIX
are just a sequence of bytes are just a sequence of bytes
there are number of types of files there are number of types of files
magic numbers can be used to determine the type of a normal filemagic numbers can be used to determine the type of a normal file
UNIX stores information about files including UNIX stores information about files including filenamesfilenames
file size (up to 4Tb) file size (up to 4Tb)
file ownership etc. file ownership etc.
Files are protected by a combination of Files are protected by a combination of users and groups: users and groups: “su”“su”
file operations file operations
file permissions file permissions
Links (both hard and soft) allow a file to have more than one name Links (both hard and soft) allow a file to have more than one name findfind command is used to command is used to
search for files search for files
which match a certain condition which match a certain condition
and perform operations on them and perform operations on them
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Commands & UtilsCommands & Utils File commands File commands
catcat print content of file(s) print content of file(s)
cdcd change directory change directory
chmodchmod Change modes (files and directories) Change modes (files and directories)
cpcp copy files copy files
diffdiff Report difference between files. patch Report difference between files. patch
lsls List files ( -r & -R ) List files ( -r & -R )
mvmv move files (you can think also as rename) move files (you can think also as rename)
odod Dump the content of file ("od -h -c" for hex dump) Dump the content of file ("od -h -c" for hex dump)
pwdpwd print working firectory print working firectory
rmrm Remove files Remove files
teetee Genrate output to files inside pipeline Genrate output to files inside pipeline
touchtouch Modify File's time stamp Modify File's time stamp
tartar Group files into TAR packet. Usually with gzip or bzip2 Group files into TAR packet. Usually with gzip or bzip2
bzip2bzip2 Compress single file with bzip2 Compress single file with bzip2
gzipgzip Compress single file with gzip Compress single file with gzip
compresscompress Old Unix .Z compression Old Unix .Z compression
Disk commands Disk commands dudu Disk usage Disk usage
dfdf Disk free, sometimes as bdf(1) Disk free, sometimes as bdf(1)
Reference Material: <External Filters, Programms and Commands in UnixReference Material: <External Filters, Programms and Commands in Unix >, >, Mendel CooperMendel Cooper
Unix Programming EnvironmentUnix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
AppendixAppendix 1. How does the kernel implement file systems? 1. How does the kernel implement file systems?
Architecture, directory cache( namei ), vnode, inode, Architecture, directory cache( namei ), vnode, inode,
I/O device management: /dev, major & minorI/O device management: /dev, major & minor
Layout of the file system: boot block, super block, inode, dir, data blockLayout of the file system: boot block, super block, inode, dir, data block
Hard link, soft linkHard link, soft link
Link: reference countLink: reference count
The link of one dir >= 2.The link of one dir >= 2.
2. modeling concepts in file systems using UML2. modeling concepts in file systems using UML User( name, uid, … )(e-uid, r-uid ), group( supplementary group), file(regular files, device files, User( name, uid, … )(e-uid, r-uid ), group( supplementary group), file(regular files, device files,
pipe, socket, etc), vnode/inode, directory, file system, access control: relationship & patternpipe, socket, etc), vnode/inode, directory, file system, access control: relationship & pattern
Mapping the model into the implementation including the layout of the file systemsMapping the model into the implementation including the layout of the file systems
3. Other issues: 3. Other issues: Tree-structured name space management: Tree-structured name space management:
Hierarchical: domain name system, the name service in CORBA, RMI in Java, Directory Hierarchical: domain name system, the name service in CORBA, RMI in Java, Directory Service( LDAP, … ), etcService( LDAP, … ), etc
Objects’ name & their reference: the name registry in many tech. Objects’ name & their reference: the name registry in many tech.
File: polymorphical & interface => AbstractionFile: polymorphical & interface => Abstraction
Link/reference count: inode, smart pointers in C++Link/reference count: inode, smart pointers in C++
Layout of the file system: the object model of Java( GC=>reference count )Layout of the file system: the object model of Java( GC=>reference count )
Top Related