240-491 Adv. UNIX: lowIO/161 Advanced UNIX v Objectives –look at low-level operations for handling...

240-491 Adv. UNIX: lowIO/16 1

Advanced UNIXAdvanced UNIX

ObjectivesObjectives– look at low-level operations for handling look at low-level operations for handling

filesfiles

240-491 Special Topics in Comp. Eng. 2Semester 2, 2000-2001

16. Low-level File I/O


OverviewOverview

1. Basic Operations1. Basic Operations

2. Why use Low-level Operations?2. Why use Low-level Operations?

3. A Simple Example3. A Simple Example

4. File Descriptors4. File Descriptors

5. File Permissions Again5. File Permissions Again

continued


6. 6. open()open() 12. 12. lseek()lseek()

7. 7. creat()creat() 13. Hotel Guests13. Hotel Guests

8. 8. close()close() 14. 14. unlink()unlink()

9. 9. read()read() 15. 15. fcntl()fcntl()

10. 10. write()write() 16. Duplicating FDs16. Duplicating FDs

11. 11. copyfilecopyfile 17. File Pointers17. File Pointers


1. Basic Operations1. Basic Operations

NameName MeaningMeaningopen()open() Opens a file for reading or Opens a file for reading or writing.writing.creat()creat() Creates an empty file for writing.Creates an empty file for writing.close()close() Closes an open file.Closes an open file.read()read() Reads data from a file.Reads data from a file.write()write() Writes data to a file.Writes data to a file.lseek()lseek() Moves to a specified byte in file.Moves to a specified byte in file.unlink()unlink() Removes a file.Removes a file.


2. Why use Low-level Operations?2. Why use Low-level Operations?

No buffering of I/ONo buffering of I/O– useful for network programming and reading/writing useful for network programming and reading/writing

to certain peripherals (e.g. tape drives)to certain peripherals (e.g. tape drives)

No conversion of I/ONo conversion of I/O– no transformation of input to integers, floats, etc.; no transformation of input to integers, floats, etc.;

output can be any byte sequenceoutput can be any byte sequence

Building blocks for more complex I/O.Building blocks for more complex I/O.


3. A Simple Example: oprd.c3. A Simple Example: oprd.c#include <stdio.h>#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>#include <unistd.h>

void main(){ int fd, nread; char buf[1024]; if ((fd = open("data-file", O_RDONLY)) < 0) { perror("open"); exit(-1); } nread = read(fd, buf, 1024); close(fd); buf[nread] = '\0'; /* so can print */ printf("buf: %s\nnread: %d\n", buf, nread);}


4. File Descriptors4. File Descriptors

A file descriptor is a small, non-negative A file descriptor is a small, non-negative integer.integer.

A file descriptor identifies an open file to A file descriptor identifies an open file to the low-level operations.the low-level operations.

Standard descriptors:Standard descriptors:00 represents represents stdinstdin11 stdoutstdout22 stderrstderr


5. File Permissions Again5. File Permissions Again

Earlier Earlier chmodchmod examples used letters to examples used letters to represent permissions:represent permissions:– e.g.e.g. chmod a+r filechmod a+r file

Most low-level operations (and Most low-level operations (and chmodchmod) ) represent permissions as octal values:represent permissions as octal values:– e.g.e.g. chmod 0644 filechmod 0644 file


File Permissions as OctalsFile Permissions as Octals

OctalOctal Symbolic Meaning of PermissionSymbolic Meaning of Permission

04000400 r-- --- ---r-- --- ---02000200 -w- --- ----w- --- ---01000100 --x --- -----x --- ---00400040 --- r-- ------ r-- ---00200020 --- -w- ------ -w- ---00100010 --- --x ------ --x ---00040004 --- --- r----- --- r--00020002 --- --- -w---- --- -w-00010001 --- --- --x--- --- --x


ExampleExample

Add octal values together to get the Add octal values together to get the complete file permission:complete file permission:

OctalOctal MeaningMeaning04000400 r-- --- ---r-- --- ---02000200 -w- --- ----w- --- ---00400040 ++ + + --- r-- ------ r-- ---00040004 --- --- r----- --- r--======== ======================06440644 rw- r-- r--rw- r-- r--

Use in Use in chmodchmod::chmod 0644 filechmod 0644 file


6. 6. open()open()

#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>

int open(char *pathname, int flag/*, mode_t mode */ );

Return file descriptor if ok, -1 for errorReturn file descriptor if ok, -1 for error

an octal value (as in chmod),used when creating a file


Some Some open()open() Flags Flags

O_RDONLYO_RDONLY Open file for reading only.Open file for reading only.O_WRONLYO_WRONLY Open file for writing only.Open file for writing only.O_RDWRO_RDWR Open file for reading & writing.Open file for reading & writing.

Possibly Possibly combinedcombined with: with:O_APPENDO_APPEND Append to file when writing.Append to file when writing.

O_CREATO_CREAT Create file if is does not exist Create file if is does not exist (requires the 3rd (requires the 3rd modemode argument).argument).

O_EXCLO_EXCL Return -1 error if file is to be createdReturn -1 error if file is to be createdand already exists.and already exists.


Using Bitwise ORUsing Bitwise OR

open()open()flags can be combined with the flags can be combined with the bitwise OR operator ‘|”:bitwise OR operator ‘|”:

fd = open(file, O_WRONLY | O_APPEND);fd = open(file, O_WRONLY | O_APPEND);

Now each Now each write(fd, buf, BUFSIZE)write(fd, buf, BUFSIZE)

means append to the end of the file.means append to the end of the file.


7. 7. creat()creat()

#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>

int creat(char *pathname, mode_t mode);int creat(char *pathname, mode_t mode);

Creates a new file, or truncates an old one. Creates a new file, or truncates an old one.

Return file descriptor if ok, -1 on error.Return file descriptor if ok, -1 on error.


modemode gives access permission of resulting new gives access permission of resulting new file:file:

06440644 (rw- r-- r--)(rw- r-- r--)

modemode only has meaning if the file is new only has meaning if the file is new– so it cannot be used to change the mode of an existing so it cannot be used to change the mode of an existing

filefile


Create a new fileCreate a new file

#include <stdio.h>#include <stdio.h>#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>

void main()void main(){ int fd;{ int fd; if ((fd = if ((fd = creatcreat(“new-file”,0644)) < 0){(“new-file”,0644)) < 0){ printf(“cannot create new-file\n”); printf(“cannot create new-file\n”); exit(1); exit(1); } } /* rest of program */ /* rest of program */


Replacing Replacing creat()creat() with with open()open()

creat()creat() only opens a file for writing; to only opens a file for writing; to read it the file must be closed and opened for read it the file must be closed and opened for reading.reading.

creat() creat() can be replaced by can be replaced by open()open()::

open(“new-file”, open(“new-file”, O_WRONLY | O_CREAT | O_TRUNC, 0644)O_WRONLY | O_CREAT | O_TRUNC, 0644)


Other VersionsOther Versions

creat()creat() version which can read and write: version which can read and write:

open(“new-file”, open(“new-file”, O_RDWR | O_CREAT | O_TRUNC, 0644)O_RDWR | O_CREAT | O_TRUNC, 0644)

Avoid truncation of old file (append Avoid truncation of old file (append instead):instead):

open(“new-file”, open(“new-file”, O_WRONLY|O_CREAT|O_APPEND, 0644) O_WRONLY|O_CREAT|O_APPEND, 0644)


One User at a TimeOne User at a Time

Create Create locklock if it doesn’t already exist, otherwise if it doesn’t already exist, otherwise fail (and return -1):fail (and return -1):

fd = open(“lock”, fd = open(“lock”, O_WRONLY|O_CREAT|O_EXCL, 0644);O_WRONLY|O_CREAT|O_EXCL, 0644);

Use Use locklock to protect access to another file to protect access to another file– open “open “accountsaccounts” only if ” only if locklock can be opened can be opened– after processing “after processing “accountsaccounts”, delete ”, delete locklock


Diagram of LockingDiagram of Locking"lock" file

"accounts" file

:if ((fd = open("lock", O_WRONLY | O_CREAT | O_EXCL, 0644)) < 0)) do nothingelse { open "accounts" : close "accounts" delete "lock"}

program 1 :if ((fd = open("lock", O_WRONLY | O_CREAT | O_EXCL, 0644)) < 0)) do nothingelse { open "accounts" : close "accounts" delete "lock"}

program 2


8. 8. close()close()

#include <unistd.h>#include <unistd.h>int close(int fd);int close(int fd);

Close a file: return 0 if ok, -1 on error.Close a file: return 0 if ok, -1 on error.

Useful since the number of open files is Useful since the number of open files is limited (~20)limited (~20)


9. 9. read()read() #include <unistd.h>#include <unistd.h>int read(int fd, void *buffer, int read(int fd, void *buffer,

unsigned int nbytes);unsigned int nbytes);

Attempts to read Attempts to read nbytesnbytes into the into the bufferbuffer array array– no conversion of bytes (characters)no conversion of bytes (characters)– may read less than specified amount (e.g. at end)may read less than specified amount (e.g. at end)

Returns the number of bytes read, 0 if end of file, -Returns the number of bytes read, 0 if end of file, -1 on error.1 on error.


The Read-Write PointerThe Read-Write Pointer

The R-W Pointer records the position The R-W Pointer records the position of the next byte in the file to be read (or of the next byte in the file to be read (or written).written).

It is implicit (hidden) in each use of It is implicit (hidden) in each use of read() read() and and write()write()

1 1+n

read(..., n)


Count Characters: countchars.cCount Characters: countchars.c

#include <stdio.h>#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>#include <unistd.h>

#define SIZE 512 /* block of chars read at a time */

int main(){ char buffer[SIZE]; int fd, j;

:

continued


long total = 0;

if ((fd = open(“foo”, O_RDONLY)) < 0){ printf(“cannot open foo\n”); exit(1); }

while((j = read(fd, buffer, SIZE)) > 0) total += j; printf(“total chars: %ld\n”, total); close(fd); return 0;}


UNIX systems are often configured to UNIX systems are often configured to read/write in blocks of 512 or 1024 bytes read/write in blocks of 512 or 1024 bytes (the (the diskdisk blocking factorblocking factor).).

Can use Can use BUFSIZBUFSIZ from from stdio.hstdio.h which which contains a default disk blocking factor.contains a default disk blocking factor.

Notes


10. 10. write()write()

#include <unistd.h>#include <unistd.h>int write(int fd, void *buffer, int write(int fd, void *buffer,

unsigned int nbytes);unsigned int nbytes);

Returns number of bytes written if ok, -1 on Returns number of bytes written if ok, -1 on error.error.

write()write() starts at the current R-W pointer starts at the current R-W pointer position, and pointer is moved as file is written.position, and pointer is moved as file is written.


ExampleExample

:int fd;char header1[512], header2[512]; :if ((fd = creat(“foo”, 0644)) != -1) : write(fd, header1, 512); write(fd, header2, 512); :


11. 11. copyfilecopyfile#include <stdio.h>#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>#include <unistd.h>

int copyfile(char *nm1, char *nm2);

int main(){ int result; result = copyfile(“test.in”, “test.out”); printf("Result code of copy: %d\n", result); return 0;}

continued


int copyfile(char *nm1, char *nm2){ int infd, outfd, nread; char buffer[BUFSIZ];

if ((infd = open(nm1,O_RDONLY))< 0) return -1; if ((outfd = creat(nm2,0644)) < 0){ close(infd); return -2; } :

continued


while ((nread = read(infd, buffer,BUFSIZ)) > 0) { if (write(outfd, buffer, nread) < nread) { close(infd); close(outfd); return -3; /* write error */ } } close(infd); close(outfd); return 0;}


copyfile()copyfile() & & BUFSIZBUFSIZ

BUFSIZBUFSIZ Real timeReal time User timeUser time System timeSystem time11 3:42.83:42.8 4.14.1 3:24.83:24.86464 0:27.30:27.3 0.00.0 0:05.10:05.1511511 0:24.00:24.0 0.00.0 0:01.90:01.9512512 0:22.30:22.3 0.00.0 0:01.00:01.0513513 0:25.10:25.1 0.00.0 0:02.40:02.440964096 0:13.30:13.3 0.00.0 0:00.90:00.981928192 0:12.90:12.9 0.00.0 0:01.10:01.1

Most saving is made by reducing no. of system Most saving is made by reducing no. of system calls.calls.

$ time copyfile


12. 12. lseek()lseek() #include <sys/types.h>#include <sys/types.h>#include <unistd.h>#include <unistd.h>long int lseek(int fd, long int offset, long int lseek(int fd, long int offset,

int start); int start);

Change position of R-W pointer: return new Change position of R-W pointer: return new file offset if ok, -1 on error.file offset if ok, -1 on error.

Start names (and numbers):Start names (and numbers):– SEEK_SETSEEK_SET 00 start of filestart of file– SEEK_CURSEEK_CUR 11 current R-W pointer posncurrent R-W pointer posn– SEEK_ENDSEEK_END 22 end of fileend of file


ExamplesExamples

:fd = open(filename, O_RDWR);lseek(fd, 0L, SEEK_END);write(fd, buffer, BUFSIZ);:

/* get filesize in bytes */long int filesize;filesize = lseek(fd, 0L, SEEK_END);


13. Hotel Guests13. Hotel Guests

The The guestsguests file has the format: file has the format:

. . . . .

name of guest, ending in '\n';always 41 chars

1 2 5000

. . . . .

3


TaskTask

Write the function:Write the function:int get_guest(int fd, int room_num, int get_guest(int fd, int room_num,

char char *name);*name);

Return 1 if the guest in Return 1 if the guest in room_numroom_num is found is found

((namename is also bound); 0 otherwise. is also bound); 0 otherwise.


CodeCode

#include <stdio.h>#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>#include <unistd.h>

#define NAMELEN 41#define NROOMS 5000

:

continued


int main(){ int fd, i; char name[NAMELEN];

if ((fd = open(“guests”,O_RDONLY))== -1){ fprintf(stderr,“Cannot read guests\n”); exit(1); } for (i=1; i <= NROOMS; i++) if (get_guest(fd, i, name) == 1) printf(“Room %d, %s\n”, i, name); else printf(“Error in room %d\n”, i); return 0;}

continued


int get_guest(int fd, int rnum, char *nm){ long int offset; int nread;

offset = (rnum-1)*NAMELEN; if (lseek(fd, offset, SEEK_SET) < 0) return 0; nread = read(fd, nm, NAMELEN); if (nread < NAMELEN) return 0; /* read() error */ else nm[nread-1] = ‘\0’; /* overwrite \n */ return 1;}


14. 14. unlink()unlink()

#include <unistd.h>#include <unistd.h>int unlink(char *pathname);int unlink(char *pathname);

Remove a file: return 0 of ok, -1 on error.Remove a file: return 0 of ok, -1 on error.– e.g.e.g. unlink(“/tmp/tmpfile”);unlink(“/tmp/tmpfile”);

Must have permissions to execute and write Must have permissions to execute and write in the file’s directoryin the file’s directory– for the for the cdcd and change to the contents and change to the contents


15. 15. fcntl()fcntl()

#include <sys/types.h>#include <sys/types.h>#include <unistd.h>#include <unistd.h>#include <fcntl.h>#include <fcntl.h>int fcntl(int fd, int cmd, int arg); int fcntl(int fd, int cmd, int arg);

Used to view/alter the behaviour of the file.Used to view/alter the behaviour of the file.

Two (of 10) Two (of 10) cmdcmd values: values:F_GETFLF_GETFL get current status flagsget current status flags

F_SETFLF_SETFL set a status flagset a status flag


Some File Status Flags for Some File Status Flags for fcntl()fcntl()

FlagFlag DescriptionDescription

O_RDONLYO_RDONLY Open for reading only (0).Open for reading only (0).O_WRONLYO_WRONLY Open for writing only (1).Open for writing only (1).O_RDWRO_RDWR Open for reading and writing (2).Open for reading and writing (2).

O_APPENDO_APPEND Append on each write.Append on each write.::


Using Using O_ACCMODEO_ACCMODE

O_RDONLYO_RDONLY, , O_WRONLYO_WRONLY and and O_RDWRO_RDWR are are not separate bits (e.g. not separate bits (e.g. notnot 0, 2, 4), and so 0, 2, 4), and so their values can be hard to test.their values can be hard to test.

One solution is to use the One solution is to use the O_ACCMODEO_ACCMODE mask.mask.


:int accmode, val;

if ((val = fcntl(fd, F_GETFL, 0)) < 0) printf(“Some error\n”);else { accmode = val & O_ACCMODE; if (accmode == O_RDONLY) printf(“read”); else if (accmode == O_WRONLY) printf(“write”); else if (accmode == O_RDWR) printf(“read write”); else printf(“access mode error”);

if (val & O_APPEND) printf(“, append”);:


Using Using F_SETFLF_SETFL

Main use is to change Main use is to change O_APPENDO_APPEND::

if (fcntl(fd, F_SETFL, O_APPEND) < 0) printf(“Setting append failed\n”);else ...


16. Duplicating File Descriptors16. Duplicating File Descriptors

#include <unistd.h>int dup(int fd);

Make a Make a newnew file descriptor ( file descriptor (fd2fd2) refer to ) refer to the same file as the same file as fdfd::

fd2 = dup(fd)

Useful for Useful for concurrentconcurrent programming, where programming, where two processes refer to the same file.two processes refer to the same file.


Duplicating DiagramDuplicating Diagram

filefd 3

fd2 = dup(fd);

filefd 3

fd24


Another ApproachAnother Approach

#include <unistd.h>int dup2(int fd, int fd2);

Make the Make the existingexisting fd2fd2 descriptor refer to descriptor refer to the same file as the same file as fdfd. .

If If fd2fd2 is already open, close its file before is already open, close its file before redirecting it.redirecting it.


Main use is for redirecting Main use is for redirecting stdiostdio::dup2(fd, 0)dup2(fd, 0)

Same as the ‘hack’:Same as the ‘hack’:close(0);close(0);dup(fd);dup(fd);


dup2(fd,0) Diagramdup2(fd,0) Diagram

filefd

program

Before After

filefd

program

stdin0


17. File Pointers17. File Pointers

Connect a file pointer to Connect a file pointer to fdfd’s file with:’s file with:

#include <stdio.h>#include <stdio.h>FILE *fdopen(int fd, char *mode);FILE *fdopen(int fd, char *mode);

modemode is usual thing: “ is usual thing: “rr”, “”, “w”w”, “, “rbrb”, etc.”, etc.

Useful for doing formatted I/O on top of Useful for doing formatted I/O on top of pipes and network comunication.pipes and network comunication.

240-491 Adv. UNIX: lowIO/161 Advanced UNIX v Objectives –look at low-level operations for handling...

Documents

Transcript of 240-491 Adv. UNIX: lowIO/161 Advanced UNIX v Objectives –look at low-level operations for handling...