Client/Server Distributed Systems

240-322 Cli/Serv.: low IO/4 1

Client/Server Distributed SystemsClient/Server Distributed Systems

ObjectivesObjectives– look at low-level operations for handling look at low-level operations for handling

filesfiles

240-322, Semester 1, 2005-2006

4. Low-level File I/O


OverviewOverview

1. Basic Operations1. Basic Operations

2. Why use Low-level Operations?2. Why use Low-level Operations?

3. A Simple Example3. A Simple Example

4. File Descriptors4. File Descriptors

5. File Permissions Again5. File Permissions Again

6. 6. open()open()

7. 7. creat()creat()

continued


8. 8. close()close()

9. 9. read()read()

10. 10. write()write()

11. 11. copyfile.ccopyfile.c

12. 12. lseek()lseek()

13. File Pointers13. File Pointers


1. Basic Operations1. Basic Operations

NameName MeaningMeaningopen()open() Opens a file for reading or Opens a file for reading or writing.writing.creat()creat() Creates an empty file for writing.Creates an empty file for writing.close()close() Closes an open file.Closes an open file.read()read() Reads data from a file.Reads data from a file.write()write() Writes data to a file.Writes data to a file.lseek()lseek() Moves to a specified byte in the file.Moves to a specified byte in the file.unlink()unlink() Removes a file.Removes a file.


Information on OperationsInformation on Operations

apropos <name>apropos <name>man -k <name>man -k <name>– e.g. e.g. apropos openapropos open

man <name>man <name>man <section number> <name>man <section number> <name>– e.g.e.g. man 2 openman 2 open

Look in Look in /usr/include/usr/include


2. Why use Low-level Operations?2. Why use Low-level Operations?

No buffering of I/ONo buffering of I/O– useful for network programming and reading/writing useful for network programming and reading/writing

to certain peripherals (e.g. tape drives)to certain peripherals (e.g. tape drives) No conversion of I/ONo conversion of I/O

– no transformation of input to integers, floats, etc.;no transformation of input to integers, floats, etc.;output can be any byte sequenceoutput can be any byte sequence

Can be used as building blocks for more Can be used as building blocks for more complex I/O.complex I/O.


3. A Simple Example3. A Simple Example (oprd.c)(oprd.c)

#include <stdio.h>#include <sys/types.h>#include <sys/stat.h>#include <fcntl.h>#include <unistd.h>

int main(){ int fd, nread; char buf[1024];

if ((fd = open("data-file", O_RDONLY)) == -1) { perror("open"); exit(-1); }

:


nread = read(fd, buf, 1024); close(fd); buf[nread-1] = '\0'; /* so can print */ printf("buf: %s\nnread: %d\n", buf, nread); return 0;}


4. File Descriptors4. File Descriptors

A file descriptor is a small, non-negative A file descriptor is a small, non-negative integer.integer.

A file descriptor identifies an open file to A file descriptor identifies an open file to the low-level operations.the low-level operations.

Standard descriptors:Standard descriptors:00 represents represents stdinstdin11 stdoutstdout22 stderrstderr


5. File Permissions5. File Permissions

One way of using One way of using chmodchmod is with letters to is with letters to represent permissions:represent permissions:

e.g.e.g. chmod a+r filechmod a+r file

Most low-level operations (and Most low-level operations (and chmodchmod) ) represent permissions as octal values:represent permissions as octal values:

e.g.e.g. chmod 0644 filechmod 0644 file


File Permissions as OctalsFile Permissions as Octals

OctalOctal Symbolic Meaning of PermissionSymbolic Meaning of Permission

04000400 r-- --- ---r-- --- ---02000200 -w- --- ----w- --- ---01000100 --x --- -----x --- ---00400040 --- r-- ------ r-- ---00200020 --- -w- ------ -w- ---00100010 --- --x ------ --x ---00040004 --- --- r----- --- r--00020002 --- --- -w---- --- -w-00010001 --- --- --x--- --- --x


ExampleExample

Add octal values together to get the Add octal values together to get the complete file permission:complete file permission:

OctalOctal MeaningMeaning04000400 r-- --- ---r-- --- ---02000200 -w- --- ----w- --- ---00400040 ++ + + --- r-- ------ r-- ---00040004 --- --- r----- --- r--======== ======================06440644 rw- r-- r--rw- r-- r--

Use in Use in chmodchmod::chmod 0644 filechmod 0644 file


6. 6. open()open()

#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>

int open(char *pathname, int flagint open(char *pathname, int flag /*, mode_t mode /*, mode_t mode

*/ );*/ );

Return file descriptor if ok, -1 for errorReturn file descriptor if ok, -1 for error


Some Some open()open() Flags FlagsO_RDONLYO_RDONLY Open file for reading only.Open file for reading only.O_WRONLYO_WRONLY Open file for writing only.Open file for writing only.O_RDWRO_RDWR Open file for both reading & writing.Open file for both reading & writing.

Possibly Possibly combinedcombined with: with:O_APPENDO_APPEND Append to file when writing.Append to file when writing.

O_CREATO_CREAT Create file if is does not exist Create file if is does not exist (requires the 3rd (requires the 3rd modemode argument).argument).

O_EXCLO_EXCL Return -1 error if file is to be createdReturn -1 error if file is to be createdand already exists.and already exists.


Using Bitwise ORUsing Bitwise OR

open() open() flags can be combined with the flags can be combined with the bitwise OR operator ‘|”:bitwise OR operator ‘|”:

fd = open(file, O_WRONLY | O_APPEND);fd = open(file, O_WRONLY | O_APPEND);

Now each Now each write(fd, buf, BUFSIZE)write(fd, buf, BUFSIZE)

means append to the end of the file.means append to the end of the file.


7. 7. creat()creat()

#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>

int creat(char *pathname, mode_t mode);int creat(char *pathname, mode_t mode);

Creates a new file, or truncates an old one. Creates a new file, or truncates an old one.

Return file descriptor if ok, -1 on error.Return file descriptor if ok, -1 on error.


modemode gives access permission of resulting gives access permission of resulting new file:new file:

06440644 (rw- r-- r--)(rw- r-- r--)

modemode only has meaning if the file is new. only has meaning if the file is new.


Create a new fileCreate a new file

#include <stdio.h>#include <stdio.h>#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>

void main()void main(){ { int fd; int fd; if ((fd = if ((fd = creatcreat(“new-file”,0644))== -1){(“new-file”,0644))== -1){ printf(“cannot create new-file\n”); printf(“cannot create new-file\n”); exit(1); exit(1); } } /* rest of program */ /* rest of program */}}


Replacing Replacing creat() creat() with with open()open()

creat()creat() only opens a file for writing; to only opens a file for writing; to read it the file must be closed and opened for read it the file must be closed and opened for reading.reading.

creat() creat() can be replaced by can be replaced by open()open()::

open(“new-file”, open(“new-file”, O_WRONLY | O_CREAT | O_TRUNC, 0644)O_WRONLY | O_CREAT | O_TRUNC, 0644)


Other VersionsOther Versions

creat()creat() version which can read and write: version which can read and write:

open(“new-file”, open(“new-file”, O_RDWR | O_CREAT | O_TRUNC, 0644)O_RDWR | O_CREAT | O_TRUNC, 0644)

Avoid truncation of old file (append instead):Avoid truncation of old file (append instead):

open(“new-file”, open(“new-file”, O_WRONLY|O_CREAT|O_APPEND, 0644) O_WRONLY|O_CREAT|O_APPEND, 0644)


One User at a TimeOne User at a Time

Create Create locklock if it doesn’t already exist, otherwise if it doesn’t already exist, otherwise fail (and return -1):fail (and return -1):

fd = open(“lock”, fd = open(“lock”, O_WRONLY|O_CREAT|O_EXCL, 0644);O_WRONLY|O_CREAT|O_EXCL, 0644);

Use Use locklock to protect access to another file to protect access to another file– open open ““accountsaccounts”” only if only if locklock can be opened can be opened– after processing after processing ““accountsaccounts””, delete , delete locklock


8. 8. close()close()

#include <unistd.h>#include <unistd.h>int close(int fd);int close(int fd);

Close a file: return 0 if ok, -1 on error.Close a file: return 0 if ok, -1 on error.

Useful since the number of open files is Useful since the number of open files is limited (~20)limited (~20)


9. 9. read()read()

#include <sys/types.h>#include <sys/types.h>#include <unistd.h>#include <unistd.h>int read(int fd, void *buffer, int read(int fd, void *buffer,

unsigned int nbytes);unsigned int nbytes);

Tries to read Tries to read nbytesnbytes into the into the bufferbuffer array array– no conversion of bytes (characters)no conversion of bytes (characters)– may read less than specified amount (e.g. at end)may read less than specified amount (e.g. at end)

Returns the no. of bytes read, 0 if end of file, Returns the no. of bytes read, 0 if end of file, -1 on error.-1 on error.


The Read-Write PointerThe Read-Write Pointer

The R-W Pointer records the position of The R-W Pointer records the position of the next byte in the file to be read (or the next byte in the file to be read (or written).written).

It is implicit (hidden) in each use of It is implicit (hidden) in each use of read() read() and and write()write()


Count Characters (countchars.c)Count Characters (countchars.c)

#include <stdio.h>#include <stdio.h>#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>#include <unistd.h>#include <unistd.h>

#define SIZE 512#define SIZE 512

int main()int main(){{ char buffer[SIZE]; char buffer[SIZE]; int fd, j; int fd, j; long total = 0; long total = 0;

::

continued


if ((fd = open(“data-file”, O_RDONLY)) == -1){if ((fd = open(“data-file”, O_RDONLY)) == -1){ perror("open"); perror("open"); exit(1); exit(1); } }

while((j = read(fd, buffer, SIZE)) > 0) { while((j = read(fd, buffer, SIZE)) > 0) { putchar('.'); putchar('.'); total += j; total += j; } } putchar('\n'); putchar('\n'); printf(“total chars: %ld\n”, total); printf(“total chars: %ld\n”, total); close(fd); close(fd); return 0; return 0;}}


UNIX systems are often configured to UNIX systems are often configured to read/write in blocks of 512 or 1024 bytes read/write in blocks of 512 or 1024 bytes (the (the diskdisk blocking factorblocking factor).).

Can use Can use BUFSIZBUFSIZ from from stdio.hstdio.h which which contains a default disk blocking factor.contains a default disk blocking factor.


10. 10. write()write()

#include <unistd.h>#include <unistd.h>int write(int fd, void *buffer, int write(int fd, void *buffer,

unsigned int nbytes);unsigned int nbytes);

Returns number of bytes written if ok, -1 on Returns number of bytes written if ok, -1 on error.error.

write()write() starts at the current R-W pointer starts at the current R-W pointer position, and pointer is moved as file is written.position, and pointer is moved as file is written.


ExampleExample

::int fd;int fd;char header1[512], header2[512];char header1[512], header2[512]; : :if ((fd = creat(“foo”, 0644)) != -1)if ((fd = creat(“foo”, 0644)) != -1) : : write(fd, header1, 512); write(fd, header1, 512); write(fd, header2, 512); write(fd, header2, 512); : :


11. 11. copyfile.ccopyfile.c#include <stdio.h>#include <stdio.h>#include <sys/types.h>#include <sys/types.h>#include <sys/stat.h>#include <sys/stat.h>#include <fcntl.h>#include <fcntl.h>#include <unistd.h>#include <unistd.h>

int copyfile(char *nm1, char *nm2);int copyfile(char *nm1, char *nm2);

void main()void main(){ { printf("result: %d\n", printf("result: %d\n", copyfile(“test.in”, “test.out”) ); copyfile(“test.in”, “test.out”) );}}

continued


int copyfile(char *nm1, char *nm2)int copyfile(char *nm1, char *nm2){{ int infd, outfd, nread; int infd, outfd, nread; char buffer[BUFSIZ]; char buffer[BUFSIZ];

if ((infd = open(nm1,O_RDONLY))== -1) if ((infd = open(nm1,O_RDONLY))== -1) return -1; return -1; if ((outfd = creat(nm2,0644)) == -1){ if ((outfd = creat(nm2,0644)) == -1){ close(infd); close(infd); return -2; return -2; } } ::

continued


while ((nread = read(infd, buffer,while ((nread = read(infd, buffer,BUFSIZ)) > 0) {BUFSIZ)) > 0) {

if (write(outfd, buffer, nread) if (write(outfd, buffer, nread) < nread) {< nread) {

close(infd); close(infd); close(outfd); close(outfd); return -3; /* write error */ return -3; /* write error */ } } } } close(infd); close(infd); close(outfd); close(outfd); return 0; return 0;}}


copyfile()copyfile() & & BUFSIZBUFSIZ BUFSIZBUFSIZ Real timeReal time User timeUser time System timeSystem time

11 3:42.83:42.8 4.14.1 3:24.83:24.86464 0:27.30:27.3 0.00.0 0:05.10:05.1511511 0:24.00:24.0 0.00.0 0:01.90:01.9512512 0:22.30:22.3 0.00.0 0:01.00:01.0513513 0:25.10:25.1 0.00.0 0:02.40:02.440964096 0:13.30:13.3 0.00.0 0:00.90:00.981928192 0:12.90:12.9 0.00.0 0:01.10:01.1

Most saving is made by reducing no. of system Most saving is made by reducing no. of system calls.calls.


12. 12. lseek()lseek() #include <sys/types.h>#include <sys/types.h>#include <unistd.h>#include <unistd.h>long int lseek(int fd, long int offset, long int lseek(int fd, long int offset,

int start); int start);

Change position of R-W pointer: return new Change position of R-W pointer: return new file offset if ok, -1 on error.file offset if ok, -1 on error.

Start names (and numbers):Start names (and numbers):– SEEK_SETSEEK_SET 00 start of filestart of file– SEEK_CURSEEK_CUR 11 current R-W pointer positioncurrent R-W pointer position– SEEK_ENDSEEK_END 22 end of fileend of file


Code FragmentsCode Fragments

::fd = open(filename, O_RDWR);fd = open(filename, O_RDWR);lseek(fd, 0lseek(fd, 0LL, SEEK_END);, SEEK_END);write(fd, buffer, BUFSIZ);write(fd, buffer, BUFSIZ);::

/* get filesize in bytes *//* get filesize in bytes */long int filesize;long int filesize;filesize = lseek(fd, 0filesize = lseek(fd, 0LL, SEEK_END);, SEEK_END);


13. File Pointers13. File Pointers

Connect a file pointer to Connect a file pointer to fdfd’s file with:’s file with:

#include <stdio.h>#include <stdio.h>FILE *fdopen(int fd, char *mode);FILE *fdopen(int fd, char *mode);

modemode is usual thing: “ is usual thing: “rr”, “”, “w”w”, “, “rbrb”, etc.”, etc.

Useful for doing formatted I/O on top of Useful for doing formatted I/O on top of pipes and network comunication.pipes and network comunication.


Code FragmentCode Fragment

int fd = open("data", O_RDONLY);int fd = open("data", O_RDONLY);FILE *fp = FILE *fp = fopenfopen(fd, "r");(fd, "r");int num;int num;fscanf(fp, "%d", &num);fscanf(fp, "%d", &num);

Client/Server Distributed Systems

Documents

Transcript of Client/Server Distributed Systems