Unix System Interface Programming Part 6.1 – System Interface Overview
description
Transcript of Unix System Interface Programming Part 6.1 – System Interface Overview
Unix System Interface Unix System Interface ProgrammingProgramming
Part 6.1 – System Interface Overview
Prepared by Xu Zhenya( [email protected] )
Draft – Xu Zhenya( 2002/10/01 )
Rev1.0 – Xu Zhenya( 2002/10/10 )
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
AgendaAgenda 1. System Interface Programming Standards1. System Interface Programming Standards 2. how a “C” program is started?2. how a “C” program is started? 3. System call and library functions3. System call and library functions 4. String functions4. String functions 5. Error handling5. Error handling 6. Memory management6. Memory management 7. man & api7. man & api 8. Debugging programs8. Debugging programs 9. Summary9. Summary Appendix: Appendix:
Understanding O.S. KernelUnderstanding O.S. Kernel Recommanded BooksRecommanded Books
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (1)Standards (1)
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (2)Standards (2) X/Open Common Applications Environment (CAE) X/Open Common Applications Environment (CAE)
Portability Guide Issue 3 (XPG3) and Issue 4 (XPG4)Portability Guide Issue 3 (XPG3) and Issue 4 (XPG4) SUS( Single UNIX Specification ), SUSv2SUS( Single UNIX Specification ), SUSv2 XNS4: Networking Services Issue4XNS4: Networking Services Issue4
ILP32 and LP64 programming environmentsILP32 and LP64 programming environments Notes:Notes:
1. The developers of SVID3( UNIX Systems Laboratories) are no 1. The developers of SVID3( UNIX Systems Laboratories) are no longer in business, and this specification defers to POSIX and longer in business, and this specification defers to POSIX and X/Open CAE. X/Open CAE.
2. Utilities: conflictions with historical Solaris utility2. Utilities: conflictions with historical Solaris utility Included /usr/xpg4/bin into PATH, and before other utilities.Included /usr/xpg4/bin into PATH, and before other utilities.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (3)Standards (3)
XPG.3
POSIX.2
POSIX.1b
POSIX.1c
BSD Interfaces
SUS( XPG4v2)
ISO-C Amendement 1
XNS4
SUSV2XNS5
LP64-clean derivative of XNS4
POSIX Standard
XPG.4
POSIX.1
SVR4
Solaris8
XPG.3
SUSV2
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (4)Standards (4) POSIX inlcues the following standards:POSIX inlcues the following standards:
1003.1 :1003.1 : System Interface( POSIX.1 )System Interface( POSIX.1 )
POSIX.1 has adopted virtually all ANSI C library calls. POSIX.1 has adopted virtually all ANSI C library calls.
However, POSIX.1 has not adopted include operations on wide However, POSIX.1 has not adopted include operations on wide characters and multi-byte characters( as used for Chinese ).characters and multi-byte characters( as used for Chinese ).
1003.1b : Real-time extensions1003.1b : Real-time extensions
1003.1c: User-level threads( pthreads, POSIX threads library )1003.1c: User-level threads( pthreads, POSIX threads library )
1003.1g: Networking standards1003.1g: Networking standards
1003.2: Shells and utilities1003.2: Shells and utilities
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (5)Standards (5) Feature test macrosFeature test macros
__EXTENSIONS__: the application with access to all interfaces and __EXTENSIONS__: the application with access to all interfaces and headers not in conflict with the specified standard. headers not in conflict with the specified standard.
Specification Compiler/Flags Feature Test Macros
ANSI/ISO C c89 none SVID3 cc -Xt nonePOSIX.1-1990 c89 _POSIX_SOURCEPOSIX.1-1990 and POSIX.2-1992 C-Language Bindings
Option
c89
_POSIX_SOURCE andPOSIX_C_SOURCE=2
POSIX.1b-1993 c89 _POSIX_C_SOURCE=199309L
POSIX.1c-1996 c89 _POSIX_C_SOURCE=199506L
CAE XPG4 c89 _XOPEN_SOURCE and _XOPEN_VERSION=4
SUSv2(includes XNS5) c89 _XOPEN_SOURCE=500
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Standards (6)Standards (6)
#if ( __STDC__ == 0 && !defined(_POSIX_C_SOURCE) && \!defined(_XOPEN_SOURCE)) || \
(defined(_XOPEN_SOURCE) && _XOPEN_VERSION - 0 >= 4) || \ defined(__EXTENSIONS__)
Case StudyCase Study
Solaris: /usr/include/sys/feature_test.hSolaris: /usr/include/sys/feature_test.h Linux: /usr/include/features.hLinux: /usr/include/features.h
Compiler options Compiler options - see C/C++ Manual: for Gnu C/C++, see the links page. - see C/C++ Manual: for Gnu C/C++, see the links page.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How a C program is started? (1)How a C program is started? (1)
kernel
Libc_init
Main
UserFunctions
exit
Functionsregistered by
atexit()
Libio_close
exec
Cal
l
Return
Cal
l
Return
_exi
t
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How a C program is started? (2)How a C program is started? (2)Address Kbytes Resident Shared Private Permissions Mapped File
08046000 8 8 - 8 read/write/exec [ stack ]
08050000 4 4 - 4 read/exec a.out
08060000 4 4 - 4 read/write/exec a.out
08061000 8 8 - 8 read/write/exec [ heap ]
DFB00000 4 4 - 4 read/write/exec [ anon ]
DFB10000 528 520 464 56 read/exec libc.so.1
DFBA4000 24 24 - 24 read/write/exec libc.so.1
DFBAA000 8 8 - 8 read/write/exec libc.so.1
DFBB0000 4 4 - 4 read/exec libdl.so.1
DFBC0000 116 116 112 4 read/exec ld.so.1
DFBED000 8 8 - 8 read/write/exec ld.so.1
DFBEF000 4 4 - 4 read/write/exec ld.so.1
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How a C program is started? (3)How a C program is started? (3) LINK EDITOR MEMORY MAP (ELF): ld –m –o who who.o
Output section Virtual address size
.interp 80500d4 11
.hash 80500e8 90
.dynsym 8050178 110
.dynstr 8050288 bb
.SUNW_ve 8050344 20
.rel.bss 8050364 10
.rel.plt 8050374 20
.plt 8050394 50
.text 80503e8 d5
.rodata1 80504c0 43
.got 8060504 1c
.dynamic 8060520 a8
.bss 80605c8 3c8
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How a C program is started? (4)How a C program is started? (4) Command line arguments &
environment variables => Limits on the length of
arguments and environment
sysconf( _SC_ARG_MAX )
exec(): copy the arguments list and environmental variables after setting the user’s stack ( crt_init() )
getenv, setenv() & putenv() Environmental table
extern char ** environ;
int main( int argc, char *argv[], char *env[] );
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 1 )System calls & library calls ( 1 )
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 2 )System calls & library calls ( 2 ) 1. malloc & sbrk1. malloc & sbrk 2. time & date: time()2. time & date: time() 3. We can conclude that: 3. We can conclude that:
A few of syscalls (256 or less)A few of syscalls (256 or less) Apps. -> syscall / libc(->syscall)Apps. -> syscall / libc(->syscall)
Making a syscallMaking a syscall When we make a system call When we make a system call parameter that is a pointerparameter that is a pointer to a data to a data
object, we must allocate space for the object and pass its address in object, we must allocate space for the object and pass its address in the call. Example: time( time_t * p_time )the call. Example: time( time_t * p_time )
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 3 )System calls & library calls ( 3 ) Make a library callMake a library call
For a return value or parameter declared as a pointer type, there For a return value or parameter declared as a pointer type, there are three possibilities:are three possibilities: Don’t Allocate space:Don’t Allocate space:
The same as the system callsThe same as the system calls
Allocate space statically: Allocate space statically: Copy the object pointed by the address: ctime(3C)Copy the object pointed by the address: ctime(3C)
Allocate space dynamically: Allocate space dynamically: MUST remember to free the space: strdup()MUST remember to free the space: strdup()
Read the man pages carefully.Read the man pages carefully.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 4 )System calls & library calls ( 4 ) 12 time( tptr ); 0x08050820: main : pushl %ebp 0x08050821: main+0x0001: movl %esp,%ebp 0x08050823: main+0x0003: subl $8,%esp /* time_t *tptr */ 0x08050826: main+0x0006: movl -8(%ebp),%eax 0x08050829: main+0x0009: pushl %eax 0x0805082a: main+0x000a: call time [PLT] <0x80506dc> 0x0805082f: main+0x000f: addl $4,%esp
13 printf( "Machine time in sec = %d\n", *tptr ); 0x08050832: main+0x0012: movl -8(%ebp),%eax 0x08050835: main+0x0015: movl (%eax),%eax 0x08050837: main+0x0017: pushl %eax 0x08050838: main+0x0018: pushl $0x8050900 0x0805083d: main+0x001d: call printf [PLT] <0x80506ec> 0x08050842: main+0x0022: addl $8,%esp
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 5 )System calls & library calls ( 5 )1. static char cbuf[26];
2. char *3. ctime( t )4. const time_t *t;5. {6. return ( asctime( localtime( t ) ) );7. }
8. char *9. asctime( t )10. const struct tm *t;11. {12. register char *cp;13. 14. /* …… */15.
16. return( cbuf );17. }
Ctime.c
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 6 )System calls & library calls ( 6 )struct sysent { char sy_narg; /* total number of arguments */#ifdef _LP64 unsigned short sy_flags; /* various flags as defined below */#else unsigned char sy_flags; /* various flags as defined below */#endif int (*sy_call)(); /* argp, rvalp-style handler */ krwlock_t *sy_lock; /* lock for loadable system calls */ int64_t (*sy_callc)(); /* C-style call hander or wrapper */};
#define NSYSCALL 256 /* number of system calls */
extern struct sysent sysent[];#ifdef _SYSCALL32_IMPLextern struct sysent sysent32[];#endif
/* See /usr/include/sys/systm.h & /etc/name_to_sysnum */
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 7 )System calls & library calls ( 7 ) fork – fork –
__asm__(" __asm__(" movl $0x2, %eax // SYS_fork, see /etc/name_to_synum movl $0x2, %eax // SYS_fork, see /etc/name_to_synum lcall $0x27, $0x0 lcall $0x27, $0x0 "); ");
Solaris Kernel Solaris Kernel 1. trap into the kernel1. trap into the kernel 2. enter the common trap handler2. enter the common trap handler
Save context : the pointer to CPU structure, the return addressSave context : the pointer to CPU structure, the return address Save sycall arguments into LWP_CBSave sycall arguments into LWP_CB if ( t_pre_sys ) the do the pre-action /* if ( t_pre_sys ) the do the pre-action /* truss, micro-statetruss, micro-state */ */ Call sysent[ 2 ].sy_callc() – sys_fork(); return;Call sysent[ 2 ].sy_callc() – sys_fork(); return; Check if here is a signal? Check if here is a signal? If ( t_post_sys) then post_actionIf ( t_post_sys) then post_action Set the returned value or errno Set the returned value or errno Return;Return;
Fast System call – gethrtime(), gethrvtime() & gettimeofday()Fast System call – gethrtime(), gethrvtime() & gettimeofday() Performance: using the registersPerformance: using the registers
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
System calls & library calls ( 8 )System calls & library calls ( 8 )System CallSystem Call Library CallLibrary Call
Section 2 of the man pagesSection 2 of the man pages Section 3 of the man pagesSection 3 of the man pages
Never allocates space for parametersNever allocates space for parameters 3 possibilities3 possibilities
Executes in the system mode ( the Executes in the system mode ( the kernel mode )kernel mode )
Executes in the user modeExecutes in the user mode
When a failure occurs: When a failure occurs:
Return -1;Return -1;
Set errno( perror() )Set errno( perror() )
When a failure occures:When a failure occures:
Often returns NULL ( See man)Often returns NULL ( See man)
Can set errno ( See man )Can set errno ( See man )
Signal can interrupt a system call. Signal can interrupt a system call. ( restartable? )( restartable? )
Async-signal safe issuesAsync-signal safe issues
Misc: Tools used to trace system calls made by a process: 1. It executes the specified command and produces a trace of the system calls
performed by theat command, the signals the command receives, and the machie faults the command incurs.
2. Solaris: truss3. Linux: strace
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
String FunctionsString Functions 1. textbook, p129 – Table6-21. textbook, p129 – Table6-2 2. Parsing an input string: 2. Parsing an input string:
strpbrk, strstr, strspn, strcspn, strtok, index, rindexstrpbrk, strstr, strspn, strcspn, strtok, index, rindex 3. Memory functions: 3. Memory functions:
memcpy, memmove, memccpy, memchr, memcmp, memsetmemcpy, memmove, memccpy, memchr, memcmp, memset Used to copy structures. But some compilers support structures assignement. Used to copy structures. But some compilers support structures assignement. Don’t use the memcmp() to compare data structures: paddingDon’t use the memcmp() to compare data structures: padding
4. String Conversion Functions: 4. String Conversion Functions: strtol, strtoll, atol, atoll, atoistrtol, strtoll, atol, atoll, atoi
5. Byte string functions:5. Byte string functions: bzero, bcopy, etcbzero, bcopy, etc
6. Notes: 6. Notes: Buffer overflow => security ( size_t count )Buffer overflow => security ( size_t count ) For programs in C++, we should use its native string manipulation class & For programs in C++, we should use its native string manipulation class &
methods.methods.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Dynamic Memory Allocation Dynamic Memory Allocation 1. Memory Allocation functions: 1. Memory Allocation functions:
malloc, free, realloc, calloc, allocamalloc, free, realloc, calloc, alloca Heap & stack( automatic storage class )Heap & stack( automatic storage class ) => Memory leaks: CASE Tools=> Memory leaks: CASE Tools => Alignment: malloc() & compiler supporting=> Alignment: malloc() & compiler supporting
2. Resource management2. Resource management Resource includes many things in our code: memory, file descriptors, locksResource includes many things in our code: memory, file descriptors, locks Resource management: allocate & free, lock & unlock, state management, etcResource management: allocate & free, lock & unlock, state management, etc Open issues: Open issues:
who should allocate and free them? Specially who free them? who should allocate and free them? Specially who free them? => answer: do allocate and free resources at the same level=> answer: do allocate and free resources at the same level
In multithreaded programs, very difficult!!!! => so be carefully.In multithreaded programs, very difficult!!!! => so be carefully. But here C++-class can be used to the basic unit to manage resources. But here C++-class can be used to the basic unit to manage resources.
In some situations, we have to implement our own memory pool for the central In some situations, we have to implement our own memory pool for the central management: performance, debugging, etc. management: performance, debugging, etc.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Error Handling ( 1 )Error Handling ( 1 ) 1. Error handling functions: 1. Error handling functions:
perror, strerrorperror, strerror
Notes: Notes:
The return value & “errno”The return value & “errno”
System calls & Library functionsSystem calls & Library functions When system calls fail they always (1) return -1; (2) set errno.When system calls fail they always (1) return -1; (2) set errno.
Library calls might set error when an error occurs. Check the man pages.Library calls might set error when an error occurs. Check the man pages.
2. terminating a process: 2. terminating a process: exit(), abort()exit(), abort()
Signals => terminate abnormally.Signals => terminate abnormally.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Error Handling ( 2 )Error Handling ( 2 ) 3. Discussion about error handling3. Discussion about error handling
1. we should handle errors at high-level, and check errors in the low-1. we should handle errors at high-level, and check errors in the low-level => but how to return the error information? level => but how to return the error information?
In Unix, return -1, and set errnoIn Unix, return -1, and set errno
If there are more information than one integer value? If there are more information than one integer value? In C++, pair in STL is one choice. In C++, pair in STL is one choice.
In multithreaded programs, TLS is one choice. In multithreaded programs, TLS is one choice.
2. carefully using exception in C++: 2. carefully using exception in C++:
Performance issues: Performance issues:
Control sequence, and our code are filled with try. Control sequence, and our code are filled with try.
Memory may not be freed. ( in java, GC can do it. Maybe it’s why Memory may not be freed. ( in java, GC can do it. Maybe it’s why Java uses exceptions everywhere? )Java uses exceptions everywhere? )
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Man Pages & API (1)Man Pages & API (1) 1. Xman, Answerbook2, etc1. Xman, Answerbook2, etc 2. Discussion:2. Discussion:
UML UML Syntax: Syntax:
Signature: internal & Signature: internal & external representationexternal representation
Type checkType check SemanticsSemantics
Parameters:value/referenceParameters:value/reference Resource?Resource?
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Man Pages & API (2)Man Pages & API (2)Corba-IDL: the famous example –ATM
interface BankServer { ::xaction HandleTransaction( inout ::xaction Transaction ); long BankID();};
interface Customers { exception CustomerException{ string s; };
// Get the data packet for a single customer any GetCustomers( in boolean metadata );
// Apply a delta packet to the customer table any ApplyCustomerUpdates( in any Delta, out long ErrorCount );};
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How to document our own code?(1)How to document our own code?(1) documenting documenting our assumptions, our approach, and our reasonsour assumptions, our approach, and our reasons for for
choosing the approach we did. choosing the approach we did. Donald KnuthDonald Knuth once observed that we should be able to read a well- once observed that we should be able to read a well-
written program just as we read a well-written book. written program just as we read a well-written book. "Self-Documenting Code," Chapter 19 by Steve McConnell"Self-Documenting Code," Chapter 19 by Steve McConnell
We also need to keep our We also need to keep our comments coordinated with the codecomments coordinated with the code. . Each function or method needs a sentence or two that clarifies the Each function or method needs a sentence or two that clarifies the
following information: following information: What the routine does What the routine does What assumptions the routine makes What assumptions the routine makes What each input parameter is expected to contain What each input parameter is expected to contain What each output parameter is expected to contain on success and What each output parameter is expected to contain on success and
failure failure Each possible return value Each possible return value
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How to document our own code?(2)How to document our own code?(2) Each part of the function that Each part of the function that isn't completely obviousisn't completely obvious from the from the
code needs a sentence or two that explains what it's doing. code needs a sentence or two that explains what it's doing. Any interesting algorithmAny interesting algorithm deserves a complete description. deserves a complete description. Any nontrivial bugsAny nontrivial bugs we've fixed in the code need to be we've fixed in the code need to be
commented with the bug number and a description of what we commented with the bug number and a description of what we fixed. fixed.
Well-placed Well-placed trace statements, trace statements, assertionsassertions, and good naming , and good naming conventions can also serve as good comments and provide conventions can also serve as good comments and provide excellent context to the code. excellent context to the code.
Comment as if we were going to be the one maintaining the code Comment as if we were going to be the one maintaining the code in five years. in five years.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
How to document our own code?(3)How to document our own code?(3) Case StudyCase Studyint32_tint32_t terminal_mngr::configure ( terminal_mngr::configure ( salsal__himkeyhimkey * * p_conf p_conf ) )
Read the configuration information from "p_conf", create terminals & configure them, Read the configuration information from "p_conf", create terminals & configure them, and add them into the and add them into the terminalterminal table. One example of the format of HiM table. One example of the format of HiM configuration is defined in conf/term1234_conf.orig. configuration is defined in conf/term1234_conf.orig.
the caller of this member method is "login" the caller of this member method is "login" executed in the main threadexecuted in the main thread
Parameters: Parameters: p_conf p_conf : sal_himkey : sal_himkey
[[inin] the pointer to the configuration inforamtion.] the pointer to the configuration inforamtion.
Returns: Returns: if successfully loading, then return 0, or a non-zero errval. if successfully loading, then return 0, or a non-zero errval.
Error CodesError Codes ERRID_TERMINAL_MNGR_CONF_NR_TERM ERRID_TERMINAL_MNGR_CONF_NR_TERM
invalid nr_terms in the configuration paratmersinvalid nr_terms in the configuration paratmers ERRID_TERMINAL_MNGR_CONF_NOSUBKEY ERRID_TERMINAL_MNGR_CONF_NOSUBKEY
no subkey for one no subkey for one terminalterminal ERRID_TERMINAL_MNGR_CONF_INVALIDTYPE ERRID_TERMINAL_MNGR_CONF_INVALIDTYPE
invalid type for one invalid type for one terminalterminal ERRID_TERMINAL_MNGR_CONF_CREATE_INSTANCE ERRID_TERMINAL_MNGR_CONF_CREATE_INSTANCE
fail to create the instance for a fail to create the instance for a terminalterminal
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Debugging programs (1)Debugging programs (1) 1. Debuggers on Unix ( Linux here )1. Debuggers on Unix ( Linux here )
Backend: ( gdb, read the page “links” to get details )Backend: ( gdb, read the page “links” to get details )
Frontend: insight, DDD, kdbgFrontend: insight, DDD, kdbg
Notes: Notes:
Commercial development environments: workshop for SolarisCommercial development environments: workshop for Solaris
Other CASE Tools: runtime memory check, etcOther CASE Tools: runtime memory check, etc
2. Suggestions – 2. Suggestions – Controlling the processControlling the process the debugger can answer all our debugging questions as long as we the debugger can answer all our debugging questions as long as we
ask it the right questions. ask it the right questions.
having one or more having one or more hypothesis in mindhypothesis in mind—something you want to prove —something you want to prove or disprove—before leveraging the debugger.or disprove—before leveraging the debugger.
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Debugging programs (2)Debugging programs (2)3. Debugging Process3. Debugging Process
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
SummarySummary In our programming practice, the following issues are the In our programming practice, the following issues are the
most important: most important: InterfaceInterface
Error handlingError handling
Resource management: ( memory )Resource management: ( memory )
Document code, add many Document code, add many ““assertassert”” into our code into our code
Control our debugging processControl our debugging process
refractoringrefractoring
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Exercise: Part6-1Exercise: Part6-1 Read the homework page to get details.Read the homework page to get details. Makefile, C/C++ Compilers, and the debugger(gdb)Makefile, C/C++ Compilers, and the debugger(gdb) Sample Output: Sample Output:
[upe@linux exercise1]$ ./shell[upe@linux exercise1]$ ./shell
myshell -> who am imyshell -> who am i
[0] : who[0] : who
[1] : am[1] : am
[2] : i[2] : i
myshell -> I am UPEmyshell -> I am UPE
[0] : I[0] : I
[1] : am[1] : am
[2] : UPE[2] : UPE
myshell -> exitmyshell -> exit
[upe@linux exercise1]$ [upe@linux exercise1]$
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Appendix - AAppendix - A Understanding O.S. Kernel – Process and its environmentUnderstanding O.S. Kernel – Process and its environment
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
1. a process is a 1. a process is a VMVM implemented by the kernel to run a implemented by the kernel to run a executable executable programprogram. .
2. define the VM – thinking about a computer2. define the VM – thinking about a computer Instruction set: the user-level instructions & system callsInstruction set: the user-level instructions & system calls Memory: address spaceMemory: address space I/O subsystem: file systemI/O subsystem: file system Interrupt: signal, asynchronous eventInterrupt: signal, asynchronous event Process-to-process communication: IPCProcess-to-process communication: IPC Misc: protection mechanisms, debugging, etcMisc: protection mechanisms, debugging, etc
3. executable programs – produced by the linker, map file3. executable programs – produced by the linker, map file Text, data segment( bbs, ro, etc ), stack, heapText, data segment( bbs, ro, etc ), stack, heap Image after loaded into the memory by exec()Image after loaded into the memory by exec()
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
4. the first step: Process control block ( 1 )4. the first step: Process control block ( 1 ) Hardware context: registers, flags, etc ( user & kernel stack pointer !!! )Hardware context: registers, flags, etc ( user & kernel stack pointer !!! ) Address space: text, data, stack, heapAddress space: text, data, stack, heap File system: hierarchical name management (current directory, umask, File system: hierarchical name management (current directory, umask,
etc), opened file descriptors( object references ) etc), opened file descriptors( object references ) Signal management: signal handlers, blocked & ignored, pendingSignal management: signal handlers, blocked & ignored, pending Current terminal: input and output textCurrent terminal: input and output text Identification: process IDIdentification: process ID
5. O.S. is a hardware resource manager for many VM. (2)5. O.S. is a hardware resource manager for many VM. (2) Scheduling class, priority management Scheduling class, priority management State management/control: the famous state machine State management/control: the famous state machine
6. relationship management ( 3 )6. relationship management ( 3 ) relationships among VMs: family( father and child ), process groups relationships among VMs: family( father and child ), process groups VM and the owner( users )/credentials: user’s ID, user’s group IDVM and the owner( users )/credentials: user’s ID, user’s group ID
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
7. it’s very important to understand the runtime scenario:7. it’s very important to understand the runtime scenario: How to enter the kernel? Interrupt, trap and exceptionHow to enter the kernel? Interrupt, trap and exception How to leave from the kernel? signal processing, schedulingHow to leave from the kernel? signal processing, scheduling Synchronization: Synchronization:
among processes: lock( mutex ), sleep/wakeupamong processes: lock( mutex ), sleep/wakeup the interrupt handlers and the remainder: the interrupt levelthe interrupt handlers and the remainder: the interrupt level
7. now we can further to a multiprocessors (SMP) (4)7. now we can further to a multiprocessors (SMP) (4) Threads & process: now two context (control block)Threads & process: now two context (control block) PCB should include threads management: PCB should include threads management:
threads list: threads list: CPU context, scheduling parameters and stack pointer (SP) CPU context, scheduling parameters and stack pointer (SP)
should move to the thread’s contextshould move to the thread’s context
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
8. Solaris’ thread and process model: 8. Solaris’ thread and process model: Process, the user-level thread, LWP and kernel threadProcess, the user-level thread, LWP and kernel thread Why choose this model:Why choose this model:
Flexibility and high-performanceFlexibility and high-performance Book, p7-10Book, p7-10
Concurrency and parallelConcurrency and parallel ULT: not require kernel resource, and fast context switch, user-level schedulingULT: not require kernel resource, and fast context switch, user-level scheduling LWP: real parallelismLWP: real parallelism
Questions: Questions: Too complicated for tuning and programmingToo complicated for tuning and programming mapping the user-level thread(Pthread)’s scheduling attributes? mapping the user-level thread(Pthread)’s scheduling attributes? Scheduling activation & SIGWAITINGScheduling activation & SIGWAITING Need more consideration here!!!! Need more consideration here!!!!
user-level scheduling user-level scheduling Computation-bound threadComputation-bound thread
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
Modeling the concepts: process, LWP and kthreadModeling the concepts: process, LWP and kthread Kthreads own the scheduling parameters specific TS, RT, IA, Kthreads own the scheduling parameters specific TS, RT, IA,
SYSSYS Process control blockProcess control block
Program: vnode, arg & envProgram: vnode, arg & env Lock: fine-grain lock for threadsLock: fine-grain lock for threads Cred: Cred: Address space: including “segments”- segdriver – page tableAddress space: including “segments”- segdriver – page table Pid, pgid:Pid, pgid: P_child: family relationshipP_child: family relationship Singal: siganl-q, single hander vector, mask of ignored Singal: siganl-q, single hander vector, mask of ignored
blockedblocked u-area: file descriptorsu-area: file descriptors See: /usr/include/sys/proc.h, /usr/include/sys/user.hSee: /usr/include/sys/proc.h, /usr/include/sys/user.h
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Understanding “process” – its environmentUnderstanding “process” – its environment
LWP Control Block: /usr/include/sys/kwp.hLWP Control Block: /usr/include/sys/kwp.h An LWP identifier A signal mask that tells the kernel which signals will be accepted Saved values of user-level registers (when the LWP is not running) system call arguments, results, and error Resource usage and profiling data Pointer to the corresponding kernel thread
Kthread control block: /usr/include/sys/thread.hKthread control block: /usr/include/sys/thread.h Reference to its own scheduling parameters – specific to the class A kernel stack some pointer to CPU structure: affinity, bind, Flags: t_schedflag, t_state, t_preempt Linked pointer to other kthreads
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Appendix - BAppendix - B Recommended booksRecommended books
Unix Programming Environment Unix Programming Environment Dept. of CSE, BUAADept. of CSE, BUAA
Appendix - BAppendix - B System Interfaces ProgrammingSystem Interfaces Programming
The practice of programming, Brian Kernighan The practice of programming, Brian Kernighan Advanced Programming in the UNIX Environment, Richard Stevens, Addison-Wesley, 1993 Advanced Programming in the UNIX Environment, Richard Stevens, Addison-Wesley, 1993 UNIX Networking Programming Volume 2: Inter-Process Communication, Richard Stevens, UNIX Networking Programming Volume 2: Inter-Process Communication, Richard Stevens,
Addison-Wesley Addison-Wesley UNIX System Programming, Keith Haviland, Dina Gray and Ben Salama, 2nd ed., Addison-Wesley, UNIX System Programming, Keith Haviland, Dina Gray and Ben Salama, 2nd ed., Addison-Wesley,
1999. 1999. Interprocess Communications in UNIX, John Gray, Prentice Hall, 1997. Interprocess Communications in UNIX, John Gray, Prentice Hall, 1997. Programming with UNIX Threads, Charles J. Northrup, John Wiley & Sons, Inc., 1997. Programming with UNIX Threads, Charles J. Northrup, John Wiley & Sons, Inc., 1997. Practical UNIX Programming: A Guide to Concurrency, Communication, and Multithreading, Practical UNIX Programming: A Guide to Concurrency, Communication, and Multithreading,
Kay Robbins and Steven Robbins, Prentice Hall, 1996.Kay Robbins and Steven Robbins, Prentice Hall, 1996. A Practical Guide to the UNIX System, Mark Sobell, 3rd ed., Benjamin Cummings, 1995. A Practical Guide to the UNIX System, Mark Sobell, 3rd ed., Benjamin Cummings, 1995. UNIX for Programmers and Users, Graham Glass & King Ables, 2nd ed., Prentice Hall, 1999. UNIX for Programmers and Users, Graham Glass & King Ables, 2nd ed., Prentice Hall, 1999.
The following three books on the Unix Kernel are absolutely classic. The following three books on the Unix Kernel are absolutely classic. The Design of the UNIX operating System, Maurice Bach, Prentice Hall, 1986. The Design of the UNIX operating System, Maurice Bach, Prentice Hall, 1986. The Design and Implementation of the 4.4 BSD Operating System, Marshall McKusick, Keith The Design and Implementation of the 4.4 BSD Operating System, Marshall McKusick, Keith
Bostic, Michael Karels and John Quarterman, Addison-Wesley, 1996. Bostic, Michael Karels and John Quarterman, Addison-Wesley, 1996. UNIX Internals: The New Frontiers, Uresh Vahalia, Prentice Hall, 1996. UNIX Internals: The New Frontiers, Uresh Vahalia, Prentice Hall, 1996.