Software Tools Design

Architecture – the design and and specification of overall system structure

• Classical architecture– Design Representations

• plan, elevation, perspective

• drawings/models

– Architectural Styles• Victorian

• Colonial

• Spanish

– Requirements,Constraints• circulation patterns

• acoustics

• lighting

• air flow

• Software architecture– Design Representations

• multiple views

• models for different roles (customer, developer)

– Architectural Styles• Client/server

• Layered

• Object-Oriented

– Requirements,Constraints• reliability, fault tolerance

• performance

• maintainabililty

Software Architecture• As size/complexity of software increases,

design/spec of overall system structure becomes more significant than the choice of algorithms/data structures

• Currently not a well-defined terminology available for software arch.– but good software engineers do use architectural

principles

• Software architecture: defines a system in terms of computational components and interactions among them.

Architectural Elements– components: define computation,data storage

• clients/servers, databases, filters,...

– connectors: define the interactions between components

• procedure calls, pipes, event multicast, …

– configurations: collections of interacting components and connectors

• diagrams of boxes and arrows, formal mathematical models

– constraints: provide semantic understanding of the style requirements

• what makes the style correct, legal, efficient, secure, reliable

Architectural Styles• Dataflow systems

– batch sequential– pipes and filters

• Call-and-return systems– main program and subroutine– OO systems– Hierarchical layers

• Independent components– communicating processes– event systems

Architectural Styles, cont.• Virtual machines

– interpreters– rule-based systems

• Data-centered systems (repositories)– databases– hypertext systems– blackboards

This class covers:• Pipes and Filters: Unix shell

• Object-Oriented: Java

• Client/Server: – Web Applications and XML, Javascript– CGI programming with Perl– Applets, Servlets

Architectural Style Questions:• What is the design vocabulary? (components,

connectors?)• What are the allowable structural patterns?• What is the underlying computational model?• What are the essential invariants of the style?• What are some common examples of its use?• What are the advantages/disadvantages of using

the style?• What are some common specializations?

Common Framework• Use a common framework in order to

compare different arch. styles• framework: collection of computational components

(components), description of interactions (connectors), set of (constraints) on how they can be combined

• For each style:• structural pattern

• underlying computation model

• essential invariants

• common examples

• advantages/disadvantages

• common specializations

Pipes & Filters• Components:

– filters (processing components): incrementally transform data (stream to stream) from a set of inputs to a set of outputs.

• Connectors:– pipes: conduits for the data stream. transmit output of

one filter to input of next filter

• Invariants: • filters (components) must be independent entities; • little context info is utilized, no state info preserved between

instantiations• filters do not know the identity of their upstream and

downstream filters.

• Examples: unix shell, compilers

Pipes & Filters• Advantages:

– neat design; support reuse; easy to maintain

– hierarchical decomposition

– black box approach

– easy to reuse; support concurrency

– support specialized analysis (deadlock, throughput)

• Disadvantages:– batch organization of processing

• not interactive

– difficult to synchronize pipes, order filters

– performance penalty : forces lowest common denominator of data (often ASCII)

– parsing/unparsing overhead

– infinite queues may be required

– possible deadlock with finite buffers

Data abstraction, OO• Components: objects/instances of ADT

• Connectors: procedure invocation

• Variations:– allow multiple interfaces

• (member function access)

– allow concurrency

• Advantages – encapsulation/info hiding

• can change implementation without affecting clients

– inheritance: shared definitions of functionality

– polymorphism: actual operation determined at runtime

– reuse/maintenance: promotes separation of concerns

• Disadvantages– objects must know id for interaction, change of id requires

notification

Event-based, Implicit Invocation• Components: objects/processes with interfaces that:

– define allowable incoming, outgoing events• Connectors: event procedure bindings

– procedures are registered with events

– components interact by announcing events

– announcing component does not know who registered for the event

– upon event occurrence, associated procedures are implicitly invoked

– order of invocation is non-deterministic

• Main invariant: event announcer does not know who will respond

• Examples: GUIs, databases

• Advantages:– strong support of reuse (event registration)

– eases system evolution (replace components without affecting interfaces)

• Disadvantages:– less control of computation, data exchange difficult

– difficult to reason about correctness

– harder to understand flow, test

Client/Server• Components: server contains processing and data

resources required by clients

• Connectors: packets of client requests and server replies

• Examples: distributed systems, web applications

• Advantages:– strong support of reuse

– support parallel computation

• Disadvantages:– complexity

– performance (data transmission)

– security

The Structure Of a Unix OS

• Major Unix OS features:

– Kernel

– Shell

– File System

• Kernel

• Manages memory and allocates it to each process

• Schedules work done by the CPU

• Organizes transfer of data from one part of machine to another

• Accepts instructions from shell and carries them out

• Enforces access permission on the file system

UNIX OS• Shell• Command interpreter• Create customized environments• Write shell scripts• Define command aliases• Manipulate command history• File and command completion• Edit the command line

Unix OS• File System

• Logical method for organizing and storing large amounts of information.

• Easy to manage.

• File: basic storage unit.

• Types:– ordinary file (stores info)– directory (holds other files and directories)– special file (represents physical devices like

printers, terminals, etc)– pipe (temporary file for command linkage)

Unix Operating System Structure

• OS mediates between the user and the computer

User

Application Programs

Kernel

Hardware

Shell

UNIX: Multi-user Multi-tasking• More than one user can run at the same time and

more than one task can run at the same time• In Unix, each program is started as a process.• A process is a program in execution.• Usually only one copy of a program, but there

may be many processes running the same program.

• To each interactive user (window):– only one process in foreground– may have several processes in background

Unix Process• The fundamental unit of a multi-user UNIX

operation.

• A program in execution, doing some work.

• Multiple users and multiple processes.

• CPU is switched rapidly from one process to the other.

• The shell uses the UNIX process structure to carry out user commands.

• Command Execution From Shell:• Typically a shell process forks a child process to

execute the command.• The child process execs the utility program named

in the command.• Now two processes are running for the user - the

shell (parent) and the utility (child) process.• Running the command in the foreground: shell

waits until the child process is complete.• Running the command in the background:

– the shell prints the process id of the child process and waits for the next command.

httpdlpd

Processes

/etc/init

kernel Process 0: Kernel bootstrap. Start process 1.

Process 1: create processes to allow login.

inetd/etc/getty

forkexec

/bin/login

exec

shell

exec

/etc/getty

forkexec

/bin/login

exec

shell

exec

condition terminal for login

check password

command interpreter

kernal mode

user mode

Unix Process• Init process

– last step in booting procedure

– create other processes to allow the users to login

• Getty process

– conditions for terminal connection

– wait for user-id

• display login on the screen

• Login process

– check password with the uid

– execute .profile or .login (depends on default shell)

– display shell prompt

• Shell process (command line interpreter)

• Shell prompt ($, %)

UNIX Process• Process environment

– Process id, parent-process-id, process-group-id– Opened files– Working directory– File creation mask– User ID, Group ID– Resource limits– Environment variables– Code

• A child process inherits parent’s environment.

Processesuse ps to see the processes that you are running.

$ ps

PID TTY TIME CMD

221 pts/4 4:15 netscape

201 pts/4 0:05 bash

215 pts/4 1:15 emacs-19– use “&” to execute a task in background

• Example: $ sort infile > outfile &

– use ctrl-Z to suspend the foreground task, and then use bg.

– use fg to move a background task to foreground.

Shell: Command Interpreter• Bourne Shell: the default shell (sh)

– original unix shell

– does not have interactive features of newer shells

– widely used for writing shell scripts

– standard on Unix System V

• C Shell (csh): available on BSD Unix and most other systems

– with syntax similar to the C language

– with many enhancement over the Bourne shell.

• Korn Shell (ksh): AT&T’s answer to C Shell

– combine features of Bourne and C shells

– very efficient

• Other shells: tcsh, bash

Day-to-Day Use

Command Function Meaningcat Display a file

conCATenatecp Copies a file CoPymv Renames a file or moves it MoVerm Delete files ReMovelpr Sends a file to a printer Line Printerlp (Sys V)ls Lists the contents of a directory LiStchmod Changes the “mode” of permissions Change MODepwd Shows the current directory Print WorkingDircd Change current directory Change Dirmkdir Create a directory MaKe DIRrmdir Delete a directory ReMove DIRps Shows the processes on the system Process Statusman Shows info. about commands Manualdf Shows file system status Disk Filedu Shows the space allocation of files Disk Utilizationgrep Search for patterns in files

Standard Command Format• command [options] [arguments]

wc [-c | -m | -C] [-lw] [file …]

• stuff in brackets is optional

• boldface words are literals (must be typed as is)

• italicized (or <> enclosed) words are args (replace appropriately)

• Options modify how the command works

• command [options] [--] [<file> …]

options ::= -option white-space options*

option ::= noargoption* argoption | noargoption+

noargoption ::= letter

argoption ::= letter whitespace string

$cc -po zap zap.c

Some Examples• ls [-alRF…] file-list

– a for listing all files including the dot files– l for long format (file type, permissions, #links,

owner, group, etc)– R for recursive, list subdirectories.– F for listing directories with a trailing /

• ps [<options>]– List the information about running processes– Example:

%ps -el # print the info about all processes (e) in the long format (l)

I/O Redirection• Redirection and Pipe

“>“ redirects standard output (screen) to a file

E.g. ls > dirlist

“<“ redirects standard input (keyboard) from a file

E.g. sort < infile > outfile

“|” pipe output of program 1 to input of program 2

E.g. who | wc

Or getit < in | check_it | process_it | format_id > out

“>>“ appends output to a file

E.g. ls -l >> oldfile

Exercise: find out the definition of “tee”.

Sequential vs. Concurrent Process• Sequential:

%date

%ps -ef OR %date; ps -ef; who

%who

• Concurrent:%pgm1 & prgm2 > file1 & pgm3

%make > log &

%sort +1 pdir; ((pr dir | lpr) & sort +1 local))

File Name Expansion• Each shell program recognizes a set of special characters

called meta characters.

• The metacharacters are used to create patterns to match filenames and command names.

• Bourne and Korn shell meta/wildcard characters

– * matches any string (including null)

– ? matches any one character

– [a-dA-D] matches any one character in the range

– [!AB123] matches a char not in this range

– \ escape

• ~<username> : (not bourne shell) the home dir of the user.

File Name Expansion• Assume we have the following files under

the current directory: 120, 235, 2#5, a.out, c22, c*2, Doc.1, Doc.2, Doc.3, one.c,two.c, three.c

ls *.c ls c*2

ls [a-z]*[!0-9] a.* *.*

ls ??? cd ~foo

ls *

Filters• Most UNIX utilities are filters• A filter is a program which

– reads input (text) from standard input only– writes output to standard output only– writes error to standard error only– may use temporary files for intermediate results

• Filters can be combined to work together using pipes

• Pipe: takes stdout of one command and uses it as stdin of another command

ls | wc

Command Alias• Assign your own name for a command • Syntax is shell dependent

alias ll ‘ls -l’ C shell

alias ll=‘ls -l’ Korn, Bourne shell• Displaying the value of an alias

alias ll (displays value)

Unix File Systems• File: a collection of data

– Just a sequence of bytes– no record or block structure required

• Directory– A file that contains information for files– distinction between a directory and a file

• system can alter the contents of a directory

– rooted tree file structure (inverted tree)• directories can contain both files and other

directories info.

5binbinetc

Unix File System Road Map

• Special files: /dev/* represents I/O devices.

/

/etc /var /bin /tmp /usr /mnt /home

sue john

passwdhosts…...

spool admprintermail(in)mail(out)uucp

messageswtmp…

libinclude

/dev

File Systems and the I-nodes• Each disk drive contains one or more file systems

– Each file system occupies a number of cylinder groups.

– Each file system has a superblock, an i-node table and files

• The superblock is at a fixed location relative to the beginning of the file system. It contains a description of the file system.

• One can find the location of the I-node table thru superblock.

• Each entry of the I-node table is an I-node, which uniquely represents a file in the file system.

• An I-node contains uid, gid, time of creation, modification, and access, permissions, location of the file on the disk.

superblock I-node table file1 file2 free file3 free

• An I-node includes the following information:– File access permissions– Owner and group ids– File link count– Time of last modification– Time of last file access– Location of blocks for each regular file and

directory file– Device id for special files

I-node 2 contains the location of the block with the root directory

chmod• Change the access permissions of a file

• chmod <permissions> <filename>– permissions can be specified as 3 octal digits,

<user,group,others>, the three bits of an octal means r,w,x

• Example: chmod 755 test

– permissions can be specified as +x, or u+x, or g+r, …

– “chmod +s test” sets the set-uid bit for file test.

• If a directory has “x” in its permission, the dir is searchable, ie., one can do “ls” on the directory.

Pathnames• Every file and directory in the file system can

be identified by a “full path name” (route from root to file)/home/sue/email/f1

• Relative path name– location relative to current directory

. Current directory

.. Parent directory

– if cwd is /home/sue:ls email

ls ./email

cd ..

ls ../fred

fred

/

home

sue

docs email

f1 f2

Some Details• cp [-ir…] file1 file2

cp [-ir…] file-list directory– i for interactive. prompt use whenever a file will be

overwritten– r for recursive. copy a directory tree

• ls [-alRF…] file-list– a for listing all files including the dot files– l for long format– R for recrusive. list the all subdirectories.– F for listing directories with a trailing /

• date [+format]– % date ‘+%h %d, 19%y’

Some Details (cont.)• wc file-list

– display the number of lines, words and characters• more file-list

– Browse through text files on page at a time.• head [-n …] file-list

– Display the first n lines of the files (default=10)• tail [+n|-n| -f| …]

– Display the last few lines in the files (default = 10)– Example:

%tail +5 foo # display the last parf of foo starting from line 5

%tail -5 foo # display the last five lines of foo

%tail +30 foo | head -15 | more #display line 30-45 of foo

%tail -f foo # wait and display the new lines appended to foo

Some Details (cont.)• cut -c list file• cut -f list [-dChar] file

– Cut out selected characters or fields from each line of a file– Examples:

%cut -c 1-5,15-20 foo # extract chars 1-5 and 5-20 from each line of foo.

%cut -f 1,3 -d” “ moo # extract field 1 and 3 from each line of moo.

• paste file1 file2– Concatenate corresponding lines of the given input files– Example (reverse two fields of the file abc)

%cut -f1 abc > abc1%cut -f2 abc > abc2%paste abc2 abc1 > xyz

Some Details (cont.)• grep, egrep, fgrep

grep [-nv...] pattern [file-list]– Search the input text files for lines

matcing the pattern

%grep Unix doc.1 # Display the lines in doc.1 that contains “Unix”

%grep -n Unix doc.* # Display the lines with line numbers

%grep -v Unix doc.1 # Display the lines which do not contain “Unix”

Some Details (cont.)• sort [-tC…] [-o outfile] [field-list] [file-

list]– sort the files

%sort +1 list1 # sort list 1 starting from field 2 to the end of the line

%sort +2-3 list2 # sort list2 on the third field

%sort -n -o list4 list3 sort list3 numerically and place the output in list4

diff • diff file1 file 2

– Display different lines that are found when comparing two files

– It prints a message that users ed-lide notation (a - append, c - change, d -delete) to describe how a group of lines has changed.

– It also describes what changes (additions, deletions and changes) need to be made to the first file to make it the same as the second file.

diff Example

file1 file2 file3

apples apples oranges

oranges oranges bananas

bananas kumquat s kiwis

peaches

diff (cont.)%diff file1 file2

3c3

<bananas

-----------------

>kumquats

%diff file1 file3

1d0

<apples

3a3,4

>kiwis

>peaches

comm file1 file2– Takes two sorted text fields and print common lines and lines

which appear exclusively in one file on separate columns.– column1: lines only in file1, column 2: lines only in file2; col 3:

comm– Example

file1 file2 %comm file1 file2

apple apple apple

banana cantaloup banana

grape grade cantaloup

orange kiwi grape

lemon kiwi

%comm -12 file1 file2

apple

grape

tr [-csd…] pattern1 pattern2– Translate input character to output character based on the

input and output patterns– Example%tr ‘[A-Z]’ ‘[a-z]’ <in >out

# xlate all letters to lower case.%tr -s ‘\012\011\040’ ‘\012\012\012’ < in > out

# xlate newline, tab and blank chars to newline chars and squeeze (-s) consecutive newline char into one

%tr -cs ‘[a-z][A-Z]’ ‘[\012*]’ < in > out# change all non-aplphbetic (-c) chars to newline

chars and squeeze consecutive newlne char into one.

The * (asterisk) tells tr to repeat the # (number sign) enough times

to make the second string as long as the first one. %tr -d ‘\040’ < in > out

# delete all blanks.

find <dir-name> <exp>– Recursively search the directory tree rooted at

<pathname> and find all files whose names satisfy <exp>

– find pathname search options action options– There are many details in the expression.– Examples:%find . -name \*.doc -print # list all files names ending

with .doc%find /etc/source -atime 2 -print

# print the names of the files under /etc/source whose lst access time was 2 days ago.

find <dir-name> <exp>%find . -name “[a-z]*” -exec rm {} \;

# remove the files under the curent directory whose names begin with a lower case letter.

%find / \(-name a.out -o -name “*.o” \) -atime +7 -exec {} \;

# remove the object and binary executable files under the root directory which have not be accessed more than 7 days.