Algorithm SC 2006

8/3/2019 Algorithm SC 2006

1/138

Algorithms forScientific Computation(An Introduction Using C)

Division of Applied Mathematics

Korea Advanced Institute of Science and Technology

2004

1


2/138

Contents

1 Introduction 4

2 Basic Data Types 8

2.1 Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Int . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Float, Double, and Long Double . . . . . . . . . . . . . . . . . . 112.4 Char . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3 Arithmetic operators 15

4 Input and Output Functions 18

4.1 printf() and scanf() statements . . . . . . . . . . . . . . . . . . . 184.2 File Input and Output . . . . . . . . . . . . . . . . . . . . . . . . 20

5 Control Flow Statements 26

5.1 Conditional Expression . . . . . . . . . . . . . . . . . . . . . . . . 265.2 if Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275.3 switch Statements . . . . . . . . . . . . . . . . . . . . . . . . . . 305.4 while Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.5 for Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325.6 Jump Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

6 Function statements 34

6.1 Local and Global Variables . . . . . . . . . . . . . . . . . . . . . 356.2 Storage Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

6.3 Recursive Functions . . . . . . . . . . . . . . . . . . . . . . . . . 386.4 Macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

7 Arrays 41

7.1 Arrays and Pointers . . . . . . . . . . . . . . . . . . . . . . . . . 417.2 Dynamic Memory Allocation Function . . . . . . . . . . . . . . . 437.3 Multi-Dimensional Arrays and Pointers. . . . . . . . . . . . . . . 44

8 Strings 46

8.1 Input and Output functions of Strings . . . . . . . . . . . . . . . 468.2 String Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 488.3 Command Line Arguments . . . . . . . . . . . . . . . . . . . . . 50

9 Structures 51

10 Linear Linked Lists 55

10.1 basic list operations . . . . . . . . . . . . . . . . . . . . . . . . . 5610.2 Stacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5910.3 Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

2


3/138


4/138

1 Introduction

Computer Systems : information processing

Structure

why C language?

- Dennis Ritchie, 1973 (AT&T Bell lab) for Unix O.S.

- American National Standards Institute(ANSI) : 1989, ANSI C

- Pros :

portability

general purpose

compact code

structured programming

flexible control structures. mid-level language

4


5/138

C programming(in Unix environment)

Structure of C program :

#header - preprocessor

main()

{

statements;

.

.

.

}

function a()

{

statements;

.

.

.

}

function b()

{

statements;

.

.

.

}

5


6/138

5 types of C statements:

declaration statements.

assignment statements.

function statements.

control statements.

null statements.

Problem solving methodology

1. problem statement

give a clear and concise problem statement.(to avoid any misunderstanding)

- eg. compute the straight-line distance between two points in a plane.2. input/ output description

the I/O diagram.

- eg.

3. hand example

simple set of data for testing the program.4. algorithm development

algorithm : a step-by-step outline of the problem solution

- eg.

(a) give values to the two points.

(b) compute the lengths of the two sides.

(c) compute the distance between the two points using the Pythagoreantheorem.

(d) print the distance between the two points.

Pseudo-code

5. testingCompare with the hand solution.

6


7/138

Sample Program

- Pseudo-code

1. read (x1, y1) and (x2, y2)

2. side_1 = x1 - x2, side_2 = y1 - y2

3. distance = sqrt (side_1*side_1 + side_2*side_2)

4. print distance

- Sample Program

/*---------------------------------------------------------------*/

/* This program computes the distance between two points. */

/* */

#include

#include

#incldue

main()

{

/* Declare and initialize variables. */

float x1, y1, x2, y2, side_1, side_2, distance;

/* Read two points. */

printf("two points: x1 y1 x2 y2 = ");

scanf("%f %f %f %f", &x1, &y1, &x2, &y2);

/* Compute two sides. */

side_1 = x1 - x2;

side_2 = y1 - y2;

/* Compute the distance */

distance = sqrt(side_1*side_1 + side_2*side_2);

/* Print the distance */

printf("distance = %6.2f\n", distance);

}

/*---------------------------------------------------------------*/

7


8/138

2 Basic Data Types

2.1 Data

Data

data

characters : char

numbers

integers : int(short, int, long)

real numbers : float, double, long double

Qualifiers - short, long, unsigned

- int : int, short(int), long(int), unsigned(int),

unsigned short, unsigned long

- double : long double

Number of bytes for data types : sizeof operator

printf("short has %d bytes\n", sizeof(short));

short has 2 bytes

Example program

/* ...*/ - comments

#include - input/output functions

(< > : standard C library)

- constants for exiting the program

- mathematical functions

(.h : header files)

main()

{...} - block

statements : end with ;

- declaration statements : data_type list of variables;

- assignment statements : identifier = expression;

- control flow statements

- function statements

- null statements

8


9/138

The declaration statements define memory locations for variables.

eg.

short x, y=1; - short : data_type.

- x, y : variables.

- y=1 : initialization.

Selection of the name of variables.

- small letter, capital letter, underline( ), number.

- number cannot come first.

- maximum number of characters(831).

- no reserved words.

eg.

valid not valid

wiggly $z**cat1 1cat

Hot tub Hot-tubKcaB dont

9


10/138

2.2 Int

int constants

- decimal number : 123 {0,1,2,...,9}

- octal number : O123 {0,1,2,...,7}

- hexadecimal number : Ox123 {0,...,9,a,b,...,f}

range of values

short (2) 215 215 1 (32768 32767)unsigned short (2) 0 216 1 (0 65535)

int (4) 231 231 1

unsigned int (4) 0 232 1

eg. unsigned (range : 0 15)

binary number decimal number

0000 00001 10010 2

......

1111 15

eg. int (range : 8 7)

2s complement method for negative integers.x = x + 1

where, x is a binary number.

binary number decimal number

0111 7...

...0001 10000 01111 -1

.

.....

1000 -8

eg. range of short(2bytes)(215 215 1)

binary number 1000 0000 0000 0000 0111 1111 1111 1111hexa-decimal number 8 0 0 0 7 f f f

10


11/138

2.3 Float, Double, and Long Double

constants :

1. 2.75 3.14 1.6e 19 (= 1.6 1019)

- Always be stored double form.

binary number (IEEE 754 standard) - float(32bits)

eg. the precision digits and the range of exponents for floats (32bits)

i. valid numbers

- fraction part (23bits)

- range of values : 20 21 222

1222 =

1(210)2.2 =

1(103)2.2 = 10

6.6

= precision digits = 6 (decimal digits)

ii. range of exponents

- exponent part (8bits)

- range of values : 0 28 1(255)

- bias : 27 1(127) = 127 128 i.e. 2127 2128

- transform to the exponent of 10 : 2128 = (210)12.8 = (103)12.8 = 1038.4

= the range of the exponent of 10 : 38 38

11


12/138

The limitations of data type (in SUNWS)

/*---------------------------------------------------------------*/

/* This program prints the system limitations. */

/* */

#include

#include

#include

main() {

/* byte size for each data type */

printf("short has %d bytes\n", sizeof(short));

printf("int has %d bytes\n", sizeof(int));

printf("long has %d bytes\n\n", sizeof(long));

printf("float has %d bytes\n", sizeof(float));

printf("double has %d bytes\n", sizeof(doluble));

printf("long double has %d bytes\n\n", sizeof(long double));

/* Print integer type maximums. */

printf("short maximum: %d\n", SHRT_MAX);

printf("int maximum: %d\n", INT_MAX);

printf("long maximum: %ld\n\n", LONG_MAX);

/* Print float precision, range, and maximum. */

printf("float precision digits: %d\n", FLT_DIG);printf("float maximum exponent: %d\n", FLT_MAX_10_EXP);

printf("float maximum: %e\n\n", FLT_MAX);

/* Print double precision, range, and maximum */

printf("double precision digits: %d\n", DBL_DIG);

printf("double maximum exponent: %d\n", DBL_MAX_10_EXP);

printf("double maximum: %e\n\n", DBL_MAX);

/* Print long precision, range, and maximum */

printf("long double precision digits: %d\n", LDBL_DIG);

printf("long double maximum exponent: %d\n", LDBL_MAX_10_EXP);

printf("long double maximum: %e\n\n", LDBL_MAX);

}/*---------------------------------------------------------------*/

12


13/138

short has 2 bytes

int has 4 byteslong has 4 bytes

float has 4 bytes

double has 8 bytes

long double has 16 bytes

short maximum : 32767

int maximum : 2147483647

long maximum :2147483647

float precision digits : 6

float maximum exponent : 38

float maximum : 3.402823e+38

double precision digits : 15

double maximum exponent : 308

double maximum : 1.797693e+308

long double precision digits : 33

long double maximum exponent : 4932

long double maximum : 1.189731e+4932

13


14/138

2.4 Char

Express by ASCII code

(7 or 8 bits)

eg. 7 bit ASCII

character binary number decimal numberA 1000001 65B 1000010 66...

......

P 1010000 80...

......

Z 1011010 90

- constants :

A B C a - single quotation.

cf. 1 : 00000011 : 0110001

- Non-existing character in the keyboard.

Use \

eg.

char beep = \007

\n \012 (newline)

\t \011 (tab)

\b \010 (backspace)\r \015 (carriage return)

\ ~~ \\

~~ \

" ~~ \"

14


15/138

3 Arithmetic operators

arithmetic operators

unary operators (one operand) : =, (data_type), -, +

binary operators (two operands)

additive operators : +, -

multiplicative operators : *, /, %

= (assignment)fahr = 0; (correct)

0 = fahr; (wrong)

The direction of assignment is from left to right.

- (sign change)

x = -1;

One or more space between = and -1.

+, -, *

x = a+b;

/

- type of integer numbers : 5/2 2 (quotient)

- type of real numbers : 5./2 2.5

% (remainder, modulus)

3%2 1, 8%2 0

(data type) cast operator

eg.

int i, j,;

i = 1.6+1.7; (i=3)j = (int)1.6+(int)1.7; (j=2)

15


16/138

increment and decrement operators : ++, --

The priority is same as unary operators.

- prefix operators

x = ++n : n = n+1; x = n;

x = --n : n = n-1; x = n;

- postfix operators

x = n++ : x = n; n = n+1;

x = n-- : x = n; n = n-1;

eg.

int x, n=1;

x = n++; (x=1, n=2)

x = ++n; (x=3, n=3)

abbreviated assignment operators : op=,The priority is same as = operators.

- op : +, -, *, /, % and bitwise operators

eg.

x += 3 (x = x+3)x *= y+1 (x = x*(y+1))

the general form of assignment statement: identifier = expression;

- expression: a constant, another variable, or the results of an operation.

eg.

5

4+20

c=3+8

6+(c=3+8)

16


17/138

Priority of operators

- basic rule:unary operators except = > binary operators > . . .

operators Associativity(when same priority)

( )-, (data_type), ++, --

*, /, % +, -

=

Priority of operands

- Basic rule : Change to the many number of bytes operands.(double > float > long > int > short)

eg.

int i, j;

float x, y;

i = 5./2*5; (i=12)

x = 5./2*5; (x=12.5)

j = 5/2*5; (j=10)

y = 5/2*5; (y=10)

eg.

f =x3 2x2 + x 6.3

x2 + 0.05005x 3.14

f = (x*x*x-2*x*x+x-6.3)/(x*x+0.05005*x-3.14);

17


18/138

4 Input and Output Functions

4.1 printf() and scanf() statements

printf() functions

printf("control string", arguments);

cf. Arguments can be omitted.

eg.

printf("output");

output

printf("%c %d\n", A, 10);

A 10

%c, %d - numeric conversion specifier

variable-type specifier meaningcharacter %c character

%s stringinteger %d decimal no.

%o octal no.%x hexadecimal no.%u unsigned no.

floating-point %f floating point typevalues %e exponent type%g shorter between %f and %e

cf. qualifiers - short : h, long : l, double : l, long double : L

- The number of conversion specifiers should be the number of outputs.

eg.

printf("%c %d\n", 65, 65);

A 65

printf("%d", A-8+5);

4

printf("%d %o %x", 10, 10, 10);

10, 12, a

printf("%f %e\n", 2.4, 2.4);

2.400000 2.400000e+00

printf("%s", string);

string

18


19/138

- user defined output forms

integer real number string%nd or %n.mf or %n.ms or

%-nd %-n.mf %-n.ms

n : no. of spaces

- : starting from left

m(in real number) : no of floating points

m(in string) : first m characters

eg.

printf("%d\n", 126); : 126

printf("%10d\n", 126); : #######126printf("%-10d\n", 126); : 126#######

eg.

printf("%f\n", 1234.56); : 1234.56####

printf("%e\n", 1234.56); : 1.23456#e+03

printf("%3.1f\n", 1234.56); : 1234.6

printf("%10.3f\n", 1234.56); : ##1234.560

printf("%10.3e\n", 1234.56); : #1.235e+03

eg.

printf("%2s\n", "string"); : string

printf("%10s\n", "string"); : ####string

printf("%-10.5s\n", "string"); : strin#####

19


20/138

scanf() functions

scanf("control string", arguments);

- Arguments should be memory addresses of variables.

- Conversion specifiers are same as printf().

eg.

char ch;

scanf("%c %d", &ch, &i);

cf. & operator: returns the memory address of assigned variable.

4.2 File Input and Output

unix command :

a.out out.file

declaration of data type of file :

FILE *fp; (*fp : file pointer - the variable that indicate

the memory address of file)

fopen() and fclose() functions

fopen("filename", "r");

r : read

w : write

a : append

- fopen() returns the memory address of file.

20


21/138

eg.

FILE *in;in = fopen("test". r");

fclose(in) : fclose(file-pointer)

- notifying the end of a task to the file

Input, Output function related to files.

- character

char ch; FILE *fp;

ch = getchar();ch = getc(fp); : read one character in the FILE

which is indicated by fp.

ch = getc(stdin) : stdin - keyboard

putchar(ch);

putc(ch, fp); : write ch to the fp

putc(ch, stdout); : stdout - screen

- string

char str[100]; FILE *fp;

gets(str);

fgets(str, max, fp); : read the first max characters in

a string of the FILE indicated by fp

and assign to str.

puts(str);

fputs(str, fp); : write the string to the FILE

indicated by fp

- printf & scanfscanf("control-string", argument_list);

fscanf(fp, "control-string", argument_list); : read from the FILE

indicated by fp

printf("control-string", argument_list);

fprintf(fp, "control-string", argument_list) : write to the FILE

indicated by fp

21


22/138

eg.

main()

{

FILE *fp;

int age;

fp=fopen("sam", "r");

fscanf(fp, "%d", &age);

fclose(fp);

fp=fopen("sam", "w");

fprintf(fp, "sam is %d years old\n", age);fclose(fp)

}

eg.

if(fp==NULL)

printf("error");

else{

...

Programming Linear Modeling (Regression)

the process that determines the linear equation that is the best fit to aset of data points in terms of minimizing the sum of the squared distancesbetween the line and the data points.

For the given sample :

(x1, y1), (x2, y2), . . . , (xn, yn)

linear model(estimator) :y(x) = mx + b

problem : for the given sample and estimator, find m and b which mini-mizes

E =1

2

ni=1

(yi y(xi))2.

22


23/138

If y is linear, E has a quadratic form.

Em

m=m= 0,

Eb

b=b= 0

m =

nk=1 xk

nk=1 yk n

nk=1 xkyk

(n

k=1 xk)2 n

nk=1 x

2k

b =

nk=1 xk

nk=1 xkyk

nk=1 x

2k

nk=1 yk

(n

k=1 xk)2 n

nk=1 x

2k

23


24/138

- ozone measurements progrmamming

1. problem statement: for a given data (file),

determine a linear model for estimating the ozone mixing ratio at aspecified altitude and

range of altitudes

2. I/O diagram

3. hand example

- data: (ppmv: parts per million value)

altitude (xk) ppmv (yk)

20 324 426 528 6

4

k=1

xk = 98,4

k=1

yk = 18,4

k=1

x2k = 2436,4

k=1

xkyk = 464

denominator = (4

k=1

xk)2 4

4k=1

x2k = 140

m = (4

k=1

xk

4k=1

yk 44

k=1

xkyk)/denominator = 0.37

b = (4

k=1

xk

4k=1

xkyk 4

k=1

x2k

4k=1

yk)/denominator = 4.6

4. algorithm:

(a) read data file

(b) compute range of altitudes, and parameters(m & b) of linear model

(c) print range of altitudes and linear model

24


25/138


26/138

5 Control Flow Statements

1. selection statements: if, if-else, if-else if, switch

2. repetition statements: while, do-while, for

3. jump statements: goto, continue, break, return

Selection and repetition statements are conditionally controlled.Jump statements are unconditionally controlled.

5.1 Conditional Expression

evaluated to be true (1) or false (0)

relational operators

> : greater than

>= : greater than or equal to

< : less than

rel. op. > eq. op. > logical op.

26


27/138

operators associativity

( )++, --, -, !, (data_type) *, /, %

+, - =

==, != && ||

=, op=

eg.

x>y+2 : (x>(y+2))

ex!=xye==zee : ((ex!=xye)==zee)a>c==d!=e : (((a>c)==d)!=e)

5>2&&4>7 : ((5>2)&&(4>7))

!0&&2>3||5>4&&-1 : (((!0)&&(2>3))||((5>4)&&(-1)))

5.2 if Statements

if statements

1) if(expr)

statement; : binary decision.

2) if(expr)

statement;else

statement; : binary decision

3) if(expr)

statement;

else if(expr)

statement;

...

else

statement; : multiway decision.

eg1.a=1; sum=0;

if(a


28/138

eg2.

j=k=m=1;

if(x>y)

if(yy)

{

if(y 1x if 1 x 1

1 if x < 1

if(x>1) if(x=-1) else if(x


29/138

conditional operator (ternary)

E1?E2:E3 operator

if E1

E2;

else

E3;

eg1.

if(y


30/138

5.3 switch Statements

switch statements : multiway decision.

switch(expr){ : (expr) - integral type (char or int)

case const_expr: statement; - const_expr (char or int)

...

case const_expr: statement;

...

default : statement;

...

}

eg.

int i;

scanf("%d", &i); i

switch(i){ 1 grater than 4

case 2: printf("%d\n", 2); 2 2

break; 3 3

case 3: printf("%d\n", 3); 4 4

break;

case 4: printf("%d\n", 4); 5 greater than 4

break; 6 greater than 4

default: printf("greater than 4\n"); ... ...

}

30


31/138

5.4 while Statements

while statements

1) while(expr) 2) do

statement; statement;

while(expr);

eg.

i=1; sum=0; i=1; sum=0;

while(i


32/138

5.5 for Statements

for statements

for(expr1; expr2; expr3)

statement;

* expr1 : initialization

* expr2 : condition for repetition

* expr3 : modification to the loop-control values

eg.

i=1; sum=0; : expr1

while(i


33/138

5.6 Jump Statements

- break statements: an early exit from

"for, while, do_while, switch"

i=1; sum=0; i=1; sum=0;

while(1){ while(i10) --> sum += i;

break; i++;

sum += i; }

i++;

}

- continue statements: next iteration of "for, while, do-while"

for(i=1, sum=0; i


34/138

6 Function statements

function statements

function: generates an output for the given input

procedure: a part of whole program function in C

definition :

return_data_type function_name(arguments)

data_type arguments;

{

statements;

:

return(expr);

}

return_data_type:

- It does not need to be declared when there is no return statement.

- If it is declared, return_data_type should be declared in main().

return statement: one value can be returned.

eg.

return(1);return(x + y);

return(x, y); not valid

eg.

main() | int sum(a)

{ | int a;

int i, j; | {

int sum( ); | int i=1, j=0;

scanf("%d", &i); | while (i


35/138

eg.

int j;

main() | void sum(a)

{ | int a;

int i; | {

void sum( ); | int i=1;

scanf("%d", &i); | while (i


36/138

eg.

{int a = 5, b = -3, c = -7

{

int b = 8;

float c = 9.9;

a = b ;

{

int c;

c = b ;

printf("%d %d %d \n", a, b, c); -> 8 8 8

}

printf("%d %d %f \n", a, b, c); -> 8 8 9.9

}printf("%d %d %d \n", a, b, c); -> 8 -3 -7

}

6.2 Storage Classes

Storage classes are specified in the declaration statement.

definition: storage class data type variables;

auto - dynamic storage in the main memory

register - temporary storage in the CPU

static - static storage in the main memory extern - referred variables from the outside of function

auto: no declaration of storage class auto

eg. int i, j; auto int i, j;

register:

- Variables are stored in the register located at CPU

fast access time.

- If not possible, variables are stored in the dynamic storage, that is, sameas auto.

eg. register int i, j;

36


37/138

static:

- Variables are maintained until the program terminates.- Initialization of variables is performed only once.- If variables are not initialized, variables are set to 0.

eg.

int i;

main()

{

int j;

for(j=1; j


38/138

6.3 Recursive Functions

A function that invokes itself.

eg. factorial computation

n! = 1 2 n =n

i=1 i iterative form

n! = n (n 1)! if n 1 recursive form

1 if n = 0

iterative form : recursive form long :

long factorial(int k) | long factorial(int k){ | {

int j; long term; | if (k == 0)

if(k ==0) return(1); | return(1);

else{ | else

term = 1; | return(k*factorial(k-1));

for(j=2; j


39/138

eg.

main(){

void up_down( ); a1

up_down(1); a2

} a3

void up_down(int n) a4

{ b4

printf("a%d\n", n); b3

if (n < 4) up_down(n+1); b2

printf("b%d\n", n); b1

}

eg. Fibonacci sequence

an = an1 + an2 if n 2

1 if n = 1, n = 0.

limnan

an1=

5+12 : golden ratio

int fibonacci(int k)

{

if (k == 0 || k ==1) return(1);

else (fibonacci(k-1) + fibonacci(k-2));

}

fibonacci(5) = ?

39


40/138

6.4 Macros

Macros are specified by a preprocessing directive.

form: #define macro_name(parameters) macro_text

usage:

The macro_name is replaced by the macro_text.

It should be described in one line.

If we need to describe the macro in the next line, \ is attached atthe end of line.

; should not be described.

no space between macto_name and parameters

eg.

#define PI 3.141592

#define degree_C(x) ((x)-32)*(5.0/9.0))

#define MAX(x,y) ((x)>y?(x):(y))

characteristics

faster execution than function statement

If the program segment is large size and frequently used, macros areinefficient.

Sometimes, we need to check the meaning of macros.

cc -E test.c

eg.

#define SQ(X) X*X -> ((X)*(X))

#define PR(X) printf("X is %d\n", X)

main()

{

int x=4;

PR(SQ(X+2)); X+2 * X+2 -> 14

PR(100/SQ(2)); 100/2*2 -> 100

PR(SQ(++X)); ++X * ++X -> 36

}

#include: the file is includes from the specified path.

#include "myfile.h": include file in the current directory.

#include "/user/kil/am320/myfile.h": the path of file is specified.

40


41/138

7 Arrays

Declaration

int a[10]; 1 dimensional array, a[0], a[1], ..., a[9]

int b[10][10]; 2 dimensional array

b[0][0], b[0][1], ..., b[0][9]

b[1][0], b[1][1], ..., b[1][9]

...

b[9][0], b[9][1], ..., b[9][9]

Initialization

Only external array and static array can be initialized.

If arrays are not initialized, the default value of each element is zero.

eg.

static int a[10]={1,2,3,4,5,6,8,9,10};

static int b[2][10]={{1,2,3,4,5,6,7,8,9,10},{2,4,6,8,10}};

7.1 Arrays and Pointers

The use of pointer.

- name of array = memory address of the first element,

that is, pointer constant.

eg.

int data[10]; data == &data[0]

pointer variable and pointer constant.

eg.

int *pti, dates[10];

dates == &dates[0]

dates+2 == &dates[2]

*(dates+2) == dates[2]

cf. * operator fetches the values of the assigned memory address.

41


42/138

pti = dates; memory address of dates is assigned to the pointer pti

pti+2 == &dates[2]*(pti+2) == dates[2]

*pti+2 == dates[0]+2

cf.

- pti+1: the memory address of the next data to the data

indicated by pti- dates+1: the memory address of the next data todates[0], that is, &dates[1]

- in general,

*(pti+i) == dates[i]

*(dates+i) == dates[i]

passing the array to the argument of function

memory address of an array is passed

eg.

main() {

int ages[50]

convert(ages);

}

convert(a) | convert(a)

int a[]; | int *a;

{ | {

: | :

a[i] | *(a+i) = a[i]

} | }

a[i] == *(a+i) == ages[i]&a[i] == a+i == &ages[i]

42


43/138

memory address is returned

eg.main()

{

char *gets();

...

}

char *gets();

{

...

return( ); : return the memory address

}

7.2 Dynamic Memory Allocation Function

Memory allocation functions

#include is required.

- malloc(size): allocation of storage for an object of size bytes

returns a pointer.

- calloc(n, size): allocation of storage for an array,

n objects of size bytes

returns a pointer.

Storage is initialized to all zeros.

- free(ptr): free preciously allocated storage for an object

indicated by ptr.

eg. programming an array using pointers.

int x[10]; the index value 10 is fixed.

int *x;

x = (int *) calloc (10, sizeof(*x));*(x+i) == x[i]

43


44/138

eg.

int n, *x;

printf("number of elements in an array = ");

scanf("%d", &n)

x = (int *)calloc(n, sizeof(*x));

printf("scanning element values\n");

for(i=0; i


45/138

pointers of pointers :

the indirection operator * is used more than one time.eg.

main()

{

static short x[2][2]={{1,2},{3,4}};

short *px[2], **pp;

px[0] = x[0];

px[1] = x[1];

pp = px;

}

eg.

**pp == X[0][0] == 1

**(pp+1) == X[1][0] == 3

*(*pp+1) == X[0][1] == 2

*(*(pp+1)+1) == X[1][1] == 4

in general,

*(*(pp+i)+j) == x[i][j]

cf. 3 dimensional array

*(*(*(pp+i)+j)+k) = x[i][j][k]

eg. programming two dimensional array using pointers

int y[10][10];

...

int **y;

y = (int **)calloc(10, sizeof(*y));

for(i=0; i


46/138

8 Strings

definition of strings:- one dimensional arrays of char- terminated by a null character \0- String constants are written between double quotes.

eg. string- If " is described, \ is used, that is, \"

declaration of strings:

1. array size is specified:eg.char name[10]="string"; orchar name[10]={s,t,r,i,n,g,\0 };

2. array size is not specified:eg.char name[]="string";

3. using pointers:eg.char *p="string"; orchar *p; p="string";

array of strings: multi-dimensional array ofchareg.

static char fruit[3][7]={"Apple","Pear","Orange"};static char *fruit[3]={"Apple","Pear","Orange"};

8.1 Input and Output functions of Strings

character input and output functions:putchar(): print one character to the screengetchar(): read one character from the keyboard

eg.

char ch;

ch = getchar();putchar(ch);

putchar(a);

putchar(65);

46


47/138

string input and output functions:

gets(s) functions:- read a string into s from stdin.- characters are placed in s until a new line character is read.- the value of s is returned.

puts(s) functions:- the string s is copied to stdout.- a new line character is placed at the end of string.- the value of s is returned.

eg.

char name[80];char *ptr;

ptr = gets(name); Hong Kil Dong

puts(ptr); Hong Kil Dong

puts(ptr+5); Kil Dong

puts(name); Hong Kil Dong

cf.

scanf("%s", name); (!= gets(name);)

printf("%s\n", name); (== puts(name);)

47


48/138

8.2 String Functions

string functions:- arguments of string functions are memory addresses of strings.- #include is required.

strlen(s): a count of the number of characters before \0 is returned.

strcmp(s1,s2): an integer is returned which is less than, equal to, orgreater than zero depending on whether s1 is lexicographically less than,equal to, or greater then s2.

strcat(s1,s2): a copy of s2, including the null character, is appendedto the end of s1 and the value of s1 is returned.

strcpy(s1,s2): s2 is copied into s1 until \0 is moved; whatever exits ins1 is overwritten and the value of s1 is returned.

eg.

char *p, *s1, *s2, *s3;

s1="abcde"; s2="fghij"; s3="hij";

printf("%d\n", strlen(s1+2)); 3

printf("%d\n", strcmp(s2+2, s3)); 0

strcat(s2, s3);

puts(s2); fghijhij

strcpy(s1+1, s2+2);

puts(s1); ahijhij

48


49/138

eg.

static char *p[2][3]= {"abc","defg", "hi","jklmo","pgstuvw","xyz"};

printf("%c\n", ***p);

***p == p[0][0][0] == a

**p[1] == p[1][0][0] == j

**(p[1]+2) == p[1][2][0] == x

(*(*(p+1)+1))[7] == p[1][1][6] == w

*(p[1][2]+2) == p[1][2][2] == z

**(p[1]+1) == **(*(p+1)+1) == p[1][1][0] == p

*(*(*(p+1)+1)+1) == p[1][1][1] == q

cf.

p == memory address of pointer array

*p == value of p[0] = memory address of the 0th row

**p == value of p[0][0] == memory address of "abc"

***p == a

49


50/138

8.3 Command Line Arguments

Two arguments, conventionally called argc and argv, can be used with main()to communicate with the operating system.

The variable argc provides a count of the number of command line argu-ments.

The array argv is an array of pointers to char, and can be thought of anarray of strings.

eg.

main(argc, argv)

int argc;

char *argv[];{

int i;

printf("argc = %d\n", argc);

for (i=0; i


51/138

9 Structures

hierarchy of a file:

file

record

field

. . .

. . .

declaration of structures:

struct structure_name{field_type field_name;

...

} variable_name;

eg.

#define MAX_TITLE 80

#define MAX_AUTHOR 40

struct book{

char title[MAX_TITLE];

char author[MAX_AUTHOR];

float value;

};struct book libry;

initialization of structure variables:

static or global variables are initialized.

eg.

static struct book libry = {

"The pirate at the damsel",

"Tenes Vivottee",

1.95

};

member operators:

To access the values of members, we use the structure member operator".".

eg.

libry.title = "KAIST Annual Report";

51


52/138

array of structures:

eg.

#define MAX_BOOK 1000

static struct book libry[MAX_BOOK];

libry[0].value = 2.0;

libry[4].title = "LaTex";

libry[2].title[4]

nested structures:

another structure is defined within the structure.

eg.

#define MAX_LENGTH 20

struct names {char first[MAX_LENGTH];

char last[MAX_LENGTH];

};

struct guy {

struct names handle; /* nested structure */

char favfood[MAX_LENGTH];

char job[MAX_LENGTH];

float income;

}

main()

{/* initialization of nested structure */

static struct guy fellow = {

{"Franco", "Wathall"},

"eggplant",

"doormat customizer",

15435.00

};

...

}

52


53/138

pointers of structures:

pointers of structures are useful in the following points:1) member of structure can be easily accessed by -> operator,2) structure variables can be easily passed to arguments of function, and3) data structures (eg. list, tree, ... ) can be described.

eg.

main()

{

static struct guy fellow[2] = {

{{"Franco","Wathall"},

"eggplant",

"doormat customizer",

15435.00

},

{{"Rodney","Swillbelly"},

"salmon mousse",

"interior decorator",

35000.00

}

};

struct guy *him;

him = fellow;

printf("him->income is $%.2f\n\n", him->income);

printf("(*him).income is $%.2f\n\n", (*him).income);

him++;printf("him->favfood is %s\n\n", him->favfood);

printf("him->names.last is %s\n\n", him->names.last);

}

him->income is $15435.00

(*him).income is $15435.00

him->favfood is salmon mousse

him->names.last is Swillbelly

cf.

him->income == (*him).income

Here, . operator has higher priority than * operator.

53


54/138

unions: unions follow the same syntax as structures but have members

that share storage.eg.

union int_or_float {

int n;

float x;

}temp;

temp.n = 4444;

printf("%10d %10.4e\n", temp.n, temp.x); 0000004444 4.3387e-29

temp.x = 4444.0;

printf("%10d %10.4e\n", temp.n, temp.x); -0536852854 4.4440e+03

...

the use of typedef: the typedef declaration allows a data type to beexplicitly associated with user defined identifier.

eg.

typedef float REAL;

REAL x, y[25], *pr; (== float x, y[25], *pr;)

variations of typedef:

eg.

typedef char *string;

string name, sign; (== char *name, *sign;)

typedef double scalar;

typedef scalar vector[10];

vector a, b, c; (== a[10], b[10], c[10];)

54


55/138

10 Linear Linked Lists

a linear linked list consists of a series of items (or nodes), where each nodecontains a pointer to the next node.

struct list {

char d;

struct list *next;

};

typedef struct list element;

typedef element *link;

link head; (= struct list *head;)

head = (link) malloc(sizeof(element));head -> d = a;

head -> next = NULL;

head -> next = (link) malloc(sizeof(element));

head -> next -> d = b;

head -> next -> next = NULL;

55


56/138

10.1 basic list operations

creating a list

counting the elements

looking up an element

concatenation two lists

inserting an element

deleting an element

creating a listcf.- stack: nodes are accessed in a last in, first out (LIFO) manner.- queue: nodes are accessed in a first in, first out (FIFO) manner.

- creating a lista function that will produce a list from a string

(1) iterative function

link s_to_l(s)

char s[ ];

{

link head = NULL, tail;

int i;

if (s[0]! = \0) {

head = (link)malloc(sizeof(element));

head -> d = s[0];

tail = head;

for (i=1; s[i]! = \0; i++){

tail -> next = (link) malloc(sizeof(element));

tail = tail -> next;

tail -> d = s[i];

}

tail -> next = NULL;

}return(head);

}

56


57/138

(2) recursive function

link s_to_l(s)char s[ ];

{

link head;

if (s[0] == \0) return(NULL);

else {

head = (link) malloc(sizeof(element));

head -> d = s[0];

head -> next = s_to_l(s+1);

return(head);

}

}

- counting the elements

int count(head)

link head;

{

if (head == NULL) return (0);

else return(1+count(head -> next));

}

- printing the elements

void print_list(head)link head;

{

if (head == NULL) prinf ("NULL");

else{

printf("%c ->", head ->d);

print_list(head -> next);

}

}

- concatenating list a and list b

void cat_list[a, b)link a, b;

{

if (a -> next == NULL) a -> next = b;

else cat_list (a -> next, b);

}

57


58/138

- inserting an element

void insert(p1, p2, q)

link p1, p2, q;

{

p1 -> next = q;

q -> next = p2;

}

- deleting an element

void delete_from_list(head, ch)

link head; char ch;

{ list temp, prev; int found = 0;

temp = head;

while (temp != NULL && !found) {

if (temp -> d == ch) found = 1;

else {

prev = temp;

temp = temp -> next;

}

if found{

if(temp == head) head = temp -> next;

else {prev -> next = temp -> next}; free(temp);

}

}

- recursive deletion of a list

void destroy_list(head)

link head;

{

if (head != NULL){

destroy_list (head -> next);

free(head);

}

}

58


59/138

10.2 Stacks

Stacks are data structures to which entries can be added or removed in alast in first out (LIFO) manner.

standard stack operations:

isempty(t): return 1 if stack is empty

otherwise 0

vtop(t): return the value of the top element

pop(t): return the value of the top element and remove it

push(t, d): place the value d onto the top of the stack

declaration of stack data type:

struct stack{

char d;

struct stack *next;

};

typedef struct stack element;

typedef element *top;

59


60/138

- int isempty(t)

top t;{

return(t == NULL);

}

- char vtop(t)

top t;

{

return(t -> d);

}

- char pop(t)

top *t;

{

top t1 = *t; char ch;

if (! isempty(t1)) {

ch = t1 -> d;

*t = t1 -> next;

free(t1);

return(ch);

}

else

return(\0);}

- void push(t, x)

top *t ; char x;

{

top temp;

temp = (top) malloc (sizeof (element));

temp -> d= x;

temp -> next = *t;

*t = temp;

}

60


61/138

eg. reversing a string using a stack

void reverse(s)

char s[ ];

{

int i;

top t = NULL;

for(i=0; s[i]! = \0, i++) push(&t, s[i]);

for(i=0; s[i]! = \0, i++) putchar(pop(&t));

}

61


62/138

10.3 Queues

Queues are data structures that can be used to store information thatneeds to be accessed in a first in, first out (F1, F0) manner.

declaration of queue data type:

struct list{

char d;

struct list *next;

};

typedef struct list element;

typedef element *link;

struct queue_struct{

link front;

link rear;

};typedef queue_struct *queue;

queue operations:

make queue: set up the queue pointers.

add to queue: add a new item to the rear of queue.

remove from queue: remove an item from the front of a queue.

62


63/138

queue make_queue( )

{ queue q;

q = (queue)malloc(sizeof(struct queue_struct));

if (q != NULL){

q -> front = NULL;

q -> rear = NULL;

}

return(q);

}

void add_to_quene(q, ch)

queue q; char ch;{

link node;

node = (link)malloc(sizeof(element));

node -> d = ch;

node -> next = NULL;

if (q -> rear == NULL)

q -> front = q -> rear = node;

else{

q -> rear -> next = node;

q -> rear = node;

}

}

char remove_from_quene(q)

queue q;

{

char ch = \0; link temp;

if (q -> front != NULL) {

ch = q -> front -> d;

temp = q -> front;

q -> front = q -> front -> next;

free(temp);

}if (q -> front == NULL)

q -> rear = NULL;

return(ch);

}

63


64/138

11 Trees

k-ary trees: every node has at most k children.

balanced tree: the leaf nodes are all either at the lowest

or next-to-lowest levels.

declaration of binary tree:

struct node{

int d

struct node *left;

struct node *rignt;

};

typedef struct node element;

typedef element *btree;

64


65/138

create a binary tree from an array

int a[10] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};

btree create_tree( );

create_tree(a, 0, 10)

btree init_node(d, p1, p2)

int d; btree p1, p2;{

btree t;

t = (btree)malloc(sizeof(element));

t -> data = d

t -> left = p1;

t -> right = p2;

return(t);

}

btree creat_tree(a, i, size)

int a[ ], i, size;

{

if (i >= size) return(NULL);

else return(int_node(a[i], create_tree(a, 2*i+1, size),creat_tree(a, 2*i+2, size)));

}

65


66/138

create_tree(a, 0, 10);

66


67/138

binary tree traversal

- inorder : left, root, right- preorder : root, left, right

- postorder : left, right, root

inorder(root)

btree root;

{

if (root != NULL){

inorder(root -> left);

printf("%d ", root ->d);

inorder(root -> right);

}}

preorder(root)

btree root;

{

if(root != NULL){

printf("%d ", root -> d);

preorder(root -> left);

preorder(root -> right);

}

}

postorder(root)

btree rooot;

{

if (root != NULL){

postorder(root -> left);

postorder(root -> right);

printf("%d ", root -> d);

}

}

67


68/138

eg.

inorder(root); 7 3 8 1 9 4 0 5 2 6

preorder(root); 0 1 3 7 8 4 9 2 5 6

postorder(root); 7 8 3 9 4 1 5 6 2 0

68


69/138

12 Sorting Methods

records: data structures which include several fields of information.

keys: the data attributes which are used for sorting

elementary methods: require O(n2) steps on the average to sort n ran-domly arrayed records

bubble sort

insertion sort

selection sort

shell sort

advanced methods: require O(n log n) steps on the average merge sort

quick sort

heap sort

12.1 Insertion Sort

insertion sort(A) algorithm: an array A[1, . . ., n] is sorted in ascendingorder.

1. for j 2 to length[A]

2. do key A[ j ]3. i j-1

4. while i>0 and A[ i ] > key

5. do A[i+1] A[ i ]

6. j i-1

7. A[i+1] key

Steps 3 through 7 indicate that insert A[ j ] into

the sorted sequence A[1, . . ., j-1]

69


70/138

eg. A = 5, 2, 4, 6, 1, 3

running time analysis cost time

1. for j 2 to length[A] c1 n

2. do key A[ j ] c2 n 1

3. i j 1 c3 n 1

4. while i > 0 and A[ i ] > key c4n

j=2 tj

5. do A[i + 1] A[ i ] c5n

j=2(tj 1)

6. i i 1 c6n

j=2(tj 1)

7. A[i + 1] key c7 n 1

T(n) = c1n + c2(n 1) + c3(n 1) + c4n

j=2 tj + c5n

j=2(tj 1)

+c6n

j=2(tj 1) + c7(n 1)

tj = j for j = 1, . . . , n 1

nj=2 tj =

nj=2j =

n(n+1)2 1

nj=2(tj 1 =

n(n1)2

T(n) = (c42 +

c52 +

c62 )n

2

+(c1+c2+c3+c42

c52

c62 +c7)n(c2+c3+c4+c7)

(worst-case) running tim of o(n2)

average-case running time of o(n2)

70


71/138

C program of insertion sort:

void insertion_sort(a, n)

double a[ ]; int n;

{

int i, j;

double key;

for(j=1; j= 0 && a[i] > key){

a[i+1] = a[i];

i = i-1;

}

a[i+1] = key;}

}

71


72/138

12.2 Quicksort

Sorting Performance

worst-case running time: O(n2)

average running time: O(n log n)

Algorithm:

1. divide procedure:the array A[p,r] is partitioned into two nonempty subarrays A[p,q]and A[q+1,r] such that each element of A[p,q] is less than or equalto each element of A[q+1,r].

2. conquer procedure:A[p,q] and A[q+1,r] are sorted by recursive calls to quicksort.

void quicksort(a, n)

int a[ ], n;

{

int k, pivot;

if(findpivot(a, n, &pivot)!= 0){

k = partition(a, n, pivot);

quicksort(a, k);

quicksort(a+k, n-k);

}}

int findpivot(a, n, pivot_ptr)

int a[ ], n, *pivot_ptr;

{

int i;

for(i=1; ia[i])?a[0]:a[i];

return(1);

}

return(0);}

72


73/138

int partition(a, n, pivot)

int a[ ], n, pivot;{

int i=0, j=n-1, temp;

while(i=pivot) j--;

if(i


74/138

Performance of quicksort

1. Worst-case partitioning:

T(n) = T(n 1) + O(n)where O(n) represents partitioning cost.

Since T(1) = O(1),T(2) = O(1) + O(2), ,T(n) =

O(k) = O(

k) = O(n2).

In general,

O(f(k)) = O(

f(k)).

)(2

nO

n

n -1

n -2

n -3

1

1

1

1

1 1

2

2

n

n

n -1

no. of operations

Figure 2: Worst-Case Partitioning

74


75/138

2. Best-case partitioning:

T(n) = 2T(n/2) + O(n) = O(n log2 n)

In general, T(n) = f(n) + aT(n/b) T(n) = O(nlogba logb n).

(proof)T(n) = f(n) + aT(n/b)= f(n) + af(n/b) + a2T(n/b2)= f(n)+af(n/b)+a2f(n/b2)+...+alogbn1f(n/blogbn1)+alogbnT(1).

Since alogbnT(1) = O(nlogba),T(n) = O(nlogba) +

ajf(n/bj ). ... (1)

f(n) = O(nlogba).aj (n/bj )

logb

a= nlogba

(a/blogba)j = nlogba

1 = nlogba logb n.

... (2)

From (1) and (2),T(n) = O(nlogba logb n).If a = b = 2, T(n) = O(n log2 n).

4

n

4

n

4

n

4

nn2log

)log( 2 nnO

2

n

2

n

n n

n

n

n

1 1 1 1

Figure 3: Best-case partitioning

75


76/138

3. Average-case partitioning:

The average-case running time is much closer to the best case.

eg.T(n) = T(9n/10) + T(n/10) + O(n)log10 n = O(log2 n) (log10 n < log2 n)log10/9 n = O(log2 n) (log10/9 n > log2 n) the total cost = O(n log2 n)

n10

1n

10

9

100

1

100

9

100

9nnnn

100

81

1

1

n n

n

n

n

n10log

n

9

10log


77/138

12.3 Heapsort

heap:

1. ascending heapa balanced binary tree in which the key value for every node is greaterthan or equal to the key value for its parent node.

2. descending heap

heapsort algorithm: making a heap from an array a[1..n]

heapify(a, n)

{

for i


78/138


79/138

Performance of heapsort

running time of heapsort :no. of calls to bubbledown

. . .


80/138

heapsort(a, n)

{ heapify(a, n) -> O(n)

i < - n

while (i > 1) {

swap(a[1], a[i])

i O(log n)

}

}

total running time of heapsort: O(n log2 n)

80


81/138

13 Searching Methods

13.1 linear and binary search methods

linear search: the sequential process of scanning through the records,starting at the first record

binary search: the procedure for locating an element in an already sortedarray

eg.

int bin_search(a, n, target)int a[ ], n, target;

{

int low, high, middle;

low = 0; high = n-1;

while(low a[middle])

low = middle+1;

else

return(middle);

}return(not_found);

}

it requires O(log2n) iterations in the worst case.eg.

int a[8]={1,4,8,10,11,12,16,17};

bin_search(a, 8, 8);

low=0, high=7

----------------

middle=3, high=2----------------

middle=1, low=2

----------------

middle=2, found

----------------

81


82/138

13.2 binary search tree

every node in the left subtree has a key value no larger than the root node.

every node in the right subtree has a key value no smaller than the rootnode.

this property applies recursively to each subtree of the complete tree.eg.

5

4 9

1 8 10

10

9

8

5

4

1

tree traversal: a process of visiting all nodes in a tree.pre-order : root,left,rightin-order : left,root,rightpost-order : left,right,root

searching in a binary search treesearching procedure :

1. check to see if the root node is empty2. compare the search key with the root key

3. if it matches, end the search

4. if it is smaller, search the left subtree

5. otherwise, search the right subtree

82


83/138

struct node{

int data;struct node *left;

struct node *right;

};

typedef struct node *bin_tree;

bin_tree find_node(t, value)

bin_tree t; int value;

{

if(empty_tree(t)) return(t);

else{

if(t->data == value) return(t);

else if(t->data > value)

return(find_node(t->left, value));else return(find_node(t->right, value));

}

}

int empty_tree(t)

bin_tree t;

{

if(t == NULL) return(1);

else return(0);

}

83


84/138

Adding a node to a binary search tree

Only one location at which the new node can be added.

Finding the location and adding the new node is required.

void add_node(t, new_node)

bin_tree t, new_node;

{

bin_tree prev;

while(t != NULL){

if(new_node->data < t->data){

prev = t;

t = t->left;}

else{

prev = t;

t = t->right;

}

}

if(new_node->data < prev->data)

prev->left = new_node;

else

prev->right = new_node;

}

eg.

7

add 5

4 9

1 8 10

7

84


85/138

Construction of a binary search tree

add_node()

the structure of the resulting tree will depend heavily on the sequencein which the nodes are added.

make a binary search tree from a sorted array

bin_tree creat_bst(a, n)

bin_tree a; int n;

{

int mid;

if(n == 0) return(NULL);

mid = n/2;

(a+mid)->left = creat_bst(a, mid);(a+mid)->right = creat_bst(a+mid+1, n-mid-1);

return(a+mid);

}

0 1 11

0 1 2 3 4 5 6 7 8 9 10 11

a

data

0 1

left right

11

l m r

l m mr rl

l l l lm m m m

6

3 9

1 5 8 11

0 2 4 7 10

(balanced) banary search tree

. . .

85


86/138

Node deletion

1. locate the node to be deleted

2. node deletion

(a) a leaf node: just remove the node.

(b) a node with only one child:the node is replaced with its child node.

(c) a node with two children:- two options:

i. replace it with the largest-key-valued node fromthe left subtree.

ii. replace it with the smallest-key-valued node fromthe right subtree.

55

917770

867437

805140

6348

(a)

(b) (c)

functions to be implemented:

delete_node( ) includes the following functions:

find_parent( )

delete_leaf( )

delete_one_child( )

delete_two_children( )

86


87/138

#define true 1

#define false 0

bin_tree delete_node(t, value, flag)

bin_tree t; int value; int *flag;

{

bin_tree parent, node;

node = find_parent(t, &parent, value);

if(node == NULL) {

*flag = false;

return(t);

}

if((node->left == NULL) && (node->right == NULL))

t = delete_leaf(t, parent, node);

else if((node->left == NULL) || (node->right == NULL))t = delete_one_child(t, parent, node);

else delete_two_children(t, node);

*flag = true;

return(t);

}

bin_tree find_parent(t, p, value)

bin_tree t; bin_tree *p; int value;

{

*p = NULL;

while(t != NULL && value != t->data){

*p = t;if(value < t->data)

t = t->left;

else

t = t->right;

}

return(t);

}

87


88/138

bin_tree delete_leaf(t, p, node)

bin_tree t, p, node;{

if(p == NULL) t = NULL;

else if(p->data < node->data)

p->right = NULL;

else

p->left = NULL;

free(node);

return(t);

}

88


89/138

bin_tree delete_one_child(t, p, node)

bin_tree t, p, node;{

if(p == NULL){

if(node->left == NULL) node = t->right;

else node = t->left;

free(t);

return(node);

}

if(node->left == NULL)

if(p->data < node->data)

p->right = node->right;

else

p->left = node->right;

elseif(p->data < node->data)

p->right = node->left;

else

p->left = node->left;

free(node);

return(t);

}

89


90/138

void delete_two_children(t, node)

bin_tree t, node;{

int temp;

bin_tree small, small_parent;

/* find the node which has the smallest

key value in the right subtree */

small = node->right;

small_parent = node;

while(small->left != NULL){

small_parent = small;

small = small->left;

}

temp = small->data;

if(small->left == NULL && small->right == NULL)

delete_leaf(t, small_parent, small);

else

delete_one_child(t, small_parent, small);

node->data = temp;

}

90


91/138

14 Hashing

Hashing involves some transformation of the search key which is used toaccess an element of the arrayeg. telephone directories, on-line dictionaries

Hash Table : an array of active records

Hash Function : mapping the record key into the appropriate array indexspreading the records fairly evenly over the hash table, and minimizingthe number of collisions (duplicate hashes) collision resolution process

linear probing

double hashing

chaining

student ID number % 8

960324

970344

980285

(in am320)

eg. h(key_value)=key_value % M

where M is the size of the hash table.

h

0

1

7

hash table

** hash entry can be a pointer, or a

data structure containing all info.

91


92/138

14.1 Linear Probing

1. set up a hash table

key_valueh 0

M-1

if null,

set that entry in the hash table

else

scan the hash table entries

to find the first null entry.

(no null entry -> hash table is full)

2. search the hash table

key_valueh 0

M-1

if the hash table entry points

to the item, search is successful.

else if null, search is unsuccessful.

else scan subsequent entries

until a null entry is found or searching

item is found.

init_hash(): allocates an initial, empty hash table

make_hash(): 1(hash table -> linear linked list)

search_hash(): 2

#define table_full -1

struct node{

int data;

struct node *next;};

typedef struct node element;

typedef element *list

92


93/138

list *init_hash(n); int n;

{ list *hash_table;

int i;

hash_table = calloc(n, sizeof(list));

if(hash_table == NULL) exit(-1);

for(i=0; idata != key)){

entry = (++entry)%n;

count++;

}

if(count == n) return(table_full);

return(entry);

}

cf. exit(status)

this function terminates a program.

all buffers are finished and files are closed

the function returns the value of status to the calling

process.

status: 0 the program ran properly

(integer) non-zero otherwise

93


94/138

list *make_hash(l, n)

list l; int n;{

list *hash_table;

int j;

hash_table = init_hash(n);

while(l != NULL){

j = search_hash(hash_table, n, l->data);

if(j == table_full) exit(2);

if(*(hash_table+j) == NULL) *(hash_table+j) = l;

else printf("Record duplicate exits\n");

l=l->next;

}

return(hash_table);}

h

l

ht

ht+j

94


95/138

14.2 Double Hashing

Repeating the hash process itself with the second hash function.

eg. settling up a hash table

entry number = h(k)

if null, set that entry in the hash table

else apply the second hash function

h(k)=M-2-k%(M-2) [1,...,M-2]

entry number = entry no.+h(k)

repeat the above procedure until a null entry is found

or table_full is found.

int search_hash(table, n, key)

list table[ ]; int n, key;

{

int entry, count;

entry = key%n;

for((count = 0; (count < n) && (table[entry] != NULL)&&

(table[entry]->data != key); count++)

entry = (entry+(n-2)-key%(n-2))%n;

if(count == n) return(table_full);

return(entry);}

95


96/138

14.3 Chaining

Set up a linked list whenever a duplicate hash is encountered.

If there is a lot of uncertainty about the problem size, chaining may bethe preferred approach.

key_valueh 0

n-1

1

int search_hash_chain(table, n, key, pre1)

list table[ ]; int n, key; list *pre1;

{

int entry;

list temp;

entry = key%n;

temp = table[entry];while((temp != NULL) && (temp->data != key)){

*pre1 = temp;

temp = temp->next;

}

return(entry);

}

96


97/138

list *make_hash_chain(l, n)

list l; int n;{

list *hash_table; int i, j;

list temp, pre2;

hash_table = init_hash(n);

while(l != NULL){

temp = l->next;

j = search_hash_chain(hash_table, n, l->data, &pre2);

if(*(hash_table+j) == NULL) ==>(i)

{*(hash_table+j) = l; l->next = NULL;}

else if(pre2->next == NULL) ==>(ii)

{pre2->next = l; l->next = NULL;}

else{printf("Record duplicate exits\n"); free(l);}

l = temp;

}

return(hash_table);

}

templ

l

a b c

templ

pre2

pre1

pre2

pre1

a

0

(i)

(ii)

b cn-1

97


98/138

Deleting a record

linear probing & double hashing: a dummy node can be allocated tothe delete node to permit searches for duplicated hash record lyingbeyond the removed record.

chaining: remove a node from a linked list

98


99/138

15 Root Finding Methods

Finding the real root of univariate function

assumptions:

1. known interval [x1, x2]initial starting value, x0 [x1, x2]

2. at most one root

3. approximate solution

- floating point arithmetic

- iterative methods (asymptotic convergence)

basic idea: find the interval [x1, x2] where f(x1) f(x2) 0

the interval [x1, x2] must contain at least one root.(odd number of roots)

15.1 Root Finding Based on Search Methods

1. linear search:

find_interval( ) returns

1 if a root is found

0 otherwise

f: target function; xstart, xend: given interval;

xleft, xright: interval containing a root;

int find_interval(f, xstart, xend, stepsize, xleft, xright)

double (*f)(), xstart, xend, stepsize, *xleft, *xright;

{

int found_root = TRUE;

*xleft = xstart;

*xright = *xleft + stepsize;

while(*xleft < xend && f(*xleft)*f(*xright) > 0.0){

*xleft = *xright;*xright += stepsize;

if(*xright > xend) *xright = xend;

}

if(f(*xleft)*f(*xright) > 0.0) found_root = FALSE;

return(found_root);

}

99


100/138

finds only the first interval containing the root.

fails to find an interval if an ever number of roots are sufficiently closeto one another

2. binary search (bisection method):

divide and conquer method -the initial interval [x1, x2] is successively halved the size of interval at the nth iteration = (x2 x1)/2

n

f: target function; x1, x2: given interval;

epsilon: tolerance (accuracy) of solution

double bisect(f, x1, x2, epsilon)double (*f)(), x1, x2, epsilon;

{

double y;

for(y=(x1+x2)/2.0; fabs(x1-y)>epsilon; y=(x1+x2)/2.0)

if(f(x1)*f(y)


101/138

15.2 Root Finding Based on Target Functions

1. secant method

algorithm

(a) fitting a straight linebetween the points, (x1, f(x1)) and (x2, f(x2)).

(b) find the root, x of the straight line.

(c) ifx is close to x2, stop.otherwise, x1 x2, x2 x, and go to step1.

pros and cons:

root can be the value outside the initial interval. there is no way to detect that it will fail to find a root

(x2, y2)y = f(x)

x2x3

x1

(x1, y1)

x4

101


102/138

C program

f: target function; x1, x2: given interval;

epsilon: tolerance; max_tries: maximum number of iterations;

found_flag: flag of finding a solution

double secant(f, x1, x2, epsilon, max_tries, found_flag)

double (*f)(), x1, x2, epsilon;

int max_tries, *found_flag

{

int count = 0;

double root = x1;

*found_flag = 1;while(fabs(x2-x1) > epsilon && count < max_tries){

root = x1 - f(x1)*(x2-x1)/(f(x2)-f(x1));

x1 = x2;

x2 = root;

count++;

}

if(fabs(x2-x1) > epsilon) *found_flag = 0;

return(root);

}

102


103/138

2. Newtons method

algorithm:

(a) initial guess xn(n = 0).

(b) find the root of the straight line that fits a tangent to f(x):

xn+1 = xnf(xn)

f(xn)

f(xn) =

f(xn) f(xn+1)

xn xn+1and f(xn+1) 0 as n

(c) if |xn+1 xn| < , stopotherwise, go to 2.

cf. f(xn)f(xn) could result in a floating-point overflow error message

different initial guess, start again

x0x1

x2

103


104/138

C program

f: target function; fprime: f(x_n);

dmin: minimum value of f(x_n); x0: starting point;

epsilon: tolerance; error: flag of finding a solution

double newton(f, fprime, dmin, x0, epsilon, error)

double (*f)(), (*fprime)(), dmin, x0, epsilon; int *error;

{

double deltax;

deltax = 2.0*epsilon;

*error = 0;

while(!(*error) && fabs(deltax) > epsilon)if(fabs(fprime(x0) > dmin){

deltax = f(x0)/fprime(x0);

x0 = x0 - deltax;

}

else

*error = 1;

return(x0);

}

104


105/138

problems in applying root finding methods

(a) false convergence in secant and Newtons methods -very small change in xn and xn+1 but far from a root check whether the returned value is a root

xi1 xi

xi+1

(b) cycling (in Newtons method)f(x) = 1

1+ex 12

selecting a different starting value.

xn+1

xn

x

y

105


106/138

16 Numerical Integration

Integration of f(x) in the range of x [a, b] is approximately computed bythe summation of weighted function values.

16.1 Trapezoidal Rule

Rectangular Rule: Integration of f(x) in the range of x [a, b] is approxi-mated by Riemannian integration formula. rarely used in practice.

y = f(x)

x

y

y = f(x)

y

x

106


107/138

Trapezoidal Rule: Integration of f(x) in the range of x [a, b] is approx-

imated by the summation of trapezoidal segments more accurate ap-proximation than the rectangular rule.

y = f(x)

y

x

double trapezoidal(f, a, b, n)

double (*f)(), a, b; int n;

{

double answer, h;

int i;

answer = f(a)/2;

h = (b-a)/n;

for(i=1; i


108/138

The error bound of trapezoidal rule is given by O(h2).

Here, the error is given by

E =

ba

f(x)dx b a

2(f(a) + f(b))

.

The Taylor series expansion of f(x) around x is

f(x) = f(x) + zf(x) +z2

2f(x) +

where z = x x.

That is,

ba

f(x)dx =

h/2h/2

(f(x) + zf(x) +z2

2f(x) + )dz

= hf(x) +1

24h3f(x) + (1)

On the other hand,

b a

2(f(a) + f(b)) =

h

2

f(x)

h

2f(x) +

1

2

h

2

2f(x) + f(x) +

h

2f(x) +

= hf(x) +1

8h3f(x) + (2)

From (1) and (2),

E =

ba

f(x)dx b a

2(f(a) f(b))

= 1

12h3f(x)

Therefore, total error is given by

ET = 1

12

(B A)3

N3

N

i=1f(x)

where h = BAN and xi =12

(xi + xi+1).

Let f = 1NN

i=1 f(x). Then,

ET = 1

12(B A)h2f O(h2).

108


109/138

16.2 Simpsons Rule

Simpsons Rule: approximating the function within each of n intervals bysome polynomials

the second-order polynomials (Simpsons rule):Sum = h6 [f(x) + 4f(x +

h2

) + f(x + h)] error bound: O(h4)

f(x)

f(x + h)

x + hx

f(x + h2

)

the third-order polynomials (Simpsons 3/8 rule):Sum = h8 [f(x) + 3f(x +

h3 ) + 3f(x +

23h) + f(x + h)]

error bound : O(h5)

f(x)

f(x + h)

x + hx

f(x + 2h3

)

f(x + h3

)

109


110/138

double simpsion(f, a, b, n)

double (*f)(double), a, b; int n;{

double answer, h, x;

int i;

answer = f(a);

h = (b-a)/n;

for(i=1; i


111/138

- The First Method:

double new_simpson(f, a, b, no, tolerance)

double (*f)(double), a, b; int no; double tolerance;

{

double check = tolerance + 1.0;

double lowval, val;

lowval = simpson(f, a, b, no);

while(check > tolerance){

no = 2*no;

val = simpson(f, a, b, no);

check = fabs((val-lowval)/val);

lowval = val;

}return(val);

}

- The Second Method

double adapt(f, a, b, n, tolerance)

double(*f)(double), a, b; int n; double tolerance;

{

double val, check;

val = simpson(f, a, b, 2*n);

check = fabs((simpson(f,a,b,n)-val)/val);if(check > tolerance)

val = adapt(f, a, (a+b)/2.0, n, tolerance)+

adapt(f, (a+b)/2.0, b, n, tolerance);

return(val);

}

Other Methods

Quadrature method (Refer Numerical Recipes in C)

Monte Carlo integration

111


112/138

17 Numerical Solutions of

Ordinary Differential Equations

the solution of the first-order ordinary differential equations (O.D.E.s)

dy

dx= g(x, y)

conversion of an math-order O.D.E. to the first-order O.D.E.

eg. y = g(x,y,y, y)

let y0 = y, y1 = y, y2 = y, then

y2

= g(x, y0, y1, y2)where y1 = y2, y

0 = y1

y =

y2y1

y0

= g(x, y0, y1, y2)y2

y1

17.1 Eulers Method

Eulers method is the most straightforward method, but not accurate.

Problem: y = g(x, y)

algorithm:

Step 1. set up the variables :x = x0, y = y0 (initial value of x and y)xlast (the largest value of x)h (an increment of x)

Step 2. output x and y

Step 3. set x = x + h

Step 4. compute y = y + h g(x, y)

Step 5. if x > last, stopotherwise, go to Step 2.

112


113/138

C program:

void euler(x0, y0, g, h, xlast)

double x0, y0, (*g)(), h, xlast;

{

for(; x0 xlast, stopotherwise, go to step 2

the local error per step: O(h3).

113


114/138

Second-order Runge-Kutta method

dydx = g(x, y), y(0) = y0

yn+1 = yn +xn+1

xng(x, y)dx

xn+1 = xn + h

applying the trapezoidal rule to the integration term:xn+1xn

g(x, y)dx = 12h[g(xn, yn) + g(xn+1, yn+1)]

yn+1 = yn + hg(xn, yn), (estimate of yn+1 using Eulers method)yn+1 = yn +

h

2 [g(xn, yn) + g(xn+1, yn+1)] (1) the local error par step: O(h3)

Error Analysis:

dydx = g(x, y)

Taylor series expansion around xn and yn:

yn+1 = yn + hg +h2

2[gx + gyg] +

h3

6[gxx + 2gxyg + gyy g2 + gxgy + g2yg] +

o(h4) (2)where gx =

gx |x=xn,y=yn , gy =

gy |x=xn,y=yn

applying Taylor series expansion of g(xn+1, yn+1) around (xn, yn) in (1)

yn+1 = yn + hg +h2

2[gx + gyg] +

h3

4[gxx + 2gxyg + gyy g2] + o(h4) (3)

by comparing (2) and (3)error in the order of h3 O(h3)

114


115/138

Fourth-order Runge-Kutta method (most widely used method)

dydx = g(x, y), y(0) = y0yn+1 = yn +

xn+1xn

g(x, y)dxxn+1 = xn + h

1. applying the simpsons 1/3 rule in the integral term:

xn+1xn

g(x, y)dx = h6 [g(xn, yn) + 4g(xn +h2

, yn + ) + g(xn+1, yn+1)]

k1 = hg(xn, yn)k2 = hg(xn +

h2

, yn +k12

)

k3 = hg(xn +h2 , yn +

k22 )

k4 = hg(xn + h, yn + k3)yn+1 = yn +

16

[k1 + 2k2 + 2k3 + k4]

the local error per step: O(h5)

2. applying the Simpsons 3/8 rule in the integral term:

xn+1xn

g(x, y)dx = h8 [g(xn, yn)+3g(xn +h3 , yn +)+3g(xn +

2h3 , yn)+

g(xn, yn)]

k1 = hg(xn, yn)k2 = hg(xn + h3 , yn +

k13

)

k3 = hg(xn +2h3

, yn +k13

+ k23

)k4 = hg(xn + h, yn + k1 k2 + k3)yn+1 = yn +

18 [k1 + 3k2 + 3k3 + k4]

the local error per step: O(h5)

115


116/138

algorithm:

Step 1. set up the variablesx = x0, y = y0,xlast,h,ta,tb,tc,td

Step 2. output x and y

Step 3. computeta = h g(x, y) estimation of y evaluated at xtb = h g(x + h/2, y + ta/2),tc = h g (x + h/2, y + tb/2) estimation of y evaluated atx + h/2td = h g(x + h, y + tc) estimation of y evaluated at x + h

Step 4. compute y = y + 16(ta + 2tb + 2tc + td)

Step 5. set x = x + h

Step 6. if x > xlast, terminateotherwise, go to step 2

C program:

void runge_kutta(x0, y0, g, h, xlast)

double x0, y0, (*g)(), h, xlast;

{

double ta, tb, tc, td;

for(; x0


117/138

18 Numerical Solution of

Simultaneous Equations

Consider only the case where the number of linear equations is same as thenumber of unknowns: n equations and n unknowns

a00 a01 a0,n1a10 a11 a1,n1

......

. . .. . .

an1,0 an1,1 an1,n1

x0x1...

xn1

=

b0b1...

bn1

Ax = b

18.1 simple Gauss-Jordan Elimination

Let us consider the following simultaneous equations:

2x0 + 3x1 + 5x2 = 10x0 + 2x1 + 4x2 = 5

4x0 + 7x1 + 3x2 = 15

2x0 +3x1 + 5x2 = 100.5x1 + 1.5x2 = 0

1x1 7x2 = 5

2x0 4x2 = 10

0.5x1 +1.5x2 = 010x2 = 5

2x0 = 120.5x1 = 0.75

10x2 = 5

x0 = 6, x1 = 1.5, and x2 = 0.5

G-J elimination by the ith diagonal element (aii):

rowj = rowj ajiaii

rowi (j = i)

117


118/138

To program G-J elimination method, fist make a working matrix by

augmenting A matrix with b (n rows, n + 1 columns).

e.g.

w =

A b

=

2 3 5 101 2 4 5

4 7 3 15

C program:

void simple_gauss(a, b, x, n)

double **a, b[], x[]; int n;

{

double **work;

double m; int i, j, k;

/* working matrix */

work = (double **)calloc(n, sizeof(*work));

for(i=0; i


119/138

problem in G-J elimination:

this method fails if any of the pivot elements is zero. Select a nonzero pivot element at each step (partial pivoting).

Select the pivot element with the greatest absolute magnitude(for the numerical stability).

If all of candidate pivot elements are zero,no solution or an infinite number of solutions exists.

other pivoting methods (in order to provide numerical stability)

implicit pivoting:

rescale all equations so that they are all same order of magnitude

use partial pivoting

the final solution is rescaled back to the original units

full pivoting :

exchanges of both rows and columns are considered in order toselect the pivot element.(exchanging columns permutes the subscripts in vector x.)

the subscripts in the final solution should be reordered.

119


120/138

matrix inversion:

G-J elimination can be used to find the matrix inverse

x0 x1 xn1

=

1 0 00 1 00 0 0...

.... . .

...0 0 1

A A1 = I

Ax0 =

10

...0

, Ax1 = 01

...0

, , Axn1 = 0...01

make a working matrix such as |A | I

|

120


121/138

18.2 LU Decomposition Methods

Any non-singular matrix A can be decomposed to the LU form.

A = LU

where L is a lower triangular matrix and U is a upper triangular matrix.

eg.

a11 a12 a13a21 a22 a23a31 a32 a33

=

1 0 0l21 1 0l31 l32 1

u11 u12 u130 u22 u230 0 u33

where lii = 1.

algorithm

1. the first row of U:u1j = a1j , j = 1, . . . , N .

the first column of L:

li1 = ai1/u11, i = 2, . . . , N .

2. the nth row of U (n 2):

unj = anj n1k=1

lnkukj , j = n , . . . , N .

the nth column of L (n 2):

lin =

ain

n1k=1

likukn

/unn, i = n + 1, . . . , N .

121


122/138

eg.

A =

2 1 31 3 2

3 1 3

u11 = 2, u12 = 1, u13 = 3,

l11 = 1, l21 = 0.5, l31 = 1.5,

u22 = 3 (1/2) 1 = 3.5, u23 = 2 (1/2)(3) = 0.5,

l22 = 1, l32 = [1 (1.5) 1]/3.5 = 0.14,

u33 = 3 (1.5)(3) (0.14)(0.5) = 1.57.

Therefore,

L =

1 0 00.5 1 0

1.5 0.14 1

and

U =

2 1 30 3.5 0.5

0 0 1.57

.

122


123/138

Solving simultaneous equations

Change simultaneous equations of Ax = y with LUx = y.Let U x = z, then Lz = y.Solve z Solve x.

eg.

- forward elimination: 1 0 0l21 1 0

l31 l32 1

z1z2

z3

= y1y2

y3

z1 = y1z2 = y2 z1l21

z3 = y3 z1l31 z2l32

- backward substitution: u11 u12 u130 u22 u23

0 0 u33

x1x2

x3

= z1z2

z3

x3 = z3/u33x2 = (z2 u23x3)/u22

x1 = (z1 u12x2 u13x3)/u11

algorithm

1. Forward elimination step:z1 = y1,

zi = yi i1j=1 lij zj, i = 2, . . . , N .2. Backward substitution step:

xN = zN/uNN,

xi = (zi N

j=i+1 uij xj )/uii, i = N 1, . . . , 1.

123


124/138

Computing the determinant

- Determinant of matrix A:det(A) =

ai1aj2ak3 arN where the summation is extended over

all permutations of the first subscripts of a.

eg.

det(A) = det

a11 a12 a13a21 a22 a23

a31 a32 a33

= a11a22a33 + a21a32a13 + a31a12a23 a11a32a23 a21a12a33 a31a22a13.

5 5 matrix: 5! = 120 terms (each term need 4 multiplications)10 10 matrix: 10! 2 108 terms (each term need 9 multiplications)

- Two important rules of determinants:

1. det(BC) = det(B)det(C)

2. det(M) = the product of all diagonal elements of Mif M is an upper or lower triangular matrix:det(A) = det(LU) = det(L)det(U) = det(U)

i.e. det(A) =N

i=1 uii

- Checking ill-conditioned problems

1. A number of problems which are solvable, yet their solutions becomevery inaccurate because of severe round-off errors.

2. Compare det(A)det(A1) with 1.

3. Compare A1(A1)1 with AA1. If there is significant deviation,increase the precision of computation.

124


125/138

18.3 Solutions ofm equations with n unknowns

Compute the solution of Ax = y:

Case 1. if m = n and rank(A) = n or det(A) = 0,unique solution of x exists.

Case 2. if m < n, rank(A) < n and det(A) = 0,A: singular matrix. no unique solution, infinite number of solutions.

eg. x + y = 1, m = 1, n = 2

Solution:x = 1 y or y = 1 xif we have m linearly independent equations for n unknowns and m < n,we have m basic variable n m free variables.

eg. G-J elimination for m < n:

2u + 3v + 1w + 4x + y = 6

2u + 3v + 1w + 1x y = 14u + 6v 1w + 1x + 2y = 5

m = 3, n = 5u v w x y RHS 2 3 1 4 1 62 3 1 1 1 14 6 1 1 2 5

G-J elimination:

u v w x y RHS 1 1.5 0 0 0.05 0.4

1 0 1.5 1.51 0.6 1.6

125


126/138

basic variables: u,w,x, free variables: v, y.u = 0.4 1.5v + 0.05yw = 1.5 + 1.5yx = 1.6 0.6y

Case 3. if m > n, no solution. We need to solve the approximate solution.

eg. linear regression

N samples are given:

(x1, y1), (x2, y2), . . . , (xN, yN).

The samples are generated by

yN =L

i=0

wixik +

whereyk: the output for the kth sample,wi: the ith weight, andxik: the ith input for the kth sample.: the random noise, usually Gaussian noise havingthe distribution of N(0, 2).

if L = 2, the estimation of output for the kth sample:

yk = w0 + w1x1k

126


127/138

yk = wTxk: the estimation of output for the kth sample

where xk = [x0k, x1k, , xLk]

T

and w = [w0, w1, , wL]

T

.

the error for the kth pattern:ek = yk yk = yk wTxk

We want to estimate w such that minimizing the mean square error (MSE):

E[e2k] =

e2kp(y|x)dy.

In practice, we use

1N

Nk=1

e2k

instead of E[e2k].

M SE = E[e2k] = E[(yk yk)2]

= E[(yk wTxk)

2]= E[y2k] + w

TE[xkxTk ]w 2E[ykxTk ]w

Let R = E[xkxTk ] (input correlation matrix) and P = E[ykxk].

Then,MSE = E[e2k] = E[y

2k] + w

TRw 2PTw.

Ew |w=w = 0

2Rw 2P = 0

w = R1P

minimum mean square error (MMSE):

MMSE = E[y2k] + wTRw 2PTw

= E[y2k] PTw

127


128/138

regression algorithm:

1. Read (x1, y1), , (xN, yN).

2. Compute R and P:

R = 1NN

k=1 xkxTk

P = 1NN

k=1 ykxk

3. Compute R1 by G-J elimination method(or LU decomposition method).

4. Compute w:

w = R1

P

128


129/138

19 Random Number Generation

19.1 Creating Uniform Random Numbers

linear congruential method

- The i + 1th random number (integer) is generated by

ri+1 = (ari + b)%m (0, , m 1)

where a, b and m are fixed constants.

- r0: seed, m: range.

eg.

a = 13, b = 1, and m = 17

r0 = 1r1 = (13 1 + 3)%17 = 16r2 = (13 16 + 3)%17 = 7r3 = (13 7 + 3)%17 = 9( cycle length=4)r4 = (13 9 + 3)%17 = 1...

- Select values of a, b and m so that the cycle length isas long as possible.

- Independent random number and uniform distribution.

- No absolute set of rules for selecting the values of a, b and m.a typical form:

a = 8x + 5

where x is a positive integer and b is an odd number.

129


130/138

ANSI C Standard(1988): rand() generate the random number

between 0 and RAND MAX (32767)

static unsigned long next=1;

int rand(void)

{

next = next*1103515245 + 12345;

return((unsigned long) (next/65536)%32768);

}

eg. generating random numbers between 0 and INT MAX:

#include

static int next=1;

Algorithm SC 2006

Documents

Transcript of Algorithm SC 2006