Testing in C

7/23/2019 Testing in C

http://slidepdf.com/reader/full/testing-in-c 1/41

1

Testing

The material for this lecture is drawn, in part, from

The Practice of Programming (Kernighan & Pike) Chapter 6

Jennifer Rexford



2

For Your Amusement

“On two occasions I have been asked [by members of Parliament!],

Pray, r" #abba$e, if yo% &%t into the machine wron$ fi$%res, will

the ri$ht answers come o%t'( I am not able ri$htly to a&&rehend the

kind of conf%sion of ideas that co%ld &rovoke s%ch a )%estion"*

‒ +harles #abba$e

“Pro$ram testin$ can be )%ite effective for showin$ the &resence

of b%$s, b%t is ho&elessly inade)%ate for showin$ their absence"*

‒ ds$er -i.kstra

“#eware of b%$s in the above code/ I have only &roved it correct,not tried it"*

-onald 0n%th‒



“Programming in the Large” Steps

-esi$n 1 Im&lement2 Pro$ram 1 &ro$rammin$ style

2 +ommon data str%ct%res and al$orithms

2 od%larity

2 #%ildin$ techni)%es 1 tools

-eb%$2 -eb%$$in$ techni)%es 1 tools

3est2 3estin$ techni)%es <-- 4e are here

aintain2 Performance im&rovement techni)%es 1 tools

3



Goals of this Lecture

5el& yo% learn abo%t62 Internal testin$

2 xternal testin$

2 7eneral testin$ strate$ies

4hy'2 It(s hard to know if a 8lar$e9 &ro$ram works &ro&erly

2 : &ower &ro$rammer s&ends at least as much time composing

test code as he;she s&ends com&osin$ the code itself

2 : &ower &ro$rammer knows how to s&end that time wisely

4



5

Program Verification

Ideally6 Prove that any $iven &ro$ram is correct

7eneral

Pro$ram

+hecker program.c

Ri$ht or 4ron$<&ecification

?



6

Program Testing

Pragmatically6 +onvince yo%rself that a specific

&ro$ram proaly works

<&ecific

3estin$<trate$y program.c

Probably Ri$ht

or

+ertainly 4ron$

<&ecification



Agenda

!"ternal testing2 #esigning data to test your program

Internal testin$

2 -esi$nin$ yo%r &ro$ram to test itself

7eneral testin$ strate$ies

7



8

Statement Testing

8=9 Statement testin$

2 “3estin$ to satisfy the criterion that each statement in a &ro$ram be

exec%ted at least once d%rin$ &ro$ram testin$"*

2 >rom the Glossary of Computerized System and Software

Development Terminology



9

Statement Testing !"ample

xam&le &se%docode6

if (condition1) statement1;else statement2 ;

…if (condition2 ) statement3 ;else statement4;…

Statement testin$6

<ho%ld make s%re both ifstatements and all ? nested

statements are exec%ted

5ow many &asses

thro%$h code are

re)%ired'



10

Path Testing

8@9 Path testin$

2 “3estin$ to satisfy covera$e criteria that each lo$ical &ath thro%$h

the &ro$ram be tested" Often &aths thro%$h the &ro$ram are$ro%&ed into a finite set of classes" One &ath from each class is then

tested"*





11

Path Testing !"ample

xam&le &se%docode6

2 <im&le &ro$rams AB maybe reasonable

2 +om&lex &ro$ram AB combinatorial ex&losion!!!

2 Path test code fra$ments

if (condition1) statement1;else statement2 ;…if (condition2 ) statement3 ;else statement4;…

Path testin$6

<ho%ld make s%re all lo$ical

&aths are exec%ted

5ow many &asses

thro%$h code are

re)%ired'



12

$oundary Testing

8C9 $oundary testin$ 8alias corner case testin$9

2 “: testin$ techni)%e %sin$ in&%t val%es at, .%st below, and .%st

above, the defined limits of an in&%t domain/ and with in&%t val%esca%sin$ o%t&%ts to be at, .%st below, and .%st above, the defined

limits of an o%t&%t domain"*





13

$oundary Testing !"ample

<&ecification62 Print the n elements of array a to stdout, in reverse order

:ttem&t6

void printBackwards(int a[] unsigned int n)! unsigned int i; for (i " n; i #" $; i--) printf(%&d'n% a[i]);

-oes it work' :&olo$ies for the

forward reference

to arrays



14

Stress Testing

8?9 Stress testin$

2 “3estin$ cond%cted to eval%ate a system or com&onent at or beyond

the limits of its s&ecified re)%irements*





15

Stress Testing

<ho%ld stress the &ro$ram with res&ect to62 %uantity of data

2 Dar$e data sets

2 Variety of data

2 3ext%al data sets containin$ nonE:<+II chars

2 #inary data sets

2 Randomly $enerated data sets

<ho%ld %se com&%ter to $enerate in&%t sets2 :voids h%man biases



16

Stress Testing !"ample &

<&ecification62 Print n%mber of characters in stdin

:ttem&t6

include <stdio.*#

int main(void)! c*ar c*ar+ount " $; w*ile (getc*ar() ," /) c*ar+ount00; printf(%&d'n% c*ar+ount); return $;

-oes it work'



17

Stress Testing !"ample '

<&ecification62 Read a line from stdin

2 <tore as strin$ 8witho%t 1'n19 in array of len$th 2332456789:

:ttem&t6

int i; c*ar s[2332456789:]; for (i " $; i < 2332456789:-; i00) ! s[i] " getc*ar(); if ((s[i] "" /) (s[i] "" 1'n1)) =reak;

s[i] " 1'$1;

-oes it work'



18

!"ternal Testing Summary

xternal testin$6 -esi$nin$ data to test yo%r &ro$ram

xternal testin$ taxonomy

8=9 <tatement testin$

8@9 Path testin$ 8C9 #o%ndary testin$

8?9 <tress testin$



Agenda


Internal testing

2 #esigning your program to test itself

7eneral testin$ strate$ies

19



20

Aside( The assert )acro

assert(int e>pr)2 If e>pr eval%ates to 3RF 8nonEGero96

2 -o nothin$

2 If e>pr eval%ates to >:D< 8Gero96

2 Print messa$e to stderr “assert at line x failed*

2 xit the &rocess

Fsef%l for internal testin$



21

Aside( The assert )acro

-isablin$asserts2 3o disable asserts, define 7B@8H

2 In code6

2 Or when b%ildin$6

A------------------------------------AA mCprogram.c A

A------------------------------------Adefine 7B@8…A 2sserts are disa=led *ere. A…

D gccEF G 7B@8 mCprogram.c Go mCprogram



22

Validating Parameters

8=9 alidate &arameters2 :t leadin$ ed$e of each f%nction, make s%re val%es of &arameters

are valid

int f(int i dou=le d)! assert(i has a reasonable value); assert(d has a reasonable value); …



23

Validating Parameters

2 xam&le

A 3eturn t*e greatest common divisor of positive integers

i and H. A

int gcd(int i int H)! assert(i # $); assert(H # $); …



24

*hec+ing In,ariants

8@9 +heck invariants2 :t leadin$ ed$e of f%nction, check as&ects of data str%ct%res that

sho%ld not vary/ maybe at trailin$ ed$e too

int isIalid (JC9Cpe o=Hect)

! … A +ode to c*eck invariants goes *ere. 3eturn (93@) if o=Hect passes all tests and $ (/26K) ot*erwise. A …

void mC/unction(JC9Cpe o=Hect)! assert(isIalid(o=Hect)); … A +ode to manipulate o=Hect goes *ere. A … assert(isIalid(o=Hect));



25

*hec+ing In,ariants

2 xam&le

2 “#alanced binary search tree insertion* f%nction

2 :t leadin$ ed$e6

2 :re nodes sorted'

2 Is tree balanced'2 :t trailin$ ed$e6

2 :re nodes still sorted'

2 Is tree still balanced'



26

*hec+ing -eturn Values

8C9 +heck f%nction ret%rn val%es2 +heck val%es ret%rned by called f%nctions

some3etIalue " f(some2rgs);if (some3etIalue "" =adIalue) A :andle t*e error A…

f(some2rgs);… Bad code (sometimes)

Good code

if (f(some2rgs) "" =adIalue) A :andle t*e error A…

Good code



27


2 xam&le6

2 scanf89 ret%rns n%mber of val%es read

2 +aller sho%ld check ret%rn val%e

int i H;

…if (scanf(%&d&d% Li LH) ," E) A :andle t*e error A

int i H;…scanf(%&d&d% Li LH);

Bad code

Good code



28


2 xam&le6

2 &rintf89 ret%rns n%mber of chars 8not val%es9 written

2 +an fail if writin$ to file and disk )%ota is exceeded

2 +aller sho%ld check ret%rn val%e'''

int i " $$$;…if (printf(%&d% i) ," M) A :andle t*e error A

int i " $$$;…

printf(%&d% i);Is this too

m%ch'

Bad code???

Good code???



29

*hanging *ode Temporarily

8?9 +han$e code tem&orarily2 3em&orarily chan$e code to $enerate artificial bo%ndary or stress

tests

2 xam&le6 :rrayEbased sortin$ &ro$ram

2 3em&orarily make array very small

2 -oes the &ro$ram handle overflow'



30

Lea,ing Testing *ode Intact

89 Deave testin$ code intact

2 -o not remove testin$ code when &ro$ram is finished

2 In the “real world* no &ro$ram ever is “finished*!!!

2 If testin$ code is inefficient6

2 mbed in calls of assert(), or

2 Fse ifdef…endif &re&rocessor directives

2 <ee :&&endix



31

Internal Testing Summary

Internal testin$6 -esi$nin$ yo%r &ro$ram to test itself

Internal testin$ techni)%es8=9 alidatin$ &arameters

8@9 +heckin$ invariants8C9 +heckin$ f%nction ret%rn val%es

8?9 +han$in$ code tem&orarily

89 Deavin$ testin$ code intact

$e.are of conflict et.een

internal testing and code clarity



Agenda


Internal testin$

2 -esi$nin$ yo%r &ro$ram to test itself

General testing strategies

32



33

Automation

8=9 :%tomate the tests2 +reate scripts to test yo%r programs

2 +reate software clients to test yo%r modules

2 +om&are im&lementations 8when &ossible9

2 ake s%re inde&endent im&lementations behave the same

2 0now what o%t&%t to ex&ect 8when &ossible9

2 7enerate o%t&%t that is easy to reco$niGe as ri$ht or wron$

:%tomated testin$ can &rovide62 %ch better covera$e than man%al testin$

2 #on%s6 xam&les of ty&ical %se of yo%r code



34

Testing Incrementally

8@9 3est incrementally2 3est as yo% com&ose code

2 :dd test cases as yo% com&ose new code

2 -o regression testing

2 :fter a b%$ fix, make s%re &ro$ram has not “re$ressed*

2 3hat is, make s%re &revio%sly workin$ code is not broken

2 Rer%n all test cases

2 Kote the val%e of a%tomation!!!

2 +reate scaffolds and stus as a&&ro&riateH



35

Testing Incrementally

/unction E

/unction N /unction M

/unction

Scaffold6 3em&orary

code that calls code

that yo% care abo%t

Stu6 3em&orarycode that is called

by code that yo%

care abo%t

+ode that

yo% care abo%t



36

$ug/#ri,en Testing

8C9 Det deb%$$in$ drive testin$

2 Reactive modeH

2 >ind a b%$ AB create a test case that catches it

2 Proactive modeH

2 -o fault in0ection

2 Intentionally 8tem&orarily!9 in.ect a b%$

2 ake s%re testin$ mechanism catches it2 3est the testin$!!!



37

General Strategies Summary

7eneral testin$ strate$ies8=9 :%tomation

8@9 3estin$ incrementally

8C9 #%$Edriven testin$



38

1ho #oes the Testing2

Pro$rammers2 1hite/o" testin$

2 Pro6 0now the code AB can test all statements;&aths;bo%ndaries

2 +on6 0now the code AB biased by code desi$n

L%ality :ss%rance 8L:9 en$ineers2 $lac+/o" testin$

2 Pro6 -o not know the code AB %nbiased by code desi$n

2 +on6 -o not know the code AB %nlikely to test all

statements;&aths;bo%ndaries

+%stomers2 Field testin$

2 Pros6 Fse code in %nex&ected ways/ “deb%$* s&ecs

2 +ons6 Often don(t like “&artici&atin$*/ diffic%lt to $enerate eno%$h

cases



39

Summary

xternal testin$ taxonomy2 <tatement testin$

2 Path testin$

2 #o%ndary testin$

2 <tress testin$

Internal testin$ techni)%es2 alidatin$ &arameters

2 +heckin$ invariants

2 +heckin$ f%nction ret%rn val%es

2 +han$in$ code tem&orarily

2 Deavin$ testin$ code intact



Summary 3cont45

7eneral testin$ strate$ies2 :%tomation

2 +om&arin$ im&lementations

2 0nowin$ what o%t&%t to ex&ect

2 3estin$ incrementally

2 Re$ression testin$

2 <caffolds and st%bs

2 #%$Edriven testin$

2>a%lt in.ection

3est the code M and the tests6

40



41

Appendi"( ifdef

Fsin$ ifdef…endif

2 3o enable testin$ code6

2 3o disable testin$ code6

…ifdef 9K95/29@35OA +ode to test feature O goes *ere. A

endif…

D gccEF G 9K95/29@35O mCprog.c Go mCprog

myprog.c

D gccEF mCprog.c Go mCprog

Testing in C

Documents

Transcript of Testing in C