Ron Patton

8/12/2019 Ron Patton

1/172

E-Book : Software Testing By Ron Patton Publisher: Sams Publishing (Chapter 1

Chapter 1. Software Testing Background

IN THIS CHAPTER

!nfamous Software Error Case Stu"ies #hat !s a Bug$

#hy %o Bugs &''ur$

The Cost of Bugs

#hat E a'tly %oes a Software Tester %o$

#hat )akes a *oo" Software Tester$

In 1947, computers were big, room-sized machines operating on mechanical relays and glowingvacuum tubes. The state o the art at the time was the !ar" II, a behemoth being built at#arvard $niversity. Technicians were running the new computer through its paces when itsuddenly stopped wor"ing. They scrambled to igure out why and discovered, stuc" between aset o relay contacts deep in the bowels o the computer, a moth. It had apparently lown intothe system, attracted by the light and heat, and was zapped by the high voltage when it landedon the relay.

The computer bug was born. %ell, o"ay, it died, but you get the point.

%elcome to the irst chapter o &o tware Testing. In this chapter, you'll learn about the historyo so tware bugs and so tware testing.

#ighlights o this chapter include

#ow so tware bugs impact our lives %hat bugs are and why they occur

%ho so tware testers are and what they do

Infamous Software Error Case Studies

It's easy to ta"e so tware or granted and not really appreciate how much it has in iltrated ourdaily lives. (ac" in 1947, the !ar" II computer re)uired legions o programmers to constantlymaintain it. The average person never conceived o someday having his own computer in hishome. *ow there's ree so tware + - !s attached to cereal bo/es and more so tware in our"ids' video games than on the space shuttle. %hat once were techie gadgets, such as pagersand cell phones, have become commonplace. !ost o us now can't go a day without logging onto the Internet and chec"ing our email. %e rely on overnight pac"ages, long-distance phoneservice, and cutting-edge medical treatments.

1
http://www.blogs.krify.com/softwaretesting/9791/http://www.blogs.krify.com/softwaretesting/9791/


2/172

&o tware is everywhere. #owever, it's written by peopleso it's not per ect, as the ollowinge/amples show.

%isney+s ,ion ing. 1//01//

In the all o 1994, the isney company released its irst multimedia + - ! game or children,The 0ion ing 2nimated &toryboo". 2lthough many other companies had been mar"etingchildren's programs or years, this was isney's irst venture into the mar"et and it was highlypromoted and advertised. &ales were huge. It was 3the game to buy3 or children that holidayseason. %hat happened, however, was a huge debacle. n ecember 5, the day a ter+hristmas, isney's customer support phones began to ring, and ring, and ring. &oon the phonesupport technicians were swamped with calls rom angry parents with crying children whocouldn't get the so tware to wor". *umerous stories appeared in newspapers and on T6 news.

It turns out that isney ailed to test the so tware on a broad representation o the manydi erent + models available on the mar"et. The so tware wor"ed on a ew systemsli"ely theones that the isney programmers used to create the gamebut not on the most commonsystems that the general public had.

!ntel Pentium 2loating-Point %i3ision Bug. 1//0

8nter the ollowing e)uation into your +'s calculator

(4195835 / 3145727) * 3145727 - 4195835

I the answer is zero, your computer is :ust ine. I you get anything else, you have an old Intelentium + $ with a loating-point division buga so tware bug burned into a computer chip andreproduced over and over in the manu acturing process.n ctober ; +ollege traced anune/pected result rom one o his e/periments to an incorrect answer by a division problemsolved on his entium +. #e posted his ind on the Internet and soon a terward a irestormerupted as numerous other people duplicated his problem and ound additional situations thatresulted in wrong answers. ?ortunately, these cases were rare and resulted in wrong answersonly or e/tremely math-intensive, scienti ic, and engineering calculations. !ost people wouldnever encounter them doing their ta/es or running their businesses.%hat ma"es this story notable isn't the bug, but the way Intel handled the situation

Their so tware test engineers had ound the problem while per orming their own testsbe ore the chip was released. Intel's management decided that the problem wasn'tsevere enough or li"ely enough to warrant i/ing it or even publicizing it.

nce the bug was ound, Intel attempted to diminish its perceived severity throughpress releases and public statements.

%hen pressured, Intel o ered to replace the aulty chips, but only i a user could provethat he was a ected by the bug.

There was a public outcry. Internet newsgroups were :ammed with irate customers demandingthat Intel i/ the problem. *ews stories painted the company as uncaring and incredulous. Inthe end, Intel apologized or the way it handled the bug and too" a charge o more than @4


3/172

n 2ugust Ath,


4/172


5/172

e ect 6ariance

roblem Inconsistency

8rror ?eature

Incident (ug

2nomaly

=There's also a list o unmentionable terms, but they're most o ten used privately amongprogrammers.>Bou might be amazed that so many names could be used to describe a so tware ailure. %hy somany It's all really based on the company's culture and the process the company uses to

develop its so tware. I you loo" up these words in the dictionary, you'll ind that they all haveslightly di erent meanings. They also have in erred meanings by how they're used in day-to-day conversation.?or e/ample, ault, ailure, and de ect tend to imply a condition that's really severe, maybeeven dangerous. It doesn't sound right to call an incorrectly colored icon a ault. These wordsalso tend to imply blame 3It's his ault that the so tware ailed.32nomaly, incident, and variance don't sound )uite so negative and are o ten used to in erunintended operation rather than all-out ailure. 3The president stated that it was a so twareanomaly that caused the missile to go o course.3roblem, error, and bug are probably the most generic terms used.

;


6/172

+alling any and all so tware problems bugs may sound simple enough, but doing so hasn't reallyaddressed the issue. *ow the word problem needs to be de ined. To "eep rom running incircular de initions, there needs to be a de initive description o what a bug is.?irst, you need a supporting term product speci ication. 2 product speci ication, sometimesre erred to as simply a spec or product spec, is an agreement among the so tware developmentteam. It de ines the product they are creating, detailing what it will be, how it will act, what

it will do, and what it won't do. This agreement can range in orm rom a simple verbalunderstanding, an email, or a scribble on a nap"in, to a highly detailed, ormalized writtendocument. In +hapter , 3The &o tware evelopment rocess,3 you will learn more aboutso tware speci ications and the development process, but or now, this de inition is su icient.?or the purposes o this boo" and much o the so tware industry, a so tware bug occurs whenone or more o the ollowing ive rules is true

1. The so tware doesn't do something that the product speci ication says it should do.2. The so tware does something that the product speci ication says it shouldn't do.

3. The so tware does something that the product speci ication doesn't mention.

4. The so tware doesn't do something that the product speci ication doesn't mention butshould.

5. The so tware is di icult to understand, hard to use, slow, or in the so tware tester'seyes will be viewed by the end user as :ust plain not right.

To better understand each rule, try the ollowing e/ample o applying them to a calculator.The speci ication or a calculator probably states that it will per orm correct addition,subtraction, multiplication, and division. I you, as the tester, receive the calculator, press theJ "ey, and nothing happens, that's a bug because o ule K1. I you get the wrong answer,that's also a bug because o ule K1.The product spec might state that the calculator should never crash, loc" up, or reeze. I youpound on the "eys and get the calculator to stop responding to your input, that's a bug becauseo ule K .&uppose that you receive the calculator or testing and ind that besides addition, subtraction,multiplication, and division, it also per orms s)uare roots. *owhere was this ever speci ied. 2nambitious programmer :ust threw it in because he elt it would be a great eature. This isn't aeatureit's really a bug because o ule K;. The so tware is doing something that the productspeci ication didn't mention. This unintended operation, although maybe nice to have, will addto the test e ort and will li"ely introduce even more bugs.The ourth rule may read a bit strange with its double negatives, but its purpose is to catchthings that were orgotten in the speci ication. Bou start testing the calculator and discoverwhen the battery gets wea" that you no longer receive correct answers to your calculations. *oone ever considered how the calculator should react in this mode. 2 bad assumption was madethat the batteries would always be ully charged. Bou e/pected it to "eep wor"ing until thebatteries were completely dead, or at least noti y you in some way that they were wea".+orrect calculations didn't happen with wea" batteries, and it wasn't speci ied what shouldhappen. ule K4 ma"es this a bug.ule KG is the catch-all. 2s a tester you are the irst person to really use the so tware. I youweren't there, it would be the customer using the product or the irst time. I you indsomething that you don't eel is right, or whatever reason, it's a bug. In the case o thecalculator, maybe you ound that the buttons were too small. !aybe the placement o the L"ey made it hard to use. !aybe the display was di icult to read under bright lights. 2ll o theseare bugs because o ule KG.* T8

6
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02


7/172

8very person who uses a piece o so tware will have di erent e/pectations and opinions as tohow it should wor". It would be impossible to write so tware that every user thought wasper ect. 2s a so tware tester, you should "eep this in mind when you apply ule KG to yourtesting. (e thorough, use your best :udgment, and, most importantly, be reasonable. Bouropinion counts, but, as you'll learn in later chapters, not all the bugs you ind can or will bei/ed.

These are greatly simpli ied e/amples, so thin" about how the rules apply to so tware that youuse every day. %hat is e/pected, what is une/pected %hat do you thin" was speci ied andwhat was orgotten 2nd, what do you :ust plain disli"e about the so twareThis de inition o a bug covers a lot o ground but using all ive o its rules will help youidenti y the di erent types o problems in the so tware you're testing.

#hy %o Bugs &''ur$*ow that you "now what bugs are, you might be wondering why they occur. %hat you'll besurprised to ind out is that most o them aren't caused by programming errors. *umerousstudies have been per ormed on very small to e/tremely large pro:ects and the results arealways the same. The number one cause o so tware bugs is the speci ication =see ?igure 1.1>.

2igure 1>1> Bugs are 'ause" for numerous reasons. but. in this sample pro?e't analysis. themain 'ause 'an be tra'e" to the spe'ifi'ation>

There are several reasons speci ications are the largest bug producer. In many instances a specsimply isn't written. ther reasons may be that the spec isn't thorough enough, it's constantlychanging, or it's not communicated well to the entire development team. lanning so tware isvitally important. I it's not done correctly, bugs will be created.

The ne/t largest source o bugs is the design. This is where the programmers lay out their planor the so tware. +ompare it to an architect creating the blueprints or a building. (ugs occur

7
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh87F5.htm#ch01fig01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh87F5.htm#ch01fig01


8/172


9/172

The costs are logarithmic that is, they increase ten old as time increases. 2 bug ound andi/ed during the early stages when the speci ication is being written might cost ne/t tonothing, or @1 in our e/ample. The same bug, i not ound until the so tware is coded andtested, might cost @1< to @1


10/172

realities to so tware testing. on't get caught in the dangerous spiral o unattainableper ection.

#hat )akes a *oo" Software Tester$In the movie &tar Tre" II The %rath o han, &poc" says, 32s a matter o cosmic history, it has

always been easier to destroy than to create.3 2t irst glance, it may appear that a so twaretester's :ob would be easier than a programmer's. (rea"ing code and inding bugs must surelybe easier than writing the code in the irst place. &urprisingly, it's not. The methodical anddisciplined approach to so tware testing that you'll learn in this boo" re)uires the same hardwor" and dedication that programming does. It involves very similar s"ills, and although aso tware tester doesn't necessarily need to be a ull- ledged programmer, having that"nowledge is a great bene it.Today, most mature companies treat so tware testing as a technical engineering pro ession.They recognize that having trained so tware testers on their pro:ect teams and allowing themto apply their trade early in the development process allows them to build better )ualityso tware. $n ortunately, there are still a ew companies that don't appreciate the challenge oso tware testing and the value o well-done testing e ort. In a ree mar"et society, thesecompanies usually aren't around or long because the customers spea" with their wallets and

choose not to buy their buggy products. 2 good test organization =or the lac" o one> can ma"eor brea" a company.#ere's a list o traits that most so tware testers should have

They are e/plorers. &o tware testers aren't a raid to venture into un"nown situations.They love to get a new piece o so tware, install it on their +, and see what happens.

They are troubleshooters. &o tware testers are good at iguring out why somethingdoesn't wor". They love puzzles.

They are relentless. &o tware testers "eep trying. They may see a bug that )uic"lyvanishes or is di icult to re-create. ather than dismiss it as a lu"e, they will try everyway possible to ind it.

They are creative. Testing the obvious isn't su icient or so tware testers. Their :ob isto thin" up creative and even o -the-wall approaches to ind bugs.

They are =mellowed> per ectionists. They strive or per ection, but they "now when itbecomes unattainable and they're o"ay with getting as close as they can.

They e/ercise good :udgment. &o tware testers need to ma"e decisions about whatthey will test, how long it will ta"e, and i the problem they're loo"ing at is really abug.

They are tact ul and diplomatic. &o tware testers are always the bearers o bad news.They have to tell the programmers that their baby is ugly. Food so tware testers "nowhow to do so tact ully and pro essionally and "now how to wor" with programmers whoaren't always tact ul and diplomatic.

They are persuasive. (ugs that testers ind won't always be viewed as severe enough tobe i/ed. Testers need to be good at ma"ing their points clear, demonstrating why thebug does indeed need to be i/ed, and ollowing through on ma"ing it happen.

10


11/172

S&2T#5RE TEST!4* !S 2


12/172

Chapter 2. The Software Development Process

I* T#I& +#2 T8

roduct +omponents &o tware ro:ect &ta

&o tware evelopment 0i ecycle !odels

To be an e ective so tware tester, it's important to have at least a high-level understanding othe overall process used to develop so tware. I you write small programs as a student orhobbyist, you'll ind that the methods you use are much di erent rom what big companies useto develop so tware. The creation o a new so tware product may involve dozens, hundreds,even thousands o team members all playing di erent roles and wor"ing together under tightschedules. The speci ics o what these people do, how they interact, and how they ma"edecisions are all part o the so tware development process.

The goal o this chapter isn't to teach you everything about the so tware development processthat would ta"e an entire boo"N The goal is to give you an overview o the all the pieces thatgo into a so tware product and a loo" at a ew o the common approaches in use today. %iththis "nowledge you'll have a better understanding o how best to apply the so tware testings"ills you learn in the later chapters o this boo".

The highlights o this chapter include

%hat ma:or components go into a so tware product %hat di erent people and s"ills contribute to a so tware product

#ow so tware progresses rom an idea to a inal productPro"u't Components%hat e/actly is a so tware product !any o us thin" o it as simply a program that wedownload rom the Internet or install rom a 6 that runs on our computer. That's a prettygood description, but in reality, many hidden pieces go into ma"ing that so tware. There arealso many pieces that 3come in the bo/3 that are o ten ta"en or granted or might even beignored. 2lthough it may be easy to orget about all those parts, as a so tware tester, you needto be aware o them, because they're all testable pieces and can all have bugs.

#hat Effort *oes !nto a Software Pro"u't$

?irst, loo" at what e ort goes into a so tware product. ?igure .1 identi ies a ew o theabstract pieces that you may not have considered.

2igure 7>1> 5 lot of hi""en effort goes into a software pro"u't>

12
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec1.html#ch02lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec2.html#ch02lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec3.html#ch02lev1sec3http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec1.html#ch02lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec2.html#ch02lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec3.html#ch02lev1sec3http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig01


13/172

&o what are all these things, besides the actual code, that get unneled into the so tware 2tirst glance they probably seem much less tangible than the program listing a programmercreates. 2nd they de initely aren't something that can be viewed directly rom the product's+ - !. (ut, to paraphrase a line rom an old spaghetti sauce commercial, 3they're in there.32t least, they should be.The term used in the so tware industry to describe a so tware product component that'screated and passed on to someone else is deliverable. The easiest way to e/plain what allthese deliverables are is to organize them into ma:or categories.

Customer Re uirements

&o tware is written to ul ill some need that a person or a group o people has. 0et's call themthe customer. To properly ill that need, the product development team must ind out whatthe customer wants. &ome teams simply guess, but most collect detailed in ormation in theorm o surveys, eedbac" rom previous versions o the so tware, competitive productin ormation, magazine reviews, ocus groups, and numerous other methods, some ormal, somenot. 2ll this in ormation is then studied, condensed, and interpreted to decide e/actly whateatures the so tware product should have.

P


14/172

choose your avorite. r, you may discuss as a group eatures you'd li"e to see in a newproduct. (est o all, you get paid or your time.!ost ocus groups are conducted in such a way that the so tware company re)uesting thein ormation is "ept anonymous. (ut, it's usually easy to igure out who they are.

Spe'ifi'ations

The result o the customer re)uirements studies is really :ust raw data. It doesn't describe theproposed product, it :ust con irms whether it should =or shouldn't> be created and whateatures the customers want. The speci ications ta"e all this in ormation plus any unstated butmandatory re)uirements and truly de ine what the product will be, what it will do, and how itwill loo".The ormat o speci ications varies greatly. &ome companies specially those developingproducts or the government, aerospace, inancial and medical industries use a very rigorousprocess with many chec"s and balances. The result is an e/tremely detailed and thoroughspeci ication that's loc"ed down, meaning that it can't change e/cept under very e/tremeconditions. 8veryone on the development team "nows e/actly what they are creating.There are development teams, usually ones creating so tware or less-critical applications, whoproduce speci ications on coc"tail nap"ins, i they create them at all. This has the distinct

advantage o being very le/ible, but there's lots o ris" that not everyone is 3on the samepage.3 2nd, what the product inally becomes isn't "nown until it's released.

S'he"ules

2 "ey part o a so tware product is its schedule. 2s a pro:ect grows in size and comple/ity, withmany pieces and many people contributing to the product, it becomes necessary to have somemechanism to trac" its progress. This could range rom simple tas" lists to Fantt charts =see?igure . > to detailed trac"ing o every minute tas" with pro:ect management so tware.

2igure 7>7> 5 *antt 'hart is a bar 'hart that shows a pro?e't+s tasks against a hori ontaltimeline>

The goals o scheduling are to "now which wor" has been completed, how much wor" is stillle t to do, and when it will all be inished.

Software %esign %o'uments

ne common misconception is that when a programmer creates a program, he simply sits downand starts writing code. That may happen in some small, in ormal so tware shops, but oranything other than the smallest programs, there must be a design process to plan out how theso tware will be written. Thin" about this boo", which re)uired an outline be ore the irst

14
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig02


15/172

words were typed, or a building, which has blueprints drawn be ore the irst concrete ispoured. The same planning should happen with so tware.

The documents that programmers create vary greatly depending on the company, the pro:ect,and the team, but their purpose is to plan and organize the code that is to be written.

#ere is a list o a ew common so tware design documents

2rchitecture. 2 document that describes the overall design o the so tware, includingdescriptions o all the ma:or pieces and how they interact with each other.

ata ?low iagram. 2 ormalized diagram that shows how data moves through aprogram. It's sometimes re erred to as a bubble chart because it's drawn by usingcircles and lines.

&tate Transition iagram. 2nother ormalized diagram that brea"s the so tware intobasic states, or conditions, and shows the means or moving rom one state to thene/t.

?lowchart. The traditional means or pictorially describing a program's logic.?lowcharting isn't very popular today, but when it's used, writing the program coderom a detailed lowchart is a very simple process.

+ommented +ode. There's an old saying that you may write code once, but it will beread by someone at least 1< times. roperly embedding use ul comments in theso tware code itsel is e/tremely important, so that programmers assigned to maintainthe code can more easily igure out what it does and how.

Test %o'uments

Test documentation is discussed in detail in +hapters 17 < but is mentioned here because it'sintegral to what ma"es up a so tware product. ?or the same reasons that programmers mustplan and document their wor", so tware testers must as well. It's not unheard o or a so twaretest team to create more deliverables than the programmers.#ere's a list o the more important test deliverables

The test plan "es'ribes the o3erall metho" to be used to veri y that the so twaremeets the product speci ication and the customer's needs. It includes the )ualityob:ectives, resource needs, schedules, assignments, methods, and so orth.

Test cases list the speci ic items that will be tested and describe the detailed stepsthat will be ollowed to veri y the so tware.

(ug reports describe the problems ound as the test cases are ollowed. These could bedone on paper but are o ten trac"ed in a database.

Test tools and automation are described in detail in +hapter 1G, 32utomated Testingand Test Tools.3 I your team is using automated methods to test your so tware, thetools you use, either purchased or written in-house, must be documented.

!etrics, statistics, and summaries convey the progress being made as the test wor"progresses. They ta"e the orm o graphs, charts, and written reports.

15
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch17.html#ch17http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch20.html#ch20http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch15.html#ch15http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch17.html#ch17http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch20.html#ch20http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch15.html#ch15


16/172

#hat Parts )ake


17/172

roduct support in o Icons and art

8rror messages 2ds and mar"eting material

&etup and installation eadme ile

%&4+T 2&R*ET T& TEST ERR&R )ESS5*ES

8rror messages are one o the most overloo"ed parts o a so tware product. rogrammers, nottrained writers, typically write them. They're seldom planned or and are usually hac"ed inwhile i/ing bugs. It's also very di icult or testers to ind and display all o them. on't leterror messages such as these creep into your so tware[View full width] Error !e"#o$rd %ot fou%d. &re'' 1 to o%ti%ue. $%+t i%'t$%ti$te the ,ideo thi% . i%dow' h$'

fou%d $% u% %ow% de,i e $%d i' i%'t$lli% $ dri,er for it. 0 $t$l E e tio% 6 h$' o urred $t7.

Software Pro?e't Staff *ow that you "now what goes into a so tware product and what ships with one, it's time tolearn about all the people who create so tware. course, this varies a great deal based onthe company and the pro:ect, but or the most part the roles are the same, it's :ust the titlesthat are di erent.The ollowing lists, in no particular order, the ma:or players and what they do. The mostcommon names are given, but e/pect variations and additions

ro:ect managers, program managers, or producers drive the pro:ect rom beginning toend. They're usually responsible or writing the product spec, managing the schedule,and ma"ing the critical decisions and trade-o s.

2rchitects or system engineers are the technical e/perts on the product team. They'reusually very e/perienced and there ore are )uali ied to design the overall systemsarchitecture or design or the so tware. They wor" very closely with the programmers.

rogrammers, developers, or coders design and write so tware and i/ the bugs thatare ound. They wor" closely with the architects and pro:ect managers to create theso tware. Then, they wor" closely with the pro:ect managers and testers to get thebugs i/ed.

Testers or M2 =Muality 2ssurance> &ta is responsible or inding and reportingproblems in the so tware product. They wor" very closely with all members o theteam as they develop and run their tests, and report the problems they ind. +hapter1 , 3&o tware Muality 2ssurance,3 thoroughly covers the di erences between so twaretesting and so tware )uality assurance tas"s.

Technical writers, user assistance, user education, manual writers, or illustratorscreate the paper and online documentation that comes with a so tware product.

17
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec1.html#PLID0http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch21.html#ch21http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch21.html#ch21http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02lev1sec1.html#PLID0http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch21.html#ch21http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch21.html#ch21


18/172


19/172

The beauty o the big-bang method is that it's simple. There is little i any planning,scheduling, or ormal development process. 2ll the e ort is spent developing the so tware andwriting the code. It's a process that is used i the product re)uirements aren't well understoodand the inal release date is completely le/ible. It's also important to have very le/iblecustomers, too, because they won't "now what they're getting until the very end.

*otice that testing isn't shown in ?igure .4 . In most cases, there is little to no ormal testingdone under the big-bang model. I testing does occur, it's s)ueezed in :ust be ore the productis released. It's a mystery why testing is sometimes inserted into this model, but it's probably toma"e everyone eel good that some testing was per ormed.

I you are called in to test a product under the big-bang model, you have both an easy and adi icult tas". (ecause the so tware is already complete, you have the per ect speci ication theproduct itsel . 2nd, because it's impossible to go bac" and i/ things that are bro"en, your :obis really :ust to report what you ind so the customers can be told about the problems.

The downside is that, in the eyes o pro:ect management, the product is ready to go, so yourwor" is holding up delivery to the customer. The longer you ta"e to do your :ob and the morebugs you ind, the more contentious the situation will become. Try to stay away rom testing inthis model.

Co"e-an"-2i )o"el

The code-and- i/ model shown in ?igure .G is usually the one that pro:ect teams all into byde ault i they don't consciously attempt to use something else. It's a step up, procedurally,rom the big-bang model, in that it at least re)uires some idea o what the productre)uirements are.

2igure 7> > The 'o"e-an"-fi mo"el repeats until someone gi3es up>

19
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig05http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig05http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fig05


20/172

2 wise man once said, 3There's never time to do it right, but there's always time to do it over.3That pretty much sums up this model. 2 team using this approach usually starts with a roughidea o what they want, does some simple design, and then proceeds into a long repeatingcycle o coding, testing, and i/ing bugs. 2t some point they decide that enough is enough andrelease the product.

2s there's very little overhead or planning and documenting, a pro:ect team can show resultsimmediately. ?or this reason, the code-and- i/ model wor"s very well or small pro:ectsintended to be created )uic"ly and then thrown out shortly a ter they're done, such asprototypes and demos. 8ven so, code-and- i/ has been used on many large and well-"nownso tware products. I your word processor or spreadsheet so tware has lots o little bugs or it:ust doesn't seem )uite inished, it was li"ely created with the code-and- i/ model.

0i"e the big-bang model, testing isn't speci ically called out in the code-and- i/ model but doesplay a signi icant role between the coding and the i/ing.

2s a tester on a code-and- i/ pro:ect, you need to be aware that you, along with theprogrammers, will be in a constant state o cycling. 2s o ten as every day you'll be given new orupdated releases o the so tware and will set o to test it. Bou'll run your tests, report thebugs, and then get a new so tware release. Bou may not have inished testing the previousrelease when the new one arrives, and the new one may have new or changed eatures.8ventually, you'll get a chance to test most o the eatures, ind ewer and ewer bugs, andthen someone =or the schedule> will decide that it's time to release the product.

Bou will most li"ely encounter the code-and- i/ model during your wor" as a so tware tester.It's a good introduction to so tware development and will help you appreciate the more ormalmethods.

#aterfall )o"el

The water all method is usually the irst one taught in programming school. It's been aroundorever. It's simple, elegant, and ma"es sense. 2nd, it can wor" well on the right pro:ect.?igure .5 shows the steps involved in this model.

2igure 7> > The software "e3elopment pro'ess flows from one step to the ne t in thewaterfall mo"el>

20


21/172

2 pro:ect using the water all model moves down a series o steps starting rom an initial ideato a inal product. 2t the end o each step, the pro:ect team holds a review to determine ithey're ready to move to the ne/t step. I the pro:ect isn't ready to progress, it stays at thatlevel until it's ready.

*otice three important things about the water all method

There's a large emphasis on speci ying what the product will be. *ote that thedevelopment or coding phase is only a single bloc"N

The steps are discreteE there's no overlap. There's no way to bac" up. 2s soon as you're on a step, you need to complete the tas"s

or that step and then move on you can't go bac". Q1R

Q1R 6ariations o the water all model loosen the rules a bit, allowingsome overlap o the steps and the ability to bac" up one step inecessary.

This may sound very limiting, and it is, but it wor"s well or pro:ects with a well-understoodproduct de inition and a disciplined development sta . The goal is to wor" out all theun"nowns and nail down all the details be ore the irst line o code is written. The drawbac" isthat in today's ast moving culture, with products being developed on Internet time, by thetime a so tware product is so care ully thought out and de ined, the original reason or its

being may have changed.?rom a testing perspective, the water all model o ers one huge advantage over the othermodels presented so ar. 8verything is care ully and thoroughly speci ied. (y the time theso tware is delivered to the test group, every detail has been decided on, written down, andturned into so tware. ?rom that, the test group can create an accurate plan and schedule.They "now e/actly what they're testing, and there's no )uestion about whether something is aeature or a bug.(ut, with this advantage, comes a large disadvantage. (ecause testing occurs only at the end,a undamental problem could creep in early on and not be detected until days be ore thescheduled product release. emember rom +hapter 1 , 3&o tware Testing (ac"ground,3 how the

21
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fn01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh44C1.htm#ch02fn01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01


22/172

cost o bugs increases over time %hat's needed is a model that olds the testing tas"s inearlier to ind problems be ore they become too costly.

Spiral )o"el

It's not )uite utopia, but the spiral model =see ?igure .7 > goes a long way in addressing manyo the problems inherent with the other models while adding a ew o its own nice touches.

2igure7>9> The spiral mo"el starts small an" gra"ually e pan"s as the pro?e't be'omesbetter "efine" an" gains stability>

The spiral model was introduced by (arry (oehm in 19A5 in his 2ssociation or +omputing!achinery =2+!> paper, 32 &piral !odel o &o tware evelopment and 8nhancement.3 It's usedairly o ten and has proven to be an e ective approach to developing so tware.

The general idea behind the spiral model is that you don't de ine everything in detail at thevery beginning. Bou start small, de ine your important eatures, try them out, get eedbac"rom your customers, and then move on to the ne/t level. Bou repeat this until you have yourinal product.

8ach time around the spiral involves si/ steps

1. etermine ob:ectives, alternatives, and constraints.. Identi y and resolve ris"s.

;. 8valuate alternatives.4. evelop and test the current level.G. lan the ne/t level.5. ecide on the approach or the ne/t level.

22


23/172

(uilt into the spiral model is a bit o water all =the steps o analysis, design, develop, test>, abit o code-and- i/ =each time around the spiral>, and a bit o big-bang =loo" at it rom theoutside>. +ouple this with the lower costs o inding problems early, and you have a prettygood development model.

I you're a tester, you'll li"e this model. Bou'll get a chance to in luence the product early by

being involved in the preliminary design phases. Bou'll see where the pro:ect has come romand where it's going. 2nd, at the very end o the pro:ect, you won't eel as rushed to per ormall your testing at the last minute. Bou've been testing all along, so the last push should only bea validation that everything is o"ay.

5*!,E S&2T#5RE %E E,&P)E4T

2 type o development process that has gained in popularity among a number o so tware companies is"nown as 2gile &o tware evelopment. Bou might hear other names or it such as apid rototyping,8/treme rogramming, or 8volutionary evelopment, among others.The goal o 2gile &o tware evelopment, as described in its mani esto at www.agilemani esto.org , is

3Individuals and interactions over processes and tools%or"ing so tware over comprehensive documentation+ustomer collaboration over contract negotiationsesponding to change over ollowing a planPThat is, while there is value in the items on the right, we value the items on the le t more.3

2gile &o tware evelopment has developed a bit o a ollowing and has had some success ul pro:ects but isar rom main stream. It sounds li"e a great philosophy i you and your pro:ect seem to be bogged down inprocess and documentation but, in my opinion, it can too easily digress into chaos. It's a method to watchand learn about and your team may consider using it. ?or now, however, I won't be riding in a :et whoseso tware was rapidly prototyped by an e/treme programmer. !aybe someday.

?or more in ormation, chec" out 2gile !odeling, Teach Boursel 8/treme rogramming in 4 #ours, and8/treme rogramming ractices in 2ction. 2ll are published by &ams ublishing.

SummaryBou now have an understanding o how so tware products are created both what goes intothem and the processes used to put them together. 2s you can see, there's no de initiveapproach. The our models presented here are :ust e/amples. There are many others and lotso variations o these. 8ach company, each pro:ect, and each team will choose what wor"s orthem. &ometimes they will choose right, sometimes they will choose wrong. Bour :ob as aso tware tester is to wor" the best you can in the development model you're in, applying thetesting s"ills you learn in the rest o this boo" to create the best so tware possible.

AuiThese )uiz )uestions are provided or your urther understanding. &ee 2ppendi/ 2 , 32nswers toMuiz Muestions,3 or the answers but don't pee"N

1. *ame several tas"s that should be per ormed be ore a programmer starts writing theirst line o code.

. %hat disadvantage is there to having a ormal, loc"ed-down speci ication

;. %hat is the best eature o the big-bang model o so tware development

23
http://www.agilemanifesto.org/http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/app01.html#app01http://www.agilemanifesto.org/http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/app01.html#app01


24/172

4. %hen using the code-and- i/ model, how do you "now when the so tware is ready torelease

G. %hy can the water all method be di icult to use

5. %hy would a so tware tester li"e the spiral model better than the others

Chapter 3. The ealities of Software Testing

I* T#I& +#2 T8

Testing 2/ioms &o tware Testing Terms and e initions

In +hapter 1 , 3&o tware Testing (ac"ground,3 and +hapter , 3The &o tware evelopmentrocess,3 you learned about the basics o so tware testing and the so tware developmentprocess. The in ormation presented in these chapters o ered a very high-level and arguably

idealistic view o how so tware pro:ects might be run. $n ortunately, in the real world you willnever see a pro:ect per ectly ollow any o the development models. Bou will never be given athoroughly detailed speci ication that per ectly meets the customer's needs and you will neverhave enough time to do all the testing you need to do. It :ust doesn't happen. (ut, to be ane ective so tware tester, you need to understand what the ideal process is so that you havesomething to aim or.

The goal o this chapter is to temper that idealism with a reality chec" rom a so tware tester'sperspective. It will help you see that, in practice, trade-o s and concessions must be madethroughout the development cycle. !any o those trade-o s are directly related to theso tware test e ort. The bugs you ind and the problems you prevent all signi icantly a ect thepro:ect. 2 ter reading this chapter, you'll have a much clearer picture o the roles, the impact,and the responsibilities that so tware testing has and you'll hope ully appreciate the behind-the-scenes decisions that must be made to create a so tware product.

The highlights o this chapter include

%hy so tware can never be per ect %hy so tware testing isn't :ust a technical problem

The terms commonly used by so tware testers

Testing 5 iomsThis irst section o this chapter is a list o a/ioms, or truisms. Thin" o them as the 3rules othe road3 or the 3 acts o li e3 or so tware testing and so tware development. 8ach o them is alittle tidbit o "nowledge that helps put some aspect o the overall process into perspective.

!t+s !mpossible to Test a Program Completely

2s a new tester, you might believe that you can approach a piece o so tware, ully test it, indall the bugs, and assure that the so tware is per ect. $n ortunately, this isn't possible, evenwith the simplest programs, due to our "ey reasons

The number o possible inputs is very large.

24
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch03lev1sec1.html#ch03lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch03lev1sec2.html#ch03lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch03lev1sec1.html#ch03lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch03lev1sec2.html#ch03lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02


25/172


26/172

The point o this e/ample is to demonstrate that it's impossible to completely test a program,even so tware as simple as a calculator. I you decide to eliminate any o the test conditionsbecause you eel they're redundant or unnecessary, or :ust to save time, you've decided not totest the program completely.

Software Testing !s a Risk-Base" E er'ise

I you decide not to test every possible test scenario, you've chosen to ta"e on ris". In thecalculator e/ample, what i you choose not to test that 1 24 1 24 2 48 It's possible theprogrammer accidentally le t in a bug or that situation. I you don't test it, a customer willeventually enter it, and he or she will discover the bug. It'll be a costly bug, too, since it wasn'tound until the so tware was in the customer's hands.This may all sound pretty scary. Bou can't test everything, and i you don't, you will li"ely missbugs. The product has to be released, so you will need to stop testing, but i you stop too soon,there will still be areas untested. %hat do you done "ey concept that so tware testers need to learn is how to reduce the huge domain opossible tests into a manageable set, and how to ma"e wise ris"-based decisions on what'simportant to test and what's not.?igure ;. shows the relationship between the amount o testing per ormed and the number o

bugs ound. I you attempt to test everything, the costs go up dramatically and the number omissed bugs declines to the point that it's no longer cost e ective to continue. I you cut thetesting short or ma"e poor decisions o what to test, the costs are low but you'll miss a lot obugs. The goal is to hit that optimal amount o testing so that you don't test too much or toolittle.

2igure D>7> E3ery software pro?e't has an optimal test effort>

Bou will learn how to design and select test scenarios that minimize ris" and optimize yourtesting in +hapters 4 through 7.

Testing Can+t Show That Bugs %on+t E ist

26
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh6014.htm#ch03fig02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04.html#ch04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch07.html#ch07http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh6014.htm#ch03fig02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04.html#ch04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch07.html#ch07


27/172

Thin" about this or a moment. Bou're an e/terminator charged with e/amining a house orbugs. Bou inspect the house and ind evidence o bugs maybe live bugs, dead bugs, or nests.Bou can sa ely say that the house has bugs.Bou visit another house. This time you ind no evidence o bugs. Bou loo" in all the obviousplaces and see no signs o an in estation. !aybe you ind a ew dead bugs or old nests but yousee nothing that tells you that live bugs e/ist. +an you absolutely, positively state that the

house is bug ree *ope. 2ll you can conclude is that in your search you didn't ind any livebugs. $nless you completely dismantled the house down to the oundation, you can't be surethat you didn't simply :ust miss them.&o tware testing wor"s e/actly as the e/terminator does. It can show that bugs e/ist, but itcan't show that bugs don't e/ist. Bou can per orm your tests, ind and report bugs, but at nopoint can you guarantee that there are no longer any bugs to ind. Bou can only continue yourtesting and possibly ind more.

The )ore Bugs 6ou 2in". the )ore Bugs There 5re

There are even more similarities between real bugs and so tware bugs. (oth types tend tocome in groups. I you see one, odds are there will be more nearby.?re)uently, a tester will go or long spells without inding a bug. #e'll then ind one bug, then

)uic"ly another and another. There are several reasons or this rogrammers have bad days. 0i"e all o us, programmers can have o days. +ode

written one day may be per ectE code written another may be sloppy. ne bug can bea tell-tale sign that there are more nearby.

rogrammers o ten ma"e the same mista"e. 8veryone has habits. 2 programmer who isprone to a certain error will o ten repeat it.

&ome bugs are really :ust the tip o the iceberg. 6ery o ten the so tware's design orarchitecture has a undamental problem. 2 tester will ind several bugs that at irstmay seem unrelated but eventually are discovered to have one primary serious cause.

It's important to note that the inverse o this 3bugs ollow bugs3 idea is true, as well. I you ailto ind bugs no matter how hard you try, it may very well be that the eature you're testing wascleanly written and that there are indeed ew i any bugs to be ound.

The Pesti'i"e Para"o

In 199


28/172

emember the spiral model o so tware development described in +hapter The test process

repeats each time around the loop. %ith each iteration, the so tware testers receive theso tware or testing and run their tests. 8ventually, a ter several passes, all the bugs that thosetests would ind are e/posed. +ontinuing to run them won't reveal anything new.To overcome the pesticide parado/, so tware testers must continually write new and di erenttests to e/ercise di erent parts o the program and ind more bugs.

4ot 5ll the Bugs 6ou 2in" #ill Be 2i e"

ne o the sad realities o so tware testing is that even a ter all your hard wor", not every bugyou ind will be i/ed. *ow, don't be disappointed this doesn't mean that you've ailed inachieving your goal as a so tware tester, nor does it mean that you or your team will release apoor )uality product. It does mean, however, that you'll need to rely on a couple o those traitso a so tware tester listed in +hapter 1 e/ercising good :udgment and "nowing when per ection

isn't reasonably attainable. Bou and your team will need to ma"e trade-o s, ris"-baseddecisions or each and every bug, deciding which ones will be i/ed and which ones won't.There are several reasons why you might choose not to i/ a bug

There's not enough time. In every pro:ect there are always too many so tware eatures,too ew people to code and test them, and not enough room le t in the schedule toinish. I you're wor"ing on a ta/ preparation program, 2pril 1G isn't going to moveyoumust have your so tware ready in time.

It's really not a bug. !aybe you've heard the phrase, 3It's not a bug, it's a eatureN3 It'snot uncommon or misunderstandings, test errors, or spec changes to result in would-be bugs being dismissed as eatures.

It's too ris"y to i/. $n ortunately, this is all too o ten true. &o tware can be ragile,intertwined, and sometimes li"e spaghetti. Bou might ma"e a bug i/ that causes otherbugs to appear. $nder the pressure to release a product under a tight schedule, itmight be too ris"y to change the so tware. It may be better to leave in the "nown bugto avoid the ris" o creating new, un"nown ones.

It's :ust not worth it. This may sound harsh, but it 's reality. (ugs that would occurin re)uently or bugs that appear in little-used eatures may be dismissed. (ugs thathave wor"-arounds, ways that a user can prevent or avoid the bug, are o ten not i/ed.It all comes down to a business decision based on ris".

28
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01


29/172

The decision-ma"ing process usually involves the so tware testers, the pro:ect managers, andthe programmers. 8ach carries a uni)ue perspective on the bugs and has his own in ormationand opinions as to why they should or shouldn't be i/ed. In +hapter 19 , 3 eporting %hat Bou?ind,3 you'll learn more about reporting bugs and getting your voice heard.

#=5T =5PPE4S #=E4 6&< )5 E T=E #R&4* %EC!S!&4$

emember the Intel entium bug described in +hapter 1 The Intel test engineersound this bug be ore the chip was released, but the product team decided that itwas such a small, rare bug that it wasn't worth i/ing. They were under a tightschedule and decided to meet their current deadline and i/ the bug in later releaseso the chip. $n ortunately, the bug was discovered and the rest, they say, is history.In any piece o so tware, or every 3 entium type3 bug that ma"es the headlines,there could be perhaps hundreds o bugs that are le t un i/ed because they wereperceived to not have a ma:or negative e ect. nly time can tell i those decisionswere right or wrong.

#hen a Bug+s a Bug !s %iffi'ult to SayI there's a problem in the so tware but no one ever discovers itnot programmers, not testers,and not even a single customeris it a bugFet a group o so tware testers in a room and as" them this )uestion. Bou'll be in or a livelydiscussion. 8veryone has their own opinion and can be pretty vocal about it. The problem isthat there's no de initive answer. The answer is based on what you and your development teamdecide wor"s best or you.?or the purposes o this boo", re er bac" to the rules to de ine a bug rom +hapter 1

1. The so tware doesn't do something that the product speci ication says it should do.. The so tware does something that the product speci ication says it shouldn't do.

;. The so tware does something that the product speci ication doesn't mention.

4. The so tware doesn't do something that the product speci ication doesn't mention butshould.

G. The so tware is di icult to understand, hard to use, slow, orin the so tware tester'seyeswill be viewed by the end user as :ust plain not right.

?ollowing these rules helps clari y the dilemma by ma"ing a bug a bug only i it's observed. Toclaim that the so tware does or doesn't do 3something3 implies that the so tware was run andthat 3something3 or the lac" o 3something3 was witnessed. &ince you can't report on what youdidn't see, you can't claim that a bug e/ists i you didn't see it.#ere's another way to thin" o it. It's not uncommon or two people to have completelydi erent opinions on the )uality o a so tware product. ne may say that the program isincredibly buggy and the other may say that it's per ect. #ow can both be right The answer isthat one has used the product in a way that reveals lots o bugs. The other hasn't.* T8(ugs that are undiscovered or haven't yet been observed are o ten re erred to as latent bugs.

I this is as clear as mud, don't worry. iscuss it with your peers in so tware testing and ind outwhat they thin". 0isten to others' opinions, test their ideas, and orm your own de inition.

29
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch19.html#ch19http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch19.html#ch19http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch01.html#ch01


30/172


31/172

pro:ect late to do some 3ad-hoc banging on the code to see what they might ind.3 Times havechanged.0oo" at the so tware help-wanted ads and you'll see numerous listings or so tware testers. Theso tware industry has progressed to the point where pro essional so tware testers aremandatory. It's now too costly to build bad so tware.To be air, not every company is on board yet. !any computer game and small-time so tware

companies still use a airly loose development modelusually big-bang or code-and- i/. (ut muchmore so tware is now developed with a disciplined approach that has so tware testers as core,vital members o their sta .This is great news i you're interested in so tware testing. It can now be a career choicea :obthat re)uires training and discipline, and allows or advancement.

Software Testing Terms an" %efinitionsThis chapter wraps up the irst section o this boo" with a list o so tware testing terms andtheir de initions. These terms describe undamental concepts regarding the so twaredevelopment process and so tware testing. (ecause they're o ten con used or usedinappropriately, they're de ined here as pairs to help you understand their true meanings andthe di erences between them. (e aware that there is little agreement in the so tware industryover the de inition o many, seemingly common, terms. 2s a tester, you should re)uently

clari y the meaning o the terms your team is using. It's o ten best to agree to a de initionrather than ight or a 3correct3 one.

Pre'ision an" 5''ura'y

2s a so tware tester, it's important to "now the di erence between precision and accuracy.&uppose that you're testing a calculator. &hould you test that the answers it returns are preciseor accurate (oth I the pro:ect schedule orced you to ma"e a ris"-based decision to ocus ononly one o these, which one would you choose%hat i the so tware you're testing is a simulation game such as baseball or a light simulator&hould you primarily test its precision or its accuracy?igure ;.4 helps to graphically describe these two terms. The goal o this dart game is to hitthe bull's-eye in the center o the board. The darts on the board in the upper le t are neitherprecise nor accurate. They aren't closely grouped and not even close to the center o thetarget.

2igure D>0> %arts on a "artboar" "emonstrate the "ifferen'e between pre'ision an"a''ura'y>

31
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh6014.htm#ch03fig04http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh6014.htm#ch03fig04


32/172

The board on the upper right shows darts that are precise but not accurate. They are closelygrouped, so the thrower has precision, but he's not very accurate because the darts didn't evenhit the board.The board on the lower le t is an e/ample o accuracy but poor precision. The darts are veryclose to the center, so the thrower is getting close to what he's aiming at, but because theyaren't closely positioned, the precision is o .The board in the lower right is a per ect match o precision and accuracy. The darts are closelygrouped and on target.%hether the so tware you test needs to be precise or accurate depends much on what theproduct is and ultimately what the development team is aiming at =e/cuse the pun>. 2 so twarecalculator li"ely demands that both are achieveda right answer is a right answer. (ut, it maybe decided that calculations will only be accurate and precise to the i th decimal place. 2 terthat, the precision can vary. 2s long as the testers are aware o that speci ication, they cantailor their testing to con irm it.

erifi'ation an" ali"ation

6eri ication and validation are o ten used interchangeably but have di erent de initions. Thesedi erences are important to so tware testing.6eri ication is the process con irming that somethingso twaremeets its speci ication. 6alidationis the process con irming that it meets the user's re)uirements. These may sound very similar,but an e/planation o the #ubble space telescope problems will help show the di erence.In 2pril 199


33/172


34/172

AuiThese )uiz )uestions are provided or your urther understanding. &ee 2ppendi/ 2 , 32nswers toMuiz Muestions,3 or the answersbut don't pee"N

1. Fiven that it's impossible to test a program completely, what in ormation do you thin"should be considered when deciding whether it's time to stop testing

2. &tart the %indows +alculator. Type 5 -5 =the comma is important>. 0oo" at theresult. Is this a bug %hy or why not

;. I you were testing a simulation game such as a light simulator or a city simulator,what do you thin" would be more important to testits accuracy or its precision

4. Is it possible to have a high-)uality and low-reliability product %hat might an e/amplebe

G. %hy is it impossible to test a program completely

5. I you were testing a eature o your so tware on !onday and inding a new bug every

hour, at what rate would you e/pect to ind bugs on Tuesday

Chapter !. "#amining the Specification

I* T#I& +#2 T8

Fetting &tarted er orming a #igh-0evel eview o the &peci ication

0ow-0evel &peci ication Test Techni)ues

This chapter will introduce you to your irst real hands-on testingbut it may not be what youe/pect. Bou won't be installing or running so tware and you won't be pounding on the "eyboardhoping or a crash. In this chapter, you'll learn how to test the product's speci ication to indbugs be ore they ma"e it into the so tware.Testing the product spec isn't something that all so tware testers have the lu/ury o doing.&ometimes you might come into a pro:ect midway through the development cycle a ter thespeci ication is written and the coding started. I that's the case, don't worryyou can still usethe techni)ues presented here to test the inal speci ication.I you're ortunate enough to be involved on the pro:ect early and have access to a preliminaryspeci ication, this chapter is or you. ?inding bugs at this stage can potentially save yourpro:ect huge amounts o time and money.#ighlights o this chapter include

%hat is blac"-bo/ and white-bo/ testing #ow static and dynamic testing di er

%hat high-level techni)ues can be used or reviewing a product speci ication

%hat speci ic problems you should loo" or when reviewing a product speci ication indetail

34
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/app01.html#app01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec1.html#ch04lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec2.html#ch04lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec3.html#ch04lev1sec3http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/app01.html#app01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec1.html#ch04lev1sec1http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec2.html#ch04lev1sec2http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch04lev1sec3.html#ch04lev1sec3


35/172

*etting Starte"Thin" bac" to the our development models presented in +hapter , 3The &o twareevelopment rocess3 big-bang, code-and- i/, water all, and spiral. In each model, e/ceptbig-bang, the development team creates a product speci ication rom the re)uirementsdocument to de ine what the so tware will become.Typically, the product speci ication is a written document using words and pictures to describe

the intended product. 2n e/cerpt rom the %indows +alculator =see ?igure 4.1> product specmight read something li"e this

The 8dit menu will have two selections +opy and aste. These can be chosenby one o three methods pointing and clic"ing to the menu items with themouse, using access-"eys =2ltJ8 and then + or +opy and or aste>, or usingthe standard %indows shortcut "eys o +trlJ+ or +opy and +trlJ6 or aste.

The +opy unction will copy the current entry displayed in the number te/t bo/into the %indows +lipboard. The aste unction will paste the value stored inthe %indows +lipboard into the number te/t bo/.

2igure 0>1> The stan"ar" #in"ows Cal'ulator "isplaying the "rop-"own E"it menu>

2s you can see, it too" )uite a ew words :ust to describe the operation o two menu items in asimple calculator program. 2 thoroughly detailed spec or the entire application could be ahundred pages long.It may seem li"e over"ill to create a meticulous document or such simple so tware. %hy not:ust let a programmer write a calculator program on his own The problem is that you wouldhave no idea what you'd eventually get. The programmer's idea o what it should loo" li"e,what unctionality it should have, and how the user would use it could be completely di erentrom yours. The only way to assure that the end product is what the customer re)uiredand toproperly plan the test e ortis to thoroughly describe the product in a speci ication.The other advantage o having a detailed spec, and the basis o this chapter, is that as a testeryou'll also have a document as a testable item. Bou can use it to ind bugs be ore the irst lineo code is written.

Bla'k-Bo an" #hite-Bo Testing

Two terms that so tware testers use to describe how they approach their testing are blac"-bo/testing and white-bo/ testing. ?igure 4. shows the di erence between the two approaches. Inblac"-bo/ testing, the tester only "nows what the so tware is supposed to dohe can't loo" inthe bo/ to see how it operates. I he types in a certain input, he gets a certain output. #edoesn't "now how or why it happens, :ust that it does.

35
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch02.html#ch02http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig02


36/172

2igure 0>7> #ith bla'k-bo testing. the software tester "oesn+t know the "etails of how thesoftware works>

TI(lac"-bo/ testing is sometimes re erred to as unctional testing or behavioral testing. on't getcaught up in what the actual terms are, as your team may use something di erent. It's your :obto understand their meaning and how they're used by your team.

Thin" about the %indows +alculator shown in ?igure 4.1. I you type 3.14159 and press the s)rtbutton, you get 1.7724531 2341 . %ith blac"-bo/ testing, it doesn't matter what gyrations theso tware goes through to compute the s)uare root o pi. It :ust does it. 2s a so tware tester,you can veri y the result on another 3certi ied3 calculator and determine i the %indows+alculator is unctioning correctly.In white-bo/ testing =sometimes called clear-bo/ testing>, the so tware tester has access to theprogram's code and can e/amine it or clues to help him with his testinghe can see inside thebo/. (ased on what he sees, the tester may determine that certain numbers are more or lessli"ely to ail and can tailor his testing based on that in ormation.* T8There is a ris" to white-bo/ testing. It's very easy to become biased and ail to ob:ectively testthe so tware because you might tailor the tests to match the code's operation.

Stati' an" %ynami' Testing

Two other terms used to describe how so tware is tested are static testing and dynamictesting. &tatic testing re ers to testing something that's not runninge/amining and reviewing it.ynamic testing is what you would normally thin" o as testingrunning and using the so tware.The best analogy or these terms is the process you go through when chec"ing out a used car.ic"ing the tires, chec"ing the paint, and loo"ing under the hood are static testing techni)ues.&tarting it up, listening to the engine, and driving down the road are dynamic testingtechni)ues.

Stati' Bla'k-Bo Testing: Testing the Spe'ifi'ation

Testing the speci ication is static blac"-bo/ testing. The speci ication is a document, not ane/ecuting program, so it's considered static. It's also something that was created using datarom many sourcesusability studies, ocus groups, mar"eting input, and so on. Bou don'tnecessarily need to "now how or why that in ormation was obtained or the details o the

36
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig01http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/~hh75EC.htm#ch04fig01


37/172

process used to obtain it, :ust that it's been boiled down into a product speci ication. Bou canthen ta"e that document, per orm static blac"-bo/ testing, and care ully e/amine it or bugs.8arlier you saw an e/ample o a product speci ication or the %indows +alculator. Thise/ample used a standard written document with a picture to describe the so tware's operation.2lthough this is the most common method or writing a spec, there are lots o variations. Bourdevelopment team may emphasize diagrams over words or it may use a sel -documenting

computer language such as 2da. %hatever their choice, you can still apply all the techni)uespresented in this chapter. Bou will have to tailor them to the spec ormat you have, but theideas are still the same.%hat do you do i your pro:ect doesn't have a spec !aybe your team is using the big-bangmodel or a loose code-and- i/ model. 2s a tester, this is a di icult position. Bour goal is to indbugs earlyideally inding them be ore the so tware is codedbut i your product doesn't have aspec, this may seem impossible to do. 2lthough the spec may not be written down, someone,or several people, "now what they're trying to build. It may be the developer, a pro:ectmanager, or a mar"eter. $se them as the wal"ing, tal"ing, product spec and apply the sametechni)ues or evaluating this 3mental3 speci ication as though it was written on paper. Bou caneven ta"e this a step urther by recording the in ormation you gather and circulating it orreview.Tell your pro:ect team, 3This is what I plan to test and submit bugs against.3 Bou'll be amazedat how many details they'll immediately ill in.TIBou can test a speci ication with static blac"-bo/ techni)ues no matter what the ormat o thespeci ication. It can be a written or graphical document or a combination o both. Bou caneven test an unwritten speci ication by )uestioning the people who are designing and writingthe so tware.

Performing a =igh-,e3el Re3iew of the Spe'ifi'atione ining a so tware product is a di icult process. The spec must deal with many un"nowns,ta"e a multitude o changing inputs, and attempt to pull them all together into a documentthat describes a new product. The process is an ine/act science and is prone to havingproblems.The irst step in testing the speci ication isn't to :ump in and loo" or speci ic bugs. The irst

step is to stand bac" and view it rom a high level. 8/amine the spec or large undamentalproblems, oversights, and omissions. Bou might consider this more research than testing, butultimately the research is a means to better understand what the so tware should do. I youhave a better understanding o the whys and hows behind the spec, you'll be much better ate/amining it in detail.

Preten" to Be the Customer

The easiest thing or a tester to do when he irst receives a speci ication or review is topretend to be the customer. o some research about who the customers will be. Tal" to yourmar"eting or sales people to get hints on what they "now about the end user. I the product isan internal so tware pro:ect, ind out who will be using it and tal" to them.It's important to understand the customer's e/pectations. emember that the de inition o

)uality means 3meeting the customer's needs.3 2s a tester, you must understand those needs totest that the so tware meets them. To do this e ectively doesn't mean that you must be ane/pert in the ield o nuclear physics i you're testing so tware or a power plant, or that youmust be a pro essional pilot i you're testing a light simulator. (ut, gaining some amiliaritywith the ield the so tware applies to would be a great help.2bove all else, assume nothing. I you review a portion o the spec and don't understand it,don't assume that it's correct and go on. 8ventually, you'll have to use this speci ication todesign your so tware tests, so you'll eventually have to understand it. There's no better time tolearn than now. I you ind bugs along the way =and you will>, all the better.TI

37


38/172

on't orget about so tware security when pretending to be the customer. Bour users willassume that the so tware is secure, but you can't assume that the programmers will handlesecurity issues properly. It must be speci iedin detail. +hapter 1; , 3Testing or &o tware&ecurity,3 discusses what you should loo" or when reviewing the product design andspeci ication or security issues.

Resear'h E isting Stan"ar"s an" *ui"elines

(ac" in the days be ore !icroso t %indows and the 2pple !acintosh, nearly every so twareproduct had a di erent user inter ace. There were di erent colors, di erent menu structures,unlimited ways to open a ile, and myriad cryptic commands to get the same tas"s done.!oving rom one so tware product to another re)uired complete retraining.Than" ully, there has been an e ort to standardize the hardware and the so tware. There hasalso been e/tensive research done on how people use computers. The result is that we nowhave products reasonably similar in their loo" and eel that have been designed withergonomics in mind. Bou may argue that the adopted standards and guidelines aren't per ect,that there may be better ways to get certain tas"s done, but e iciency has greatly improvedbecause o this commonality.+hapter 11 , 3$sability Testing,3 will cover this topic in more detail, but or now you should

thin" about what standards and guidelines might apply to your product.* T8The di erence between standards and guidelines is a matter o degree. 2 standard is muchmore irm than a guideline. &tandards should be strictly adhered to i your team has decidedthat it's important to comply with them completely. Fuidelines are optional but should beollowed. It's not uncommon or a team to decide to use a standard as a guidelineas long aseveryone "nows that is the plan.

#ere are several e/amples o standards and guidelines to consider. This list isn't de initive. Boushould research what might apply to your so tware

+orporate Terminology and +onventions. I this so tware is tailored or a speci iccompany, it should adopt the common terms and conventions used by the employees othat company.

Industry e)uirements. The medical, pharmaceutical, industrial, and inancialindustries have very strict standards that their so tware must ollow.

Fovernment &tandards. The government, especially the military, has strict standards.

Fraphical $ser Inter ace =F$I>. I your so tware runs under !icroso t %indows or 2pple!acintosh operating systems, there are published standards and guidelines or how theso tware should loo" and eel to a user.

&ecurity &tandards. Bour so tware and its inter aces and protocols may need to meetcertain security standards or levels. It may also need to be independently certi ied thatit does, indeed, meet the necessary criteria.

2s a tester, your :ob isn't to de ine what guidelines and standards should be applied to yourso tware. That :ob lies with the pro:ect manager or whoever is writing the speci ication. Boudo, however, need to per orm your own investigation to 3test3 that the correct standards arebeing used and that none are overloo"ed. Bou also have to be aware o these standards andtest against them when you veri y and validate the so tware. +onsider them as part o thespeci ication.

38
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch13.html#ch13http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch11.html#ch11http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch13.html#ch13http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch11.html#ch11


39/172

Re3iew an" Test Similar Software

ne o the best methods or understanding what your product will become is to researchsimilar so tware. This could be a competitor's product or something similar to what your teamis creating. It's li"ely that the pro:ect manager or others who are speci ying your product havealready done this, so it should be relatively easy to get access to what products they used in

their research. The so tware li"ely won't be an e/act match =that's why you're creating newso tware, right >, but it should help you thin" about test situations and test approaches. Itshould also lag potential problems that may not have been considered.&ome things to loo" or when reviewing competitive products include

&cale. %ill there be ewer or greater eatures %ill there be less or more code %illthat size di erence matter in your testing

+omple/ity. %ill your so tware be more or less comple/ %ill this impact your testing

Testability. %ill you have the resources, time, and e/pertise to test so tware such asthis

MualityS eliability. Is this so tware representative o the overall )uality planned oryour so tware %ill your so tware be more or less reliable

&ecurity. #ow does the competitor's so tware security, both advertised and actual,compare to what you'll be o ering

There's no substitute or hands-on e/perience, so do whatever you can to get a hold o similarso tware, use it, bang on it, and put it through its paces. Bou'll gain a lot o e/perience thatwill help you when you review your speci ication in detail.TIon't orget about reading online and printed so tware reviews and articles about thecompetition. This can be especially help ul or security issues as you may not li"ely see thesecurity laws as you casually use the application. They will, though, be well "nown in thepress.

,ow-,e3el Spe'ifi'ation Test Te'hni ues2 ter you complete the high-level review o the product speci ication, you'll have a betterunderstanding o what your product is and what e/ternal in luences a ect its design. 2rmedwith this in ormation, you can move on to testing the speci ication at a lower level. Theremainder o this chapter e/plains the speci ics or doing this. Q1R

Q1R The chec"lists are adapted rom pp. 94- 9G and ;


40/172

2ccurate. Is the proposed solution correct oes it properly de ine the goal 2re thereany errors

recise, $nambiguous, and +lear. Is the description e/act and not vague Is there asingle interpretation Is it easy to read and understand

+onsistent. Is the description o the eature written so that it doesn't con lict withitsel or other items in the speci ication

elevant. Is the statement necessary to speci y the eature Is it e/tra in ormation thatshould be le t out Is the eature traceable to an original customer need

?easible. +an the eature be implemented with the available personnel, tools, andresources within the speci ied budget and schedule

+ode- ree. oes the speci ication stic" with de ining the product and not theunderlying so tware design, architecture, and code

Testable. +an the eature be tested Is enough in ormation provided that a tester couldcreate tests to veri y its operation

%hen you're testing a product spec, reading its te/t, or e/amining its igures, care ully considereach o these traits. 2s" yoursel i the words and pictures you're reviewing have theseattributes. I they don't, you've ound a bug that needs to be addressed.

Spe'ifi'ation Terminology Che'klist

2 complement to the previous attributes list is a list o problem words to loo" or whilereviewing a speci ication. The appearance o these words o ten signi ies that a eature isn't yetcompletely thought outit li"ely alls under one o the preceding attributes. 0oo" or these

words in the speci ication and care ully review how they're used in conte/t. The spec may goon to clari y or elaborate on them, or it may leave them ambiguousin which case, you've ounda bug.

2lways, 8very, 2ll, *one, *ever. I you see words such as these that denote somethingas certain or absolute, ma"e sure that it is, indeed, certain. ut on your tester's hatand thin" o cases that violate them.

+ertainly, There ore, +learly, bviously, 8vidently. These words tend to persuade youinto accepting something as a given. on't all into the trap.

&ome, &ometimes, ten, $sually, rdinarily, +ustomarily, !ost, !ostly. These wordsare too vague. It's impossible to test a eature that operates 3sometimes.3

8tc., 2nd &o ?orth, 2nd &o n, &uch 2s. 0ists that inish with words such as these aren'ttestable. 0ists need to be absolute or e/plained so that there's no con usion as to howthe series is generated and what appears ne/t in the list.

Food, ?ast, +heap, 8 icient, &mall, &table. These are un)uanti iable terms. Theyaren't testable. I they appear in a speci ication, they must be urther de ined toe/plain e/actly what they mean.

40


41/172

#andled, rocessed, e:ected, &"ipped, 8liminated. These terms can hide largeamounts o unctionality that need to be speci ied.

I PThenP=but missing 8lse>. 0oo" or statements that have 3I PThen3 clauses but don'thave a matching 38lse.3 2s" yoursel what will happen i the 3i 3 doesn't happen.

Summary2 ter completing this chapter, you may have decided that testing a speci ication is a verysub:ective process. #igh-level review techni)ues will lush out oversights and omissions, andlow-level techni)ues help assure that all the details are de ined. (ut, these techni)ues aren'treally step-by-step processes to ollow, or two reasons

This is an introductory boo" whose aim is to get you rapidly up the testing curve. Thematerial presented here will do :ust that. 2rmed with the in ormation presented in thischapter, you will ma"e a big dent in any so tware spec you're given to test.

The ormat o speci ications can vary widely. Bou'll be able to apply the techni)uesrom this chapter whether you're pulling the spec out o someone's brain, loo"ing at ahigh-level diagram, or parsing through sentences. Bou will ind bugs.

I you're interested in pursuing more advanced techni)ues or reviewing speci ications, do someresearch on the wor" o !ichael ?agan. %hile at I(!, !r. ?agan pioneered a detailed andmethodical approach called so tware inspections that many companies use, especiallycompanies creating mission-critical so tware, to ormally review their so tware speci icationsand code. Bou can ind more in ormation on his website www.m agan.com .

AuiThese )uiz )uestions are provided or your urther understanding. &ee 2ppendi/ 2 , 32nswers toMuiz Muestions,3 or the answersbut don't pee"N

1. +an a so tware tester per orm white-bo/ testing on a speci ication. +ite a ew e/amples o !ac or %indows standards or guidelines.

;. 8/plain what's wrong with this speci ication statement %hen the user selects the+ompact !emory option, the program will compress the mailing list data as small aspossible using a #u man-sparse-matri/ approach.

4. 8/plain what a tester should worry about with this line rom a spec The so tware willallow up to 1


42/172

ata Testing

&tate Testing

ther (lac"-(o/ Test Techni)ues

"ay, now or the good stu N This chapter covers what most people imagine when they thin"o so tware testing. It's time to crac" your "nuc"les, sit in ront o your computer, and startloo"ing or bugs.

2s a new so tware tester, this may be the irst :ob you're assigned to do. I you're interviewingor a so tware test position, you will no doubt be as"ed how you'd approach testing a newso tware program or a new program's eatures.

It's very easy to :ump right in, start pounding on "eys, and hope that something brea"s. &uch anapproach might wor" or a little while. I the so tware is still under development, it's very easyto get luc"y and ind a ew bugs right away. $n ortunately, those easy pic"ings will )uic"lydisappear and you'll need a more structured and targeted approach to continue inding bugsand to be a success ul so tware tester.

This chapter describes the most common and e ective techni)ues or testing so tware. Itdoesn't matter what "ind o program you're testingthe same techni)ues will wor" whether it's acustom accounting pac"age or your company, an industrial automation program, or a mass-mar"et shoot-'em-up computer game.

Bou also don't need to be a programmer to use these techni)ues. 2lthough they're all based onundamental programming concepts, they don't re)uire you to write code. 2 ew techni)ueshave some bac"ground in ormation that e/plains why they're e ective, but any code samplesare short and written in a simple macro language to easily demonstrate the point. I you're intoprogramming and want to learn more low-level test techni)ues, a ter you inish reading thischapter, move on to +hapter 5 , 38/amining the +ode,3 and +hapter 7 , 3Testing the &o twarewith O- ay Flasses,3 the white-bo/ testing chapters.

Topics covered in this chapter include

%hat is dynamic blac"-bo/ testing #ow to reduce the number o test cases by e)uivalence partitioning

#ow to identi y troublesome boundary conditions

Food data values to use to induce bugs

#ow to test so tware states and state transitions

#ow to use repetition, stress, and high loads to locate bugs

2 ew secret places where bugs hide

%ynami' Bla'k-Bo Testing: Testing the Software #hile Blin"fol"e"

42
http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec4.html#ch05lev1sec4http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec5.html#ch05lev1sec5http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec6.html#ch05lev1sec6http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch06.html#ch06http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch07.html#ch07http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec4.html#ch05lev1sec4http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec5.html#ch05lev1sec5http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch05lev1sec6.html#ch05lev1sec6http://c/Documents%20and%20Settings/Vivek%20Jain/Local%20Settings/Temp/ch06.html#ch06http://c/Documents%20and%20Settings/V

Ron Patton

Documents

Transcript of Ron Patton