Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is...
Transcript of Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is...
![Page 1: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/1.jpg)
Software EngineeringTesting and Debugging — Testing
Prof. Dr. Peter Thiemann
Universitat Freiburg
02.07.2009
![Page 2: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/2.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 3: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/3.jpg)
Recap
I Testing – detect the presence of bugs by observing failures
I Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 4: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/4.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failure
I In order to know when a program fails we must have aspecification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 5: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/5.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 6: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/6.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specifications
I Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 7: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/7.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficult
I Even with good test cases, absence of failures does not implyabsence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 8: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/8.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 9: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/9.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugs
I This seems preferable, but software verification is in mostcases economically impossible
I On the other hand, testing is not just a way of finding bugsduring the development –
I Making testing a part of the development makes your claimsabout the software more credible
![Page 10: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/10.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossible
I On the other hand, testing is not just a way of finding bugsduring the development –
I Making testing a part of the development makes your claimsabout the software more credible
![Page 11: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/11.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –
I Making testing a part of the development makes your claimsabout the software more credible
![Page 12: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/12.jpg)
Recap
I Testing – detect the presence of bugs by observing failuresI Debugging – find the bug causing a certain failureI In order to know when a program fails we must have a
specification (except for obvious failure such as a programcrash)
I A great deal of care is needed when writing specificationsI Finding a good set of test cases is often difficultI Even with good test cases, absence of failures does not imply
absence of bugs (the number of ways to use the program ishuge or infinite)
I Program Verification is about making sure there are no bugsI This seems preferable, but software verification is in most
cases economically impossibleI On the other hand, testing is not just a way of finding bugs
during the development –I Making testing a part of the development makes your claims
about the software more credible
![Page 13: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/13.jpg)
Contents of Testing part
I Specifications (informal)I Test Cases
I How to write a test caseI How to come up with a good test suite (collection of test
cases)
![Page 14: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/14.jpg)
Contents of Testing part
I Specifications (informal)
I Test CasesI How to write a test caseI How to come up with a good test suite (collection of test
cases)
![Page 15: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/15.jpg)
Contents of Testing part
I Specifications (informal)I Test Cases
I How to write a test caseI How to come up with a good test suite (collection of test
cases)
![Page 16: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/16.jpg)
Contents of Testing part
I Specifications (informal)I Test Cases
I How to write a test case
I How to come up with a good test suite (collection of testcases)
![Page 17: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/17.jpg)
Contents of Testing part
I Specifications (informal)I Test Cases
I How to write a test caseI How to come up with a good test suite (collection of test
cases)
![Page 18: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/18.jpg)
Specifications
I Program specifications tell what a piece of code should do
I But also what it requires to be able to do its job
I A specification can be seen as contract between theimplementor and the user of the implemented code.
I A specification consists of two parts:I Requires (precondition) – what the user should fulfill before
calling the codeI Ensures (postcondition) – what the implementor promises
about the result of the execution (provided requires werefulfilled)
![Page 19: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/19.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
![Page 20: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/20.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires:Ensures: Result is the minimum element in a
![Page 21: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/21.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires: a is non-nullEnsures: Result is the minimum element in a
![Page 22: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/22.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires: a is non-nullEnsures: Result is a minimum element in a
![Page 23: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/23.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires: a is non-nullEnsures: Result is equal to the minimum element in a
![Page 24: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/24.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires: a is non-nullEnsures: Result is less than or equal to all elements in a
![Page 25: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/25.jpg)
Specification Example
public s ta t i c int find_min( int [] a) { ... }
Specification
Requires: a is non-null
Ensures: Result is less than or equal to all elements in aand equal to at least one element in a
![Page 26: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/26.jpg)
Specification Example
public s ta t i c int find_min( int [] a) {int x, i;x = a[0];for (i = 1; i < a.length;i ++) {i f (a[i] < x) x = a[i];
}return x;
}
Specification
Requires: a is non-null
Ensures: Result is less than or equal to all elements in aand equal to at least one element in a
![Page 27: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/27.jpg)
Specification Example
public s ta t i c int find_min( int [] a) {int x, i;x = a[0];for (i = 1; i < a.length;i ++) {i f (a[i] < x) x = a[i];
}return x;
}
Specification
Requires: a is non-null and contains at least one element
Ensures: Result is less than or equal to all elements in aand equal to at least one element in a
![Page 28: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/28.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
![Page 29: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/29.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
Specification
Requires: a is non-nullEnsures: Result is the minimum element in a
![Page 30: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/30.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meant
I Imprecise – more can be said about the behaviourI Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
Specification
Requires: a is non-nullEnsures: Result is a minimum element in a
![Page 31: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/31.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
![Page 32: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/32.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weak
I Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
Specification
Requires: a is non-nullEnsures: Result is less than or equal to all elements in a
![Page 33: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/33.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
Specification
Requires: a is non-null and contains at exactly one ele-ment
Ensures: Result is less than or equal to all elements in aand equal to at least one element in a
![Page 34: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/34.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is said
I Precondition is too weakI Postcondition is too strong
![Page 35: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/35.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weak
I Postcondition is too strong
Specification
Requires: a is non-null
Ensures: Result is less than or equal to all elements in aand equal to at least one element in a
![Page 36: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/36.jpg)
What can be wrong about a specification?
I Badly stated – does not make sense
I Vague – unclear what is meantI Imprecise – more can be said about the behaviour
I Postcondition is too weakI Precondition is too strong
I Incorrect – too much is saidI Precondition is too weakI Postcondition is too strong
Specification
Requires: a is non-null and contains at least one element
Ensures: Result is less than or equal to all elements ina and equal to at least one element in a, andresult is greater than 0
![Page 37: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/37.jpg)
What can go wrong when writing specifications?
Are all these cases of “bad” specifications?
I It’s clear that we don’t invalid or incorrect specifications.
I Vague specifications is a matter of whether they can bemisunderstood.
I But imprecise specifications is not such a bad thing
![Page 38: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/38.jpg)
Example: Strong or Weak Precondition
Example
What does this method do?
public s ta t i c int [] insert( int [] x, int n){
int [] y = new int [x.length + 1];int i;for (i = 0; i < x.length; i++) {
i f (n >= x[i]) break;y[i] = x[i];
}y[i] = n;for (; i < x.length; i++) {
y[i+1] = x[i];}return y;
}
![Page 39: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/39.jpg)
Example, cont’d
Example
What does this method do?
public s ta t i c int [] insert( int [] x, int n){ ... }
Specification
Requires:Ensures:
![Page 40: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/40.jpg)
Example, cont’d
Example
What does this method do?
public s ta t i c int [] insert( int [] x, int n){ ... }
Specification
Requires: x is non-null.Ensures: Result is equal to x with n inserted in it.
![Page 41: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/41.jpg)
Example, cont’d
Example
What does this method do?
public s ta t i c int [] insert( int [] x, int n){ ... }
Specification
Requires: x is non-null.
Ensures: Result is equal to x with n inserted in it andresult is sorted in ascending order.
![Page 42: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/42.jpg)
Example, cont’d
Example
What does this method do?
public s ta t i c int [] insert( int [] x, int n){ ... }
Specification
Requires: x is non-null and sorted in ascending order.
Ensures: Result is equal to x with n inserted in it andresult is sorted in ascending order.
![Page 43: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/43.jpg)
Specification of a Class
Class invariantI A class invariant is a condition about the state of each class
instance that should be maintain throughout its existenceI We will focus on weak invariants
I It should hold between calls to methods of the class,I but not during the execution of such methods
Class specification consists of
I Class invariant
I Requires and ensures of the methods
![Page 44: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/44.jpg)
Specification of a Class
Class invariantI A class invariant is a condition about the state of each class
instance that should be maintain throughout its existenceI We will focus on weak invariants
I It should hold between calls to methods of the class,I but not during the execution of such methods
Class specification consists of
I Class invariant
I Requires and ensures of the methods
![Page 45: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/45.jpg)
Example, class invariant
public c la s s HashSet {private Object [] arr;int nobj;
public void insert(Object o) { ... }...
}
Class InvariantI nobj should be equal to the number of non-null elements in
arr, and
I for each index i in range of arr suc that arr[i ] is non-null, allelements between indices arr[i ].hash() and i are non-null, and
I there are no two non-null elements of arr that are equal
![Page 46: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/46.jpg)
Example, class invariant
public c la s s HashSet {private Object [] arr;int nobj;
public void insert(Object o) { ... }...
}
Class Invariant
I nobj should be equal to the number of non-null elements inarr, and
I for each index i in range of arr suc that arr[i ] is non-null, allelements between indices arr[i ].hash() and i are non-null, and
I there are no two non-null elements of arr that are equal
![Page 47: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/47.jpg)
Example, class invariant
public c la s s HashSet {private Object [] arr;int nobj;
public void insert(Object o) { ... }...
}
Class InvariantI nobj should be equal to the number of non-null elements in
arr, and
I for each index i in range of arr suc that arr[i ] is non-null, allelements between indices arr[i ].hash() and i are non-null, and
I there are no two non-null elements of arr that are equal
![Page 48: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/48.jpg)
Example, class invariant
public c la s s HashSet {private Object [] arr;int nobj;
public void insert(Object o) { ... }...
}
Class InvariantI nobj should be equal to the number of non-null elements in
arr, and
I for each index i in range of arr suc that arr[i ] is non-null, allelements between indices arr[i ].hash() and i are non-null, and
I there are no two non-null elements of arr that are equal
![Page 49: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/49.jpg)
Example, class invariant
public c la s s HashSet {private Object [] arr;int nobj;
public void insert(Object o) { ... }...
}
Class InvariantI nobj should be equal to the number of non-null elements in
arr, and
I for each index i in range of arr suc that arr[i ] is non-null, allelements between indices arr[i ].hash() and i are non-null, and
I there are no two non-null elements of arr that are equal
![Page 50: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/50.jpg)
Testing
![Page 51: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/51.jpg)
Software testing
What we look at hereI Systematic – general rules for how to write test cases
I Repeatable – be able to run tests over and over again
What we don’t look at hereI Running your program to see if anything goes wrong
I Letting a lot of people run your program to see if anythinggoes wrong (Beta-testing)
![Page 52: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/52.jpg)
Software testing
What we look at hereI Systematic – general rules for how to write test cases
I Repeatable – be able to run tests over and over again
What we don’t look at hereI Running your program to see if anything goes wrong
I Letting a lot of people run your program to see if anythinggoes wrong (Beta-testing)
![Page 53: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/53.jpg)
Testing on Different Levels
I Unit Testing – testing a small unit of a systemRequires that the behaviour of the unit has been specificied.In Java this often corresponds to testing a method or a class.
I Integration Testing – testing the interaction between two ormore units
I System Testing – testing a whole system against thespecification of its externally observable behaviourSystem testing is mostly useful for convincing about thecorrectness. Less useful for finding bugs because the infectionphase going from defect to failure is usually complex anddifficult to unwind.
The code that is being tested is called the IUT (implementationunder test).
![Page 54: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/54.jpg)
Testing on Different Levels
I Unit Testing – testing a small unit of a systemRequires that the behaviour of the unit has been specificied.In Java this often corresponds to testing a method or a class.
I Integration Testing – testing the interaction between two ormore units
I System Testing – testing a whole system against thespecification of its externally observable behaviourSystem testing is mostly useful for convincing about thecorrectness. Less useful for finding bugs because the infectionphase going from defect to failure is usually complex anddifficult to unwind.
The code that is being tested is called the IUT (implementationunder test).
![Page 55: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/55.jpg)
Testing on Different Levels
I Unit Testing – testing a small unit of a systemRequires that the behaviour of the unit has been specificied.In Java this often corresponds to testing a method or a class.
I Integration Testing – testing the interaction between two ormore units
I System Testing – testing a whole system against thespecification of its externally observable behaviourSystem testing is mostly useful for convincing about thecorrectness. Less useful for finding bugs because the infectionphase going from defect to failure is usually complex anddifficult to unwind.
The code that is being tested is called the IUT (implementationunder test).
![Page 56: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/56.jpg)
Testing on Different Levels
I Unit Testing – testing a small unit of a systemRequires that the behaviour of the unit has been specificied.In Java this often corresponds to testing a method or a class.
I Integration Testing – testing the interaction between two ormore units
I System Testing – testing a whole system against thespecification of its externally observable behaviourSystem testing is mostly useful for convincing about thecorrectness. Less useful for finding bugs because the infectionphase going from defect to failure is usually complex anddifficult to unwind.
The code that is being tested is called the IUT (implementationunder test).
![Page 57: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/57.jpg)
Testing Non-Functional RequirementsNot considered further
I Performance testing or load testing
I Stability testing
I Usability testing
I Security testing
![Page 58: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/58.jpg)
When Should Testing Take Place?
The sooner a bug is found, the better.
So, testing should start early.Extreme case: Test-driven program development
but
Testing early means a lot of unit testing which requires a lot ofspecifications.
But writing specifications for the units of a system is alreadyneeded for a large project when programming by contract.
Tested units may be replaced later on, making the testsuseless.
On the other hand, writing and running tests often gives a deepunderstanding of the program. The need to replace the unit mayhave been realized during the testing activities.
![Page 59: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/59.jpg)
When Should Testing Take Place?
The sooner a bug is found, the better.
So, testing should start early.Extreme case: Test-driven program development
but
Testing early means a lot of unit testing which requires a lot ofspecifications.
But writing specifications for the units of a system is alreadyneeded for a large project when programming by contract.
Tested units may be replaced later on, making the testsuseless.
On the other hand, writing and running tests often gives a deepunderstanding of the program. The need to replace the unit mayhave been realized during the testing activities.
![Page 60: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/60.jpg)
When Should Testing Take Place?
The sooner a bug is found, the better.
So, testing should start early.Extreme case: Test-driven program development
but
Testing early means a lot of unit testing which requires a lot ofspecifications.
But writing specifications for the units of a system is alreadyneeded for a large project when programming by contract.
Tested units may be replaced later on, making the testsuseless.
On the other hand, writing and running tests often gives a deepunderstanding of the program. The need to replace the unit mayhave been realized during the testing activities.
![Page 61: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/61.jpg)
Systematic testing
I Use precise methods to design correct tests
I Each individual test is called a test case
I Organize collections of related test cases in test suites
I Use precise methods to make sure that a test suite has a goodcoverage of the different cases of usage
![Page 62: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/62.jpg)
Systematic testing
I Use precise methods to design correct tests
I Each individual test is called a test case
I Organize collections of related test cases in test suites
I Use precise methods to make sure that a test suite has a goodcoverage of the different cases of usage
![Page 63: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/63.jpg)
Systematic testing
I Use precise methods to design correct tests
I Each individual test is called a test case
I Organize collections of related test cases in test suites
I Use precise methods to make sure that a test suite has a goodcoverage of the different cases of usage
![Page 64: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/64.jpg)
Systematic testing
I Use precise methods to design correct tests
I Each individual test is called a test case
I Organize collections of related test cases in test suites
I Use precise methods to make sure that a test suite has a goodcoverage of the different cases of usage
![Page 65: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/65.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 66: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/66.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 67: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/67.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 68: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/68.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 69: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/69.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 70: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/70.jpg)
Repeatable testing
The basic idea is to write code that performs the tests.
I A tool can automatically run a large collection of tests
I The testing code can be integrated into the actual code, thusstored in a organised way
I When a bug has been fixed, the tests can be rerun to check ifthe failure is gone
I Whenever the code is extended, all old test cases can be rerunto check that nothing is broken (regression testing)
JUnit is a small tool for writing and running test cases. It provides:
I Some functionality that is repeatedly needed when writing testcases
I A way to annotate methods as being test cases
I A way to run test cases automatically in a batch
![Page 71: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/71.jpg)
Testing influences code
Apart the obvious – that testing should result in removal of bugs
I When you write specifications and test cases for units, theresponsibilities of the different parts become clearer, whichpromotes good OO programming style (low coupling)
I In order to be able to test programs automatically, separatingthe IO and functionality becomes important
![Page 72: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/72.jpg)
Testing influences code
Apart the obvious – that testing should result in removal of bugs
I When you write specifications and test cases for units, theresponsibilities of the different parts become clearer, whichpromotes good OO programming style (low coupling)
I In order to be able to test programs automatically, separatingthe IO and functionality becomes important
![Page 73: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/73.jpg)
Testing influences code
Apart the obvious – that testing should result in removal of bugs
I When you write specifications and test cases for units, theresponsibilities of the different parts become clearer, whichpromotes good OO programming style (low coupling)
I In order to be able to test programs automatically, separatingthe IO and functionality becomes important
![Page 74: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/74.jpg)
What does a test case consists of?
Test caseI Initialisation (of class instance and input arguments)
I Call to the method of the IUT
I A test oracle which decides if the test succeeded or failed
I The test oracle is vital to run tests automatically
![Page 75: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/75.jpg)
What does a test case consists of?
Test caseI Initialisation (of class instance and input arguments)
I Call to the method of the IUT
I A test oracle which decides if the test succeeded or failed
I The test oracle is vital to run tests automatically
![Page 76: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/76.jpg)
Demo
Small demo showing basics of how to use JUnit
![Page 77: Software Engineering Testing and Debugging --- Testing · I Finding a good set of test cases is often di cult I Even with good test cases, absence of failures does not imply absence](https://reader034.fdocuments.us/reader034/viewer/2022042210/5eae46b7d53a6946a62c7ffa/html5/thumbnails/77.jpg)
Summary, and what’s next?
Summary
I Specifications (motivation, contracts, pre- and postconditions,what to think about)
I Testing (motivation, different kinds of testing, role in softwaredevelopment, junit)
What’s next?I More examples of test cases, presenting aspects of writing test
cases and features of JUnit
I How to write a good test case?
I How to construct a good collection of test cases (test suite)?