Hearing screening on the path through life - Dr. Dr. h.c. Monika Lehnhardt
Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search...
-
Upload
christina-walker -
Category
Documents
-
view
217 -
download
2
Transcript of Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search...
![Page 1: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/1.jpg)
SE 3800NOTE 5
BEYOND FUNCTIONAL TESTING
Dr. Rob Hasker
Dr. Brad Dennis
![Page 2: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/2.jpg)
ReliabilitySystems Engineering: • Linear System: response to
complex input can be described as a sum of responses to simpler inputs• Eg: Wave propagation• Nonlinear examples: AC power
flow, almost any complex system
• Reliability for linear systems: product of reliability of each component
A: 90% Reliable
B: 90%
C: 90%
Reliability: .93 = .73
![Page 3: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/3.jpg)
Reliability in Software
Car that works 73% of the time: find a shop or dealer
How many components in software?
How can we achieve high reliability for SW?
![Page 4: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/4.jpg)
Review – what is
Integration, system, acceptance testing? What is performance testing?
How do you do it?What can you learn from this type of testing?What can't you learn from this type of testing?
Stress testAnswers what?What do you want to happen?
○ Goal: graceful degradation - bend but don't break
![Page 5: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/5.jpg)
More testing methods
How do you test security? Websites:
Database injection, cross-site scriptingFuzz checkers – inserting random data
How to test usability? In general, how to test non-functional
requirements?Must identify testable component!
![Page 6: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/6.jpg)
Testing strategy Why might we not be able to run every test
every time?How should we prioritize tests?
Regression set minimization: identifying a subset that would reveal most bugsResearch question!Should be able to run all unit tests, majority of
system tests Less frequently: performance, other tests
Maybe these are just run once per day
![Page 7: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/7.jpg)
Qualities of good tests
Thorough – coverageStart with statement coverage
RepeatableGenerate small data sets that have
predictable outputsTest runs should not depend on each other
UnderstandableMaintainableEasy to identify failed case
![Page 8: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/8.jpg)
Testing
What about “grandma testing”?
![Page 9: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/9.jpg)
Testing
What about “grandma testing”?Sexist? Ageist?
"Local grandmother finally uses printer"
CES 2009: hall for products for retirees
![Page 10: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/10.jpg)
Testing
What about “grandma testing”?Sexist? Ageist?
"Local grandmother finally uses printer"
Is grandma-using-the-computer themodern equivalent of racist jokes?
CES 2009: hall for products for retirees
![Page 11: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/11.jpg)
Testing
What about “grandma testing”?Sexist? Ageist?Fuzz testing tools
![Page 12: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/12.jpg)
Testing
What about “grandma testing”?Sexist? Ageist?Fuzz testing tools
What if need large data sets?Test oraclesVersion matching
![Page 13: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/13.jpg)
Testing
What about “grandma testing”?Sexist? Ageist?Fuzz testing tools
What if need large data sets?Test oraclesVersion matching
Key: differentiate between robustness, correctness
![Page 14: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/14.jpg)
System Testing Focus: functional testing
that is, are there errors in features? In some sense, this is the only test that
matters tests all functionality seen by customerwhy shouldn't we do all testing at this level?
○ harder to test deeply○ harder to know how to fix system if test fails
Acceptance testing: system testing w/ focus on common scenarios
![Page 15: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/15.jpg)
Functional/unit testing
How is functional testing different from unit testing?Scope: unit test exercises pieces, functional
is end-to-end How are the the same?
Written by developer organizationLikely to use mocks to isolate components
![Page 16: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/16.jpg)
Acceptance/functional testing
How does acceptance testing differ from functional testing?Functional:
○ Internal, verify featuresAcceptance:
○ External, validate requirements○ Rare to use mocks – only for very low
probability/high impact errors that must be tested○ Production environment (as close as possible)
![Page 17: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/17.jpg)
The Real Differences Lies in What You Are Asking Does this feature meet the customer
requirement?Acceptance Test.
Does this feature work? Functional Test.
Does this feature work with the database?Integration Test.
Does this method do what I want it to do?Unit Test.
![Page 18: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/18.jpg)
What does a manual test look like?Step Req Pass Conditions Pass?
1. Select the Biology program.
UIR-2
System displays biology classes w/ first class BIOLOGY 1150, Section 01, Title GENERAL BIOLOGY, Instructor Block, Anna, Filled/Seats 52/53, Class# 1311, Credits 5, Meets BOE 0221 MWF 8:00-8:52
P / F
2. Double-click on Class# 1330
UIR-1System includes Class# 1330 in schedule at bottom
P / F
3. Scroll down to Class# 1331 (BIOLOGY 1650, Section 01)
UIR-9System displays Class# 1331 with a pink background
P / F
4. UIR-9All sections listed between #1311 and #1331 have a white background
P / F
5. Select the GENENG program.
UIR-2 System displays general engineering courses P / F
6. UIR-9
GENENG 1030 sections 01-07 have a pink backgroundall other sections of GENGENG 1030 have a white background
P / FP / F
7. UIR-9All sections of GENGENG 1000 have a white background
P / F
![Page 19: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/19.jpg)
What does a manual test look like?Step Req Pass Conditions Pass?
1. Select the Biology program.
UIR-2
System displays biology classes w/ first class BIOLOGY 1150, Section 01, Title GENERAL BIOLOGY, Instructor Block, Anna, Filled/Seats 52/53, Class# 1311, Credits 5, Meets BOE 0221 MWF 8:00-8:52
P / F
2. Double-click on Class# 1330
UIR-1System includes Class# 1330 in schedule at bottom
P / F
3. Scroll down to Class# 1331 (BIOLOGY 1650, Section 01)
UIR-9System displays Class# 1331 with a pink background
P / F
4. UIR-9All sections listed between #1311 and #1331 have a white background
P / F
5. Select the GENENG program.
UIR-2 System displays general engineering courses P / F
6. UIR-9
GENENG 1030 sections 01-07 have a pink backgroundall other sections of GENGENG 1030 have a white background
P / FP / F
7. UIR-9All sections of GENGENG 1000 have a white background
P / F
• Lots of detail
• Must document passes
manually
• Maintenance issue when
steps depend on each other
![Page 20: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/20.jpg)
Who should write acceptance tests? Obvious answer: client
Probably manual testsNot much help to development team
Alternative: behavior-driven developmentFunctional, acceptance tests written up frontSimple model: acceptance criteria for each
story = collection of system-level tests
![Page 21: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/21.jpg)
Getting clients to write tests Why can’t clients write our tests?
Barrier: testing language Solution: Cucumber
Testing language: specify start conditions, expected results
Supports tables when have several examples
Developer: provides hooks defining how the tests are executed
![Page 22: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/22.jpg)
Client-written tests Wins:
Test documents what system supposed to do in a verifiable manner!
Expected behavior is very concreteCustomer gets to control behavior from start
ProblemsExtra time to write tests – if client not writing tests,
probably should use other methods○ But then client may not really want system!
Must maintain tests○ But manual tests need maintenance, too
![Page 23: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/23.jpg)
Acceptance tests in
https://cucumber.io/ Supports Ruby, Java, .NET, C++, all
web applications Underlying language: Gherkin
![Page 24: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/24.jpg)
Steps: 1. describe behavior
Feature: AdditionIn order to avoid silly mistakesAs a math-challenged personI want to be told the sum of two numbers
Scenario: Add two numbersGiven I have entered 50 into the calculatorAnd I have entered 70 into the calculatorWhen I press addThen the result should be 120 on the
screen
![Page 25: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/25.jpg)
Steps: 1. describe behavior
Feature: AdditionIn order to avoid silly mistakesAs a math-challenged personI want to be told the sum of two numbers
Scenario: Add two numbersGiven I have entered 50 into the calculatorAnd I have entered 70 into the calculatorWhen I press addThen the result should be 120 on the
screen
These actually all have
the same effect.
![Page 26: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/26.jpg)
2. Step definition in Ruby
Given /I have entered (.*) into the calculator/ do |n| calculator = Calculator.new calculator.push(n.to_i)end
Run and watch it fail:
![Page 27: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/27.jpg)
2. Write code to make it pass
class Calculator def push(n) @args ||= [] @args << n endend
Run see step pass:
![Page 28: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/28.jpg)
3. Repeat until “green like a cuke”
![Page 29: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/29.jpg)
Behavior-driven development Basic model: test-driven development Great for starting, probably not for
regular workEspecially friendly to people who are
learning about syntax
![Page 30: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/30.jpg)
Gherkin Syntax
Each feature described in single .feature file
3 elements:Feature: a name – free form syntaxDescription: documentation
○ Any line not starting w/ a keywordScenario: concrete example
![Page 31: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/31.jpg)
Gherkin Syntax Keywords:
FeatureScenarioGiven, when, then,
and, but: stepsBackgroundScenario outlineExamples
Feature: Refund item
Sales associates required by law to refund purchases
Scenario: Jeff returns a faulty microwave Given Jeff has bought a microwave for $100 And he has a receipt # don’t assume this is true! When he returns the microwave Then Jeff should be refunded $100
![Page 32: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/32.jpg)
Gherkin Syntax Keywords:
FeatureScenarioGiven, when, then,
and, but: stepsBackgroundScenario outlineExamples
Feature: Refund item
Sales associates required by law to refund purchases
Scenario: Jeff returns a faulty microwave Given Jeff has bought a microwave for $100 And he has a receipt # don’t assume this is true! When he returns the microwave Then Jeff should be refunded $100
On same line
Documentation- no keyword, multiline
Can also comment with #
![Page 33: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/33.jpg)
Gherkin Syntax Keywords:
FeatureScenarioGiven, when, then,
and, but: stepsBackgroundScenario outlineExamples
Feature: Refund item
Sales associates required by law to refund purchases
Scenario: Jeff returns a faulty microwave Given Jeff has bought a microwave for $100 And he has a receipt # don’t assume this is true! When he returns the microwave Then Jeff should be refunded $100
• No distinctions between these• Chose for readability• Single when (or break up)• Then: compare actual to expected
![Page 34: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/34.jpg)
Gherkin Syntax Keywords:
FeatureScenarioGiven, when, then,
and, but: stepsBackgroundScenario outlineExamples
Background: Given a $100 microwave was sold on 2015-08-26 And today is 2015-08-26
Steps which are assumed before each scenario
![Page 35: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/35.jpg)
Scenario outline Parameterize values, specify with table:
Scenario Outline: feeding a suckler cow Given the cow weighs <weight> kg When we calculate the feeding requirements Then the energy should be <energy> MJ And the protein should be <protein> kg
Examples: | weight | energy | protein | | 450 | 26500 | 215 | | 500 | 29500 | 245 | | 575 | 31500 | 255 | | 600 | 37000 | 305 |
![Page 36: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/36.jpg)
Scenario outline Parameterize values, specify with table:
Scenario Outline: feeding a suckler cow Given the cow weighs <weight> kg When we calculate the feeding requirements Then the energy should be <energy> MJ And the protein should be <protein> kg
Examples: | weight | energy | protein | | 450 | 26500 | 215 | | 500 | 29500 | 245 | | 575 | 31500 | 255 | | 600 | 37000 | 305 |
One or more examples sections
Required header row
‘|’ is a key symbol
![Page 37: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/37.jpg)
Implementing tests
C++ code implementing this step:
Scenario: Given I have 48 cukes in my belly
Given(“^I have (\\d+) cukes in my belly$”) { REGEX_PARAM(int, cukes); USING_CONTEXT(MyAppCtx, context); std::cout << “Cukes: “ << cukes;}
![Page 38: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/38.jpg)
Implementing tests
C++ code implementing this step:
Scenario: Given I have 48 cukes in my belly
Given(“^I have (\\d+) cukes in my belly$”) { REGEX_PARAM(int, cukes); USING_CONTEXT(MyAppCtx, context); std::cout << “Cukes: “ << cukes;}
Capture groupUsually regular expressiond+: one or more digitsMatches and, then, etc.
Declaration
Boiler plate
![Page 39: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/39.jpg)
Key: regular expressions
See n05-3800-regex.html Flexible tool for capturing words with
specific patterns Will generally use several simple cases:
\d+, \w+^, $: anchor to start and endbackslash to escape special characters
![Page 40: Dr. Rob Hasker Dr. Brad Dennis. Reviewing CI Setup So what do the steps do? Key concept: search path Windows: for each command, run through PATH to.](https://reader034.fdocuments.us/reader034/viewer/2022042718/5697bfd31a28abf838cac060/html5/thumbnails/40.jpg)
Review
Unit vs. functional vs. acceptance testing
Testing non-functional requirements: performance, security, etc.
Behavior-driven development with Cucumber
Next class period: exercise on writing tests in Cucumber