Measurements &milestones for monitoring and controlling
-
Upload
dhiraj-singh -
Category
Technology
-
view
1.000 -
download
3
description
Transcript of Measurements &milestones for monitoring and controlling
![Page 1: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/1.jpg)
• All process should have metrics.
• Metrics in testing domain – track test progress ,evaluate quality , test effectiveness & when to stop testing .
![Page 2: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/2.jpg)
Meaningful metrics begin with questions –
Which measure to collect? Purpose ? Who collects them ? Forms & Tools used ? Who analyses the data ? Who has the access to report ?
![Page 3: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/3.jpg)
Begin with a test plan . Work done is compared with the work
planned To support monitoring , Test plans
should contain Testing milestones .
![Page 4: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/4.jpg)
Milestones are tangible events that are expected to occur at a certain time in the project’s lifetime.
Gives the status of the project .
![Page 5: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/5.jpg)
• Test Milestones can be used to monitor the progress of the testing efforts associated with a software project .
• Guideposts or Goals that needs to be met .
• Test Manager uses current testing effort data to find how close the milestone achievement is
![Page 6: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/6.jpg)
• Milestone usually appear in the Scheduling Component of the test plan.
• Different levels of testing have different
milestones .
![Page 7: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/7.jpg)
Completion of the master test plan ?
Completion of Branch Coverage ? ( Unit test )
Execution of all planned system tests ?
Completion of Test summary Report ?
![Page 8: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/8.jpg)
Every milestone has to be scheduled for completion during a certain period of time
A group of test members are assigned for achieving the milestone on time and allotted budget .
![Page 9: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/9.jpg)
Test planners need to be sure that milestone are plausible & meaningful .
Milestones should be descripted in simple terms and ambiguities should be avoided.
Monitoring Data are collected that relates to the current status of the testing cycle .
![Page 10: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/10.jpg)
• Graphs Can be used to show the trends or the phases of the monitoring process over a limited period of time .
• The graphs can be in form of bar charts and can be represented for days , weeks or months based on the scalability of the test to be done .
![Page 11: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/11.jpg)
Project & test leaders present up-to-date measurements , graphs and plots showing the testing efforts.
Testing milestones achieved and those which are not .
Test logs , test incident reports & other test-related documents
Managers question the team .
![Page 12: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/12.jpg)
Testing Status
Tester Productivity
Testing costs
Errors , Faults & Failures .
![Page 13: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/13.jpg)
Identifying the current state of the testing process .
Manager needs to determine if the testing tasks are being completed on time and within budget .
![Page 14: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/14.jpg)
• Which tasks are on time ?
• Which are behind Schedule , and by how much ?
• Have the scheduled milestones for this date been met ?
![Page 15: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/15.jpg)
Test Mangers measures how much coverage has been actually achieved and how much is left .
Compare it with the planned coverage.
![Page 16: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/16.jpg)
• White box testing – Degree of statement , branch , data flow , basis path ( planned , actual )
• Actual / planned degree of coverage
![Page 17: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/17.jpg)
Black Box testing – Number of requirements to be tested , equivalence classes identified , equivalence classes actually covered
No of features actually covered/ total no of features to be covered .
![Page 18: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/18.jpg)
To monitor the progress of test case development , and can be applied to all levels of testing .
Estimated Test cases- No of planned test cases
Test cases complete – No of available test cases .
Change in requirements – Unplanned test cases .
![Page 19: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/19.jpg)
• Test manager will want to determine if the execution process is going according to plan .
• Appropriate measures No of available test cases executed No of available test cases exe & passed No of unplanned test cases No of unplanned test cases exe and
passed .
![Page 20: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/20.jpg)
For new release where there can be regression testing
No of planned regression tests exe No of planned regression test exe and
planned
![Page 21: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/21.jpg)
Imp that Test managers To monitor progress of the development of the test harness code needed for unit and integration test – Timely manner
Size – applied by manager to help estimate the amt of effort needed to develop a s/w pgm .
![Page 22: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/22.jpg)
• Size can be measured in different ways Lines of code Function points Feature points
• Commonly used is Loc
![Page 23: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/23.jpg)
Monitor the test harness development effort over time – Available LOC / planned LOC .
![Page 24: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/24.jpg)
Productivity of the staff and how it changes the Project progresses.
Measuring productivity in s/w domain is difficult task .
Since developers are involved in many activities , many are complex
![Page 25: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/25.jpg)
The existing method of measuring productivity of developers and testers are not credible .
Productivity of tester can be measured using a combination of
No of test cases developed /unit time No of test cases exe No of Loc Test Harness developed /unit
time No of defects detected in unit time.
![Page 26: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/26.jpg)
Good Technique – budget and resource monitoring is called earned value tracking .
To calculate Planned earned value Total estimated time of budget for
overall testing effort Estimated time or budget for each
testing task
![Page 27: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/27.jpg)
• Earned value can be calculated separately for each level of testing
This will help in monitoring the budget and resource usage for individual testing phase .
Actual time for each testing task . Earned value for testing task to date .
![Page 28: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/28.jpg)
Useful for Evaluating product quality
Evaluating testing effectiveness
Making stop-test decisions .
![Page 29: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/29.jpg)
Defect casual analysis
Defect prevention
Test process improvement
Development process improvement .
![Page 30: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/30.jpg)
• Total number of incident reports.
• Number of incident reports resolved / unresolved.
• Number of defects found of each given type.
• Number of failures .
![Page 31: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/31.jpg)
• Allows managers to determine if test resources have been used wisely and productivity to remove defects and evaluate product quality .
• Evaluation is made in several ways ,• Before release – no of defects found
during testing / number of defects estimated .
• After release – no of defects found / 1000 lines of new and changed code
![Page 32: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/32.jpg)
• Defect removal ratio Pre-ship fault density / post- ship fault
density .
![Page 33: Measurements &milestones for monitoring and controlling](https://reader033.fdocuments.us/reader033/viewer/2022061221/54bdb3e94a795902778b4595/html5/thumbnails/33.jpg)
• Measurement suggested by Graham
No of defects detected in a given test phase/ total number of defects found in testing.
Detect removal leverage = no of defects found integration test / no of defects found in unit test .