Paper 13D
-
Upload
softwarecentral -
Category
Documents
-
view
485 -
download
0
Transcript of Paper 13D
![Page 1: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/1.jpg)
The Structured Testing Methodology for Software Quality Analyses of
Networking Systems
Vladimir Riabov, Ph.D.Associate Professor
Department of Mathematics & Computer Science
Rivier College, Nashua, NH
E-mail: [email protected]
56th Northeast Quality Council Conference
Mansfield, Massachusetts, October 17-18, 2006
![Page 2: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/2.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
2
Agenda:• Structured Software Testing Methodology & Graph Theory:
Approach and Tools;• McCabe’s Software Complexity Analysis Techniques;• Results of Code Complexity Analysis for two industrial projects in
Networking;• Study of Networking Protocols Implementation;• Predicting Code Errors;• Test and Code Coverage;• Conclusion: What have the Graph Theory and Structured Testing
Methodology done for us?
Developing Complex Computer Systems“If you don’t know where you’re going, any road will do,” - Chinese Proverb
“If you don’t know where you are, a map won’t help,” - Watts S. Humphrey
“You can’t improve what you can’t measure,” - Tim Lister
![Page 3: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/3.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
3
McCabe’s Structured Testing Methodology Approach and Tools
• McCabe’s Structured Testing Methodology is:- a unique methodology for software testing developed in 1976
[IEEE Transactions on Software Engineering, Vol. SE-2, No. 4, 1976, pp. 308-320];
- based on the Theory of Graphs;- approved as the NIST Standard (1996) in the structured testing;- a leading tool in computer, IT, and aerospace industries (HP, GTE, AT&T, Alcatel, GIG, Boeing, NASA, etc.) since
1977;- provides Code Coverage Capacity.
• Author’s Experience with McCabe IQ Tools since 1998:- leaded three projects in networking industry that required Code
Analysis, Code Coverage, and Test Coverage;- completed BCN Code Analysis with McCabe Tools;- completed BSN Code Analysis with McCabe Tools;- studied BSN-OSPF Code Coverage & Test Coverage.
![Page 4: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/4.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
4
McCabe’s Publication on the Structured Testing Methodology (1976)
NIST Standard on the Structured Testing Methodology (1996)
![Page 5: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/5.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
5
McCabe’s Structured Testing Methodology
• The key requirement of structured testing is that all decision outcomes must be exercised independently during testing.
• The number of tests required for a software module is equal to the cyclomatic complexity of that module.
• The software complexity is measured by metrics: - cyclomatic complexity, v - essential complexity, ev - module design complexity, iv - system design, S0 = iv - system integration complexity, S1 = S0 - N + 1 for N
modules - Halstead metrics, and 52 metrics more.
• The testing methodology allows to identify unreliable-and- unmaintainable code, predict number of code errors and maintenance efforts, develop strategies for unit/module testing, integration testing, and test/code coverage.
![Page 6: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/6.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
6
Basics: Analyzing a Software Module
• For each module (a function or subroutine with a single entry point and a single exit point), an annotated source listing and flowgraph is generated.
• Flowgraph is an architectural diagram of a software module’s logic.
1 main()2 {3 printf(“example”);4 if (y > 10)5 b();6 else7 c();8 printf(“end”);9 }
Statement CodeNumber
main Flowgraph
node:statement or blockof sequential statements
condition
end of condition
edge: flow of controlbetween nodes
1-3
4
5 7
8-9
Battlemap
main
b c printf
![Page 7: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/7.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
7
if (i) ; if (i) ; else ; if (i || j) ;
do ; while (i); while (i) ; switch(i) { case 0: break; ... }
Flowgraph Notation (in C)
if (i && j) ;
![Page 8: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/8.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
8
Flowgraph and Its Annotated Source Listing
Module: marketing
Annotated Source Listing
Program : corp4 09/23/99File : ..\code\corp4.iLanguage: instc_nppModule Module Start Num ofLetter Name v(G) ev(G) iv(G) Line Lines------ ----------------------------------------------------------- ----- ------ B marketing 2 1 2 16 10
16 B0 marketing()17 {18 int purchase;1920 B1* B2 purchase = query("Is this a purchase");21 B3 if ( purchase == 1 )22 B4* B5 development();23 else24 B6* B7 B8 support();25 B9 }
0
1*
2
3
4*
5
6*
7
8
9
Origin information
Node correspondence
Metric information
Decision construct
![Page 9: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/9.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
9
Would you buy a used car from this software?
• Problem: There are sizeand complexity boundariesbeyond which softwarebecomes hopeless– Too error-prone to use– Too complex to fix– Too large to redevelop
• Solution: Control complexityduring development andmaintenance– Stay away from the boundaries.
![Page 10: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/10.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
10
Important Complexity Measures
• Cyclomatic complexity: v = e - n + 2 (here: e = edges; n = nodes) – Amount of decision logic
• Essential complexity: ev– Amount of poorly-structured logic
• Module design complexity: iv– Amount of logic involved with subroutine calls
• System design complexity: S0 = iv – Amount of independent unit (module) tests for a system
• System integration complexity: S1 = S0 - N + 1 – Amount of integration tests for a system of N modules.
![Page 11: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/11.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
11
• Cyclomatic complexity, v - a measure of the decision logic of a software module.– Applies to decision logic embedded within written
code.– Is derived from predicates in decision logic.– Is calculated for each module in the Battlemap.– Grows from 1 to high, finite number based on the
amount of decision logic.– Is correlated to software quality and testing quantity;
units with higher v, v > 10, are less reliable and require high levels of testing.
Cyclomatic Complexity
![Page 12: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/12.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
12
Cyclomatic Complexity 1
4
2
6
7
8
9
11
13
14
15
3 5
10 12
region method regions = 11Beware of crossing lines
R1 R2
R3 R4
R5
R6R7
R8R9
R10
R11
19
23
1
23
45
67
89
10
11
12
13
1415
16
17
18
2021
22
23
24
edges and node methode = 24, n = 15v = 24 - 15 + 2 = 11v = 11
=2
=1
=1
=2
=1
=1
=1
=1predicate methodv = + 1v = 11
(Measure of independent logical decisions in the module)
![Page 13: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/13.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
13
Branching out of a loop Branching in to a loop
Branching into a decision
Branching out of a decision
Essential Complexity - Unstructured Logic
![Page 14: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/14.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
14
Essential Complexity, ev
• Flowgraph and reduced flowgraph after structured constructs have been removed, revealing decisions that are unstructured.
v = 5 Reduced flowgraphv = 3
Therefore ev of the original flowgraph = 3
Superimposedessential flowgraph
![Page 15: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/15.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
15
Essential Complexity, ev
Good designs
Can quicklydeteriorate!
v = 10 ev = 1
v = 11 ev = 10
• Essential complexity helps detect unstructured code.
![Page 16: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/16.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
16
Module Design Complexity, iv
main
progeprogd
iv = 3
Therefore,
iv of the original flowgraph = 3
Reduced Flowgraph
v = 3
proge()
progd()
main v = 5
proge()
progd()
• Example:
main(){
if (a == b) progd();if (m == n) proge();switch(expression){case value_1:
statement1;break;
case value_2:statement2;break;
case value_3:statement3;
}}
do not impact calls
![Page 17: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/17.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
17
Module Metrics Reportv, number of unit test paths for a module
Total number of test paths for all modules
iv, number of integration tests for a module
Average number of testpaths for each module
Page 1 10/01/99 Module Metrics Report
Program: less Module Name Mod # v(G) ev(G) iv(G) File Name------------- ----- ------ ----- ----- ------------------CH:fch_get 118 12 5 6 ..\code\CH.ICH:buffered 117 3 3 1 ..\code\CH.Ich_seek 105 4 4 2 ..\code\CH.Ich_tell 108 1 1 1 ..\code\CH.Ich_forw_get 106 4 1 2 ..\code\CH.Ich_back_get 110 6 5 5 ..\code\CH.Iforw_line 101 11 7 9 ..\code\INPUT.Iback_line 86 12 11 12 ..\code\INPUT.Iprewind 107 1 1 1 ..\code\LINE.Ipappend 109 36 26 3 ..\code\LINE.Icontrol_char 119 2 1 1 ..\code\OUTPUT.Icarat_char 120 2 1 1 ..\code\OUTPUT.Iflush 130 1 1 1 ..\code\OUTPUT.Iputc 122 2 1 2 ..\code\OUTPUT.Iputs 100 2 1 2 ..\code\OUTPUT.Ierror 83 5 1 2 ..\code\OUTPUT.Iposition 114 3 1 1 ..\code\POSITION.Iadd_forw_pos 99 2 1 1 ..\code\POSITION.Ipos_clear 98 2 1 1 ..\code\POSITION.IPRIM:eof_bell 104 2 1 2 ..\code\PRIM.IPRIM:forw 95 15 8 12 ..\code\PRIM.IPRIM:prepaint 94 1 1 1 ..\code\PRIM.Irepaint 93 1 1 1 ..\code\PRIM.Ihome 97 1 1 1 ..\code\SCREEN.Ilower_left 127 1 1 1 ..\code\SCREEN.Ibell 116 2 1 2 ..\code\SCREEN.Ivbell 121 2 1 2 ..\code\SCREEN.Iclear 96 1 1 1 ..\code\SCREEN.Iclear_eol 128 1 1 1 ..\code\SCREEN.Iso_enter 89 1 1 1 ..\code\SCREEN.Iso_exit 90 1 1 1 ..\code\SCREEN.Igetc 91 2 1 2 ..\code\TTYIN.I------------- ----- ------ ----- ----- ------------------Total: 142 93 82Average: 4.44 2.91 2.56Rows in Report: 32
![Page 18: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/18.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
18
Low Complexity Software• Reliable
– Simple logic• Low cyclomatic complexity (v < 10)
– Not error-prone– Easy to test
• Maintainable– Good structure
• Low essential complexity (ev < 4)
– Easy to understand– Easy to modify
![Page 19: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/19.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
19
Moderately Complex Software
• Unreliable– Complicated logic
• High cyclomatic complexity (v >> 10)
– Error-prone– Hard to test
• Maintainable– Can be understood– Can be modified– Can be improved
![Page 20: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/20.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
20
Highly Complex Software• Unreliable
– Error prone– Very hard to test
• Unmaintainable– Poor structure
• High essential complexity (ev >> 10)
– Hard to understand– Hard to modify– Hard to improve
![Page 21: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/21.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
21
McCabe QA
McCabe QA measures software quality with industry-standard metrics
– Manage technical risk factors as software is developed and changed
– Improve software quality using detailed reports and visualization
– Shorten the timebetween releases
– Develop contingency plans to address unavoidable risks
![Page 22: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/22.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
22
Processing with McCabe QA Tools
BUILDLevel
TESTLevel
ANALYSISLevel
Preprocess Compile& Link
Run& Test
CM
ClearCase
PARSE
src files
src
*.E
Battlemap
Flowgraphs
Reports
Text Graphics
Test Plan
Instrumented srcinst-src; inst-lib.c
inst-src
Inst-lib.c
inst.exe
Output
IMPORT
Trace File
CoverageAnalysis
CoverageReport
Project CodeTraditional Procedures
New McCabe’s Procedures
![Page 23: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/23.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
23
Project B: Backbone™ Concentration Node
![Page 24: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/24.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
24
Project B: Backbone Concentration Node
• This system has been designed to support carrier networks. It provides both services of conventional Layer 2 switches and the routing and control services of Layer 3 devices.
• Nine protocol-based sub-trees of the code (3400 modules written in the C-programming language for BGP, DVMRP, Frame Relay, ISIS, IP, MOSPF, OSPF2, PIM, and PPP protocols) have been analyzed.
![Page 25: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/25.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
25
Annotated Source Listing for the OSPF-module Flowgraph for the OSPF-module
![Page 26: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/26.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
26
Cyclomatic Test Paths for the OSPF-module 1st Test Flowgraph for the OSPF-module
![Page 27: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/27.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
27
Module Metrics for the OSPF Protocol Suite Halstead Metrics for the OSPF Protocol Suite
![Page 28: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/28.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
28
Example 1: Reliable and Maintainable Module Example 2: Unreliable Module that difficult to maintain
![Page 29: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/29.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
29
Example 3: Absolutely Unreliable and Unmaintainable Module
Summary of Modules’ Reliability and Maintainability
![Page 30: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/30.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
30
Project-B Protocol-Based Code Analysis
• Unreliable modules: 38% of the code modules have the Cyclomatic Complexity more than 10 (including 592 functions with v > 20);
• Only two code parts (FR, ISIS) are reliable;• BGP and PIM have the worst characteristics (49% of
the code modules have v > 10);• 1147 modules (34%) are unreliable and
unmaintainable with v > 10 and ev > 4;• BGP, DVMRP, and MOSPF are the most unreliable
and unmaintainable (42% modules);• The Project-B was cancelled.
![Page 31: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/31.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
31
Project-B Code Protocol-Based Analysis (continue)
• 1066 functions (31%) have the Module Design Complexity more than 5. The System Integration Complexity is 16026, which is a top estimation of the number of integration tests;
• Only FR, ISIS, IP, and PPP modules require 4 integration tests per module. BGP, MOSPF, and PIM have the worst characteristics (42% of the code modules require more than 7 integration tests per module);
• B-2.0.0.0int18 Release potentially contains 2920 errors estimated by the Halstead approach. FR, ISIS, and IP have relatively low (significantly less than average level of 0.86 error per module) B-error metrics. For BGP, DVMRP, MOSPF, and PIM, the error level is the highest one (more than one error per module).
![Page 32: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/32.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
32
Comparing Project-B Core Code Releases
![Page 33: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/33.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
33
Comparing Project-B Core Code Releases
• NEW B-1.3 Release (262 modules) vs. OLD B-1.2 Release (271 modules);• 16 modules were deleted (7 with v >10);• 7 new modules were added (all modules are reliable with v < 10, ev = 1);• Sixty percent of changes have been made in the code modules with
the parameters of the Cyclomatic Complexity metric more than 20.• 63 modules are still unreliable and unmainaitable;• 39 out of 70 (56%) modules with v >10 were targeted for changing
and remained unreliable;• 7 out of 12 (58%) modules have increased their complexity v > 10;• Significant reduction achieved in System Design (S0) and System
Integration Metrics (S1):S1 from 1126 to 1033; S0 from 1396 to 1294.
• New Release potentially contains less errors: 187 errors (vs. 206 errors) estimated by the Halstead approach.
• The Project-B was cancelled.
![Page 34: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/34.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
34
Project C: Broadband Service Node
• Broadband Service Node (BSN) allows service providers to aggregate tens of thousands of subscribers onto one platform and apply customized IP services to these subscribers;
• Different networking services [IP-VPNs, Firewalls, Network Address Translations (NAT), IP Quality-of-Service (QoS), Web steering, and others] are provided.
![Page 35: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/35.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
35
Project-C Code Subtrees-Based Analysis
• THREE branches of the Project-C code (Release 2.5int21) have been analyzed, namely RMC, CT3, and PSP subtrees (23,136 modules);
• 26% of the code modules have the Cyclomatic Complexity more than 10 (including 2,634 functions with v > 20); - unreliable modules!
• All three code parts are approximately at the same level of complexity (average per module: v = 9.9; ev = 3.89; iv = 5.53).
• 1.167 Million lines of code have been studied (50 lines average per module);
• 3,852 modules (17%) are unreliable and unmaintainable with v > 10 and ev > 4;
• Estimated number of possible ERRORS is 11,460;• 128,013 unit tests and 104,880 module integration tests should be
developed to cover all modules of the Project-C code.
![Page 36: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/36.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
36
Project-C Protocol-Based Code Analysis
• NINE protocol-based areas of the code (2,141 modules) have been analyzed, namely BGP, FR, IGMP, IP, ISIS, OSPF, PPP, RIP, and SNMP.
• 130,000 lines of code have been studied.• 28% of the code modules have the Cyclomatic Complexity more than
10 (including 272 functions with v > 20); - unreliable modules!• FR & SNMP parts are well designed & programmed with few possible
errors. • 39% of the BGP and PPP code areas are unreliable (v > 10).• 416 modules (19.4%) are unreliable & unmaintainable (v >10 & ev >4).• 27.4% of the BGP and IP code areas are unreliable & unmaintainable.• Estimated number of possible ERRORS is 1,272;• 12,693 unit tests and 10,561 module integration tests should be
developed to cover NINE protocol-based areas of the Project-C code.
![Page 37: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/37.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
37
Correlation between the Number of Error Submits, the Number of Unreliable Functions (v > 10), and the Number of Possible Errors for Six Protocols
0
100
200
300
400BGP
FR
IP
ISIS
OSPF
RIP Submits
UnreliableFunctions
Possible Errors
![Page 38: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/38.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
38
Correlation between the Number of Customer Reports, the Number of Unreliable Functions (v > 10), and the
Number of Possible Errors for Five Protocols
0
100
200
300
400BGP
FR
ISISOSPF
RIPCustomer Reports
UnreliableFunctions
Possible Errors
![Page 39: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/39.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
39
Project-C: Code Coverage
![Page 40: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/40.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
40
Project-C: Test Coverage
![Page 41: Paper 13D](https://reader033.fdocuments.us/reader033/viewer/2022052321/55502937b4c9058f2f8b497e/html5/thumbnails/41.jpg)
56th NEQC Conference, 2006
Complexity Metrics for Networking Software Studies
41
The Structured Testing Methodology (based on the Theory of Graphs) has done for us:
• Identified complex code areas (high v).• Identified unreliable & unmaintainable code (v >10 & ev >4).• Predicted number of code errors and maintenance efforts [Halstead B,
E-, and T-metrics].• Estimated manpower to develop, test, and maintain the code.• Developed strategies for unit/module testing, integration testing.• Provided Test & Code Coverage [paths vs. lines].• Identified “dead” code areas.• Improved Software Design and Coding Standards.• Improved Reengineering Efforts in many other projects.• Validated Automated Test Effectiveness.