Evaluating Off-The-Shelf Architectural Components
description
Transcript of Evaluating Off-The-Shelf Architectural Components
Evaluating Off-The-Shelf Architectural Components
Kendra Cooper Lawrence Chung Weimin Ma
[email protected] [email protected] [email protected]
The Component-Aware Technology (CAT) Lab.
Department of Computer Science
The University of Texas at Dallas
What do I need ???
What is the type of HACS Controller?
What is the type of AC Controller?
An Example:Home Appliance Control System (HACS)
Evaluating Home Appliance Control System (HACS) Components
How to effectively evaluate components, simple and composite?
UML Class Diagram as Model of Software Components
Figure 1: Results for matching and ranking OTS components
Architectural model for each component set/vendor is represented in UML class diagram.
Figure 3: UML class diagram model of component set 4
Figure 2: UML class diagram model of component set 1
Outline
A Scenario for Using Software Components
Conclusions
Similarity Measure
Criteria-Based User Query
Evaluation of Components
The Notion of Distance
The Notion of Criteria
Case-based scheme Evaluation of Simple Components
(E.g., Evaluate AC Controller; Evaluate HACS Controller)
Evaluation of Relationships
Evaluation of Composite Components
Analytic hierarchy process scheme
User
Evaluator
1.2. Is there an AC Controller with feature “Set Temp”?
3. Searching …
4. Component set 3>>4>{1, 2}
5. Is there a HACS Controller with feature “Check Status”?
6. Searching …
7. Component set 3>>4>>1>>2
1.1. System controller
11. A possible ranking: Component set 3>>{4, 2}>>1
8. Do HACS Controller and AC Controller fit together well?
9. Component set {2, 3}>>{1, 4}
10. Aggregate the three evaluations.
A Scenario for Using Software Components
Component Sets/Vendors
1,2
3,4
>>: better match
Distance is 1.Distance is 1.
Distance is 1.
Similarity Measure: The Notion of Distance
Distance is 1.
Distance is 2.?
Criterion a: number of figures.Criterion b: number & shape of figures.
Criteria are necessary for software components evaluation!
Simple Component: Composite Component:
Distance is 1.
Distance is 2.Distance is 3.?
Type of relationship & number of extra intermediaries
Criterion a: number of figures.Criterion b: number & shape of figures.
No context consideration.
Distance is 0. ?
Similarity Measure: The Notion of Criteria
Criteria-Based User Query
Unstructured user query:
Using the criteria and through communication, the evaluator can turn it into criteria-based structure (à la Case-Based Reasoning):
(Class Name) AC Controller(Attribute) Temperature(Operation) setTemperature
(Class Name) HACS Controller(Attribute) Status(Operation) checkStatus
I want a HACS system whose
HACS controller has feature “Check Status”
and AC controller has feature “Set Temp”.
(Association)
Sub-query 1: Sub-query 2:
Sub-query 1:(Class Name) AC Controller(Attribute) Temperature(Operation) setTemperature
Component Set 1:(Class Name) AC Conditioner(Attribute) temp(Operation)
Component Set 2:(Class Name) AC Conditioner(Attribute) temp(Operation)
Component Set 3:(Class Name) AC Controller(Attribute) temperature(Operation) settemp
Component Set 4:(Class Name) Temperature Controller(Attribute) temperature(Operation) setTemperature
S_m: 1 - (½ + 1 + 0)/3 = 0.5.
S_m: 1- (½ + 1 + 0)/3 = 0.5.
S_m: 1 - (1 + 1 + 1)/3 = 0.
S_m: 1 – (½ + 1 + 1)/3 = 0.167.
Evaluation of Components for “AC Controller” Based on Case-based Scheme
Figure 2: UML class diagram model of component set 1Criteria
Sim_component_set (“AC Controller”, component set 1)
"[
1i
i in feature, AC Controller, component set 1i
priority
X %_m(i)] priority
i in feature, AC Controller, component set 1
Sub-query 2:(Class Name) HACS Controller(Attribute) Status(Operation) checkStatus
Component Set 1:(Class Name) System Control(Attribute) (Operation) getHistory
Component Set 2:(Class Name) System Control(Attribute) (Operation)
Component Set 3:(Class Name) HACS Interaction Device(Attribute)(Operation) checkstatus
Component Set 4:(Class Name) HACS Controller(Attribute) (Operation)
S_m: 1 – (½ + 0 + ¼)/3 = 0.75.
S_m: 1 – (½ + 0 + 0)/3 = 0.833.
S_m: 1 – (½ + 0 + 1)/3 = 0.5.
S_m: 1 – (1 + 0 + 0)/3 = 0.667.
Evaluation of Components for “HACS Controller” Based on Case-based Scheme
Figure 2: UML class diagram model of component set 1
Criteria
Sim_component_set (“AC Controller”, component set 1)
"[
1i
i in feature, AC Controller, component set 1i
priority
X %_m(i)] priority
i in feature, AC Controller, component set 1
Sub-query 1 and Sub-query 2
Component Set 1:(# of Relationships) 2 (Relationship Type) {1 Association, 1 generalization}
Component Set 2:(# of Relationships) 1(Relationship Type) {1 association}
Component Set 3:(# of Relationships) 1(Relationship Type) {1 association}
Component Set 4:(# of Relationships) 2(Relationship Type) {1 association, 1 generalization}
Distance: 2
Distance: 0 Distance: 0
Distance: 2
Evaluation of Relationship between Components
Criteria
Figure 3: UML class diagram model of component set 4
Figure 2: UML class diagram model of component set 1
Similarity (Component Set 1):0.75
Similarity (AC Controller)0.5 Similarity (relationship)
1
Similarity (HACS Controller)0.75
Evaluation of Composite Components
Ranking of component sets for user’s query: 3>>{4, 2, 1}
? 3>>{4, 2}>>1
Ranking of “AC Controller”: 3 >> 4>> {1, 2}Ranking of “HACS Controller”: 3>> 4>> 1>> 2Ranking of relationship between components: {2, 3} >> {4, 1}
(Sim(AC Controller) + Sim(Relationship between AC-HACS) + Sim(HACS Controller))/
(#component+#relationships),
where Sim(Relationship between AC-HACS) = (#extra-relationships+#relationship-mismatch)/
(#relationship-in-query+#possible-relationship-attributes)
SetComponentSim )1__(
....
Component Sets/Vendors = {1, 2, 3, 4}
Analytic Hierarchy Process (AHP) EvaluationComponent Evaluation
Class Class Relationship
Class nameAttribute
NameOperation
Name
Component Set 1
Component Set 2
Component Set 3
Component Set 4
Class Name Attribute Name
Operation Name
Class Relationship
Priorities
Class Name 1 2 4 4/3 0.4
Attribute Name
1/2 1 2 2/3 0.2
Operation Name
1/4 1/2 1 1/3 0.1
Class Relationship
3/4 3/2 3 1 0.3
Example analytic hierarchy of criteria
Example relative priorities of criteria
Advantage of AHP:Assist in identifying and weighting selection criteriaRelative priority or weighting for alternatives
Pairwise Comparison of Criteria
Class Name
Com_Set_1 Com_Set_2 Com_Set_3 Com_Set_4 Priorities
Com_Set_1 1 1 2/3 2/3 0.2Com_Set_2 1 1 2/3 2/3 0.2
Com_Set_3 3/2 3/2 1 1 0.3
Com_Set_4 3/2 3/2 1 1 0.3
For Class Name, Attribute Name, and Operation Name,
))1__,(__,(()1__,( SetComACsetcomponentSimCriterionMeasureSetComCriterionMeasure
.5.02
5.05.0
Attribute Name
Com_Set_1 Com_Set_2 Com_Set_3 Com_Set_4 Priorities
Com_Set_1 1 1 1 1 0.25Com_Set_2 1 1 1 1 0.25
Com_Set_3 1 1 1 1 0.25
Com_Set_4 1 1 1 1 0.25
Operation Name
Com_Set_1 Com_Set_2 Com_Set_3 Com_Set_4 Priorities
Com_Set_1 1 X 1/8 ¼ 0.08Com_Set_2 X X X X 0.00
Com_Set_3 8 X 1 2 0.62
Com_Set_4 4 X 1/2 1 0.30
Class Relationship
Com_Set_1 Com_Set_2 Com_Set_3 Com_Set_4 Priorities
Com_Set_1 X X X X 0.0Com_Set_2 X 1 1 X 0.5
Com_Set_3 X 1 1 X 0.5
Com_Set_4 X X X X 0.0
,0.0)2__,_( SetComNameOperationMeasureSo Com_Set_2’s priority is zero and not involved in pairwise comparison.
,0.0)1__,Re_( SetComlationshipClassMeasure
So Com_Set_1 and Com_Set_4’s priorities are zero and not involved in pairwise comparison.
2/)))1__,(__,( SetComHACSsetcomponentSimCriterionMeasure
For Class relationship,
)1__,Re_( SetComlationshipClassMeasure
)1__(}))__(Re({ SetCominiprelationshSimiSetCominlationshipSimMax
.022
,0.0)4__,Re_( SetComlationshipClassMeasure
Tree Representation of AHP Process
Class Name(0.4)
CS1 0.2CS2 0.2CS3 0.3CS4 0.3
Attribute Name(0.2)
CS1 0.25CS2 0.25CS3 0.25CS4 0.25
Operation Name(0.1)
CS1 0.08CS2 0.00CS3 0.62CS4 0.30
Class
Class Name(0.4)
Attribute Name(0.2)
Operation Name(0.1)
CS1 0.2 0.25 0.08CS2 0.2 0.25 0.00CS3 0.3 0.25 0.62CS4 0.3 0.25 0.30
Class Relationship(0.3)
CS1 0.0CS2 0.5CS3 0.5CS4 0.0
Component Evaluation
Class Name(0.4)
Attribute Name(0.2)
Operation Name(0.1)
Class Relationship
(0.3)
Priorities
CS1 0.2 0.25 0.08 0.0 0.138CS2 0.2 0.25 0.00 0.5 0.280CS3 0.3 0.25 0.62 0.5 0.382CS4 0.3 0.25 0.30 0.0 0.200
Ranking of component sets for user’s query: 3>>2>>4>>1 ?
Conclusions
Hierarchy of criteria for component evaluation
Cased-based scheme for similarity-based evaluation
Contributions
Application to a suite of UML diagrams
Future Work
Technique for evaluating architectural components using UML class diagrams
Refinements of distance and criteria (context information)
AHP scheme evaluation