Uncertainty Management With Partial Models
-
Upload
michalis-famelis -
Category
Documents
-
view
426 -
download
3
description
Transcript of Uncertainty Management With Partial Models
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Uncertainty Management With Partial Models
Michalis Famelis
University of Toronto
October 2nd, 2012,
MODELS’12 Doctoral Symposium
1 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Hi, I’m Michalis Famelis
• 3rd year PhD student.
• ...“a couple years to finish”
• Supervisor: Marsha Chechik
• MSc: Model Management with RelationTypes, CS UofT 2010.
• Undergrad: Model analysis andtransformation. ECE NTUA 2008.
2 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Uncertainty in SE
“The reality of today’s software systems requires us to consider
uncertainty as a first-class concern in the design, implementation,
and deployment of those systems.” [Garlan, 2010]
• Self-adaptive systems[Esfahani et al., 2011, Cheng and Garlan, 2007]
• Probabilistic systems [Hinton et al., 2006]
• Imperfect information in requirements[Noppen et al., 2007]
• Requirements clarifications [Knauss et al., 2012]
• Risk management [Islam and Houmb, 2010]
3 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Design Uncertainty
Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
4 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
What is Design UncertaintyExample: a simple class diagram.
What does the modeler know?
5 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
What is Design UncertaintyExample: a simple class diagram.
What does the modeler not know?
5 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
What is Design UncertaintyExample: a simple class diagram.
What does the modeler not know?
5 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
What is Design UncertaintyExample: a simple class diagram.
What does the modeler not know?
5 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
What is Design UncertaintyExample: a simple class diagram.
What does the modeler not know?
5 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Dealing with Uncertainty: Options
Classical:
1 Wait until uncertainty is lifted.Inefficient; under-utilization of resources.
2 Plan for all possible solutions.Intractable: they may be too many.
3 Make a decision. Now.Artificial; risk undo or premature commitments.
We propose:
4 “Live” with uncertainty, defer resolution until moreinformation becomes available.
6 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Design Uncertainty
Uncertainty pervasive in MDSE.
But also:Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
7 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Formalisms to RepresentUncertainty
ModalBehavioralSpecification
e.g.:MTSs [Larsen’88]
DMTSs [Larsen’90]
• Restricted to one model type only.
• Only optional elements.
• Over-approximation of set:Information loss
VariabilityModeling
e.g.:FTSs [Classen’10]TVL [Boucher’10]
Clafer [Bak’10]
• Different conceptual domain (SPLs).
• Not rich enough. Eg:– No variable meta-attributes.
(e.g. variable container class)
– No open world option.
• More heavyweight (maintain an SPLfor ephemeral uncertainty?)
8 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Uncertainty: a Set of PossibleRefinements
If we remove all uncertainty, we have a concrete refinement.
9 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• Element is optional.
• Element can be multiplied to many copies.
• Element can be merged with others.
• Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• May: Element is optional.
• Element can be multiplied to many copies.
• Element can be merged with others.
• Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• May: Element is optional.
• Abs: Element can be multiplied to many copies.
• Element can be merged with others.
• Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• May: Element is optional.
• Abs: Element can be multiplied to many copies.
• Var: Element can be merged with others.
• Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• May: Element is optional.
• Abs: Element can be multiplied to many copies.
• Var: Element can be merged with others.
• OW: Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Modeling Uncertainty with PartialModels
Explicating uncertainty in a partial model.
• May: Element is optional.
• Abs: Element can be multiplied to many copies.
• Var: Element can be merged with others.
• OW: Model is incomplete.
10 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
“Extended“ Partial Models
Additional constraints to correlate points of uncertainty.e.g. May Model: variant presented in [ICSE’12]:
11 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Semantics of Partial Models
12 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Semantics of Partial Models
12 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Semantics of Partial Models
12 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Compared to Related Formalisms
Partial models vs Modal behavioral specification formalisms
• Language/metamodel independent.
• Many kinds of partiality.
• Exact representation of set.
Partial models vs Variability modeling formalisms
• Specific to uncertainty.(c.f. uncertainty-removing refinement).
• Rich formalism.
• Ultimately disposable.
13 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Status
Uncertainty pervasive in MDSE.
But also:Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
14 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Completed Work I
Explicating uncertainty
• May models (M-elements + May formula) [ICSE’12]
• MAVO models + FOL semantics [FASE’12]
• Relational-algebraic encoding of MAVO [MoDeVVa’12]
Checking properties
• Verifying May models with SAT [ICSE’12]
• Verifying MAVO models with Alloy [FASE’12]
• Verifying MAV- models with Alloy, CSP, SMT and ASP[MoDeVVa’12]
15 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Completed Work II
Uncertainty-removing refinement
• Semantics and verification [FASE’12]
• Property-driven refinement [ICSE’12]
Tooling
• Python and MathSAT [ICSE’12]
• Alloy [FASE’12]
• Translations from annotated Ecore to RA andfrom RA to SMT, CSP, ASP, Alloy [MoDeVVa’12]
16 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Status
Uncertainty pervasive in MDSE.
But also:Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
17 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Lifting Transformations [MiSE’12]
May models + Graph transformations
Q1: How do we transform M directly to N?
– Lifted semantics of transformations, using logic.(“transfer predicates”)
Q2: Are the concretizations of N exactly the models n1 . . . nk?
– Check equivalence of encodings using SAT.(“correctness criterion”)
18 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Transformations: Next Steps
• Compositionally test Correctness Criterion.– Reuse Alloy-based verification method from [VOLT’12]?
• Systematically create Transfer Predicates using FOL.– Predicates for atomic operations as building blocks.– May be hard to simplify resulting formulas.
• Handle expanding/contracting model vocabularies.– E.g. adding a new element vs. concretizing.
• Reuse results from Category Theory? [Ehrig et al., 2006]– Generalize to full MAVO.– What are the right morphisms?
19 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Tool support
Full MAVO support for Ecore-based model editors.
• As an EMF Profile? [Langer et al., 2011]
• Via RAMification of metamodels? [Kuhne et al., 2009]– Would solve issues with concrete syntax
(e.g. when the base model is not well-formed)
Model management support
• Integration with the Model Management Tool Framework.[Salay et al., 2007]
• Inherit macro-management tools for partial models– name-based matching– structural merging– etc...
• “Push-button” integration of reasoning.
20 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Status
Uncertainty pervasive in MDSE.
But also:Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
21 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Guidelines for EncodingUncertainty
“Placement” “Alternatives”
“Bottom-up” patterns of uncertainty.
• Help guide efficient/correct encoding.
• Support for choice of suitable reasoners.
22 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Towards a Methodology for DesignUncertainty
We know how to do reasoning, refinement, transformationsunder uncertainty.
We need to help identify what sorts of reasoning, refinement,etc. are necessary in each SE setting.
“Top-down” patterns of uncertainty:
• How does it appear in SE?
• What are its sources?
• What are its characteristic properties?
• What are refinements/transformations are sensible?
23 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
A Half Baked Example:“Disagreement Pattern”
Potential sources:
• Conflicting stakeholder requirements.
• Merge conflicts after commits.
• View-updates.
Characteristic properties:
• Is the conflict reconciliable?
• Have we reached agreement?
Sensible refinement and transformation checks:
• Does a refactoring expose/conceal the source ofdisagreement?
• Is a series of refinements a negotiation?(“are they making progress removing uncertainty?”)
24 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Outline
“....enable MDSE with design uncertainty:”
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
How do we go about evaluating all this?
25 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Evaluation of Individual Aspects
Experiments using randomly generated inputs
• To validate feasibility and Scalability of Property Checking[ICSE’12][MoDeVVa’12]
• Experimental driver in MMTF for rapid experiment setups.
Case Studies
• Triangulated results of random experimentation in [ICSE’12].
• Realistic bugfix of real world bug in UMLet
Theorem Proving using Alloy
• Used to verify partial model refinements in [VOLT’12]
• Technique could be adapted for proving transformations
26 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Evaluation of the Entire Approach
Plan for a Power Window Case Study
• Comprehensive case study from the Automotive domain.
• Management of uncertainty throughout the MDE lifecycle.Requirements to implementation via transformations.
However:
• Is such a case study enough?
• Best method for evaluation not clear.
• How to evaluate “green-field” research?
27 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Summary
Uncertainty pervasive in MDSE.
Not well studied:Uncertainty about design decisions – the contents of a model.
Ultimately, we need to capture and support design uncertainty.
Enable MDSE with design uncertainty:
• Create representations
• Check properties and give feedback
• Do uncertainty-removing refinement
• Enable transformations
• Create tooling support
• Create methodological support
28 / 33
Questions?
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Bibliography I
Bak, K., Czarnecki, K., and Wasowski, A. (2011).
Feature and meta-models in clafer: Mixed, specialized, and coupled.In SLE’10, pages 102–122.
Boucher, Q., Classen, A., Faber, P., and Heymans, P. (2010).
Introducing TVL, a text-based feature modelling language.In VaMoS’10, pages 159–162.
Cheng, S.-W. and Garlan, D. (2007).
Handling uncertainty in autonomic systems.In ASE’07.
Classen, A., Heymans, P., Schobbens, P.-Y., Legay, A., and Raskin, J.-F. (2010).
Model checking lots of systems: efficient verification of temporal properties in software product lines.In ICSE’10, pages 335–344.
Ehrig, H., Ehrig, K., Prange, U., and Taentzer, G. (2006).
Fundamentals of Algebraic Graph Transformation.EATCS. Springer.
Esfahani, N., Kouroshfar, E., and Malek, S. (2011).
Taming uncertainty in self-adaptive software.In ESEC/FSE ’11, pages 234–244.
Famelis, M., Ben-David, S., Chechik, M., and Salay, R. (2011).
“Partial Models: A Position Paper”.In Proceedings of MoDeVVa’11, pages 1–6.
30 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Bibliography II
Famelis, M., Chechik, M., and Salay, R. (2012a).
“Partial Models: Towards Modeling and Reasoning with Uncertainty”.In Proceedings of ICSE’12.
Famelis, M., Salay, R., and Chechik, M. (2012b).
The semantics of partial model transformations.In MISE at ICSE’12, pages 64 –69.
Garlan, D. (2010).
Software engineering in an uncertain world.In FoSER ’10, pages 125–128.
Hinton, A., Kwiatkowska, M., Norman, G., and Parker, D. (2006).
Prism: A tool for automatic verification of probabilistic systems.In Hermanns, H. and Palsberg, J., editors, Tools and Algorithms for the Construction and Analysis ofSystems, volume 3920 of Lecture Notes in Computer Science, pages 441–444.
Islam, S. and Houmb, S. H. (2010).
Integrating risk management activities into requirements engineering.In RCIS’10, pages 299–310.
Knauss, E., Damian, D., Poo-Caamano, G., and Cleland-Huang, J. (2012).
Detecting and classifying patterns of requirements clarifications.In RE’12.
Kuhne, T., Mezei, G., Syriani, E., Vangheluwe, H., and Wimmer, M. (2009).
Explicit transformation modeling.In MODELS’09, pages 240–255.
31 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Bibliography III
Langer, P., Wieland, K., Wimmer, M., and Cabot, J. (2011).
From uml profiles to emf profiles and beyond.In Objects, Models, Components, Patterns, volume 6705, pages 52–67.
Larsen, K. G. and Thomsen, B. (1988).
“A Modal Process Logic”.In Proceedings of LICS’88, pages 203–210.
Larsen, K. G. and Xinxin, L. (1990).
“Equation Solving Using Modal Transition Systems”.In Proc. of LICS’90, pages 108–117.
Noppen, J., van den Broek, P., and Akşit, M. (2007).
Software development with imperfect information.Soft Comput., 12(1):3–28.
Saadatpanah, P., Famelis, M., Gorzny, J., Robinson, N., Chechik, M., and Salay, R. (2012).
Comparing the effectiveness of reasoning formalisms for partial models.In MoDeVVa’12.
Salay, R., Chechik, M., Easterbrook, S., Diskin, Z., McCormick, P., Nejati, S., Sabetzadeh, M., and
Viriyakattiyaporn, P. (2007).An eclipse-based tool framework for software model management.In Proceedings of the 2007 OOPSLA workshop on eclipse technology eXchange, eclipse ’07, pages55–59.
Salay, R., Chechik, M., and Gorzny, J. (2012a).
“Towards a Methodology for Verifying Partial Model Refinements”.In Proceedings of VOLT’12.
32 / 33
UncertaintyManagementWith Partial
Models
M. Famelis
Introduction
ModelingUncertainty
ManagingUncertainty
Completed Work
Current Work
Future Work
Evalutation
Conclusion
Bibliography IV
Salay, R., Famelis, M., and Chechik, M. (2012b).
“Language Independent Refinement using Partial Modeling”.In Proceedings of FASE’12.
33 / 33