Partial Models: Towards Modeling and Reasoning with Uncertainty

71
Partial Models: Towards Modeling and Reasoning with Uncertainty M.Famelis , R.Salay, M.Chechik, Introduction Intuition Motivating Example Modeling Uncertainty Partial Models Semantics Reasoning With Uncertainty Property Checking Diagnosis Evaluation Experiments Case Study Conclusion Partial Models: Towards Modeling and Reasoning with Uncertainty Michalis Famelis , Rick Salay, and Marsha Chechik University of Toronto June 7, 2012 ICSE’12, Zurich, Switzerland 1 / 29

Transcript of Partial Models: Towards Modeling and Reasoning with Uncertainty

Page 1: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models: Towards Modeling andReasoning with Uncertainty

Michalis Famelis, Rick Salay, and Marsha Chechik

University of Toronto

June 7, 2012ICSE’12, Zurich, Switzerland

1 / 29

Page 2: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 3: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 4: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 5: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 6: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 7: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 8: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 9: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 10: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Intuition: Sudoku

Created with GNOME Sudoku 2.32.0.

2 / 29

Page 11: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Enough About Sudoku

Source: Wikimedia,

3 / 29

Page 12: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Goal: Uncertainty in Software

4 / 29

Modeling

Explicate points of uncertaintyCorrelate points of uncertainty

Reasoning

Check propertiesGive feedback to facilitate diagnosis

Page 13: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Designing a P2P ApplicationTrying to design a P2P client application.

What do I know?

5 / 29

Page 14: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Designing a P2P ApplicationTrying to design a P2P client application.

What do I not know?

5 / 29

Page 15: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Designing a P2P ApplicationTrying to design a P2P client application.

What do I not know?

5 / 29

Page 16: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Designing a P2P ApplicationTrying to design a P2P client application.

How can I explicate my uncertainty and reason in its presence?

5 / 29

Page 17: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Contribution

Modeling Uncertainty

• Encode uncertainty in Partial Models.

• Semantics: sets of conventional models.

Reasoning in the Presence of Uncertainty

• Check properties.

• Give feedback to facilitate diagnosis.

Evaluation of Reasoning

• Reasoning with Partial models vs. reasoning with a set ofconventional models

6 / 29

Page 18: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

In Paper / Not In Talk

Presentation of these would take too much time:

• Encoding conventional models in logic and back

• Construction algorithm of Partial Models

• Propositional Normal Form (PNF)

• Graphical Normal Form (GNF)

• Diagnostic cores

• “Property-driven” refinement.

• Translation from PNF to GNF and vice versa

• Evaluation of diagnostic cores and property-drivenrefinement

• Random generation of experimental inputs

7 / 29

Page 19: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Contribution

Modeling Uncertainty

• Encode uncertainty in Partial Models.

• Semantics: sets of conventional models.

Reasoning in the Presence of Uncertainty

• Check properties.

• Give feedback to facilitate diagnosis.

Evaluation of Reasoning

• Reasoning with Partial models vs. reasoning with a set ofconventional models

8 / 29

Page 20: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 21: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 22: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 23: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 24: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 25: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Partial Models

9 / 29

Page 26: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Semantics of Partial Models

10 / 29

Page 27: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Semantics of Partial Models

10 / 29

Page 28: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Semantics of Partial Models

10 / 29

Page 29: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Related IdeasBehavioral modeling:

• Modal Transition Systems (MTSs) [Larsen’88].

• Disjunctive MTSs [Larsen’91].

Software Product Lines:

• Variability in the metamodel [Morin’09].

• Featured Transition Systems [Classen’10].

Partial Models:• Language-independent

not just behavioral models!

• May formula: exact encodingthorough reasoning

• Focus on systematic management of uncertaintyuncertainty-reducing refinement [VOLT’12]

transformations [MiSE’12]

11 / 29

Page 30: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Contribution

Modeling Uncertainty

• Encode uncertainty in Partial Models.

• Semantics: sets of conventional models.

Reasoning in the Presence of Uncertainty

• Check properties.

• Give feedback to facilitate diagnosis.

Evaluation of Reasoning

• Reasoning with Partial models vs. reasoning with a set ofconventional models

12 / 29

Page 31: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

1) Property Checking

Property can be:

• True: holds for all concretizations

• False: holds for none

• Maybe: true for some, false for others

To check a property:

- Encode model and property in propositional logic.

- Use SAT solver.

ΦM ∧ Φp ΦM ∧ ¬Φp Property pSAT SAT MaybeSAT UNSAT True

UNSAT SAT FalseUNSAT UNSAT (model inconsistent)

13 / 29

Page 32: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 33: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 34: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 35: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 36: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 37: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 38: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 39: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 40: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 41: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 42: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 43: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 44: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 45: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 46: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Property Checking: Example

14 / 29

Page 47: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

2) Diagnosis

Feedback:

A concretization of the Partial Model for which theproperty does not hold.

Reuse the results of property checking:

ΦM ∧ Φp ΦM ∧ ¬Φp Property pSAT SAT MaybeSAT UNSAT True

UNSAT SAT FalseUNSAT UNSAT (model inconsistent)

15 / 29

Page 48: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Diagnosis: Example

16 / 29

Page 49: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Diagnosis: Example

16 / 29

Page 50: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Contribution

Modeling Uncertainty

• Encode uncertainty in Partial Models.

• Semantics: sets of conventional models.

Reasoning in the Presence of Uncertainty

• Check properties.

• Give feedback to facilitate diagnosis.

Evaluation of Reasoning

• Reasoning with Partial models vs. reasoning with a set ofconventional models

17 / 29

Page 51: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Questions

Reasoning with Partial modelsvs

Reasoning with a set of conventional models

Is there a speedup?

How is speedup affected by changing:

• model size

• levels of uncertainty?

To get answers:

1) Experiments with random inputs.

2) Real-world case study.

18 / 29

Page 52: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Questions

Reasoning with Partial modelsvs

Reasoning with a set of conventional models

Is there a speedup?

How is speedup affected by changing:

• model size

• levels of uncertainty?

To get answers:

1) Experiments with random inputs.

2) Real-world case study.

18 / 29

Page 53: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 54: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 55: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 56: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 57: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 58: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 59: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 60: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 61: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Experiments

19 / 29

Page 62: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Case Study

Why Case Study?

Triangulate experimental results (randomly inputs)

with observations from a real-world scenario.

Case Study details:

• Real-world software project: UMLet.

• Real-world bug from UMLet bugzilla.

• Realistic bug fixes.

• Two properties from literature [V.D.Straeten’03].

• 27,261 elements (XL model size)

• 220 concretizations (XL uncertainty size)

20 / 29

Page 63: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Case Study

21 / 29

Page 64: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Results of Evaluation

Reasoning with Partial modelsvs

Reasoning with a set of conventional models

Is there a speedup?

– Yes, it is consistently faster than reasoning with the set.

How is speedup affected by changing model size and levels ofuncertainty?

– Speedup decreases with model size.

– Speedup increases with uncertainty.

– No slowdowns!

22 / 29

Page 65: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Summary

Modeling Uncertainty

• Encode uncertainty in Partial Models.

• Semantics: sets of conventional models.

Reasoning in the Presence of Uncertainty

• Check properties.

• Give feedback to facilitate diagnosis.

Evaluation of Reasoning

• Reasoning with Partial models vs. reasoning with a set ofconventional models

23 / 29

Page 66: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

The Big Picture

24 / 29

Page 67: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Next Steps

25 / 29

Page 68: Partial Models: Towards Modeling and Reasoning with Uncertainty

Questions?

Page 69: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Language Independent!

Class Diagram example from [MiSE’12].

27 / 29

Page 70: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Bibliography I

P. Classen, A.and Heymans, P.Y. Schobbens, A. Legay, and J.F. Raskin.“Model Checking Lots of Systems: Efficient Verification of TemporalProperties in Software Product Lines”.In Proc. of ICSE’10, pages 335–344, 2010.

M. Famelis, Shoham Ben-David, Marsha Chechik, and Rick Salay.“Partial Models: A Position Paper”.In Proc. of MoDeVVa’11, pages 1–6, 2011.

M. Famelis, R. Salay, and M. Chechik.“The Semantics of Partial Model Transformations”.In Proc. of MiSE’12, 2012.

K. G. Larsen and B. Thomsen.“A Modal Process Logic”.In Proc. of LICS’88, pages 203–210, 1988.

P. Larsen.“The Expressive Power of Implicit Specifications”.In Proc. of ICALP’91, volume 510 of LNCS, pages 204–216, 1991.

M.Famelis, R.Salay, and M. Chechik.“The Semantics of Partial Model Transformations”.In Proc. of MiSE’12, pages 546–560, 2012.

28 / 29

Page 71: Partial Models: Towards Modeling and Reasoning with Uncertainty

PartialModels:Towards

Modeling andReasoning

withUncertainty

M.Famelis,R.Salay,

M.Chechik,

Introduction

Intuition

MotivatingExample

ModelingUncertainty

Partial Models

Semantics

ReasoningWithUncertainty

PropertyChecking

Diagnosis

Evaluation

Experiments

Case Study

Conclusion

Bibliography II

B. Morin, G. Perrouin, P. Lahire, O. Barais, G. Vanwormhoudt, and J. M.Jezequel.“Weaving Variability into Domain Metamodels”.J. Model Driven Engineering Languages and Systems, pages 690–705, 2009.

R.Salay, M. Chechik, and J.Horkoff.“Managing Requirements Uncertainty with Partial Models”.In Proc. of RE’12, pages 546–560, 2012.

R. Salay, M. Chechik, and J. Gorzny.“Towards a Methodology for Verifying Partial Model Refinements”.In Proc. of VOLT’12, 2012.

R. Salay, M. Famelis, and M. Chechik.“Language Independent Refinement using Partial Modeling”.In Proc. of FASE’12, 2012.

R. V. D. Straeten, T. Mens, J. Simmonds, and V. Jonckers.“Using Description Logic to Maintain Consistency between UML Models”.In Proc. of UML’03, pages 326–340, 2003.

29 / 29