An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan...
-
Upload
amberlynn-patterson -
Category
Documents
-
view
213 -
download
0
Transcript of An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan...
![Page 1: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/1.jpg)
An Integrated Data Model Verifier with Property Templates
Jaideep Nijjar Ivan Bocic Tevfik Bultan
{jaideepnijjar, bo, bultan}@cs.ucsb.edu
University of California, Santa BarbaraDepartment of Computer Science
Verification Lab
FormaliSE 2013
![Page 2: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/2.jpg)
Web Software Everywhere• Commerce, entertainment, social interaction
• We will rely on web apps more in the future
• Web apps + cloud will make desktop apps obsolete
![Page 3: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/3.jpg)
Model-View-Controller Pattern
DB
• Benefits of the MVC pattern:• Separation of
concerns • Modularity• Abstraction
• MVC pattern has become the standard way to structure web applications• Ruby on Rails• Zend for PHP• CakePHP• Struts for Java• Django for Python• …
ViewModel
Controller
![Page 4: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/4.jpg)
MVC Application
• Ruby on Rails
Data Model
• ActiveRecords
Formal Model
• Sets and Relations
Verification
A Data Model Verification Approach
MVC Design Principles
Automatic Extraction
Add data model properties
• Bounded and Unbounded
![Page 5: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/5.jpg)
iDaVer: Integrated Data Model Verifier
formal data model + property + technique
(expressed using Templates)
Active Records
Property Failed +
Counterexample
Property Verified
Unknown
Model Extraction
Choice of technique
Verification
Bounded (Alloy)
Unbounded (SMT Solver)
Unbounded (FOL Thm Prover)
Properties
![Page 6: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/6.jpg)
Outline
• Motivation• Overview of Approach• Rails Data Models
• Basic Relations• Options to Extend Relations• Formalization of Semantics
• Property Templates• Verification Techniques• Case Study• Conclusions and Future Work
![Page 7: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/7.jpg)
A Rails Data Model Exampleclass User < ActiveRecord::Base
has_and_belongs_to_many :roleshas_one :profile, :dependent => :destroyhas_many :photos, :through => :profile
endclass Role < ActiveRecord::Base
has_and_belongs_to_many :usersendclass Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroy has_many :videos, :dependent => :destroy, :conditions => "format='mp4'"endclass Tag < ActiveRecord::Base
belongs_to :taggable, :polymorphic => trueendclass Video < ActiveRecord::Base belongs_to :profile has_many :tags, :as => :taggableendclass Photo < ActiveRecord::Base ...
Role
*
0..1
1
User
Profile
*
1
Video
*
1
Taggable
*
Tag
1
* 1Photo
*
1
format=.‘mp4’
![Page 8: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/8.jpg)
Rails Data Models
• Data model verification: Analyzing the relationships between data objects
• Specified in Rails using association declarations inside the ActiveRecord files• The basic relationships
• One-to-one• One-to-many• Many-to-many
• Extensions to the basic relationships using Options• :through, :conditions, :polymorphic, :dependent
![Page 9: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/9.jpg)
The Three Basic Relationships in Rails• One-to-One
.
• One-to-Many
class User < ActiveRecord::Base has_one :profileend.
class Profile < ActiveRecord::Base belongs_to :userend
class Profile < ActiveRecord::Base has_many :videosend.
class Video < ActiveRecord::Base belongs_to :profileend
User
Profile
1
Profile
Video
*
1
1
![Page 10: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/10.jpg)
The Three Basic Relationships in Rails
• Many-to-Manyclass User < ActiveRecord::Base has_and_belongs_to_many :usersend
class Role < ActiveRecord::Base has_and_belongs_to_many :rolesend
User
Role
*
*
![Page 11: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/11.jpg)
Options to Extend the Basic Relationships
• :through Option• To express transitive relations
• :conditions Option• To relate a subset of objects to another class
• :polymorphic Option• To express polymorphic relationships
• :dependent Option• On delete, this option expresses whether to delete the associated
objects or not
![Page 12: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/12.jpg)
The :through Optionclass User < ActiveRecord::Base
has_one :profilehas_many :photos, :through => :profile
endclass Profile < ActiveRecord::Base belongs_to :user has_many :photosendclass Photo < ActiveRecord::Base belongs_to :profileend
Profile
User Photo
*
*
0..1 1
1
1
![Page 13: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/13.jpg)
The :dependent Option
• :delete directly delete the associated objects without looking at its dependencies
• :destroy first checks whether the associated objects themselves have associations with the :dependent option set
class User < ActiveRecord::Base has_one :profile, :dependent => :destroyend
class Profile < ActiveRecord::Base belongs_to :user has_many :photos, :dependent => :destroyend
PhotoProfileUser *1 10..1
![Page 14: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/14.jpg)
Formalizing Rails Semantics
• S: The sets and relations of the data model (data model schema)• e.g. { Photo, Profile, Role, Tag, Video, User} and the relations
between them
• C: Constraints on the relations• Cardinality constraints, transitive relations, conditional relations,
polymorphic relations
• D: Dependency constraints • Express conditions on two consecutive instances of a relation such
that deletion of an object from the first instance leads to the other instance
Formal data model: M = <S, C, D>
![Page 15: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/15.jpg)
Outline
• Motivation• Overview of Approach• Rails Data Models• Property Templates• Verification Techniques• Case Study• Conclusions and Future Work
![Page 16: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/16.jpg)
Property Templates
• User-friendly and makes it easy to express properties• Manually writing properties is a time-consuming and error-prone
process• Requires familiarity with input modeling language of solver
• Templates are language-neutral• Do not require familiarity with SMT-LIB, Spass and Alloy
languages, and understanding of output specifications• Make it easy to rerun the tool and switch the verification
technique, by not requiring the user to rewrite the property
• Eight property templates available for the most common data model properties
![Page 17: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/17.jpg)
Property Templates
• alwaysRelated[classA, classB]• To check that objects from classA are always related to objects of
classB• someMultipleRelated[classA, classB]
• To check that it is possible for objects of classA to be related to more than one object of classB
• someUnrelated[classA, classB]• To check that it is possible for an object of classA to not be
related to any objects from classB• transitive[classA, classB, classC]
• To check that the relation between classA and classC is the composition of the relations between classA and classB, and classB and classC
![Page 18: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/18.jpg)
Property Templates
• noDangling[classA, classB]• To check that when an object of classA is deleted, there are no
objects from classB that are left with a dangling reference to the deleted object
• deletePropagates[classA, classB]• To check that when an object of classA is deleted, related objects
in classB are also deleted• noDeletePropagation[classA, classB]
• To check that when an object of classA is deleted, related objects in classB are not deleted
• noOrphans[classA, classB]• To check that deleting an object from classA does not cause
related objects in classB to be orphaned • An orphan is an object that is related to no other object
![Page 19: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/19.jpg)
Outline
• Motivation• Overview of Approach• Rails Data Models• Property Templates• Verification Techniques
• Bounded Verification with Alloy• Unbounded Verification with SMT Solver• Unbounded Verification with FOL Theorem Prover
• Case Study• Conclusions and Future Work
![Page 20: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/20.jpg)
iDaVer
SMTSolver
instance or unsat or unknown
formulaSMT-LIB Encoder
Results Interpreter
UNBOUNDED VERIFICATION
AlloyAnalyzer
instance or unsat
formulaAlloy Encoder
Results Interpreter
BOUNDED VERIFICATION
formal data model + property
Active Records
Model Extraction
Choice of techniqueProperties
Theorem Prover
proof found or completion foundor timeout
formulaFOL Encoder
Results InterpreterProperty Failed +
Counterexample
Property Verified
Unknown
![Page 21: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/21.jpg)
Bounded Verification
UNBOUNDED VERIFICATION
AlloyAnalyzer
instance or unsat
formulaAlloy Encoder
Results Interpreter
BOUNDED VERIFICATION
formal data model + property
Active Records
Model Extraction
Choice of techniqueProperties
SMTSolver
instance or unsat or unknown
formulaSMT-LIB Encoder
Results Interpreter
Theorem Prover
proof found or completion foundor timeout
formulaFOL Encoder
Results InterpreterProperty Failed +
Counterexample
Property Verified
Unknown
![Page 22: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/22.jpg)
Alloy Language
• A declarative specification language for object modeling
• Based on first-order logic
• Set-based representation of objects• Defines sets of objects using signatures (sigs)• Defines relations using fields inside the signatures• Add additional constraints about the model as facts
• Well-suited for formally specifying data models• Can add assertions to specify properties about the
specification
![Page 23: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/23.jpg)
Alloy Analyzer
• Automated verification of Alloy specifications is undecidable if the domains are not bounded
• To ensure decidability, Alloy Analyzer restricts the domains to a finite scope• User-specified • A finite bound on the sizes of the domains
• SAT-based bounded verification• Alloy Analyzer translates the Alloy verification query to a Boolean
logic formula satisfiability query • Then invokes an off-the-shelf SAT-solver
![Page 24: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/24.jpg)
Sample Translation to Alloy
class User < ActiveRecord::Base has_one :profileend
class Profile < ActiveRecord::Base belongs_to :userend
sig Profile {}sig User {}one sig State {
profiles: set Profile,users: set User,relation: Profile lone -> one User
}
![Page 25: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/25.jpg)
Bounded Verification of Data Models
• Automatically translate the formal data model extracted from the Active Records and the property to Alloy
• User may specify a bound or use the default
• Use Alloy Analyzer to perform bounded verification
• Possible outputs:• Assertion holds within the
given bound• A counterexample proving
falsified assertions
Formal data model + Property+ Bound
Alloy Encoder
instance or unsat
formula
AlloyAnalyzer
BOUNDED VERIFICATION
Results Interpreter
Property Fails + Counterexample
Property Verifies (within bound)
![Page 26: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/26.jpg)
Unbounded with SMT Solver
UNBOUNDED VERIFICATION
AlloyAnalyzer
instance or unsat
formulaAlloy Encoder
Results Interpreter
BOUNDED VERIFICATION
formal data model + property
Active Records
Model Extraction
Choice of techniqueProperties
Theorem Prover
proof found or completion foundor timeout
formulaFOL Encoder
Results InterpreterProperty Failed +
Counterexample
Property Verified
Unknown
SMTSolver
instance or unsat or unknown
formulaSMT-LIB Encoder
Results Interpreter
![Page 27: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/27.jpg)
Satisfiability Modulo Theories (SMT)
• SMT-solvers automatically check the satisfiability of first-order formulas with respect to a set of theories
• Typical theories include: Linear Arithmetic, Arrays, Bit Vectors, Equality with Uninterpreted Functions• Only implicit universal quantification
• Theory of Equality with Uninterpreted Functions.• Language: Variables, Constants, Uninterpreted function symbols,
Predicate ‘=‘, and Boolean connectives• Uninterpreted functions have no properties except their
signature and functional consistency: a = b => F(a) = F(b)• Example formula: F(x) = F(G(y)) ˅ H(x,y) = 1
![Page 28: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/28.jpg)
SMT Solvers
• There are many SMT solvers out there that support popular theories
• However many of them are not suitable for us • we need support for quantified formulas to handle the property
templates• Microsoft’s Z3 supports quantified expressions in the theory of
uninterpreted functions• Uses heuristics for eliminating quantifiers in formulas• May return ‘unknown’ during satisfiability check
![Page 29: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/29.jpg)
SMT-LIB
• Defines a standard input language for SMT Solvers• Defines theories and logics in which formula can be written• Lisp-like format: Specifications are a series of s-expressions• Declare types using declare-sort command
• (declare-sort User)
• Declare uninterpreted functions using declare-fun command• (declare-fun foo (Domain) Range)
• Quantifier commands: (forall )and (exists )• Add constraints and properties using assert command• Check satisfiability by using (check-sat) command
![Page 30: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/30.jpg)
Sample Translation to SMT-LIB
class User < ActiveRecord::Base has_one :profileend
class Profile < ActiveRecord::Base belongs_to :userend
(declare-sort User)(declare-sort Profile)(declare-fun relation (Profile) User)(assert (forall ((p1 Profile)(p2 Profile))
(=> (not (= p1 p2)) (not (= (relation p1) (relation p2) ))
) ))
![Page 31: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/31.jpg)
Unbounded Verification using SMT Solvers
SMTSolver
instance or unsat or unknown
formula SMT-LIB Encoder
Results Interpreter
UNBOUNDED VERIFICATION
• Automatically translate formal data model and property into the theory of uninterpreted functions with quantification (SMT-LIB)
• Use the SMT solver Z3 to perform satisfiability check
• For failing assertion properties, our tool interprets outputs and forms a counterexample
• Unknowns (and timeouts) possible since the theory is undecidable
Formal data model + Property
Property Failed +
Counterexample
Property Verified
Unknown
![Page 32: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/32.jpg)
Unbounded Verification
UNBOUNDED VERIFICATION
AlloyAnalyzer
instance or unsat
formulaAlloy Encoder
Results Interpreter
BOUNDED VERIFICATION
formal data model + property
Active Records
Model Extraction
Choice of techniqueProperties
SMTSolver
instance or unsat or unknown
formulaSMT-LIB Encoder
Results Interpreter
Property Failed +
Counterexample
Property Verified
Unknown
Theorem Prover
proof found or completion foundor timeout
formulaFOL Encoder
Results Interpreter
![Page 33: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/33.jpg)
FOL Theorem Provers
• Rails data models and properties are expressible in first-order logic (FOL) with equality and quantifiers• Note that this is an undecidable theory
• There are automated theorem provers for first-order logic• They use search strategies to find proofs • However due to undecidability of the FOL they cannot always
give a definite answer• We use the FOL theorem prover, Spass
![Page 34: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/34.jpg)
Modeling Active Records using FOL
• Model object types by declaring a unary predicate• Returns true if an object is a member of that class
• Model relations between data objects using binary predicates• Returns true if the two objects are related
• Axioms are used to express constraints on the data model, e.g.• Specifying cardinality, dependency, and transitivity constraints on
relations• Specifying that predicates denoting classes not related by
inheritance are mutually exclusive • Conjectures are used to model the property to be checked
• To verify the property holds on the data model, send the following formula to the theorem prover:
axioms => conjecture
![Page 35: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/35.jpg)
Sample Translation to Spass
list_of_symbols. sorts[Profile, User]. predicates[(relation, 2)]. end_of_list.list_of_formulae(axioms). formula(forall([Profile(a)], not(User(a)))). formula(forall([User(a)], not(Profile(a)))). formula(forall([a, b], implies( relation(a, b), and(Profile(a), User(b))))). formula(forall([a, b1, b2], implies( and(relation(b1,a), relation(b2,a)), equal(b1,b2)))). formula(forall([a, b1, b2], implies( and(relation(a,b1), relation(a,b2)), equal(b1,b2)))). formula(forall( [Profile(a)], exists([b], relation(a, b)) )).end_of_list.
![Page 36: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/36.jpg)
Unbounded Verification of Data Models using Theorem Provers
Theorem Prover
formula FOLEncoder
Results Interpreter
UNBOUNDED VERIFICATION
• Automatically translate formal data model and property into Spass’s input language (FOL)
• Use the theorem prover Spass to see if formula provable• Interpret results to determine
whether property holds• Does not produce
counterexamples since theorem provers are not designed to do so
• If Spass times out (due to undecidability of theory), iDaVer returns ‘Unknown’
Formal data model + Property
Property FailedProperty Verified
Unknown
proof found or completion found
or timeout
![Page 37: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/37.jpg)
Outline
• Motivation• Overview of Approach• Rails Data Models• Property Templates• Verification Techniques• Case Study• Conclusions and Future Work
![Page 38: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/38.jpg)
Case Study• LovdByLess, a social networking application
• LOC: 3787
• iDaver input:• Path of the directory containing the Active Record files• Name of the file containing the properties to check
(Expressed using property templates!)• Verification technique
• Number Active Record files: 13
![Page 39: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/39.jpg)
Case Study
• Check: alwaysRelated[Photo, Profile]• Solver: Spass
+ Unbounded verification− No sample instance− May report unknown or timeout
![Page 40: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/40.jpg)
Case Study
• Check: someUnrelated[ForumTopic, ForumPost]• Solver: Z3
+ Unbounded verification + Sample instance − May report unknown or timeout
![Page 41: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/41.jpg)
Case Study
• Check: deletePropagates[Profile, Photo]• Solver: Alloy
![Page 42: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/42.jpg)
Case Study
• Check: deletePropagates[Profile, Photo]• Solver: Alloy
+ Counterexample data model instance+ Always returns a result (for small domains)− Bounded− Slower
![Page 43: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/43.jpg)
Summary of Technique Pros and Cons
Alloy SMT Solvers Theorem Provers
Unbounded Verification No Yes Yes
Produces Counterexample Yes Yes No
GeneratesUnknown No Yes Yes
Speed Slow Fast Fast*
*In particular, Spass was slightly faster than Z3 in our experiments and timed out less frequently
![Page 44: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/44.jpg)
Outline
• Motivation• Overview of Approach• Rails Data Models• Property Templates• Verification Techniques• Case Study• Conclusions and Future Work
![Page 45: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/45.jpg)
Conclusions and Future Work
• It is possible to extract formal specifications from MVC-based data models and analyze them
• We were able to find data model errors in real-world applications using some of these techniques (ISSTA’11, ASE’12)
• Integration of multiple automated verification tools makes overall approach more flexible
• Property templates simplify property specification • We have some recent work on
• automated property inference (ISSTA’13)• analyzing actions that update the data store (submitted)
• Main goal: Verifiable data model specification!
![Page 46: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/46.jpg)
Questions?
![Page 47: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/47.jpg)
Related Work
• Verification of Web Applications • [Krishnamurti et al, Springer 2006 ] focuses on correct handling of
the control flow given the unique characteristics of web apps • Works such as [Hallé et al, ASE 2010] and [Han et al, MoDELS
2007] use state machines to formally model navigation behavior• In contrast to these works, we focus on analyzing the data model
• Formal Modeling of Web Applications• WebAlloy [Chang, 2009]: user specifies the data model and
access control policies; implementation automatically synthesized• WebML [Ceri et al, Computer Networks 2000]: a modeling
language developed specifically for modeling web applications; no verification
• In contrast, we perform model extraction (reverse engineering)
![Page 48: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/48.jpg)
Related Work• Verification of Ruby on Rails applications
• Rubicon [Near et al, FSE 2012] verifies the Controller whereas we verify the Data Model
• Requires manual specification of application behavior, whereas we verify manually written properties
• Limited to bounded verification• Data Model Verification using Alloy
• [Cunha and Pacheco, SEFM 2009] maps relational database schemas to Alloy; not automated
• [Wang et al, ASWEC 2006] translates ORA-SS specifications to Alloy, and uses the Analyzer to produces instances of the data model to show consistency
• [Borbar et al, Trends 2005] uses Alloy to discover bugs in browser and business logic interactions
![Page 49: An Integrated Data Model Verifier with Property Templates Jaideep Nijjar Ivan Bocic Tevfik Bultan {jaideepnijjar, bo, bultan}@cs.ucsb.edu University of.](https://reader035.fdocuments.us/reader035/viewer/2022081603/56649e975503460f94b9abe1/html5/thumbnails/49.jpg)
Related Work
• Unbounded Verification of Alloy Specifications using SMT Solvers• [Ghazi et al, FM 2011], approach not implemented• More challenging domain since Alloy language contains
constructs such as transitive closures • Specification and Analysis of Conceptual Data Models
• [Smaragdakis et al ASE 2009, McGill et al ISSTA 2011] use Object Role Modeling to express data model and constraints
• Focus is on checking consistency and producing test cases efficiently
• Using Patterns to Facilitate Formal Property Specification• First proposed for temporal logic properties [Dwyer et al ICSE 1999]• The templates we present are not temporal and are specific to
data model analysis