CS 290C: Formal Models for Web Software Lecture 1: Introduction Instructor: Tevfik Bultan.
Data Model Analysis - UCSBbultan/courses/272/lectures/DataModel.pdf · Data Model Analysis Tevfik...
Transcript of Data Model Analysis - UCSBbultan/courses/272/lectures/DataModel.pdf · Data Model Analysis Tevfik...
DataModelAnalysisTevfikBultan
UniversityofCaliforniaSantaBarbara
Jointworkwith
Jaideep Nijjar andIvanBocic
WebApplicationDependability
2
WebApplicationDependability
3
WebApplicationDependability
President Obama: “I want to go in and fix myself, but I don't write code"
• TRACKS:Atodolistapplication
WebApplicationDependability
5
Context Recurring Todo
Feed the Dog EDIT
WebApplicationArchitecture
• ModelViewController(MVC)pattern:RubyonRails,Zend forPHP,CakePHP,StrutsforJava,Django forPython,…• ObjectRelationalMapping(ORM)ActiveRecord,Hibernate,…
6
RESTful Controller
View
OOP Rel DbORM
DataModel
AnExampleRailsDataModel
7
class User < ActiveRecord::Base
has_many :todos
has_many :projects
end
class Project < ActiveRecord::Base
belongs_to :user
has_many :todos
has_many :notes
end
class Todo < ActiveRecord::Base
belongs_to :user
belongs_to :project
end
class Note < ActiveRecord::Base
belongs_to :project
end
class ProjectsController < ApplicationController
def destroy
@project = Project.find(params[:project_id])
@project.notes.each do |note|
note.delete
end
@project.delete
respond_to(...)
end
end
Static Data Model
Data Model Updates: Actions
StaticDataModel
• ActiveRecord classdeclarations• setsofobjects
• ActiveRecord associationdeclarations• has_one,has_many,belongs_to,has_and_belongs_to_many
• Associationdeclarationscanbeusedtodeclarethethreebasictypesofrelationsbetweenclasses• one-to-one• one-to-many• many-to-many 8
ExtensionstoStaticDataModel• :throughOption
• Toexpressrelationswhicharecompositionofotherrelations
• :conditionsOption• Torelateasubsetofobjectstoanotherclass
• :polymorphicOption• Toexpresspolymorphicrelationships
• :dependentOption• Ondelete,thisoptionexpresseswhethertodeletetheassociatedobjectsornot
9
The:throughOptionclass User < ActiveRecord::Base
has_one :profile
has_many :photos, :through => :profileend
class Profile < ActiveRecord::Base
belongs_to :user
has_many :photos
end
class Photo < ActiveRecord::Base
belongs_to :profile
endProfile
User Photo
*
*
1 1
1
1
10
The:dependentOption
• :delete directlydeletetheassociatedobjectswithoutlookingatitsdependencies
• :destroy firstcheckswhethertheassociatedobjectsthemselveshaveassociationswiththe:dependent optionset
class User < ActiveRecord::Basehas_one :profile, :dependent => :destroy
end
class Profile < ActiveRecord::Basebelongs_to :userhas_many :photos, :dependent => :destroy
end
PhotoProfileUser *1 11
11
DataModelVerification
• Formalizethestaticdatamodelas• Asetofclasses• A setofrelationsbetweenthoseclasses• Asetofconstraintsontherelationsthatareimposedbytheassociationdeclarations
• Givenaformaldatamodelwecanautomaticallycheckifagivenpropertyholdsforthedatamodel• Automatedverificationdetermines:Dotheconstraintsofthedatamodelimplytheproperty?
12
DataModelVerification
AlloyEncoder
instanceorunsat
formula
formaldatamodel+property
AlloyAnalyzer
Property
ActiveRecord
SMTSolver
instanceorunsatorunknown
formulaSMT-LIBEncoder
PropertyFailed+Counterexample
PropertyVerified
Unknown
ModelExtraction
ResultsInterpreter
ResultsInterpreter
BOUNDED VERIFICATION UNBOUNDED VERIFICATION
nBound
bound
13
HowAutomatedisAutomatedVerification?• Allexceptonestep:Propertyspecification• Example:ItispossibletohaveaUserwhodoesnothaveanyPhotos.• InAlloy:
pred prop{alls:PreState |someu:User|allp:Photo|
• InSMT-LIB:
• Canwemakeiteasier?14
(pnotin(s.photo_user).u)}
(assert(exists((aPolymorphicClass))(forall ((pPhoto))(and(isUser a)(not(=p(auser_photop)))))))
PropertyTemplates
• Propertytemplatesforpropertyspecification• Language-neutral• DonotrequirefamiliaritywithSMT-LIBandAlloy
• Examplepropertytemplate:• noOrphans[classA,classB]
• Tocheckthatdeleting anobject fromclassA does notcause relatedobjects inclassB tobeorphaned
• Easilyreruntoolandswitchtheverificationtechnique,withouthavingtorewritetheproperty
• Wedevelopedsevenpropertytemplatesforthemostcommondatamodelproperties
15
CanWeDoMore?
AutomaticPropertyInference
• Automaticallyinferpropertiesbasedondatamodelschema• Datamodelschema:Adirected,annotatedgraphthatrepresentstherelations
• Lookforpatternsinthedatamodelschemaandinferapropertyifapatternthatcorrespondstoapropertyappears
• Forexample,orphanprevention
17
0 1 n. . .
. . .
CanWeDoEvenMore?
• noOrphans(X,Y)propertyfailingmeansdeletinganobjectfromclassXcreatesanorphanchainthatstartswithassociatedobjectinclassY
• Repair:Set:dependent optionto:destroy onassociationdeclarationinclassXandonremainingrelationsinthechainthatstartswithclassY
Set :dependent => :destroy on all relations in chain
X Y N. . .
. . .
18
AutomatedDataModelRepair
Summary
ModelExtraction
Verification
VerificationResults
ActiveRecords
FormalDataModel+Properties
PropertyInference
FormalDataModel
19
DataModelRepair
for failingproperties
ExperimentResultsApplication PropertyType #Inferred #Timeout #Failed
LovdByLessdeletePropagates 13 0 10
noOrphans 0 0 0
transitive 1 0 1
SubstructdeletePropagates 27 0 16
noOrphans 2 0 1
transitive 4 0 4
TracksdeletePropagates 15 0 6
noOrphans 1 0 1
transitive 12 0 12
FatFreeCRMdeletePropagates 32 1 19
noOrphans 5 0 0
transitive 6 2 6
OSRdeletePropagates 19 0 12
noOrphans 1 0 1
transitive 7 0 7
TOTAL 145 3 96
20
PropertyType#DataModel&Application
Errors
#DataModelErrors
#Failures DuetoRails
Limitations
#FalsePositives
deletePropagates 1 9 0 0
noOrphans 0 0 0 0
transitive 0 0 0 1
deletePropagates 1 3 5 7
noOrphans 0 1 0 0
transitive 0 1 0 3
deletePropagates 1 1 3 1
noOrphans 0 0 0 1
transitive 0 7 0 5
deletePropagates 0 18 1 0
noOrphans 0 0 0 0
transitive 0 0 0 6
deletePropagates 0 12 0 0
noOrphans 0 1 0 0
transitive 0 7 0 0
TOTAL 3 60 9 28
21
WhatAboutDataModelActions?
22
class User < ActiveRecord::Base
has_many :todos
has_many :projects
end
class Project < ActiveRecord::Base
belongs_to :user
has_many :todos
has_many :notes
end
class Todo < ActiveRecord::Base
belongs_to :user
belongs_to :project
end
class Note < ActiveRecord::Base
belongs_to :project
end
class ProjectsController < ApplicationController
def destroy
@project = Project.find(params[:project_id])
@project.notes.each do |note|
note.delete
end
@project.delete
respond_to(...)
end
end
Static Data Model
Data Model Updates: Actions
VerificationofDataModelActions
23
AbstractDataStores
24
class Userhas_many :todoshas_many :projects
end
class Projectbelongs_to :userhas_many :todoshas_many :notes
end
class Todobelongs_to :userbelongs_to :project
end
class Notebelongs_to :project
end
class User {0+ Todo todos inverseof user0+ Project projects inverseofuser
}
class Project {0..1 User user0+ Todo todos inverseof project0+ Note notes inverseof project
}
class Todo {0..1 User user0..1 Project project
}
class Note {0..1 Project project
}
Rails Abstract Data Store
AbstractDataStores
25
def project_destroy@project = Project.find(
params[:project_id])@project.notes.each do |note|
[email protected]_to(...)
end
action project_destroy() {at_project =
oneof(allof(Project))foreach note: at_project.notes {
delete note}delete at_project
}
invariant(forall{ |project|!project.user.empty?
})invariant(forall{ |user|
user.projects.todos.include?(user)})
forall(Project project:not empty(project.user)
)forall(User user:
user in user.projects.todos.users)
Our library allows developers to specify invariants in native Ruby
Extraction
Extractionishardforactions• Dynamictypesystem• Metaprogramming• Eval• GhostMethodssuchas: User.find_by_name(‘Rob’)
Observations• Theschemaisstatic• Actiondeclarationsarestatic• ORMclassesandmethodsdonotchangetheirsemantic
duringexecution• eveniftheimplementationcodeisgenerateddynamically
26
ExtractionviaInstrumentedExecution
• Boot-uptheRailsruntimeinasimulatedenvironment• Withoutopeningsocketsorconnectingtothedatabase
• Prepareactionmethodsforextraction• ORMoperationswillrecordtheirinvocationinsteadof
communicatingwiththedatabase• Methodcallspropagateinstrumentationjustbefore
execution• Extractionispathinsensitive,executingbothbranches
subsequently
• TriggeranHTTPrequestthattriggersanaction27
VerificationviaTranslationtoFOL
• ApredicateisgeneratedforeachclassandassociationUser(o) meansthato isaninstanceofUserProject_user(t) meansthatt representsanassociationbetweenaProjectobjectandUserobject
• Typesystemconstraintsbecomeaxioms∀u:User(u)→¬(Project(u)∨ Todo(u)...)
• Cardinalityofassociationsisexpressedthroughaxiomseg.0..1:∀t1,t2:(Project_user(t1)∧ Project_user(t2)∧
Project_user_lhs(t1)=Project_user_lhs(t2))→Project_user_rhs(t1)=Project_user_rhs(t2)
28
TranslationofStatementstoFOL
• Anactionisasequentialcompositionofstatements.
• Statements• Astateisrepresentedwithapredicatedenotingallentities
thatexistinastate• Astatementisamigrationbetweenstates
e.g.,acreate Note statement:¬pre_state(newly_created())¬∃t:post_state(t)∧ Note_project_lhs(t)=newly_created()∀o:(post_state(o)↔(pre_state(o)∨ o=newly_created())) 29
TranslationofLoopstoFOL
• WeonlysupportForEach loops(fornow)• Theycorrespondtouniversalquantification
• Statementscanexecutemultipletimesinloops• Contextstodifferentiateiterations
• Orderingofiterations• Iterationinterdependence
30
InductiveVerification
• Inv(s) isaformuladenotingthatallinvariantsholdinstates
• Action(s,s’) isaformuladenotingthattheactionmaytransitionfromstates tostates’
Checkif: ∀s,s’:Inv(s)∧ Action(s,s’)→Inv(s’)
31
Experiments
Experimentedon3 opensourceRailsapplications• FatFreeCRM,Tracks,Kandan• 272 actions, 23 invariants
Identified4 bugs• Reportedtooriginaldevelopers• Allimmediatelyconfirmedand,since,fixed• Missedbypreviousverificationeffortsontheseapplications
32
Experiments
33
Publications•JaideepNijjar andTevfikBultan.BoundedVerificationofRubyonRailsDataModels.InProc.InternationalSymposiumonSoftwareTestingandAnalysis(ISSTA),pages67–77,2011.•JaideepNijjar andTevfikBultan.UnboundedDataModelVerificationUsingSMTSolvers.InProc.27thIEEE/ACMInt.Conf.AutomatedSoftwareEngineering(ASE),pages210–219,2012.•JaideepNijjar,IvanBocic andTevfikBultan.AnIntegratedDataModelVerifierwithPropertyTemplates.In Proc.1stFMEWorkshoponFormalMethodsinSoftwareEngineering(FormaliSE 2013).•JaideepNijjar andTevfikBultan.DataModelPropertyInferenceandRepair.InProc.InternationalSymposiumonSoftwareTestingandAnalysis(ISSTA),pages202—212,2013.•IvanBocic,andTevfikBultan.InductiveVerificationofDataModelInvariantsforWebApplications.InProc.InternationalConferenceonSoftwareEngineering(ICSE),2014•JaideepNijjar,IvanBocic,andTevfikBultan.DataModelPropertyInference,VerificationandRepairforWebApplications.(SubmittedtoACMTranslationsonSoftwareEngineeringandMethodology).
34