CONNIE HEITMEYER Center for High Assurance Computer Systems Naval Research Laboratory Washington, DC...
-
Upload
heather-boone -
Category
Documents
-
view
215 -
download
0
Transcript of CONNIE HEITMEYER Center for High Assurance Computer Systems Naval Research Laboratory Washington, DC...
CONNIE HEITMEYER
Center for High Assurance Computer SystemsNaval Research Laboratory
Washington, DC
Workshop on the Verification Grand ChallengeSRI International
February 21-23, 2005
PROGRAMMINGPROGRAMMINGMETHODOLOGYMETHODOLOGY
22/24/05
• Background: The SCR toolset• Programming Methodology
– State of the art– State of the practice
• Mathematics vs. Engineering– Components of a well-founded software engineering
discipline
• How to deal with legacy code– An alternative proposal
OUTLINE
32/24/05
SCR TOOLSET: HISTORY AND OVERVIEW
• I am the chief designer of the SCR toolset, which supports the specification of requirements in the SCR tabular notation– Based on original A-7 requirements method originated by Dave Parnas et al. in the late 1970s
• Major objective of the SCR method and tools To produce a validated, verified requirements specification
• Attributes of a high quality requirements spec– Precise and unambiguous– Readable by practitioners AND customers– Complete and consistent; correctness validated by domain experts– Freedom from implementation bias
• Benefits of a high quality requirements spec - solid basis for– Automated test case generation (satisfying some coverage criterion)– Automated generation of provably correct, efficient code– Developing a new version of the system
• Other major benefit - serves as a precise means of communication between all of the stakeholders: the customers, the future users, the developers, etc.
• SCR tools have been used in over 30 university courses, conference tutorials, and industry tutorials
SCR TOOLSSCR TOOLS
THEOREM PROVER(TAME)
INVARIANTGENERATOR
ANALYSIS ANALYSIS TOOLSTOOLS
SCR SCR TOOLSETTOOLSET
PROPERTYCHECKER (Salsa)
CONSISTENCY
CHECKER
DEPENDENCYGRAPH BROWSER
SPECIFICATIONEDITOR
MODEL CHECKER
system spec
SIMULATOR
modes
events
mon vars
cont vars
conditionsterms
• most mature tools• Distributed to 200+
org’ns in industry, govt., and academia
• Focus on safety-critical, security-critical systems
• TAME is an interface to PVS designed to prove properties of state machine models
TEST CASE GENERATOR
CODE GENERATOR
Research Prototypes Research Prototypes
Current focus: Optimized, provablyCurrent focus: Optimized, provably correct codecorrect code
52/24/05
APPLICATIONS TO PRACTICAL SYSTEMS
Since 1999, at least three sites of Lockheed Martin have been using the SCR tools to develop critical avionics software
• Largely embedded software -- tends to have highly complex control logic, functions with many parts, simple data types
• Representative applications: flight navigation, flight guidance, autopilot logic, flight management, traffic and collision avoidance, and security testing
• Tools also being applied to software for the Joint Strike Fighter• LM uses specs developed with tools to automatically generate a suite of test cases
S. D. Allen, Lockheed MartinJanuary 10, 2005
We currently are supporting close to 1500 models … and have found theSCRTool suite to be an invaluable aid in finding requirements defects, as well as validating the functional behaviour of our software requirements.
We currently are supporting close to 1500 models … and have found theSCRTool suite to be an invaluable aid in finding requirements defects, as well as validating the functional behaviour of our software requirements.
CommentCommentfrom afrom a(mostly)(mostly)satisfiedsatisfieduseruser
62/24/05
• Many useful principles and guidelines have been formulated for application in software development
• Design for ease of change• Stepwise refinement• Separation of concerns• Specifications that are simultaneously easy to understand, precise and unambiguous and that avoid
implementation bias• Requirements that describe the needed software behavior as a relation on environmental
quantities…
• Lots of specification and modeling languages, methods, methodologies, and tools have been proposed by researchers
• Languages: Z, B language, CSP, CCS, SCR, RSML, Aslan, Statecharts, Esterel, Lustre, Signal, TRIO, I/O automata,…
• Methodologies: Object-Oriented (UML, Eiffel, Alloy, …); Aspect- Oriented; Extreme Programming; Agile Programming; ASMs; Model-Driven Design
• Problem: No consensus among software engineering and other researchers on which specification/modeling languages, methods, and methodologies practitioners should use to develop reliable software
– No agreement on what constitutes a good model– Few good examples of high quality specifications
PROGRAMMING METHODOLOGY:STATE OF THE ART
72/24/05
• The overwhelming majority of existing software, even software for safety-critical, security-critical systems, has been built and is being built in an ad hoc, unsystematic fashion
• The vast majority of software developers continue to focus on code– While they may write requirements and design documents, these documents are almost
universally incomplete, inconsistent, incorrect, and poorly organized.
• The vast majority of software developers do not develop specifications of the required software behavior– Few specifications, let alone high quality specs, are developed– Specifications that do exist are often low-level and filled with design and implementation bias
• Very few software developers use theorem provers or model checkers– Many are undereducated -- they cannot do formal proofs nor do they even understand
propositional logic let along 1st order logic– For the ones who could potentially do formal proofs or apply a model checker, the view is
that verification tools are NOT cost-effective and there is little evidence to prove them wrong• Two bright spots
– Increasing use of Java– Growing popularity of model-driven design
PROGRAMMING METHODOLOGY:STATE OF THE PRACTICE
82/24/05
MATHEMATICS VS.ENGINEERING
MATHEMATICALMATHEMATICALRESOURCESRESOURCES
(e.g., theories, models, (e.g., theories, models, and algorithms)and algorithms)
Theories for deductive verification,Theories for deductive verification,Theory underlying model checking, Theory underlying model checking, SAT solvers, BDDs, etc.SAT solvers, BDDs, etc.Automata theory and models Automata theory and models Theories underlying decision Theories underlying decision procedures and their combinationprocedures and their combinationTools based on fundamental researchTools based on fundamental research……
LONG-TERM GOALLONG-TERM GOALSemi-Automatic Transformation of a Semi-Automatic Transformation of a
SpecificationSpecificationinto a Provably Correct, Efficient Programinto a Provably Correct, Efficient Program
Semi-Automatic Transformation of a Semi-Automatic Transformation of a SpecificationSpecification
into a Provably Correct, Efficient Programinto a Provably Correct, Efficient Program
MATHEMATICALLY MATHEMATICALLY WELL-FOUNDEDWELL-FOUNDED
SOFTWARESOFTWAREENGINEERINGENGINEERING
DISCIPLINEDISCIPLINE
MethodsMethodsModelsModels
Languages/Templates Languages/Templates Industrial-Strength ToolsIndustrial-Strength Tools
Claim: Developing a well-founded software engineering discipline is ofClaim: Developing a well-founded software engineering discipline is ofcomparable importance to scientific advancescomparable importance to scientific advances
92/24/05
• Specification languages customized for particular domains (e.g., control systems, office systems, web programs, protocols, …)
• Powerful, robust, integrated tools that are easy to use– Model checkers coupled with simulators– Simulators with nice graphical interfaces
• Templates for specifying system-level (and other) properties• Proof systems that produce understandable proofs, e.g., a step in the mechanized
proof corresponds to a step in a hand proof– Example: TAME interface to PVS uses PVS strategies to raise level of theorem proving
to level of human reasoning– From a TAME proof, a human-understandable proof can be constructed [Archer+, ASE,
2002].• Compilers that generate provably correct, efficient code from high-level
specifications• Automatic generation of invariants satisfying some specified coverage criterion• Specification-based automatic test case generation• Good examples of high quality specs and models• Libraries of programs and program modules (including specs, properties satisfied,
and proofs of correctness)
A WELL-FOUNDED SOFTWARE ENG. DISCIPLINE: COMPONENTS
102/24/05
BENEFITS OF A SYSTEM-LEVEL SPEC 1. Flexibility in code generation – Can choose from many desired target languages – Possibly, more efficient code 2. Easier to understand/reason about the software – Higher level of abstraction 3. Improve potential for interoperability – Easier to know how to use the software as part of a larger system 4. Easier to modify and maintain the software 5. Easier to assess the software – e.g., generate good test suites
6. The process of developing the spec will expose bugs (although how to eliminate them may be tricky)
HOW TO DEAL WITH LEGACY CODE:AN ALTERNATE SOLUTION
EXTRACT A SPEC FROM THE CODE
112/24/05
EXTRACTING A SPEC FROM THE CODE:TECHNICAL APPROACH
4. Constructreplacement code
YES
2. Synthesizeoperational spec
test cases, operational speclegacy code
conforms?
operational spec
3. Test conformance ofoperational spec with
legacy code
Iterate until spec and
legacy codeconform NO
conforms?
1. Generate elements neededto build operational spec
domain experts,system users,system/user documentation
system modes, transitions, ...scenarios, system invariant props.
inputs/outputs,program units, …
legacy codecode annotations