Assurance techniques for code generators
description
Transcript of Assurance techniques for code generators
Assurance techniques for code generators
Ewen DenneyUSRA/RIACS, NASA Ames
Bernd FischerECS, U Southampton
Assurance problem
• Safety/mission-critical software requires assurance that it meets a certain level of “quality”
• What are the issues in assuring automatically generated code?– Different forms of assurance– Different assurance techniques– Diverse generator paradigms
Forms of assurance
What exactly might we need to assure?
• Compliance with requirements
• Compliance with spec/model
• Certification standards
• Coding standards
• Absence of run-time errors
• Traceability
• Appropriate documentation
Minimize “automation surprises”
Correctness
Reliability
Legibility
Code generators in practicePractitioner survey carried out in March 2006
(Code Generators in Safety-critical Applications, J. Schumann, E. Denney); 23 responses from NASA and industry.
• How are ACGs used for safety-critical applications at NASA and in industry?
• Which are the primary application areas and domains?
• Which tools are used?• Challenges, benefits and problems?• How could ACGs be extended to be more
useful in safety-critical applications?
Tools and languages
The Big Three:
• Real-Time Workshop
• MatrixX
• SCADE
Domains and criticality levels
• Principle domains:– control– modeling/simulation
• Many highly critical applications
• ACG used for – production code (74%)– prototyping (52%)– simulation (48%)– testing (30%)– glue/interface code (30%)
System components
Weaknesses
• Steep Learning Curve– applicable problems, features, correct usage,
architecture, implied methodology, semantic ambiguities, …
– substantial impact on development process
• ACG customization– necessary in 1/3 of cases– often (2/3) done by tool vendor
• ACG bugs– in 2/3 of applications, bugs were found in ACG
Qualification• A code generator is qualified
– with respect to a given standard– for a given project
if there is sufficient evidence about the generator itself so that V&V need not be carried out on the generated code to certify it
• Must be done for every project, version• Can obtain verification credit• Generators are rarely qualified• Examples: ASCET-SE (IEC 61508), SCADE,
VAPS (DO-178B)
Certification and V&V
• Auto-generated code must be certified for safety-critical use
• Techniques used:– testing (90%)– static analysis (58%)– simulation (52%)– manual review (48%)
• No formal verification
• No review of generator code
Safety properties
Generator features
Domain-specific analyses
Mostly numeric issues:
• stability (root locus, Lyapunov)
• robustness
• convergence
• transience
Some domain-specific design rules:
• “forbidden” constructs
• block structure
Documentation
• Design information
• Code derivation
• Configuration management information(to “replay” generation)
• Safety information
• Tracing information
• Interface definitions, requirements
• User manuals
• Installation information
Should be customizable
Traceability
• Most important: model code
• Secondary: code V&V artifacts
Tool integration
Also
• workflow and process tools
• tools for integrating legacy code
Survey summary
• Integrated modeling, analysis, and simulation tools are most common in control domain
• In-house extensions common for modeling and verification issues
• Natural synergy between code generation and certification activities– perceived but not realized– autocode often treated like manual code
• Iterative customization of generator should be seen as integral part of development process
Assurance techniques
• Testing the generator (qualification)– for all specs, blocks, configurations, backends, …
• Post factum verification / certification– verify / certify generated programs individually
• Correctness by construction– generator inherently guarantees certain
properties
• Documentation
• Traceability
Discussion questions
• What are the interesting assurance artifacts, properties, etc. in your target domains?
• What are suitable notions of documentation, traceability, development process?
• What assurance techniques have you tried?
• How is the generative knowledge represented (templates, transformation rules, etc.) and how can it be combined with assurance information?
• Can we apply Design for Verification (D4V) to generators?