How to Select a Static Analysis Tool
-
Upload
parasoftmitchell -
Category
Technology
-
view
844 -
download
8
description
Transcript of How to Select a Static Analysis Tool
How to Select a Static Analysis Tool
2011
Agenda for this session
Define static analysisLayout strategy for evaluating and choosing a static analysis tool that will actually work in the fieldList possible evaluation criteria
Parasoft Proprietary and Confidential
Parasoft Proprietary and Confidential
About Parasoft
Founded in 198727 Patents for automated quality processesBuild quality into the processStatic Analysis tools since 1994
Parasoft Capabilities
TechnologiesQuality Policy ManagementTask Management Code Analysis – Pattern BasedCode Analysis – Flow BasedCode Analysis - MetricsCode ReviewUnit Testing FrameworkMemory Error DetectionRuntime AnalysisMessage/Protocol TestingApplication Behavior EmulationFunctional TestingLoad Testing
Parasoft Proprietary and Confidential
What IS Static Analysis?
Variety of methodsPeer Review / Manual Code Review / Code InspectionPattern-based code scannersFlow-based code scannersMetrics-based code scannersCompiler / build output
Parasoft Proprietary and Confidential
First things first
More organizations are adopting formal policies regarding static analysisMany companies use a bake-off to choose toolsBake-offs are not very useful to select the best tool
Parasoft Proprietary and Confidential
Assess Your Needs
What pains do you plain to address?FDA, MISRA, PCI, etc.
Is your current development process stable, repeatable, and streamlined?Have you tried static analysis before?
Why did it fail – how can you prevent a repeatHow is your organization structured?
Corporate wide config or varied by group/projectWill analysis apply to all projects? New Code? Legacy?Where do you want to be in the future?
Parasoft Proprietary and Confidential
Compile Candidate List
Get recommendationsPerform due diligence
Even if the tool comes highly recommendedEven if the tool has been used by someone in the groupYour code, process, culture, and environment are unique
Keep the big picture in sight
Parasoft Proprietary and Confidential
Explore Vendors
You’re committing to a relationship with a vendorWhat is their vision?What best practices do they have?
Do they have a coherent strategy for the enterpriseIf they don’t have best practices you’ll need to develop them
ReputationWho uses tool?Case studies
Parasoft Proprietary and Confidential
Talk with Vendors
Evaluations are disruptive – get data upfrontAre your visions in sync with vendor?Explain what problems you want to solve?
Find out if they think static analysis will resolve itCan they set objective criteria to assess success
Describe your environmentHow have they helped others like you?
Explain your vision for deployment and adoption for the next 2-3 years
Do they believe its feasible?
Parasoft Proprietary and Confidential
Handling vision mismatch
Vendor should accept requests that fit their general businessIf they vendor disagrees with your strategy do they have a convincing explanation and alternative?Does the vendor bend over backward? Even for unreasonable requests?
Parasoft Proprietary and Confidential
Pilot Top Candidates
Setup test bed and run preliminary testsFamiliarize yourself with the toolIdentify obstacles
Be ready to assist those doing the actual pilotSelect one group
Real project – not static legacy codeEngineers who like new things
Parasoft Proprietary and Confidential
Work with pilot users
Don’t just give pilot users the program and expect useful resultsExplain
How to use it in your workflowWhat parts of the application to testWhat code should be testedWhat to look for while using the tool
Parasoft Proprietary and Confidential
Pilot tasks
Pilot users should have a list of tasksHow did the tool make their lives better?What could make it even better?How did the tool make their lives worse?How bad was the learning curve?
Parasoft Proprietary and Confidential
Compare Post-pilot Candidates
Zero in on required functionalityEvaluate vendors response to requests and issuesJudge what the relationship will be like
Parasoft Proprietary and Confidential
Evaluation Criteria: Rules
Number of built-in rules you’re really willing to enforceQuality of built-in rules you’re really willing to enforceDepth and breadth of analysisFeasible means to reduce noiseFew to no false positivesTolerable number of missed negativesEase of adjusting built-in rulesEase of adding custom rulesLevel of complexity possible in new rulesVendors plan for adding new rules
Parasoft Proprietary and Confidential
Evaluation Criteria: Workflow
IDE integrationBatch modeViolation reporting / review mechanismAutomated assignment of errors to responsible developersLegacy code identification and supportRule severity customizationAbility to suppress violation reportingAutomated violation correction
Parasoft Proprietary and Confidential
Evaluation Criteria: Scalability
Scalable usage modelEase of updating the rule set team-wide or organization wideAbility to support tiered rule setsExtensibility – APISupport for additional languages and verification methods (unit test, code review, etc)Speed of analysis (end-to-end)
Parasoft Proprietary and Confidential
Evaluation Criteria: Vendor
Product stability Having some issues is inevitable
Defect reportsFeature requestsOverall support
Parasoft Proprietary and Confidential
The 2 Most Important Questions
Will our engineers really adopt it and use it?Can you make the tool work on real code with zero noise?Will it scale?Is the work-flow practical?
Is this a long-term solution?Evaluations consume a lot of time and effortDon’t settle for “it’s good enough”Will it help reach your corporate goals?Time spent now will reward you laterAvoid continuous product evaluations
Parasoft Proprietary and Confidential
Q&A
Parasoft Proprietary and Confidential