National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product...
-
Upload
hortense-rose -
Category
Documents
-
view
212 -
download
0
Transcript of National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product...
![Page 1: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/1.jpg)
National Instruments Confidential
Optimizing Test System Development
Wyatt Meek
VISTA Product Manager
![Page 2: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/2.jpg)
Agenda
• Boeing Test Engineering Priorities• V I Engineering Overview• Addressing Common Software Engineering Issues
– Configuration Management– Project Management– Reuse
• Discussion
![Page 3: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/3.jpg)
Boeing Priorities
• What are the top initiatives within the test engineering group?
• How do you currently measure success/failure? • What issues does your group currently face?• Are your suppliers required to follow
process/development guidelines?
![Page 4: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/4.jpg)
History of V I Engineering
• Founded in 1992• Select NI Alliance Member • Four Locations
– Farmington Hills, MI (HQ)– St. Paul, MN– Indianapolis, IN– Los Angeles, CA
• Business Units– Engineering Services– Engineering Information Management Systems (EIM)– VISTA
![Page 5: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/5.jpg)
Multi-Industry Experience
![Page 6: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/6.jpg)
Simplified Waterfall Model for Software Design
research/definition
requirementsspecification
design coding testingship/
maintain
![Page 7: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/7.jpg)
research/definition
requirementsspecification
design coding testingship/
maintain
All Too Often…
LabVIEW makes it very easy to quickly put something together
Developers do not follow a process
![Page 8: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/8.jpg)
Case Study
Source: CMU/SEI-94-TR-013
![Page 9: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/9.jpg)
Case Study
Source: CMU/SEI-94-TR-013
![Page 10: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/10.jpg)
Common Software Engineering Issues
• Process Issues– Configuration Management – Project Management– Requirements Management– Quality Assurance (V&V)
• Development Issues– Scalability – Reusability
This leads to poor code quality and inefficient use of resources
![Page 11: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/11.jpg)
Addressing the Issues
• SEI (Software Engineering Institute www.sei.cmu.edu)– SW-CMM– CMMI
• IEEE Software Engineering Standards (www.standards.ieee.org/software)– 828, 830, 1028, 1220,1058.1
• ISO/IEC– 9001:2000, 12207, 15288
• Industry Guidelines– FDA– DoE– FAA
![Page 12: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/12.jpg)
Process Implementation
• Process Analysis and Definition– Prioritize improvement initiatives– Define development process– Develop templates and guides– Select automation tools
• Implement Automation Tools• Training
– Software development process– Development and process improvement tools– Common programming techniques and architecture
![Page 13: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/13.jpg)
Challenges with Process Implementation for LabVIEW and TestStand• LabVIEW File Handling
– Embedded Source and Object– Static Linkage
• Metrics – Complexity– Size
• Design Methodology– Style – Patterns
![Page 14: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/14.jpg)
Configuration Management
![Page 15: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/15.jpg)
Configuration Management
• Why use Configuration Management?– To know what you’ve built (configuration)– To identify and track needed changes– To track status of configuration items– To deliver known and verified configurations
• Four main Configuration Management Functions– Configuration identification– Configuration control– Configuration status accounting– Configuration authentication
![Page 16: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/16.jpg)
CM Terminology
• SCC – Source Code Control• Configuration Item – files placed in SCC• Baseline – specific versions of CI’s at a specific time in the
product’s lifecycle (ie Beta Release, Rev 2.0.1)• Status Accounting – knowing which version belongs in a
baseline and why• Label – assigned as an attribute to a specific version of a file or
application, may be used to show status accounting (Alpha, Beta, Reviewed, Verified, etc.)
![Page 17: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/17.jpg)
Example CM Plan and Process
• Configuration Items (specifications, code, daq settings, support files, exe’s)
• Configuration Control– User roles (developer, lead developer, project manager, operations)– Permissions (read/write access, labels)– Change procedures
• Status Accounting– Labeling procedures
• Authentication– Audits– Integrity checks
![Page 18: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/18.jpg)
CM Recommendations
• Treat read only VIs as locked • Establish common development environment• Do not allow multiple checkouts (merging impossible)• Avoid branching SCC projects (cross linking and merging) • Do not use LLBs during development (can not locate
modifications within)• Use labels during development (assign status)• Deploy exe or use integrity check
![Page 19: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/19.jpg)
CM Integration Solutions - demos
• Basic– LV 6.0.2 PDS built in SCC– SCC repository independent of LV
• Intermediate– LV 6.0.2 PDS and newer built in SCC with VSS or Perforce
• Advanced– VISTA CM Tools for LV 6.1 (Base, Full, PDS) and newer
with VSS or ClearCase (others in development)
![Page 20: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/20.jpg)
Project Management
![Page 21: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/21.jpg)
Importance of Metrics
• Confirm Estimates• Project Monitoring• Cost Justification • Risk Analysis • Impact of Change • Track Improvement Initiatives
– Reuse– Productivity– Diagnose root cause of project failure
“You can’t manage what you can’t measure”“You can’t manage what you can’t measure”
![Page 22: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/22.jpg)
Standard Metrics (text based languages)
• Hours• SLOC• Function Points• Cyclomatic Complexity• EVA (Earned Value Analysis)
![Page 23: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/23.jpg)
LabVIEW Metrics
• Hours• SLOC Equivalent Metric
– Nodes (VI Metrics)– GOBs (Graphical OBjects calculator)
• Cyclomatic Complexity (possible, but not yet available)• EVA
![Page 24: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/24.jpg)
TestStand Metrics
• Hours• SLOC Equivalent Metric
– Lines in seq file• EVA
![Page 25: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/25.jpg)
Metrics Trials
• LabVIEW Developer Exams– GOBs more reliable than nodes
• V I Engineering– ~10 GOBs/hr for product development and regulated industries– ~25 GOBs/hr for automotive solutions
• Ball Aerospace– 8-10 GOBs/hr for 2 week to 2 man year projects
• General Dynamics (CMMI Level 3)– 8.7 GOBs/hr avg for 8000 man hours of work
![Page 26: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/26.jpg)
Variables
• Style • Design Patterns / Architecture used (GOOP vs event
driven vs traditional)• Quality Level Requirement
– Industry (regulated vs non-regulated)– Application Type (product, manufacturing test system, DVT,
etc.)
![Page 27: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/27.jpg)
Establishing a Project Plan with Metrics
• Review Requirements• Decompose Requirements into Tasks/Components
– Complete preliminary design– MS Project example WBS
• Estimate Development Tasks based on Historical Metrics and Risk (spreadsheet example)
• Select Lifecycle • Identify Quality Goals• Assign Estimate to Non-development Tasks based on
Historical Metrics based on Quality Goals and Lifecycle
![Page 28: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/28.jpg)
Establishing Project Control
• Identify Project Deliverables for each SLM Phase• Build Master Project Schedule• Assign Planned Value to Deliverables with Project
Schedule– SW deliverables back-fire GOBs from hours estimates
![Page 29: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/29.jpg)
Project Monitoring with Metrics
• Track % complete for Deliverables– Use GOBs, SLM phase, status on tasks for objective
feedback on software deliverables• Roll Up % Complete to EV• Track EV vs. Planned Value, Actual Cost
![Page 30: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/30.jpg)
Project Monitoring with Metrics
• Track % complete for Deliverables– Use GOBs, SLM phase for objective
feedback on software deliverables
• Roll Up % Complete to EV• Track EV vs. Planned Value,
Actual Cost• Manage by Budget at Completion
vs. Estimate at Completion Variance, etc.
Development by LifeCycle PhaseWorkspace: Stopwatch Project Rev2.0
Design19%
Development23%
Test28%
Implementation19%
Rework11%
Design
Development
Test
Implementation
Rework
![Page 31: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/31.jpg)
Project Monitoring with Metrics
• Track % complete for Deliverables– Use GOBs, SLM phase for
objective feedback on software deliverables
• Roll Up % Complete to EV• Track EV vs. Planned Value,
Actual Cost• Manage by Budget at
Completion vs. Estimate at Completion Variance, etc.
GOBS vs TimeStopwatch Project
0
200
400
600
800
1000
1 2 3 4 5 6 7 8
Months
GO
BS
(100
0)
Core Classes
Plug Ins
Database
Total
![Page 32: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/32.jpg)
Reuse
![Page 33: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/33.jpg)
33
Coding for Reuse in LabVIEW
• Benefits of LabVIEW Reuse• What is Required for Effective Reuse with LabVIEW?• Reuse Tips
![Page 34: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/34.jpg)
Benefits of LabVIEW Reuse
• Decreased Development Time• Reduced Risk• Improved Quality• Lower Overall Cost
![Page 35: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/35.jpg)
Requirements for Reuse
• Reuse Development Philosophy– Reuse must be a part of the organizational culture
• Coding Standards / Style Guides– Coding style needs to be familiar to entire team
• Training• Source Code Control
![Page 36: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/36.jpg)
36
Reuse Development Philosophy
• Always Evaluate Reuse Potential• Generalize Code; Avoid project specific code• Document for Reuse and Maintenance• Have Reuse Reviews at the Beginning and the End of the
Project
![Page 37: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/37.jpg)
CM and Reuse
• Avoid VIs with the Same Name• Put LabVIEW Reuse Libraries in user.lib (treat them
similar to vi.lib)– Organize VIs based on code categories– Limit check-out/check-in access to reuse librarians
• Categorize LabVIEW Libraries– General reuse, product line reuse, product specific
• Maintain Backward Compatibility– Create a new VI before losing backward compatibility
![Page 38: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/38.jpg)
38
Example Reuse Library Directory Structure in user.lib
• ..\VIE Library.lib\ArrayTools.llb• ..\VIE Library.lib\ConfigUtilities.llb• ..\VIE Library.lib\ControlFunctions.llb• ..\VIE Library.lib\Dialogs.llb• ..\VIE Library.lib\ErrorTools.llb• ..\VIE Library.lib\FileUtilities.llb• ..\VIE Library.lib\GUI.llb• ..\VIE Library.lib\GUI.llb\Soft
Keyboard.llb• ..\VIE Library.lib\NumericTools.llb• ..\VIE Library.lib\OS Utilities• ..\VIE Library.lib\OS Utilities\Windows.llb
• ..\VIE Library.lib\SharedTYPE.llb• ..\VIE Library.lib\Sound.llb• ..\VIE Library.lib\StateMachine.llb• ..\VIE Library.lib\StringTools.llb• ..\VIE Library.lib\Time.llb• ..\VIE Module.lib\AI and Calibration.llb• ..\VIE Module.lib\DTS Engine.llb• ..\VIE Module.lib\Formula Evaluator.llb• ..\VIE Module.lib\SecurityModule.llb• ..\VIE Module.lib\System Integrity Module.llb
• (.llb extensions are used for directory names to be consistent if converted to VI library files)
![Page 39: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/39.jpg)
39
LabVIEW Style Affects Reuse
• Using a common style and good VI documentation can significantly improve reuse
• LabVIEW Development Guidelines, chapter 6– devstyle.pdf in ..LabVIEW\Search the LabVIEW Bookshelf…
• Use a code review checklist• Avoid overly complex code• Use templates to help standardize coding style
between multiple developers
![Page 40: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/40.jpg)
40
Architecture Reuse Tips
• Use layered architectures; limit interaction to adjacent layers
• Avoid project specific dependencies; use loose coupling– Use Graphical Object-oriented Programming (GOOP)– Avoid global variables– Use standard configuration tools– Separate user-interface VIs from the main application
• Think in terms of reusable components– Encapsulate functionality and data (use GOOP)– Remember that how features specific to an architecture are
used may limit reuse to that architecture.
![Page 41: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/41.jpg)
VISTA
Software engineering approach to the management, development and release of LabVIEW and TestStand software.
•Process Consulting•Training•Development and Process Tools
![Page 42: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/42.jpg)
VISTA Configuration and Project Management Tool
• Workspace• Metrics• CM Functionality
– LV and TS integration– Baselines– Build with integrity (exe or native)– File comparison– File load check
![Page 43: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/43.jpg)
VISTA Productivity and Reuse Tools
• Help Generator• Renaming and
Relocation Tool • Documentation Tool• Reuse Library• Style Guide
![Page 44: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/44.jpg)
VISTA Quality Assurance
• Node/Path Tracker• Cyclomatic Complexity• Traceability Tool
![Page 45: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/45.jpg)
VISTA GOOP Tools
• GOOP Inheritance Toolkit– GOOP Wizard 3 with
Inheritance– Icon Editor– Development Distribution
Tool
![Page 46: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/46.jpg)
GOOP Background
• Scalability – this non-OOP is not easy to grow or maintain
![Page 47: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/47.jpg)
GOOP Background
• Scalability – layered, OOP approach
![Page 48: National Instruments Confidential Optimizing Test System Development Wyatt Meek VISTA Product Manager.](https://reader035.fdocuments.us/reader035/viewer/2022070407/56649e395503460f94b2a75e/html5/thumbnails/48.jpg)
How Can V I Engineering Help You?
• Process Assessment• Custom and Advanced Training• Tools • Project Support• Architecture Design
Learn more at www.vista.viengineering.com