The Still-Forgotten Obama Lie by Nick Chase [email protected] ...
Lightning Talks Presented at STAREast 2005 By Matt Heusser … and the gang [email protected].
51
-
date post
21-Dec-2015 -
Category
Documents
-
view
216 -
download
0
Transcript of Lightning Talks Presented at STAREast 2005 By Matt Heusser … and the gang [email protected].
Simple RA Form
Requirement or issue Likelihood of failure
Impact of failure
I need to walk down the hall without tripping.
High Medium
Think about your experiences, use old bug reports, trust your gut!
CONTEXT CHANGES EVERYTHING!
Instead of a hall- the red carpet at the Oscars where I am nominated for Best Actress, Director, Producer and Screenwriter.
High HIGHEST POSSIBLE!
Jennifer BrendlingerConcurrent Technologies Corporation
CMMI for testing: A critical [email protected]
What is CMMI? Capability Maturity Model (CMM) Integration
Serve as a guide for improvement of organizational processes http://www.sei.cmu.edu/cmmi/ March 2003 – Concurrent Technologies Corporation (CTC) was
appraised at Maturity Level 3 of the CMMI-SE/SW, Version 1.1, Staged Representation – http://www.ctc.com
Disciplines – Bodies of knowledge Systems Engineering – Development of total systems Software Engineering – Development of software systems
Staged Representation Proven sequence of improvements, each serving as a
foundation for the next
Maturity Levels Maturity Level 1: Initial
Processes are ad hoc and chaotic. Success depends on heroics of the people, not on the use of proven processes.
Maturity Level 2: Managed Requirements are managed, and processes are planned, performed,
measured and controlled. Maturity Level 3: Defined
Processes are well defined and understood and are described in standards, procedures, tools and methods.
Maturity Level 4: Quantitatively Managed Selected subprocesses are controlled using statistical and other
quantitative techniques. Maturity Level 5: Optimizing
Processes are continually improved based on a quantitative understanding of the common causes of variation inherent in processes.
Organizational Processes Process Areas
Related practices that satisfy a set of goals Verification – Ensure that selected work products meet their specified
requirements Validation – Demonstrate that a product or product component fulfills
its intended use when placed in its intended environment Specific Goals (SG)
Describe what must be implemented to satisfy the process area
Specific Practices (SP) Activities performed to achieve a specific goal
Generic Goals (GG) Enable the organization to institutionalize best practices
Verification & Validation Verification
SG1 – Preparation for verification is conducted. SG2 – Peer reviews are performed on selected work products. SG3 – Selected work products are verified against their
specified requirements. Validation
SG1 – Preparation for validation is conducted. SG2 – The product or product components are validated to
ensure that they are suitable for use in their intended operating environment.
GG3 – The process is institutionalized as a managed process. Company processes and procedures
Work Products Used to Enhance Testing and Comply with CMMI Verification and Validation (V&V) Plan Operations & Maintenance (O&M) Manual System Test Procedures Requirements Traceability Matrix Discrepancy Reports Change Requests Test Run Logs Peer Review Reports Meeting Minutes Trip Reports Metrics
Bottom Line - Benefits of CMMI For the Testing Process
Established plans to follow “Enforced” cooperation, dialog, reporting and follow-up
For the Project Quality work product that meets its requirements and
fulfills its intended use
For the Organization Everyone buys in to the processes and procedures No grassroots efforts No chaos
Reversing the Outsource/Offshore Trend
with Technology
Rick Vatcher
CEO, AccordSQA
The Promise – The Truth
• SQA Outsourcing Saves Money– Outsourcing Yields Poor Results (May 4, 2005
USATODAY)
– 70% had a bad experience and 44% didn’t save money.
• SQA skilled workforce more plentiful in developing countries– India experiencing skilled labor crisis (Ostfold SW)
Take a Look at the SQA Industry
• SQA Tools developed by SQA Professionals
• Automated, Easy to use, Easy to learn, LESS TEDIOUS
• Knowledge based tools replacing Manual Repetition
• Business Process Part of Process – Elevating Roles
• Integral Part of the Development Process
Next-Generation SQA Accelerate the Process – Eliminates Offshore Gains
0
500
1000
1500
2000
2500
3000
3500
NextGeneration
2ndGeneration
Test Maintenance
Test Creation
83% Less Cost! - 83% Less Time!83% Less Cost! - 83% Less Time!
MAN-HOURSNext Next
Generation Generation TechnologyTechnology
2nd Generation
Tool
Test Development * 480480 3000
Maintenance Cycles 5050 150
Costs @ $40/Hour $21.2K$21.2K $126.0K
•1500 test cases/scripts (2 hours per script) equivalent to 45 SmarteScript test scenarios (10.5 hours per scenario)
Software Tool Advancements Driving Best Practices in Resource Allocation
•Next-Gen Tools Close the Gap–Save money
–Speed Time-to-Market–Elevate Resources
…Ultimately Reversing Outsourcing Trend
New bug(Customer, internal)
Team Triage(Eng, QE, Doc, Support)
Submit bug fix(Eng)
Verify bug fix(QE)
Add a test(QE)
Close(QE)
Close(QE)Need more info
PassFail
Not a bug
Sheryl Stockless, The MathWorks, Inc.
Star East 2005
Verification
New bug(Customer, internal)
Team Triage(Eng, QE, Doc, Support)
Submit bug fix(Eng)
Verify bug fix(QE)
Close(QE)
Close(QE)
Need more info
PassFail
Not a bug
Validate bug(QE)
Sheryl Stockless, The MathWorks, Inc.
Star East 2005
Validation and Verification
What Any QA department Need to meet SOX Requirements
• Defined Control Points– Requirements– Test Plan– Test Cases– Go/No Go document– Implementation Plan
• Processes around the Control Points– Don’t over process
• Accountability and Traceability– Sign off and acceptance from required stakeholders– Traceability of Control Points, acceptance to projects/releases*Applicable for anyone who is testing applications to get involved and understand audit
requirements
Automated Testing using Distributed Computing
Darshan DomahSenior Software Test Engineer
Vital Images Inc.
5/19/2005
STAREAST 2005
Distributed Test Environment
DTE
WinRunnerPythonTcl
•Set up Test •Start Test•Monitor Test
•Results Reporting •Drive GUI of Application•Uses Analog Mode
tclient/tserver
Overview of tclient/tserver• 2 Tier Client/Server• TCP/IP and Sockets • Multiple servers, dedicated ports• Used to:
– reset box– bring SW– execute queue of tests– monitor tasks– get results
• Some Commands– Start application remotely
• tclsh83 tclient.tcl –c –bgnd –h HOST1 vitrea.exe
– Start a winrunner test remotely• tclsh83 tclient.tcl –c –wrun –n e:/Orientation –i 2 –t 3600 –h HOST5 –o
z:/orientation/testresults
Overview of Application
• 3D Visualization for medical imaging-Vitrea• Uses: diagnosis, surgery planning, post operation analysis
– Testing• Patient Orientation • Reliability Uptime
CT and MRI data from scanners
Vitrea interprets
data
Recreates 2D images and 3D
volumes of anatomy
Medical personnel (Radiologists & Physicians)use software capabilities to analyze rendered images,
make diagnosis and create reports
Communicating the Value of Exploratory Testing
David Gilbert
President
Sirius Software Quality Associates
Exploratory Testing IS High Value
• Due to lack of co-dependencies, it can begin much earlier.– No requirements needed– Full functionality not needed– Can work around instabilities– Formal Stubs and Harnesses not always needed
Exploratory Testing IS High Value
• Due to flexibility, it should have the highest defect to time testing ratio of any other testing process– Not business process constrained– Looks for the cause and classification of defects, not
just the repeatable specific case
But Value is Relative…
• …and based on PERCEPTION!– A painting from your child is very valuable to you.– The Mona Lisa has value universally.
• What is the difference?• Quality of Presentation!
Why is Automation Held in Such High Esteem?
• Even though we all know many automation efforts do not successfully test their projects well, they are perceived as– Organized– Structured– Professional– Repeatable– Reportable– Manageable
We Must Present Our Results Professionally
• To be taken seriously and given respect, we must meet the same level of professional management and presentation.
• This can be VERY labor intensive, using– ScreenGrabbers– Spreadsheets– HomeGrown Databases– Word Processors– Lots of cut and paste– Reams of paper for printing results
We Must Present Our Results Professionally
• Or use a tool, with features such as …– Keylogging– Screenshots– Video– Organized Test Cases– Integrated Issue Management– Graphs and Reports
Motivation
• 5000+ bugs during product development life cycle• Subjective and not deterministic process for triaging
bugs• Testers were confused about what kind of bugs were still
being accepted – they wanted to be able to focus on the right set of problems to investigate
• Bad for morale and team engagement• Bugs will occur through out the whole project• Quality should go up and not down• Risk of regression gets higher with time• Not all features get ready at the same time• Aspects to be considered: cost, risk, impact to teams
(test, UA, etc.)
Proposal
• Rules of game clear and public to all team members• Bug is opened, field is set in the bug database by tester, might be updated by
developer, feature team decide on it, escalating to Management team if necessary• Bug bar considers: feature status, moment in project schedule and bug details, like
scope, risk, cost, etc.• Triage process and sets right priority for all accepted bugs accordingly• Clear communication of results
Priority Name Definition
1 Wish These might not be even considered bugs.
2 ImprovementThis bug is hard to find, not noticeable, causes minor problems (or none)and can be ignored.
3 BugThis is a bug, but might be obscure, rare or have small impact. Normally ithas easy workaround.
4 KBThis bug might be very visible and affect functionality significantly. In case ithas a workaround, it is not obvious or simple.
5 PSSThis bug will cause Support calls. Publishing KB article is not enough, sinceworkaround may not exist or be too complicated.
6 QFE Serious bug. If a customer finds this bug, we will have to issue a QFE.
7 Ship Stopper
Reasonable region of a feature is not working as expected because of thebug, and there is no workaround. We can not ship the product with this bugactive.
Example7 Wish
6 Improvement X
5 Bug X
4 KB X X X
3 PSS X X X X
2 QFE X X X X X X X
1 Ship Stopper X X X X X X X
Areas ReportsImport Export Forms Performance PDA
SBA Integration
User Assistance
5 Bug
4 KB X X
3 PSS X X X
2 QFE X X X X
1 Ship Stopper X X X X X X X
Areas ReportsImport Export Forms Performance PDA
SBA Integration
User Assistance
3 PSS
2 QFE X X
1 Ship Stopper X X X X X X X
Areas ReportsImport Export Forms Performance PDA
SBA Integration
User Assistance
After …
• Team engagement and commitment – high morale
• Number of regressions (metrics) went down significantly
• Number of opened bugs did not go down, but accepted bugs did
• Product was shipped in timely manner
Does Your Boss’s Boss Know Your Value?
Sure, Finding Bugs is Important
And helping them will help you!
But, I bet your work could help others.
Publicizing Your Contribution is Important
And the pen is mightier than the bug report.
So Start. Write Now.
• Start a blog.
• Post a paper to Stickyminds.
• Give brown bag presentations to your team.
• Offer to speak at local QA group meetings.
• Send in an abstract for STAR West 2005!
Here’s A Poem To Inspire You!
The codfish lays ten thousand eggs,
The homely hen lays one.
The codfish never cackles
To tell you what she's done.
And so we scorn the codfish,
While the humble hen we prize,
Which only goes to show you
That it pays to advertise.
- Anonymous
Lightning talks will be at the better software conference.
Call for presentations!
http://www.sqe.com/lightningtalks.asp
