Strategies of Testing, Not Schools.ppt - RBCS,...

Strategies of Testing, Not SchoolsA Powerful Way to Effective, Efficient Testing

If t b th t t If you outnumber the enemy ten to one, surround them.If you outnumber them five to one, attack themthem.If you outnumber them two to one, divide them.If you are equal, then find an advantageous If you are equal, then find an advantageous battle.If you are fewer, defend against them.If you are much weaker, evade them.y

Wh t’ W ith th S h l ?What’s Wrong with the Schools?“Seven principles” okay, but also common sense for h

p p y,the most partHowever, a tree should be judged by its fruit:

No one claims membership in other schoolsTh t t t d th hi i ft t tiThe concept started the schism in software testingThe only papers on schools are by CDTs (and are unremittingly negative about other schools)The most prescriptive pronouncements emanate from CDTs (just follow them on Twitter)(just follow them on Twitter)Builds a Tower of Babel by re-naming established terms (e.g., “checking” vs. “verification”)Often used as an excuse to be exceptionally rudeH b d bl h h dHas been used to establish an orthodoxy

To quote Bret Pettichord, schools “can be used to dismiss ideas you don’t agree with,” which is usually what I seewhat I see

Strategies, Not Schoolswww.rbcs-us.com

Copyright (c) RBCS 2008-2014

Dena Pauletti

Line

Dena Pauletti

Line

St t i B tt W t Thi kStrategies: a Better Way to ThinkStrategy: “a careful plan or method for Strategy: a careful plan or method for achieving a particular goal, usually over a long period of time”Schools: “a group of persons who hold Schools: a group of persons who hold a common doctrine or follow the same teacher”When you choose to belong to a school, you are bound to its doctrines/teachersWhen you choose strategies you do so When you choose strategies, you do so to serve your needs and those of your stakeholders



Dena Pauletti

Line

Dena Pauletti

Line

T t St t iTest StrategiesA test strategy is a general, project-independent A test strategy is a general, project independent pattern for how testing is doneThe various test strategies discussed here are in use by test teams around the worldSome work sometimes, some don’t work sometimes but all have worked in some sometimes, but all have worked in some situationsWhat do you need to know about test ystrategies, how to select them, and how to blend them for success?Let’s seeLet s see…



Dena Pauletti

Line

Dena Pauletti

Line

A l ti l T t St t iAnalytical Test StrategiesIncludes two of the most common Includes two of the most common strategies

R i t b d t tiRequirements-based testingRisk-based testing

A l h b d f hAnalyze the test basis to identify the test conditions



Dena Pauletti

Line

Dena Pauletti

Line

A l ti l K Ch t i tiAnalytical: Key CharacteristicsBenefits: Alignment with test basis, measurability g , yof testing, defect prevention, and transparency of test coverageFactors for success: Document-focused strategies Factors for success: Document focused strategies require the document(s), while stakeholder-focused strategies require input from the stakeholders Risks: For document-focused strategies, unmanaged changes, absent, or low-quality documents, while, for stakeholder-focused strategies, the inability to engage the stakeholdersStakeholders are involved, and the analysis occurs before the software is delivered for testingg



Dena Pauletti

Line

Dena Pauletti

Line

M d l b d T t St t iModel-based Test StrategiesInclude:Include:

Operational profiling for reliability and performance performance Models such UML for functional aspects

D l d l f th i t Develop a model of the environment, the inputs, and system’s behavior, t i ll f t d f t l typically from study of actual or anticipated situations



Dena Pauletti

Line

Dena Pauletti

Line

M d l b d K Ch t i tiModel-based: Key CharacteristicsBenefits: Testing consistent with real-Benefits: Testing consistent with realworld usageFactors for success: An accurate model of Factors for success: An accurate model of real-world usage and available toolsRisks: Insufficient data, statistical inaccuracies, improper selection of tools, and a focus on positive paths Model-based strategies involve the key test stakeholders in validation of the model and its constructionmodel and its construction



Dena Pauletti

Line

Dena Pauletti

Line

M th di l T t St t iMethodical Test StrategiesUse standard set of test conditions Use standard set of test conditions These predetermined test conditions

i l dcan include:A quality standard A checklist Logical test conditions

The test conditions don’t vary across iterations or releases



Dena Pauletti

Line

Dena Pauletti

Line

M th di l K Ch t i tiMethodical: Key CharacteristicsBenefits: Consistent testing of defined Benefits: Consistent testing of defined attributesFactors for success: Adequate current Factors for success: Adequate, current test conditions, and stable test objectRisks: An insufficient or outdated test Risks: An insufficient or outdated test basisMethodical strategies involve key test Methodical strategies involve key test stakeholders only in initial definition test conditions



Dena Pauletti

Line

Dena Pauletti

Line

P pli t T t St t iProcess-compliant Test StrategiesFollow a set of processes defined by Follow a set of processes defined by othersP t i ll ddProcesses typically address:

DocumentationProper test basis and test oracleOrganization of the test team



Dena Pauletti

Line

Dena Pauletti

Line

P pli t K Ch t i tiProcess-compliant: Key CharacteristicsBenefits: Leveraging skills and experience Benefits: Leveraging skills and experience of standard creators (e.g., IEEE Standards or Agile approaches)F f Th l d Factors for success: The selected process must align with existing test problemsRisks: Improper understanding of process Risks: Improper understanding of process, improper implementation of process, and misapplication of processStakeholder involvement depends on the process (e.g., Agile requires daily engagement)engagement)



Dena Pauletti

Line

Dena Pauletti

Line

R ti T t St t iReactive Test StrategiesTest approach evolves rapidlyTest approach evolves rapidlyTests derive from practical experience

th th f l d l rather than formal models Test team designs and implements tests once test item is receivedReact to the system under testyPre-existing structure (e.g., fault attacks) can be usedattacks) can be used



Dena Pauletti

Line

Dena Pauletti

Line

R ti K Ch t i tiReactive: Key CharacteristicsBenefits: Finds different defects, low cost per Benefits: Finds different defects, low cost per defect found, continuous re-focusing of testing, robust given incomplete test basis Factors for success: Skilled and experienced Factors for success: Skilled and experienced testers with deep understanding of the application and technologies Ri k I ffi i t kill l k f k l d f Risks: Insufficient skill, lack of knowledge of system and/or subject, limited demonstrable coverage, poorly defined test oracleFrequently used in combination with formalized strategies to measure coverage and leverage knowledgeg g



Dena Pauletti

Line

Dena Pauletti

Line

C lt ti T t St t iConsultative Test StrategiesRely on the input of one or more key Rely on the input of one or more key stakeholders E t l t k h ld d t i t t External stakeholders determine test conditions to coverStakeholders have complete control over conditions



Dena Pauletti

Line

Dena Pauletti

Line

C lt ti K Ch t i tiConsultative: Key CharacteristicsBenefits: Consulted stakeholders can receive their desired coverageFactors for success: Consulted stakeholders need an accurate concept of what should be tested, how an accurate concept of what should be tested, how much, and in what order Risks: Conflicting priorities between stakeholders, incorrect definition of the test conditions, no means incorrect definition of the test conditions, no means for self-checking the direction, wrong set of stakeholders Consultative strategies are often used in concert Consultative strategies are often used in concert with other strategies to reduce risk of testing gapsConsultative strategies are also used by outsource testing service providers testing service providers



Dena Pauletti

Line

Dena Pauletti

Line

Dena Pauletti

Sticky Note

Marked set by Dena Pauletti

Dena Pauletti

Sticky Note

Accepted set by Dena Pauletti

R i T t St t iRegression-averse Test StrategiesManage the risk of regression through Manage the risk of regression through testingE t i t ti i Extensive automation is common Automation can occur at one or more levelsFor iterative lifecycles, regression risk is y ghigher, so ongoing regression risk management is very importantg y p



Dena Pauletti

Line

Dena Pauletti

Line

R i K Ch t i tiRegressive-averse: Key CharacteristicsBenefits: Minimize risk of regression in key areas g ywhile supporting quick releases of new versions Factors for success: Successful, efficient, maintainable automation at the unit, integration, maintainable automation at the unit, integration, system, and/or system integration levels Risks: Insufficient or unavailable tools, unautomatable tests, insufficient skills, poorly unautomatable tests, insufficient skills, poorly tested new features, changing scopeStakeholders want features to continue to work Test automation crucial for Agile projectsTest automation crucial for Agile projectsIdeally, developers design for testability and participate in automation architecture and scripts



Dena Pauletti

Line

Dena Pauletti

Line

Bl di St t iBlending StrategiesDo you have to pick a single strategy?Do you have to pick a single strategy?NO!Disparate strategies complement each otherEach strategy’s benefits can balance other strategies’ risksgSelect a smart set of strategies and blend them for optimum successblend them for optimum success

Strategies, Not Schools www.rbcs-us.com


Dena Pauletti

Line

Dena Pauletti

Line

C l iConclusionsThe concept of “test schools” simply binds the The concept of test schools simply binds the tester to a single test strategyAbandoning this failed paradigm frees testers to evaluate other test strategies which are in to evaluate other test strategies which are in common useAll have their strengths and weaknesses, so

t d t d d t l t you must understand your needs to select a strategyWrong choices in strategy are a frequent cause g gy qof test team failureStakeholders should be involved in the strategyBlend strategies for optimum successBlend strategies for optimum success

Strategies, Not Schools www.rbcs-us.com


Dena Pauletti

Line

Dena Pauletti

Line

T C t t RBCSFor twenty years, RBCS has delivered consulting, outsourcing, and training services to clients looking to improve their software and hardware testing and quality

To Contact RBCSto clients looking to improve their software and hardware testing and quality. Employing the industry’s most experienced and recognized consultants, RBCS trains and improves test groups, tests products, and provides expert test staff for hundreds of clients worldwide. Ranging from Fortune 20 companies to start-ups, g g p pRBCS clients save time and money through improved product development, decreased tech support calls, improved corporate reputation and more. To learn more about RBCS, visit www.rbcs-us.com.Add RBCS IAddress: RBCS, Inc.

31520 Beck RoadBulverde, TX 78163-3911USA

Phone: +1 (830) 438-4830E-mail: [email protected]: www.rbcs-us.com



Dena Pauletti

Line

Dena Pauletti

Line

Strategies of Testing, Not Schools.ppt - RBCS,...

Documents

Transcript of Strategies of Testing, Not Schools.ppt - RBCS,...