Test Destiny the Psyche of a Professional Tester

8/14/2019 Test Destiny the Psyche of a Professional Tester

1/38

Optimizing the Testing

Effort with Keyword -

Driven FrameworksBrian Massey, IBM

March 31 April 2, 2009


2/38

Quic k Bio

Product Manager No worries not here to sale you anything (but to ensure the

kids get fed I have to mention them)

As part of Rational Services organization helped develop anddeploy frameworks for our customers

Established Software Quality Centers of Excellence for Companies

Experience as Developer, Tester, Architect, Dev Manager, QA

Manager

Used to build Air Defense Systems for a living

Developed and tested embedded systems

Father of 6 so prepared for anything


3/38

IBM Software Group Rational software

The Psyche of A Professional Tester

STPCon Spring 2009

Market Impact on Business Innovation

Delicate balance for CIOs

Market Pressures

IBM CEO studies of 2006 & 2008

Tremendous drive for business transformation and change

Increasing gap between expectation of change & organizationshistory to manage it

Recent global Secular Bear Market may take years torecover

Survival means drastic changes in expenditures

Advice

Manage what you can control economy is not one of

them Spending where it counts for long term growth

Growth & Earnings Assumptions dont trust your models

Focus on quality

Reduce debtSources: Sequoia Capital Report Oct 2008 & IBM CEO Study May 2008


4/38



STPCon Spring 2009

Who are the testers in your organization?

Does your test organization lack clout?

Why are testers perceived negatively?

Are testers junior level people -- hired as a tester as theywork their way up in the organization?

Is the skill set for the art we love lower than it should be?

When the quality people perform the quality job is it onlytemporary?

Do testers get the respect and training needed?


5/38


6/38



STPCon Spring 2009

Application Failure - The Costs

Typically 5 to 15 flaws in every 1,000 lines of code

Dealing with defects: (DOD & SEI)

Finding each bug takes up about 75 minutes Fixing bugs takes two to nine hours per bug

On the outside, thats ~150 hours,or roughly $30,000, to cleanseevery 1,000 lines of code

Software errors cost U.S. economy $60 billion annually (0.6% ofthe gross domestic product) (NIST)

$22.2 billion could be eliminated with more effective identification(NIST)

40% of unplanned downtime is due to application failure (Gartner)


7/38



STPCon Spring 2009

The Quality Zone

There is a new dimension beyond that which is known tosoftware development. It is the middle ground betweendeployment and development, between art and science, and

it lies between the pit of developments fears, and the summitof their knowledge. This is the dimension of quality. It is anarea which we call ...


8/38


9/38



STPCon Spring 2009

Tester Personality Profiles -- Junior Tester

Description Entry level/Junior tester

The Good Can follow well written manual scripts

Rarely deviates from scripts

Happy doing mundane tasks

May have lots of enthusiasm Often believes they are making a difference

Duh Oh,Found a

defect

The Bad and Ugly May have trouble writing scripts

Rarely deviates from scripts

Has no power or authority

Not very technical

Very little expected of them


10/38



STPCon Spring 2009

Tester Profiles Coop Tester

Big Mac Inc.Will Test for Happy Meals


11/38



STPCon Spring 2009

Tester Profiles Coop Tester

Description Temporary tester

The Good May be highly trained (college coop)

Often tries to make a difference

Tells it like it is (only temporary)

May want to move up in organization

The Bad and Ugly Usually only temporary

Not very well trained (unskilled) Often filling a role because the thought is Anyone can test

Not well respected (contractor and a tester)


12/38



STPCon Spring 2009

Tester Profiles Techno Geek/Caffeine Freak Tester

The Good Finds more bugs than anyone Solves the difficult problems

Runs a lot of tests

Can write very creative scripts Results are often solution oriented

The Bad and Ugly

Probably wont be able to replicate Defect reports are very creative

Usually does not follow scripts (findsmore interesting things)

Test because they were told to

Description Developers or other high strung individuals


13/38



STPCon Spring 2009

Tester Personality Profiles NASA Tester

Description AKA -- Test Cop

The Good Loves to find and report defects

Very process driven

Empowered by management

Loves their job

Intelligent and systematic

The Bad and Ugly

The person developers love to hate Bad perception by others -- any

process deviation could get you shot

Usually detail oriented and standards

driven

Fail it theymisspelled their

comments


14/38



STPCon Spring 2009

Tester Personality Profiles Apathetic Tester

The Good Very methodical

Writes great, highly reproducible bugreports

Often has a wealth of untapped knowledge

Could get great benefit from implementingtheir ideas

The Bad and Ugly Has their own ways of doing things

Nothing is really high priority

Will resist change (especially automation)

Yep Itscorn

Description Experienced tester who has fought many battles(losing most)


15/38



STPCon Spring 2009

The Facts (Our Data)

Developed questionnaire to obtain data set

Surveyed individuals from approximately 15 companies

Surveyed Managers, Testers and Developers

Followed up surveys with interviews


16/38



STPCon Spring 2009

Questions asked (from questionnaire)

How the Software world perceives testing

Why were ineffective testers ineffective

Top attributes and attitudes that made testers effective

How effective testers could improve

How many testers they worked with in their career How many of those were effective testers

Rank characteristics of testers in order of importance

Training accomplished by the respondent


17/38



STPCon Spring 2009

How do you think the software engineering world

perceives testing?

Not altogether as important as it really is.

People that are hired to point out flaws in product andtherefore diminish the perceived ability of the developer.

I believe this is determined by the culture of each

company you ask. Ive been at some places wheresoftware engineering felt that the test organization wascompletely useless and others where they thanked theirlucky stars that they had them.


18/38



STPCon Spring 2009

How do you think the software engineering world

perceives testing?

Necessary evil or as a curse.

Something that can be cut out when we run short ontime.

Initially, they do not value testing until they have

something that blows up in production and then theydevelop a great sense of the need for someone to checkbehind them and give them that additional view point.


19/38



STPCon Spring 2009

Results

Those surveyed deemed that approximately 57% or lessof testers they worked with were effective

Number ranged from 5% to 100% on individual responses

Individuals with more experience reported 2/3 theeffectiveness of testers compared to those with lessexperience (47% vs. 71%)

Developers thought 63% testers effective

QA thought 64% testers effective

Managers thought 43% testers effective Perceived value of QA QA 3.9, Dev 2.5, Management

4.3, Overall 3.9 (out of 10)


20/38



STPCon Spring 2009

Results

Henry, Joel and Bob Blasewitz. 1994. Do we really need SQA to produce quality software? No! Well maybe.

It depends. Yes! ACM Software Engineering Notes 19 (2): 63-64


21/38



STPCon Spring 2009

Why were testers ineffective?

Bored didnt like their job

Couldnt think independently

Poor communication skills

Submissive personality

Tired of battling developers

Unfamiliar with X (too many to list)

Unable to use a computer

Did not understand Y (too many to list)

Didnt use enough common sense


22/38



STPCon Spring 2009

What makes a Tester effective (attitudes)?

Work on being part of team

Remove personal preference from real problems

Like to break things

Strong desire (passion) for quality

Ability to think about a problem/requirement from variousperspectives

Enjoys a challenge

Non-defeatist/problem solving attitude High degree of confidence

STPC S i 2009


23/38



STPCon Spring 2009

What makes a Tester effective (attributes)?

Understanding of software development life cycle

Understand programming

Good people skills

Able to work with programmers to fix problems

Analytical methodology (scientific)

Problem solving skills

Embrace new technology

Becomes an expert on what is being tested


24/38

IBM S f G R i l f

STPC S i 2009


25/38



STPCon Spring 2009

Characteristics

23 Characteristics of Individuals

Ranking was 1 to 23 (1 being the top characteristic)

Lower the number the more important the characteristic

The bottom indicates less important (not unimportant)

Following charts show average of all characteristics withbreakdown of Development, Testing, and Managementperceptions

IBM S ft G R ti l ft

STPC S i 2009


26/38



STPCon Spring 2009

Top Characteristics of Tester

Overall Developer QA Manager

1 Organized 7.7 Communication Skills 4.3 Problem Solver 8.1 Organized 4.4

2 CommunicationSkills 8.1 Understands UsersPerspective 5.0 Independent 9.1 CommunicationSkills 6.6

3 Problem Solver 9.0 Organized 5.7 Out of the box thinker 9.4 Self Motivated 7.3

4 Self Motivated 9.3 Diplomatic 6.3 Communication Skills 9.7 Problem Solver 8.9

5Understands

Users Perspective9.4 Self Motivated 7.0

Understands UsersPerspective

10.1 Intelligent 9.0

Very diverse attitude on what ranks at the top

Diplomacy more important to developers

QA lacked conformity

8.1

Development/Management had clear winners

4.44.3

Organized top overall but not in top 5 for QA

Organized

IBM S ft G R ti l ft

STPCon Spring 2009


27/38



STPCon Spring 2009

Less desired characteristics (what was on the bottom)

Overall Developer QA Manager

19 Intuitive 12.9 Process Centric 15.3 Good People Person 13.4 Diplomatic 15.0

20 Diplomatic 15.2 Flexible 17.0 Diplomatic 17.3 Self Sufficient 16.4

21ProgrammingExperience

16.4 Effective Modeler 17.0 Detail Oriented 17.3EffectiveModeler

18.1

22 Effective Modeler 18.3Programming

Exp 19.7 Programming Exp 17.6 Independent 19.0

23 Detail Oriented 19.4 Detail Oriented 21.3 Effective Modeler 18.6Detail

Oriented22.5

Very clear losers (Effective Modeler & Detail Oriented)

QA/management want QA to tell it like it is (non-diplomatic)

19.4

Detail Oriented at the Bottom

15.3

Surprise -- Process less important todevelopers

Management ranked programming experiencehigher than Testers or Developers

19.7 17.6


STPCon Spring 2009


28/38



STPCon Spring 2009

Want more data Survey from Qaforums.com

Suppose that you have a friend who has some experiencein software testing but is now unemployed. She has beenunemployed for long enough that she is willing to spend

some money taking courses and writing exams in order toget a certificate.

Q1 -- Which type of certification do you think would MOST

likely help her land a testing job? (Up to 2 choices)

Q2 -- Which type of certification do you think wouldLEAST likely help her land a testing job? (Up to 2 choices)


STPCon Spring 2009


29/38



STPCon Spring 2009

Want more data Results from Survey

Certifications Most Helpful Least Helpful

Programming 65% 13%

Networking 3% 58%

Testing 23% 39%

Test Tool 74% 6%

Project Management 16% 19%

From general forum (Everyone)

From very senior level testers Rated network knowledge more valuable (very important for

load/stress testing and much other client/server testing)

Rated project mgmt knowledge more valuable

Rated the testing certificates as less valuable.


STPCon Spring 2009


30/38



STPCon Spring 2009

So who is John Doe?

Short lived Fox TV Show

Had knowledge of literally everything in the world, yet nomemory of who -- or even what -- he is

Mostly saw the world in black and white

Had all the answers except info on himself

Helped cops solve crimes uses his special gift to helpthem solve "impossible" crimes each week

While watching the show I thought he would make a great

SW Tester


STPCon Spring 2009


31/38



STPCon Spring 2009

Tester Personality Profiles -- John Doe

Systematic and Ad Hoc

Thorough

Well trained

Problem solver

Loves his jobWell respected by peers and organization

Provides value to the project

One of the top individuals on the project


STPCon Spring 2009


32/38



STPCon Spring 2009

Tester Personality Profiles -- John Doe (as a Tester)

John Doe loves to break things and is also capableenough to determine why (knows everything)

John Doe is a problem solver, methodical, and ad-hoc atthe same time (sees the world B/W and Color)

John Doe leads the organization to obtain a higher

perception of quality John Doe takes responsibility for improving the testing

process

John Doe uses special gifts to help developers find andresolve defects


STPCon Spring 2009


33/38



STPCon Spring 2009

Organizational Challenge

What is your mission?

Where does quality fit into your mission?

What is quality?

What is an error?

What is the cost of an error?


STPCon Spring 2009


34/38



STPCon Spring 2009

Organizational Challenge

Why not do what other industries do

Need to change SW industry mind set about quality

Must have skilled professionals doing the quality job

Have tools and roles that streamline all types of testing

Do more than say test it

Realize the value of quality software

Build quality into the product

Realize quality is function of people and process, not tools


STPCon Spring 2009


35/38



STPCon Spring 2009

Professional Challenge

We can fix the problem, we have the ability andresponsibility

Need to justify our existence Realize the more we know the better we test

Provide value to your project

Get involved early quality begins at requirements

Help build quality into the product

"Quality is not an act. It is a habit" -- Aristotle


STPCon Spring 2009


36/38



STPCon Spring 2009

Professional Challenge

Stand up and say Hello, Im John Doeand I am a Tester

Before After

We canchange. Be

proud totest.Will test for

food.


STPCon Spring 2009


37/38



STPCon Spring 2009

The Quality Zone

There is a new dimension beyond that which is known tosoftware development. It is the middle ground betweendeployment and development, between art and science, and

it lies between the pit of developments fears, and the summitof their knowledge. This is the dimension of quality. It is anarea which we call ...


38/38

Thank you!

For more information:

Brian MasseySenior Produc t Ma nager

IBM/ Rationa l Software678-248-4523

[email protected]/rational

38

Test Destiny the Psyche of a Professional Tester

Documents

Transcript of Test Destiny the Psyche of a Professional Tester