Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft

XBOSoft, Inc. All Rights Reserved. 1

XBOSOFT Inside ISO 29119 Exposed with Jon Hagar, Senior Consultant #ISO29119 #InsideISO

XBOSoft Dedicated to software quality improvement

Founded in 2006

We speed products to market with our expert:

• Software QA consulting

• Software testing

Global team with offices in San Francisco, Beijing and Amsterdam

“Thorough, accurate and fast”


House Rules Participants other than the speakers are muted

Questions via the GoToWebinar control on the right side of your screen or through Twitter @XBOSoft

Questions may be asked throughout the webinar - we’ll try to answer them at the end

You will receive info on recording after the webinar

#ISO29119

#InsideISO


Meet Our Speakers

Phillip Lew CEO and Founder , XBOSoft • Relevant specialties and

passions o Software quality process,

evaluation, measurement and improvement

o Software quality in use / UX design

o Mobile User Experience and usability

o Cycling and travel


Karla Smith Marketing Manager, XBOSoft • Emails you about our

industry hot topic webinars

• Invites you to download our latest trends in software testing whitepapers

XBOSoft Team

Jon Hagar


• More than thirty years’ experience in software

and testing • Systems-software tester consultant and trainer

• Supports software product integrity, testing,

verification, and validation with a specialization in embedded, IoT, and mobile software systems

• Projects supported include control system (avionics and automotive), mobile-smart devices, and ground systems (IT and PC).

• Publishes and writes regularly including his book: Software Test Attacks to Break Mobile and Embedded Devices, CRC press.

• Lead editor/author on numerous international standards including ISO 291119 software testing.

Copyright 2013 Jon Hagar Grand Software Testing GST

Jon D. Hagar, Senior Consultant [email protected]

Author: Software Test Attacks to Break Mobile and Embedded Devices

Copyright 2015, Jon D. Hagar Grand Software Testing, LLC – “Software Test Attacks to Break Mobile and Embedded Devices”

6

mailto:[email protected]


Current Situation

Controversy (https://www.youtube.com/watch?v=RCl8lGxY0WU)

Many people do NOT know much about it

Starting to go into use in places in the world


Scope

Purpose of standards Motivation for ISO 29119 standards Overview of ISO29119 Applicability Thoughts

Q&A


Some Preliminaries Under disclosure, I am the IEEE Project Editor for

ISO/IEEE/IEC 29119 Software Testing Standard

For many organizations: Standards are a fact of life for many industry professionals

not just testers

Standards form part of the business for many companies , customers, and countries

We end up in a mixed relationship with standards

Standards should be viewed as a baseline


What are standards?

Guideline documents—not compulsory unless mandated

Agreements because they often reflect a certain level of consensus within a community

“Guideline documentation that reflects agreements on products, practices, or operations by nationally or internationally recognized industrial, professional, trade associations or governmental bodies” - ISO


How can a standard “help?”

Standards are a basis for: Communication – common terminology (trading language) Professionalism and criticism Conformance/compliance schemes Benchmark of ‘industry practice’ (not best, may not be good

for all, etc.) Contracts and regulations Interoperability and consistency. . .

Standards describe a current ‘body of knowledge’ that provides the basis for a professional discipline


Poll

Would you consider using a standard if it solved problems your team might have and could be tailored to better fit your needs? Yes No


Motivation for ISO29119 Conflicts in definitions, processes and procedures

“One ring to rule them all” — old testing standards to be replaced by ISO 29119 e.g., IEEE 829, IEEE 1008, BS7925-1/-2, IEEE 1028

Lacking in current standards

Organizational areas e.g., Test Policy and Organizational Test Strategy

Project Test Management BS7925 only covers unit testing General processes Common functional techniques missing Coverage of non-functional testing

Part of the larger ISO/IEEE family of systems and software standard, e.g. ISO 12207, 15288, IEEE1012


Standardization Bodies

http://www.iec.ch/

http://www.omg.org/

http://www.bsi-global.com/British_Standards/index.xalter

http://www.sis.se/Default.aspx

http://www.brandsoftheworld.com/download/brand/61142.html


ISO/IEC/IEEE 29119 –Structure and History

BS7925-1

BS7925-2 IEEE 829

Concepts & Vocabulary Part 1

Process Assessment

Testing Techniques

Part 4

Documentation

Part 3 Part 2

Processes

Keyword-Driven Testing

Part 5 ISO/IEC 33063

ISO 12207 ISO 15288 Directives

IEEE 1008


Part 1: Concepts & Vocabulary

SOFTWARE TESTING CONCEPTS

Scope, Conformance, Normative References

TESTING IN DIFFERENT LIFE CYCLE MODELS

ROLES AND RESPONSIBILITIES IN TESTING

ANNEXES – Metrics, Examples, Bibliography

DEFINITIONS

Test

: A

ppro

ach,

Bas

is,

Met

hods

-

Ris

k Bas

ed T

estin

g


Part 2: Testing Processes

TEST MANAGEMENT PROCESSES

ORGANIZATIONAL TEST PROCESS

DYNAMIC TEST PROCESSES





Instantiating Testing Processes

Ref: S. Reid


Organizational Test Specification

Develop Organizational

Test Specification

(OT1)

Monitor and Control Use of Organizational

Test Specification

(OT2)

Updated Organizational Test Specification

Update Organizational

Test Specification

(OT3)

Controlled Organizational Test Specification

Organizational Test Processes




TEST PLANNING

TEST MONITORING &

CONTROL

TEST COMPLETION

ORGANIZATIONAL TEST

DOCUMENTATION

FEEDBACK ON ORGANIZATIONAL TEST DOCUMENTATION

TEST PLAN UPDATES

TEST PLAN

TEST COMPLETION REPORT


TEST MANAGEMENT

PROCESSES

TEST PLAN, TEST COMPLETION REPORT, TEST MEASURES

TEST MEASURES

TEST PLAN, CONTROL DIRECTIVES

TEST PLAN, CONTROL DIRECTIVES

Test Management Processes


Organize Test Plan

Development Identify &

Estimate Risks

Design Test Strategy

Determine Staffing and Scheduling

Document Test Plan

Schedule, Staffing Profile

Test Strategy

Analyzed Risks

Scope

Identify Risk Treatment Approaches

Gain Consensus on

Test Plan

Approved Test Plan

Draft Test Plan

Test Plan Publish

Test Plan

Understand Context

Treatment Approaches

Test Planning Processes


Part 2 Standard Primarily has a Risk-based Process Flavor

Risk based testing is a primarily “good” practice

Other bases/approaches (Agile, exploratory, model-based, math-based, etc.) are included

All parts may need “tailoring” (few efforts would be “full” conformance)

Use of the standard implies skills, thinking, dependence on context (“it depends”), etc.


Test Monitoring & Control Process

Monitor

Report

measures

Control Set-Up

[testing complete]

[testing incomplete]

control directives

test progress

info

test progress info

test plan

test measures

test status report

test control info

...TEST PROCESSES... Dynamic/Management



TEST SPECIFICATION

[NO ISSUES NOTICED]

TEST ENVIRONMENT READINESS REPORT

TEST ENVIRONMENT

REQUIREMENTS

INCIDENTREPORT

TEST DESIGN & IMPLEMENTATION

TEST ENVIRONMENT

SET-UP

TEST EXECUTION

TEST INCIDENT

REPORTING

(LEVEL) TEST MANAGEMENT PROCESS

[ISSUE NOTICED or RETEST RESULT]

TEST RESULTS

CONTROL DIRECTIVES TEST

MEASURES

(LEVEL) TEST PLAN

Dynamic Test Processes


Part 3 – Test Documentation

TEST DOCUMENTATION

ANNEXES - EXAMPLES


Sel

ect

a su

bset

of

docs


Part 3: Test Documentation

Organizational test documentation Test policy Test strategy

Project test documentation Project test plan Test project completion report

Test Level documentation Test plan Test specification Test results Anomaly reports Level test status report Test environment report Test level completion report

Appendices Examples of documents at each level of testing

Not all need to be produced when tailoring is used- Most will NOT produce all of these


Part 4 – Test Techniques (possible publish in 2015)

TEST COVERAGE MEASUREMENT


ANNEXE – TESTING OF QUALITY CHARACTERISTICS

ANNEXE – SELECTION OF TECHNIQUES

ANNEXE – TEST TECHNIQUE EFFECTIVENESS

TEST DESIGN TECHNIQUES

Functional Structural


Part 5- Keyword-Driven Testing (possible publish in 2016-17)

Part 5 addresses:

Concept

Applicability

Interfaces

Approach

Part 5 in review and ballot


Applicability of ISO/IEEE 29119 It is intended to be a generic standard to support different

environments and domains of testing when selected

Some domains are more standards oriented such as: Safety-related Telecoms International business and large companies

Some domains are more regulated world-wide such as:

Financial – banks, stock markets, etc. Medical

Can support various lifecycle models

Tailored to sequential, iterative and agile (highly tailored) Applicability to new as well as maintenance projects


Organization where the Standard Might Be Used

Providing users a common baseline of testing concepts, terms, and ideas Trading Language

Government organizations and regulators needing an industry approved starting baseline

Purchasers and procurers of software products to assess third parties Developers of software, e.g. to support unit level testing. Training course syllabus developers and providers University lecturers wishing to teach from an ISO standards University researchers wishing to conduct research Software testing tool vendors Organizations and regulators using the standard for accreditation International business and organization who use other ISO standards


Do Testers Need Standards? – Not all, but some may Standards support common communication within the topic

Common reference points Starting point for usage

Maturity of the standard is an issue but a baseline serves as

sounding board and common reference point for “Scientific” method An international benchmark Thinkers and researchers can prove/disprove benchmark(s)

Part of being in a profession (but only part)

Standards are not for everyone


ISO29119 Standard - My Viewpoint Any software standard is based on limited history (rear

looking) and developed based on consensus of many voices Represent only some of viewpoints of industry

Will expand to include more testing ideas

Not perfect

Most using projects/professionals will use “tailoring”


My Thoughts on how to tailor Thinking testers with a context-driven/based viewpoint

Keep “good” (worthwhile)

Remove or change what is not worthwhile

Use standards as a professional reference work, but

know when and how to break the “rules” via tailoring

Get stakeholder buy in – Fight to Educate


Summary First attempt at a comprehensive software test

standard

Even in the presence of standards, a thinking tester is always needed

Get it read it, tailor it (as needed), and form your own opinions by “testing it” yourself Send me your thoughts and changes

Not universal, best, or comprehensive


For more information

[email protected] For more information and details

http://softwaretestingstandard.org/ WG26 website

http://www.jtc1-sc7.org/ Access to official documents released by WG 26

http://www.jtc1-sc7.org/




Stuart Reid

Testing Solutions Group 117 Houndsditch

London EC3 UK

Tel: 0207 469 1500 Fax: 0207 623 8459

www.testing-solutions.com

Major Reused Reference Point:

http://www.testing-solutions.com/



Post your questions on Twitter and we'll answer them @XBOSoft Join us to keep updated on all our webinars, reports and whitepapers: facebook.com/xbosoft +xbosoft linkedin.com/company/xbosoft We post regularly on our blog – check us out! http://xbosoft.com/software-quality-blog/ Download our free whitepapers, available here: http://xbosoft.com/knowledge-center/ You will receive an email with information on today’s webinar slides and recording. Any further queries regarding our services or ideas for future webinars, please email us! [email protected]

Q+A

www.xbosoft.com

http://xbosoft.com/software-quality-blog/






http://xbosoft.com/knowledge-center/software-testing-white-paper/








http://www.xbosoft.com

Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft

Technology

Transcript of Inside ISO 29119 Exposed Webinar with Jon D. Hagar - XBOSoft