Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques

© 2015 Cognizant

© 2015 Cognizant

Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques

October, 2015

Susan Schanta, Director

© 2015 Cognizant 2

Agenda

2

Topic Page

Section 1: Improving Elicitation Sessions through the use of Elicitation Checklists 4

Common Challenges 5

Introduction to Industry Standards 7

Industry Requirement Definitions 8

S.M.A.R.T. Requirements 9

Create Common Language for WHAT vs. HOW 10

Defining Elicitation 11

Pre-Elicitation Planning & Meeting 12

Meeting Etiquette 14

Elicitation Checklists 15

Building Elicitation Checklists 18

Section 2: Removing Ambiguous Language from Requirements 19

Ambiguity Reviews 20

Requirements Review Phase 21

Ambiguity Types 22

© 2015 Cognizant 3

Agenda

3

Topic Page

Section 2: Removing Ambiguous Language from Requirements (Continued)

Ambiguity Examples 23

Ambiguity Words 24

Section 3: Measure Requirements Effectiveness through Metrics 25

Impact of Poor Requirements in the Industry 26

Measuring Requirements Within & Across Projects 27

Ambiguity Review Metrics 28

References 31

© 2015 Cognizant 4

Improving Elicitation Sessions through the use of Elicitation Checklists

© 2015 Cognizant 5

Common BA Challenges Strategic Approach

BA Ownership – Role & Responsibilities

Business Analysts do not own the Elicitation and Requirements Documentation Process

BA Team lacks involvement in full software lifecycle

Assess and implement a standard approach that positions the BA as the requirements owner based on industry best practices with customization where needed to address your specific needs

Development Driven Requirements

Requirements are written in technical speak

Technically driven requirements may not address business need

Create guidelines for authoring requirements that define the user’s interaction with the system

Assess current Technology documentation to determine best approach to pair it with requirements that define business needs

Elicitation & Documentation Standards

Lack of standard requirements templates

Lack of standard methodology

Lack of End to End Impact Assessment of Proposed Functionality

Apply industry best practices as documented in BABOK@ and Karl Wiegers Software Requirements

Introduce Ambiguity Reviews to measure adoption of standards

Develop BA Style Guide based on lessons learned to drive continual improvement

Common BA Challenges

© 2015 Cognizant 6

Common BA Challenges Strategic Approach

Requirements Quality

Lack of defined Elicitation Session approach leads to inconsistent requirements

Multiple interpretations applied to written requirements

Technology drives the approach where gaps exist in requirements

Define methodology to elicit requirements Identify SMEs vs. Approvers Conduct Pre-Elicitation Meeting to establish topics, timelines and meeting

participants

Introduce Ambiguity Reviews to measure adoption of standards

Develop BA Style Guide based on lessons learned to drive continual improvement

Financial Impact

Delayed project release due to: Scope Creep Requirement Change Requests Defect Rates

Set benchmarks to create platform for financial measurement Establish benchmark of evidence to measure financial impact

Measure of Effectiveness

Increased Change Requests to Address Requirement Gaps

Increased Defect Leakage to QA/UAT/Production

Negative Impact to Upstream/Downstream Systems

Limited Requirements to TC traceability

Create scorecard to measure improvement rates Defect leakage rates from Requirements to Development to QA/QA to

UAT/UAT to Production Increase in requirements traceability Rate of Change Requests Impact to upstream/downstream system interfaces

Strategic Approach

© 2015 Cognizant 7

Introduction to Industry Standards

Use Standard Set of Industry Defined Templates − Business Requirements state the functional goal − User Requirements describe the interaction between the user and system − Functional Requirements state the system behavior required to fulfill the user requirement

Drive Requirements Gathering through a Defined Elicitation Process

− Pre-meetings with business stakeholders − Early analysis to define scope of Elicitation Sessions − Document process paths

Introduction of Ambiguity Reviews

− Requirements are clear, concise and measurable − Lessons learned are formulated and applied to continuing to improve the Requirements Phase

Requirements Traceability Matrix

− Requirements are traced to test cases to ensure implementation and exercise of application functionality prior to production launch to ensure fulfillment, stability and reliability

Standard Measurement

− Success measured through metrics • Requirements measurability through Ambiguity Reviews • Requirement Defects • Volume of Change Requests

A Culture of Excellence in the Requirements Phase

© 2015 Cognizant 8

Industry Requirement Definitions

Business Requirement A business requirement is a goal of the organization requesting the system. User Requirement A user requirement is a task that the user must be able to accomplish using the system. • User requirements • User stories • Use cases Functional Requirement A functional requirement is a conversation between the system and a user or other system requesting/providing information. A functional requirement states the change or addition of a system feature to satisfy the user requirements. Nonfunctional Requirement A nonfunctional requirement is a quality attribute that the system must have. These attributes are not system features (functional requirements), but they do influence how the functionality of the system is implemented. Nonfunctional requirements deal with some aspect of usability, performance, or security or are operational in nature.

Laying the Foundation for a Common Understanding…

© 2015 Cognizant 9

S.M.A.R.T. Requirements

Specific requirements are precise

− The requirement has the appropriate level of detail and states exactly what is required

− The definition of the requirement must result in only one interpretation no matter who is reviewing it

− The use of conjunctions (and, or, but) are avoided to prevent misinterpretation

Measurable requirements can be verified as complete

− The behavior described in the requirement provides an expected outcome

− Ambiguous adjectives/adverbs have been removed to remove a subjective interpretation to the requirement

− Test scenarios can be derived from the requirement

Attainable requirements are actionable

− The requirement is not gold-plated or too grandiose to be fulfilled

− The requirement can be achieved given current environments

Realistic requirements are appropriate in relation to other requirements

− Can the requirement be fulfilled given the current systems?

− Can the requirement be satisfied within the project budget and resource constraints?

Time-Bound (Traceable) requirements have time parameters

− The requirement provides a specific time when the requirement needs to be completed or executed

− The requirement avoids ambiguous references such as quickly, fast or immediately

Requirements are deemed measureable when they are S.M.A.R.T

© 2015 Cognizant 10

Create Common Language for “What” vs. “How”

“What the user is able to do with the system.”

Requirements are a detailed instruction of WHAT the system will do Business requirements state WHAT will be achieved by implementing stated requirements User requirements state WHAT the flow will be for scenarios, conditions and outcome required Describes the Happy Path or the ideal flow to perform a function Describes alternate flows for exception conditions and special circumstances Describes constraints that restrict the function Describes error conditions resulting in failure

Functional requirements detail WHAT feature will be built into the system Nonfunctional requirements detail WHAT type of performance is expected from the system and directly

related to customer and user satisfaction. Technical Design details HOW the system will fulfill the business, user and functional requirements Technical requirements describe HOW the requirements will be fulfilled


Defining Elicitation

Elicitation is a collaborative process to identify requirements through elicitation sessions with business users. The Business Analyst engages with stakeholders and subject matter experts to define the AS IS and TO BE process. The process of elicitation allows the Business Analysts to collect, analyze and define requirements based on the information collected in these sessions.


Pre-Elicitation Planning & Meeting

Gather and review available documentation

Hold Pre-Elicitation Meeting

− Work with business sponsor to write Scope Statement to clarify what is included or not included

Define Discussion Topics

− Review Elicitation Checklist

− Identify SMEs by topic

− Estimate time necessary to elicit requirements for each topic

− Identify key stakeholders needed at the elicitation meeting and ascertain who provides approval

Select Elicitation Session format

Establish Guidelines for Meeting Etiquette

Identify action items needed for follow-up

Create agendas for Elicitation Sessions

Send invitations to meeting attendees

Setting Expectations for Elicitation

Subject: E&E Congressional Group Setup – PLEASE DO NOT FORWARD

***************************************************************

Invited: Joe Smith, John Doe, Sally Mill, Jane Green, Sam Jones, Mike Smith, Donna White

Objective: Define special processing rules for Congressional Group Setup

Schedule:

11:00 to 11:30 Incoming 820 Enrollment File differences

11:30 to 12:00 New Business Rules for processing Congressional members

12:00 to 12:30 Arrears processing for Congressional members

Roles/Responsibilities

Business Analyst/Meeting Facilitator Peter Graves

Requirements Recorder Martin Landau Primary Business User Greg Morris

Timekeeper Martin Landau

If you believe this invitation should have included someone who is not invited, please contact the meeting organizer.


Requirements Pre-Elicitation PlanningB

usi

ne

ss S

po

nso

rB

usi

ne

ss A

nal

yst

Requirements Phase

Start Pre-Elicitation Planning

Gather Information & Schedule

Meeting

Meeting with Business Sponsor to

Confirm Scope & Plan Elicitation

Elicit Scope of Enhancements

& Modifications

Identify SMEs by Topics &

Sub-Topics & Time Needed

Assign Stakeholder Approvals

Capture Scope, SMEs and Approval Stakeholders

Present Elicitation

Checklists to Business Sponsor

Review Elicitation

Checklists w/Business Sponsor

Set Expectations

for SME preparedness

Schedule Elicitation Sessions (Restrict Access)

Establish Agenda for Elicitation Sessions

Determine if Follow-up

Action Items Needed

End Pre-Elicitation Planning


Meeting Etiquette Streamlining Elicitation Session Participation

Facilitating the Elicitation Session:

Two BA’s will be present – 1 for elicitation/1 to capture requirements and meeting notes

Opening the Meeting:

Establish the protocol for each meeting at the beginning of the session by reviewing the:

Agenda topics

Time allowance for each topic

Process to park items outside of scope for the discussion

Conference Call Etiquette:

When key stakeholders are participating in a conference call format, establish protocol for phone etiquette:

All phones must be put on mute when the participant is not speaking

The Business Analyst reserves the right to put everyone on mute if people begin talking over each other without respecting the person who was speaking in order to regain control of the discussion

− The Business Analyst will bring the team back to the discussion at hand

The Business Analyst will return the floor to the person who was speaking when the interruption occurred


Elicitation Checklists Establish Expectation for SME Preparedness

Present the Elicitation Checklists targeted for use in sessions • Review checklist to set expectation for the content to be covered in

Elicitation Sessions • Questions will be grouped based on subject matter by topic


Elicitation Questions for User/Functional Requirements

Does the requirement state what triggers the action performed in the requirement?

Does the happy path user requirement detail the workflow and expected outcome?

Does the requirement identify the user performing the action?

Does the requirement state the permissions required to perform the operation?

Are the requirements boundaries clear? For instance, if a payment file is received monthly, what day of the month is it received?

Are the business rules stated that determine whether a condition is true?

Does the requirement address error conditions with the expected outcome? Are error messages given?

Does the requirement include exceptions such as responses to abnormal situations including error handling and recovery?

Does the requirement address alternate flows with an expected outcome?

Does the requirement address exceptions with an expected outcome?

Are user requirements created for alternate flows with the applicable workflow and expected outcome?

Are user requirements provided for error conditions with the applicable workflow and expected outcome?

Are valid inputs and outputs stated where required?

Is the sequence of operations or processing steps provided?

Is the effect of parameters changing considered?

Is the relationship of outputs to inputs stated, including

Input/ output sequences

Formulas for input to output conversion

‘As Is’ and ‘To be’ Process Flow diagrams

Business Rule & Workflow Focus – What is the Expected Outcome?


Elicitation for Nonfunctional Requirements

• What are the initial estimated users of the application?

• What is the expected growth rate (in terms of users) of the application annually?

• What is the number of users accessing the site per day, peak hours, off-peak hours, Monday morning, Friday evening (or any specific day depending on the nature of site)?

• What is an acceptable response time for web enabled application that includes back end processing?

• What is an acceptable response time for processing handled by the internal enterprise router, firewall and web server?

• What is an acceptable response time when calling an external system?

• What are peak periods and non-peak periods daily, weekly, monthly and annually?

• What are the transaction volumes and number of users that define peak processing time?

• What are the transaction volumes and number of users that define non-peak processing time?

• Will the users perform complex queries during peak periods?

• Will the users perform complex calculations during peak periods?

Avoid use of words such as fast, efficiently, immediately…


Building Elicitation Checklists

Internal Sources for Elicitation Checklists

Clarification Trackers

Change Requests

Defects

Internal Interviews with Project Stakeholders

Internal reviews with Business Analyst Team

Internal reviews with Development

Internal reviews with Quality Assurance

Internal reviews with Performance Engineers

Web Searches

http://www.bridging-the-gap.com/what-questions-do-i-ask-during-requirements-elicitation/

http://www.batimes.com/articles/questions-for-eliciting-user-requirements.html

http://www.bbc.co.uk/guidelines/dq/pdf/is/is_requirements_questionnaire.pdf

http://reqtest.com/requirements-blog/how-to-use-interviews-to-gather-requirements/

http://www.seilevel.com/wp-content/uploads/RequirementsGatheringTools.pdf

Using Past Clarifications to Improve Future Requirements






















































Removing Ambiguous Language from Requirements


Ambiguity (am·bi·gu·i·ty) Why do an Ambiguity Review?

20

Ambiguity can be defined as… 1. Use of words that allow alternative interpretations 2. An unclear, indefinite, or equivocal word, expression, meaning, etc. 3. The possibility of interpreting an expression in two or more distinct ways An Ambiguity Review is a formal review process that focuses on the identification of ambiguities in language, structure and logic of a requirement. The process is a collaboration between the reviewer and the Business Analyst who authored the requirements document. Review a document from the consumer’s viewpoint Assumes the reader has no knowledge of the subject Why do an Ambiguity Review? Limit Scope Creep Increase Requirements Traceability to Test Cases Reduce Defect Leakage to Production Improves Estimation Accuracy Reduce Cost of Maintenance How much is your organization spending to maintain systems today?

Reduces redo work Increases velocity of Test Case creation Increase BA productivity and work product

Fortune 50 Insurance Company #1 Reduced ambiguities from 11 to 3 per

Requirement Reduced Defect Leakage from 35% to 8% Introduced BA Style Guide to drive a

consistent approach Fortune 50 Insurance Company #2 63% of all defects were traced to

Requirement Defects


Requirements Review Phase


Ambiguity Type Description

Ambiguous Term Terms (Phrase or Word) used in requirements which can be interpreted by the reader in multiple ways e.g. frequently, occasionally, efficiently

Conflicting Requirement Requirements which contradict each other – either in the same document or across multiple documents. e.g. The field name is Effective Date but the data type is defined as an integer

Glossary Word or acronym used in requirements that is new or not commonly used but has not been defined in the Glossary/Definitions section.

Grammar, Spelling & Wording

Grammar, spelling corrections and proposed wording improvements to increase clarity of the requirement

Incomplete Requirement Incomplete requirement or statement describing conditions when information is not fully detailed preventing design or test validation e.g. The system shall handle 15-25% increase in the second year

Missing Requirement Missing requirements that were not documented or may not have been elicited from the business user. e.g. Missing requirements – alternative flows, business rules, exceptions and error conditions (Questions – What, When, Where)

Unclear Requirement Requirements or statements requiring further clarification to allow the reader user to fully understand the requirement (Questions – How, Why, What do you mean by…)

(Wiegers, Bender) (Bender/Wiegers)


Ambiguity Examples

Ambiguity of Reference & Ambiguous Statements A condition when a requirement uses words such as pronouns, adjectives, adverbs and verbs that can be interpreted differently based on the reader’s view. Boundary Ambiguity A condition when the author uses terms - among or up to. The scope of the requirement is ambiguous because the stated requirement can be interpreted in multiple ways.

The payment file shall be sent monthly. Ambiguity: What is the name of the payment file? Ambiguity: What day of the month is the payment file sent? Ambiguity: Is the payment file sent on the business day before or after if the date falls on a weekend or holiday? Ambiguity: What happens if there is no data for the payment file? Ambiguity: What happens if the payment file generation fails?

(Wiegers, Bender)

1. If an employee makes less than $20,000 per year, the employer pays 100% of the healthcare premium. 2. If an employee makes more than $20,000 per year, the employer pays 50% of the healthcare premium for the employee. Ambiguity: What if an employee makes exactly $20,000?

(Bender/Wiegers)


Ambiguity of Reference Ambiguous Adjectives Ambiguous Adverbs Ambiguous Variables Ambiguous Verbs

above ordinary infrequently the database derive

below rare intuitively the field determine

it same just about the file edit

such seamless more often than not the frame enable

the previous several more or less the information improve

them similar mostly the message indicate

these some nearly the module manipulate

they standard normally the page match

this the complete not quite the rule maximize

those the entire often the screen may

Ambiguous Adjectives transparent on the odd occasion the status might

all typical ordinarily the system minimize

any usual rarely the table modify

appropriate valid roughly the value optimize

custom Ambiguous Adverbs seamlessly the window perform

efficient accordingly seldom Ambiguous Verbs process

every almost similarly adjust produce

few approximately sometime alter provide

frequent by and large somewhat amend support

improved commonly transparently calculate update

infrequent customarily typically change validate

intuitive efficiently usually compare verify

invalid frequently Ambiguous Variables compute

many generally the application convert

most hardly ever the component create

normal in general the data customize

(Bender/Wiegers)


Measure Requirements Effectiveness through Metrics


Impact of Poor Requirements in the Industry

Countless whitepapers have been written documenting the impact of poor requirements on a company’s enterprise success.

• As high as 60% in time and cost premiums are paid on projects with poor requirements

• Fewer than one third of companies are well-equipped to produce quality requirements

• Poor requirements consume approximately 41.5% of the average IT budget due to maintenance and redo work

• Only 20% of projects are delivered on time with 28% within budget

Impact of Poor Requirements

Impact of Requirements to Product Quality* Defects found during requirements $250 Defects found during design $500 Defects found during coding and testing $1,250 Defects found after release $5,000 * May 5, 2009, Capers Jones, Capers Jones & Associates LLC, “A Short History of the Cost per Defect Metric v1.1”

Analysts report that as many as 71% of software projects that fail

do so because of poor requirements management, making it the

single biggest reason for project failure…”

CIO Magazine

Great requirements are all about great process – not about the

document itself. If the process for getting the requirements is

poor, it is likely the document will be poor. If the process is

excellent, it is likely the document will be excellent.

Keith Ellis, IAG Consulting

There is an abundance of stories of failed projects. Research has

shown the reasons for IT project failure in the USA, indicated that

project success rates were only 34%, with the rest of projects

being either “challenged” in some way or failing outright.

Sergey Korban, BA Times


Measuring Requirements Within & Across Projects

• Maintenance as % of IT Budget

• Ambiguities per Requirement

• Ambiguity Heat Map

• Schedule variance due poor or missing requirements

• Project Budget: Estimated vs. Actual

• Volatility due to Scope Changes

• Volatility due to Missing Requirements

• Volatility due to Changes in Requirements

• Volatility due to Technical & Design Changes • Volatility due to Clarifications Required

• Rate of Introduction of Requirements

• Rate of Change Requests

• Rate of Defects with Root Cause as Requirements

• Rate of Post-Launch Defects

• Frequency of Change of Requirements

• Process Design

• Process Compliance

• Tool Compliance

Requirements Metrics


Ambiguity Review Metrics

Top 10 Ambiguities by Type Total

Ambiguous Term 55

Grammar Spelling & Wording 79

Incomplete Requirements 59

Unclear Requirements 57

Total Ambiguities 250

Patterns within the Requirements Document


Program Level Ambiguity Review Metrics Patterns across the Requirements Document


References

Writing High Quality Requirements By Karl E. Wiegers, 2006 The Ambiguity Review Process By Richard Bender Assessing the Impact of Poor Requirements on Companies By Keith Ellis Executive Guide to Evaluating Business Requirements Quality By Keith Ellis Getting Consensus on Business Requirements – Tips and Traps By Keith Ellis The Quest for Good Requirements By Dr. Martin Schedlbauer, 2011 How to Prevent the Negative Impacts of Poor Requirements By Sergey Korban, April 30, 2013 The Business Value of Better Requirements By Karl E. Wiegers, 2006


References

S.M.A.R.T. Requirements By Mike Mannion, Barry Keepence; Software Engineering Research Group, Napier University, April 1995 Tools for Requirements Gathering Sessions By Seilevel Measuring Requirements May Reinforce Bad Behavior By Jeffrey, March 26, 2012 What Questions Do I Ask During Requirements Elicitation? By Laura Brandenburg Questions for Eliciting User Requirements By Karl Wiegers, December 6, 2011 Information Security Requirements Gathering Questionnaire By Andy Leigh, December 15, 2004 How to use interviews to gather requirements By Ulf Eriksson, October 29 October 2012

Thank you for your time Susan Schanta Director Cognizant Technology Solutions 500 Frank W. Burr Blvd. Teaneck, NJ 07666 (201) 478-0571 [email protected] www.cognizant.com

mailto:[email protected]

http://www.cognizant.com/

Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques

Documents

Transcript of Driving Ambiguities Out of Requirements through Stronger Elicitation Techniques