Post on 25-Aug-2018
In the Beginning . . . - Chal lenges in
Requirements El ic i tat ion
Naudé Scribante
Department of Engineering and Technology Management, University of Pretoria
nscribante@npsc.co.za
Leon Pretorius
Department of Engineering and Technology Management, University of Pretoria
leon.pretorius@up.ac.za
Siebert Benade
Department of Engineering and Technology Management, University of Pretoria
siebert.benade@up.ac.za Copyright © 2015 by Naudé Scribante. Published and used by INCOSE with permission.
Abstract. Requirements elicitation remains one of the most crucial activities in the system
life-cycle. Any errors that occur during this process will most likely lead to cost and schedule
overruns and may in a worst case scenario even lead to a failed project. This paper identifies
where the requirements elicitation process fits into the overall system life-cycle as well as the
characteristics of good requirements before presenting the results of a literature survey as well
as the practical experiences of the author on factors that can negatively influence the quality of
the requirements elicited.
Keywords: Requirements Elicitation, Requirements Engineering, Requirements Management,
Requirements Failures
Introduction
The success of a project is determined greatly by how it is started or initiated. It is estimated that
approximately 60% of the projected life-cycle cost is committed at the end of the system
planning and conceptual design stage (Blanchard and Fabrycky 1990). They further state that
“Regardless of the system type and size, one begins with an identified need and a complete
feasibility study for the purposes of establishing a set of requirements, constraints, and design
criteria” (Blanchard and Fabrycky 1990). From this it is thus clear that the starting point for
any project is a clear set of requirements, constraints, and design criteria that can be traced back
to the identified need.
To understand the importance of these elicited requirements, constraints and design criteria,
one has to look at what becomes of them once they have been elicited. On the one hand, it may
be that the acquiring party 1 does not have an in-house capability to take the elicited
requirements and to develop a system or solution that will satisfy these requirements. In such a
case they will have to contract with an external organization to develop and produce the
required system or solution. In this case the elicited requirements will form part of the Request
1 Acquirer or Acquiring Party: The stakeholder that procures or acquires a product or service (ISO/IEC/IEEE 2011)
for Proposal (RFP) that will be issued to potential suppliers which will then again use them to
base their technical and commercial proposal on with which they aim to solve the need of the
acquiring party. The elicited requirements should also form a critical part of the contractual
documentation as the final delivered system will be tested and accepted against them.
On the other hand, where the acquiring party does have an in-house capability to produce the
system or solution that will satisfy the need, the requirements elicited will serve as input to the
further system engineering process.
Blanchard and Fabrycky also identify in their consumer-to-consumer process model that the
responsibility for establishing this clear set of requirements, constraints, and design criteria lies
with the acquiring party (Blanchard and Fabrycky 1990). In an ideal world, the organization
that has the need that must be fulfilled, will also have the necessary requirements engineering
skills to drive the requirements elicitation process so as to end with a complete set of
requirements that is a true reflection of need as was identified by the stakeholders. Where the
acquiring party does not have the capability to perform the necessary activities, they will need
to make use of an external consultant or a third party to assist them with this process.
This paper identifies where the requirements elicitation process fits into the overall system
life-cycle as well as the characteristics of good requirements before presenting the results of a
literature survey as well as the practical experiences of the author on factors that can negatively
influence the quality of the requirements elicited.
The requirements elicitation process
In order to understand the challenges in the requirements elicitation process, it is important to
first define what requirements elicitation entails and where it fits into the overall system
life-cycle process.
In the INCOSE System Engineering Handbook (INCOSE 2011), the four process groups that
support the system engineering process is identified. These process groups are the
organizational project-enabling processes, the agreement processes, the project processes, and
the technical process.
Within the technical process group, the stakeholder requirements definition process is defined.
The context diagram for this process is shown in Figure 1.
Figure 1: Stakeholder requirements definition process context diagram, from (INCOSE 2011)
The INCOSE System Engineering Handbook define the purpose of the stakeholder
requirements definition process as “… to define the requirements for a system that can provide
the services needed by the users and other stakeholders in a defined environment. It identifies
stakeholders, or stakeholder classes, involved with the system throughout its life-cycle, and
their needs, expectations, and desires. It analyses and transforms these into a common set of
stakeholder requirements that express the intended interaction the system will have with its
operational environment and that are the reference against which each resulting operational
service is validated” (INCOSE 2011).
The activities that are identified in the stakeholder requirements definition process are the
following (INCOSE 2011):
Elicit Stakeholder Requirements
Define Stakeholder Requirements
Analyse and Maintain Stakeholder Requirements
Within the elicit stakeholder requirements activity, two further sub-activities are defined:
Identify stakeholders who will have an interest in the system throughout its entire
life-cycle.
Elicit requirements – what the system must accomplish and how well.
Requirements elicitation is defined by Zowghi and Coulin as “… all about learning and
understanding the needs of users and project sponsors with the ultimate aim of communicating
these needs to the system developers. A substantial part of elicitation is dedicated to uncovering,
extracting, and surfacing the wants of the potential stakeholders.” (Zowghi and Coulin 2005)
An expanded definition of requirements elicitation is that provided by Ahmad. She defines it as
“… the process to systematically extract and identify the requirements of the system from a
combination of human stakeholders, the system’s environment, feasibility studies, market
analysis, business plans, analysis of competing products and domain knowledge.” (Ahmad
2008)
From these various definitions is can thus be seen that requirements elicitation forms part of the
stakeholder requirements definition process as defined in (INCOSE 2011).
Characteristics of good requirements
During the requirements elicitation process, many requirements will be provided by the
stakeholders of which the quality will vary greatly. Care must be taken to ensure that only good
quality requirements are included in the requirement set. Fortunately requirements have certain
inherent characteristics that can be used to distinguish good requirements from bad
requirements. Hooks (1993) identified four criteria that can be used to test a requirement:
a. Need. If there is doubt about the necessity of a requirement, then ask: What is the
worst thing that could happen if these requirements were not included? If you do not
find an answer of consequence, then you probably do not need the requirement.
b. Verification. As you write a requirement, determine how you will verify it.
Determine the criteria for acceptance. This step will help to insure that the
requirement is verifiable.
c. Attainable. To be attainable, the requirement must be technical feasible and fit within
the budget, schedule and other constraints.
d. Clarity. Each requirement should express a single thought, be concise, and simple. It
is important that the requirement not be misunderstood – it must be unambiguous.
Halligan (2012) identified the following additional characteristics over and above the
characteristics identified by Hooks:
a. Correctness: An absence of errors of fact in the specified requirement.
b. Completeness: The inclusion of all necessary information such that if the requirement
is met, the need will also be satisfied.
c. Consistency: A requirement is not in conflict with any other requirement, nor is it
inconsistent internally.
d. Non-ambiguity: There is only one semantic interpretation of a requirement.
e. Connectivity: All of the terms within a requirement are adequately linked to other
requirements and to word and term definitions, causing each individual requirement to
properly relate to each other requirement in a set.
f. Singularity: A requirement cannot sensibly be expressed as two or more requirements.
g. Modifiability: The necessary changes to a requirement can be made completely and
consistently.
h. Balance: A set of requirements forms part of an optimal solution to a higher level
problem.
i. Functional Orientation: The set of requirements state what the system is to do, how
well it is to do it, necessary external interface characteristics, environmental conditions,
constraints and any other required qualities.
The following additional characteristics in addition to those indicated above have been
identified (Faulk 1997):
a. Implementation Independence: The requirement should be free of design and
implementation decisions.
Factors that can negatively influence the quality of the
requirements elicited
From the previous sections it can be seen that while the requirements elicitation process may
seem to be a set of simple activities, only as one delves into the detail does the true complexity
come to light like the proverbial tip of the iceberg.
During the requirements elicitation process other factors can negatively influence the quality of
the requirements elicited as result of that one or more of the criteria for good requirements are
not met.
Use of external specialists or consultants to assist with the elicitation process
Depending on the on availability of the necessary requirements engineering skills within the
company or organization to manage or perform the elicitation process, it may be decided to
make use of external experts or consultants to assist with the elicitation process. While such an
action may bring in the missing skills that are required, it may also decrease the quality of the
elicited requirements due to a number of factors that are discussed in the following paragraphs.
Domain Knowledge. In order to increase the quality of the elicited requirements, the external
specialist or consultant will require a certain amount of knowledge of the nature of the business
and the needs that must be solved. While this person may be able to get a more in-depth
knowledge about the business by interviewing the relevant people, this may not be sufficient in
many cases. The level of familiarity that the external specialist or consultant requires of the
business does not only include familiarity with the operation of the business, but also extends to
domain knowledge that describes the nature and culture of the organization (including specific
terminology and abbreviations that may be used within the organization).
A number of actions can be taken by the external specialist or consultant to improve their
domain knowledge. In a study done by Hadar, Soffer, and Kenzi (2014) they identify a number
of recommendations for analysts. They recommend that analysts who lack domain knowledge
learn the domain terminology before starting with the requirements elicitation sessions. They
should in addition engage support within the organization for the purposes of both obtaining a
complete understanding of the customer’s needs and communicating with the customer during
the elicitation process.
An alternative method that the external specialist or consultant may use to enhance their
understanding of the nature of the client’s business is described by Kaiya and Saeki (2006).
They suggest that the external specialist or consultant describe the domain knowledge in terms
of a domain ontology2.
On the other side of the coin Hadar, Soffer, and Kenzi (2014) found that analysts that do have
the necessary domain knowledge should avoid fixation and preconceptions that may lead to an
incomplete and inaccurate understanding of customer’s needs.
2 In computer science and information science, an ontology is a formal naming and definition of the types, properties, and interrelationships of the entities that really or fundamentally exist for a particular domain of discourse. It is thus a practical application of philosophical ontology, with a taxonomy. (Wikipedia 2015)
Customer introduced misinformation. Depending on the organizational structure it
may happen that the person, or the team, that is driving the requirements elicitation process may
not have access to, or include the actual end-user (“the guy on the ground”) that will be using or
operating the system. This type of situation is more likely to occur in organizations where a
strong hierarchical structure exists such as a large corporate/government organization or in a
military organization. In such a case the external specialist or consultant may only interact with,
or have access, to middle or senior management that may have a perception of what the needs
are but do not have actual first-hand experience.
An additional risk area that the consultant or analyst should avoid is the situation where certain
representatives of the customer may be trying to influence the outcome of the requirements
elicitation process in a certain direction. This situation will typically occur where the
representative is trying to favour a specific solution during the requirements elicitation process.
In order to counter this type of situation, the external specialist or consultant should ensure that
they are familiar with the business of the organization, as was described in the previous section.
This familiarity should enable the external specialist or consultant to identify all of the relevant
stakeholder or groups of stakeholders and ensure that they are included in the requirements
elicitation process. Techniques that allow for the cross verification of stated requirements
should also be used so as to try and eliminate ambiguities and identify manipulated
requirements.
Consultant (analyst)-induced misinformation. Another source of error that may
influence the accuracy of the requirements elicited from stakeholders are the so-called
“consultant-induced misinformation effect” as described by Appan and Browne (2012), where
misinformation is defined as distorted, false, or other erroneous and misleading information. In
their paper they identify that this misinformation effect may lead to the user to recall
misinformation that may have been introduced by the analyst rather than their true beliefs and
knowledge of fact. The overall effect of this misinformation is to reduce the correctness of the
requirements elicited.
In order to reduce the chance of consultant-induced misinformation, Appan and Browne (2012)
have identified that the choice of the elicitation technique is important, given the relative
strengths and weaknesses of interviews and surveys to yield accurate information. If the
consultant does decide to use an interview as an elicitation technique, they must take care to
remain neutral during the requirements elicitation process to reduce the so-called “demand
effect”. This “demand effect” relates to the effect where people being interviewed tend to
respond with an answer that they think that the person conducting the interview wants to hear,
rather than to answer from fact.
Implementation independence. One characteristics of a good requirement that was
identified earlier in this paper, is that the requirements must be implementation independent. If
this is not done, the risk exists that the requirements presented at the end of the process may be
skewed to favour a specific solution. This type of error may occur where the person responsible
for the requirements elicitation process has been exposed to a particular type of solution or may
even be representing a specific organization that may have a vested interest in a particular
solution.
Care must be taken when identifying external specialist or consultant to assist with such a
requirements elicitation process, so as to try and select an independent organization or
individual. This is specifically true in a competitive acquisition environment to prevent the
external specialist or consultant from becoming the supplier of the system or solution since they
are the only qualifying organization.
Attainability. It is often found that in a competitive acquisition environment such as in
government organizations that a preliminary set of requirements are issued in the form of a
Request for Information (RFI) to potential suppliers. The purpose of this exercise is normally to
enable the acquiring party to establish a budget for future acquisition as well as to benchmark
their requirements against what is available in the market.
While this may be a good method to ensure that the final set of requirements reflect the “State of
the Art” available in the market, care must be taken that the responses of the individual vendors
or suppliers are maintained in context. The risk here is that the team responsible for eliciting the
requirements may attempt to create a so-called “Best of Breed” set of requirements by picking
the best specifications of the various vendors and combining them. In many cases, the
specifications of specific items are interrelated and are an optimum combination to achieve a
specific behaviour. Where these “Best of Breed” type specifications are eventually put out for
acquisition, they may not be attainable and could lead to a failure of the project.
The effect of the human side of the stakeholder in requirements elicitation
The role of human nature in requirements elicitation. It was already previously identified
that the need that must be satisfied is represented by the various stakeholders. Ahmad (2008)
states that when dealing with these stakeholders, it is inevitable that conflict will occur due to
the fact that each one of them as individuals, have their own perspectives and perceptions of
what the need is. However as stakeholders and thus a representative of the end users, they may
have different concerns, priorities and responsibilities. This duality in the nature of the
stakeholder is shown in Figure 2.
Figure 2: Duality in the nature of the stakeholder
Fuentes-Fernández, Gómez-Sanz, and Pavón (2010) identifies that the human context within
which a system will operate, as being fundamental for its requirements. While this may not
seem to be related to the requirements of the system, it may however play a big role in achieving
its successful adoption of the system. They further identify that a gap may exist in the skill set of
those that are performing the requirements elicitation as they may rather have a background in a
technical discipline and may not be trained to elicit this kind of information.
The social nature of requirements elicitation. The social nature of the requirements
elicitation process are identified by Chakraborty, Sarker, and Sarker (2010). They identify that
the requirements elicitation process involves collaboration between the various stakeholders as
well as those responsible for the requirements elicitation process. During this collaboration
process, knowledge regarding the system requirements are shared and discussed in order to
construct a shared understanding of the requirements. Chakraborty, Sarker, and Sarker (2010)
reason that collaboration and knowledge sharing within the requirements elicitation process can
been characterised as difficult due to the fact that the various groups contribute very different
kinds of knowledge and experience to this activity, and due to this, trust among the different
parties cannot be guaranteed.
Communication. Communication plays a vital role in the requirements elicitation process. In
addition to the normal communication aspects that are inherent in the human and social nature
of requirements elicitation, specific problems may be encountered where projects are executed
over international borders. In these type of projects the project teams responsible for the
requirements elicitation process may not only have to deal with a lack of face-to-face
communication, but also with other issues such as different time zones and cultural diversity
that may lead to misunderstanding and even conflict in the process (Aranda, Vizcaíno, and
Piattini 2010).
An additional problem that may be encountered in multi-national projects is that where the
main communication language used in the project execution may not be the native language of
all of the participants. This type of situations could very well lead to misunderstanding of
Need
Requirement 1
Requirement 2
Requirement n
Expressed by Expressed by
Exp
resse
d b
y
Stakeholder
Stakeholder
Stakeholder
Repre
sended b
y
Represended by
Represended by
Own perspective and
perception of the problemOwn perspective and
perception of the problem
}Own
Concerns
Priorities
Responsibilities
questions or discussions in the requirements elicitation process with the resulting effect of
incorrect, incomplete or ambiguous requirements being identified.
Legacy requirements
In many cases where new projects are initiated, the purposes of these projects are to either
upgrade or replace existing system in order to adapt to new business challenges or changes in
the technological landscape. In these cases, it is important to as such “rediscover” the legacy
requirements upon which the current systems are based.
Rayson, Garside, and Sawyer (1999) identifies a risk that key requirements that are implicit in
the legacy systems may go unsupported in the new or upgraded systems. They identify a further
risk that also exists in that if the functionality of the legacy system is known, but not the original
requirements that drove the functionality; this may lead to costly solutions where redundant
functionality is retained at the risk of not having it available when needed. This risk exists
because business change often takes place against a background of poor organizational
memory.
If the external specialist or consultant fails to take legacy requirements into consideration, this
can typically lead to defects in terms of the completeness and consistency of the elicited
requirements.
Conclusion
The fields of requirements engineering and requirements elicitation have been the subject of
study for an extended period of time and many papers and books have been written on the
subject. While requirements elicitation may seem to be a simple process, it is clear from the
literature that there are many hidden traps that can influence the process from yielding the
required desired result.
From the various literature sources it can be concluded that the requirements elicitation process
forms part of the stakeholder requirements definition process as is defined in the System
Engineering Handbook. This is important as it confirms that the requirements elicitation
process should occur at the start of the project.
A number of different literature sources provided an overlapping sample of characteristics of
good requirements that made it possible to define a representative list of characteristics.
A number of different factors that can influence the quality of the requirements elicited was
identified both from literature as well as from the practical experience of the author. The one
common thread that that could be identified through most of the factors discussed was that the
human involvement was the most prone to introducing quality defects in the elicitation process.
This is in a large part due to the fact that the requirements engineering and requirements
elicitation involves a much wider field of study than just the obvious technical aspects. The
person tasked with managing the elicitation process must thus not only be skilled technically
but must also have the added skills in dealing with the various human aspects required to ensure
the accurate elicitation of the requirements.
References
Ahmad, Sabrina. 2008. “Negotiation in the Requirements Elicitation and Analysis Process.” In
Proceedings of the 19th Australian Software Engineering Conference, 683–89.
doi:10.1109/ASWEG.2008.6.
Appan, Radha, and Glenn Browne. 2012. “The Impact of Analyst-Induced Misinformation on
the Requirements Elicitation Process.” MIS Quarterly 36 (1): 85–106.
Aranda, Gabriela N., Aurora Vizcaíno, and Mario Piattini. 2010. “A Framework to Improve
Communication during the Requirements Elicitation Process in GSD Projects.”
Requirements Engineering 15 (4): 397–417. doi:10.1007/s00766-010-0105-9.
Blanchard, Benjamin S., and Wolter J. Fabrycky. 1990. System Engineering and Analysis. 2nd
ed. Englewood Cliffs, New Jersey, 07632: Prentice-Hall International, Inc.
Chakraborty, Suranjan, Saonee Sarker, and Suprateek Sarker. 2010. “An Exploration into the
Process of Requirements Elicitation : A Grounded Approach.” Journal of the Association
for Information Systems 11 (4): 212–49.
Faulk, S.R. 1997. “Software Requirements: A Tutorial.” In Software Requirements
Engineering 2nd Edition, edited by R Thayer and M Dorfman, 1–22. IEEE Computer
Society Press.
Fuentes-Fernández, Rubén, Jorge J. Gómez-Sanz, and Juan Pavón. 2010. “Understanding the
Human Context in Requirements Elicitation.” Requirements Engineering 15 (3): 267–83.
Hadar, Irit, Pnina Soffer, and Keren Kenzi. 2014. “The Role of Domain Knowledge in
Requirements Elicitation via Interviews: An Exploratory Study.” Requirements
Engineering 19 (2): 143–59. doi:10.1007/s00766-012-0163-2.
Halligan, Robert (Project Performance International). 2012. “Requirements Analysis That
Works” PPI-005261: 1–8.
Hooks, Ivy. 1993. “Writing Good Requirements (A Requirements Working Group Information
Report).” In Proceedings of the Third International Symposium of the INCOSE - Volume
2, 2:1–11. http://scowen.asu.edu/Additional Reading/Writing Good Requirements.pdf.
INCOSE. 2011. Systems Engineering Handbook. Edited by Cecilia Haskins, Michael Krueger,
David Walden, and R. Douglas Hamelin. INCOSE,. 3.2 ed.
ISO/IEC/IEEE. 2011. “Systems and Software Engineering — Life Cycle Processes —
Requirements Engineering (ISO/IEC/IEEE 29148).” ISO/ IEC/ IEEE. International
Organization for Standardization, International Electrotechnical Commission, IEEE.
doi:10.1109/IEEESTD.2011.6146379.
Kaiya, H., and M. Saeki. 2006. “Using Domain Ontology as Domain Knowledge for
Requirements Elicitation.” 14th IEEE International Requirements Engineering
Conference (RE’06). doi:10.1109/RE.2006.72.
Rayson, Paul, Roger Garside, and Pete Sawyer. 1999. “Recovering Legacy Requirements.”
http://comp.eprints.lancs.ac.uk/187/1/rgs99_refsq.pdf.
Wikipedia. 2015. “Ontology (information Science).” Accessed July 23.
https://en.wikipedia.org/wiki/Ontology_(information_science).
Zowghi, Didar, and Chad Coulin. 2005. “Requirements Elicitation: A Survey of Techniques,
Approaches, and Tools.” In Engineering and Managing Software Requirements, 19–46.
Springer Berlin Heidelberg.
Biography
Naudé Scribante graduated from the University of Pretoria in 1986 with a B Eng (Electronic
Engineering) degree. He subsequently completed his B Eng (Hons) (Electronic Engineering)
degree in 1989 and his M Eng (Engineering Management) degree in 1993, all from the
University of Pretoria. He started his engineering career at the Ground-to-Air Missile division
of Denel Dynamics and subsequently moved to GEW Technologies in 2000 where he is
currently the Chief Systems Engineer in the EW Systems Department. He is a registered
Professional Engineer with the Engineering Council of South Africa. He is currently enrolled
for a PhD degree in Engineering Management at the University of Pretoria under the study
leadership of Professor Leon Pretorius.
Leon Pretorius has more than 37 years professional, engineering, academic, research and
academic management experience. He is currently Professor in the GSTM at the University of
Pretoria. He is also coordinator of the Research Group for Systems Energy and Innovation at
the University of Pretoria. He has authored or co-authored more than 180 peer reviewed
research papers in international journals and conference proceedings. He was supervisor for a
total of approximately 170 Master and Doctoral theses. This includes the supervision of more
than 130 successful Master’s dissertations and 42 completed Doctoral theses. He is a registered
Professional Engineer (Pr Eng) with the Engineering Council of South Africa (ECSA). He is
furthermore an Honorary Fellow of the SAIMechE, member of ASME as well as member of
IEEE. He is rated as researcher by the National Research Foundation (NRF) in South Africa. He
has received ESKOM Tesp Grants for more than 20 years.
Siebert Benade participated in and managed a variety of projects for 34 years in government,
parastatal, private sector and academic environments. He worked in different areas, e.g.
product/system development, logistics, procurement, projects and information management
areas for 13 years. He was also involved in business consulting for 7 years primarily in
Enterprise Architecture, process and information design and modelling. He is Programme
Director of the Master’s Programmes in Engineering, Project and Technology management at
the GSTM at the University of Pretoria since 2000. He teaches System Engineering on Masters
Level and runs different company-specific Continuing Education programmes. He was
instrumental in obtaining international accreditation for academic programmes.
Siebert supervised more than 50 Masters students’ research projects. He participated in 34
accredited conferences and have ten papers published (or co-published) in peer reviewed
journals.
He obtained a PhD in Engineering Management from the University of Pretoria in 1997. He
was INCOSE President of the South-African Chapter during 2009.