Teaching Requirements Engineering with Virtual Stakeholderswithout Software Engineering Knowledge

Gregor Gabrysiak, Holger Giese, Andreas Seibel and Stefan NeumannSystem Analysis and Modeling Group

Hasso Plattner Institute at the University of PotsdamProf.-Dr.-Helmert-Str. 2-3, 14482 Potsdam, Germany

{prename.surname}@hpi.uni-potsdam.de

Abstract—Teaching requirements engineering (RE) is a diffi-cult endeavor since it must reflect the reality as close as possible.Also, a sense of risk for the students should be included, whilethe supervisors need to be in control all the time. Relatedliterature agrees on the importance of stakeholders for REeducation.

While different potential groups of people were consideredas stakeholders already, we propose a novel approach of usingstudents of non-software engineering disciplines to introduce asemantic gap between software engineering students and thesevirtual stakeholders.

In this paper, we describe our results and experiencesin embedding RE education within a software engineeringlecture for undergraduate students. Also, we report on ourefforts in setting up and conducting requirements engineeringsessions with virtual stakeholders without software engineeringknowledge.

Keywords-Requirements Engineering Education, VirtualStakeholders, Role Playing, Software Engineering

I. INTRODUCTION

The importance of requirements engineering is usually notreflected in today’s software engineering curriculum and theavailability of corresponding lectures and seminars. Thus,graduating students are lacking essential skills necessary tocreate successful software systems, as the skills required byrequirements engineers differ significantly from those of asoftware engineer (cf. [1], [2]).

To create a realistic learning experience, the environmentshould not be overly controlled, since this may create a falsesense of security [3]. Thus, the related literature focusses oncreating such a realistic learning experience, mainly throughthe usage of different role playing approaches [4] to simulatestakeholders (referred to as virtual stakeholders [3]) or byexplicitly exposing the students to the unknowable to fosteran attitude of acceptance [5].

This lack of education is not a lack of motivation. Ratherthe effort necessary for setting up and supervising a suc-cessful and stable requirements engineering course is usuallyquite time consuming and costly. To obtain requirements, theneeds and desires of stakeholders need to be elicited [6].Thus, one of the most commonly discussed issues in re-quirements engineering education is the availability of stake-

holders. The presence of a stakeholder provides the studentswith the possibility to develop for somebody who, e.g., hasa real need and is not a member of the faculty or a fellowstudent. To experience the whole requirements engineeringphase, at least one iteration of elicitation, specification, andvalidation is necessary, which needs to be supervised andevaluated to be able to comment on the students’ results.Also, to emphasize and enforce the effect of feedback duringiteration cycles, surprising results might help. Alas, shockingresults are not recommended [7].

As argued by Polajnar et al. [3], a realistic softwareengineering experience is not only a prerequisite for graspingthe intent of software engineering methods, but also aprerequisite for proper understanding of why the disciplineexists. Thus, our intent was to enable students to experiencethe need for requirements engineering methods by experi-encing the problems these methods try to solve first hand.For one iteration cycle, the goals of our efforts were thefollowing:

G1 the students should experience a semantic gap duringelicitation;

G2 the students should experience consistency issues whensynthesizing information gathered during an interview;

G3 the students should experience the usual problems whenvalidating requirements specified in a way that does notsupport a suitable view for stakeholders.

To achieve our goals, we embedded a requirements en-gineering session consisting of elicitation, specification andvalidation of requirements into an undergraduate lecture onmodeling complex software systems. Based on the lecture’sassignment provided by the authors, groups of students wereexposed to a virtual stakeholder in order to elicit additionalrequirements, to synthesize and specify them, and to validatethem together with this stakeholder.

The contributions of this paper consist of a discussionof our experiences from the introduction of a stakeholdermeeting experience within an undergraduate lecture. Further-more, we present our experiences with virtual stakeholderswho do not have a software engineering background andare usually unfamiliar with the application domain – a

978-1-4244-8786-8/10/$26.00 c© 2010 IEEE Published by the IEEE Computer Society, DOI: 10.1109/REET.2010.5633109

category of stakeholders which is usually considered un-suitable. Moreover, we report on our efforts for planningand preparing requirements elicitation sessions with suchstakeholders.

This paper starts by discussing related work concerningthe usage of real and virtual stakeholders in Section II. After-wards, Section III reports on guidelines for planning sessionswith virtual stakeholders without a software engineeringbackground. Then, the established undergraduate lecturewithin which we introduced a requirements engineeringexperience is discussed in Section IV. The conduct of theplanned education is then presented in Section V. The resultsare finally discussed in Section VI and conclusions are drawnin Section VII.

II. RELATED WORK

Macaulay and Mylopoulos [2] argue that there is aneducational dilemma in teaching requirements engineeringsince it is required to provide a solid foundation in thesubject of matter while exposing them to the inherent uncer-tainties, inconsistencies and idiosyncrasies associated withreal requirements problems. The common way of exposingstudents to such uncertainties is to arrange stakeholdermeetings for elicitation, validation or negotiation for them,since meeting with stakeholders is an important part ofrequirements engineering activities [8].

As pointed out by Polajnar et al. [3], there are two distinctkinds of stakeholders available for teaching requirementsengineering – real and virtual stakeholders.

Real stakeholders have many advantages: they know theirapplication domain, they usually are not software engineersthemselves, thus leading to a semantic gap between them andthe students. Additionally, they are also usually interestedin the results, which motivates them to negotiate with thestudents. However, if a real stakeholder is not satisfied atany point in time, the cooperation might be terminated.While this sense of risk is realistic, coping with the worstcase scenario might not be feasible. Consequently, reportsof successful stable lectures throughout multiple iterations,similar to the cooperation with a local company discussedby Gnatz et al. [9], are rare.

Virtual stakeholders, on the other hand, are normally notfamiliar with the application domain. The same, however, isalso true for an optional software engineering background.These are two important factors which can be consideredwhen looking for persons who are suited to enact virtualstakeholders. However, they usually cannot expect to gainanything from the results of the elicitation and validationsessions. Thus, it might be hard to motivate them accord-ingly for the role they are going to play.

This classification into real and virtual stakeholders wasextended by Callele and Makaroff [7] who introduce a moredetailed distinction between these two kinds of stakehold-ers. Due to the commitment necessary to provide a real

stakeholder, they are instead most commonly simulated bystudents. However, while students of the same class (cf. [1],[10]) or graduate students with industrial experience areconsidered as alternatives, students of other faculties arenot mentioned. Although external students usually needto be motivated differently, e.g. financially, and also needto be prepared for the role of a virtual stakeholder moreintensively than software engineering students, it seemsworthwhile in our opinion. Firstly, it explicitly introducesa semantic gap between the students and the virtual stake-holders, which is more realistic than somebody who hasthe same technical knowledge as the students, even if thesestakeholders try to hide it. Secondly, providing the studentswith “new faces” is also important. The interaction betweenstudents who know each other, e.g., from another lecture,cannot be compared to how they have to behave in front ofsomebody they never saw before.

III. PREPARING VIRTUAL STAKEHOLDERS

Especially for students from other faculties who enactvirtual stakeholders it is important to consider what theyknow and how truthfully they can answer questions duringthe elicitation. As discussed in Section II, students enactingvirtual stakeholders do usually not have enough applicationdomain knowledge to answer elicitation questions truthfully,let alone authenticly. This observation holds for students ofall faculties, although software engineering students as wellas faculty members tend to concentrate on technical detailsor requirements engineering methods (cf. [1], [7]). Thus, itis necessary to introduce the required knowledge to thesepotential virtual stakeholders in order to prepare them toanswer elicitation questions as truthfully as possible. Duringthis discussion, we refer to a stakeholder’s knowledge asfacts. Therefore, stakeholders have knowledge in form offacts that can be elicited by requirements engineers. Thesefacts are either already known to the stakeholder, or theyneed to be learned before the elicitation starts. Otherwise, therequirements engineers are not able to elicit the necessaryinformation from the respective stakeholder.

Due to the fact that virtual stakeholders rarely have thenecessary domain knowledge to be perceived as knowl-edgable by the requirements engineers they need to beprepared for the elicitation. By introducing them to theproblem domain, they can memorize the important factsthat need to be elicited. The amount of facts and implicitrequirements introduced or taught to them is presented asthe set Preparation P in Figure 1. Presenting them morefacts leads to an increased P .1 Consequently, without anypreparation session whatsoever, they have little consistentknowledge about the problem domain.

Furthermore, each person has an individual backgroundwhich is essentially the sum of all experiences, i.e. facts,

1In this paper, we use set theory only to discuss the relationships betweenthe presented sets.

Figure 1: The outcome of an elicitation session with avirtual stakeholder depends on what was presented duringthe preparation session and the individual experience of thevirtual stakeholder in relation to what needs to be elicited.

acquired so far. This is represented by the set PersonalExperience XP in Figure 1. This set also includes the pro-fessional background of each person. If a software engineeris asked about a technical detail, the answer strongly dependson whether or not the problem was already considered and,thus, is somehow included in the personal experience. Moreimportantly, individual virtual stakeholders might alreadyhave insights into the application domain through suchpersonal experience. This allows them to answer certainquestions truthfully, even without being explicitly preparedto answer them. E.g., this might be the case if the elicitationsession considers call center agents and some of the studentsenacting virtual stakeholders have already worked in a callcenter. In any case, enquiring about related experienceshelps to avoid inconsistencies with presented facts and alsoto benefit from potentially more realistic insights into theproblem domain.

The goal of the preparation session is to introduce thefacts necessary to authentically enact a virtual stakeholderto somebody who in turn learns them. Thus, a fact p ∈ Pwhich can be elicited later on has become an element ofXP since it has been learned by the virtual stakeholder.Usually, such a preparation is conducted for multiple virtualstakeholders simultaneously. This is due to the fact, that it isnot feasible to prepare each virtual stakeholder individually,since conducting multiple sessions takes much longer andthe results of such sessions are not necessarily consistent.Thus, a preparation should always conducted with groupsof potential virtual stakeholders.

For an elicitation as part of a lecture, the students gatherinformation according to what they need to know to fulfill

their assignment. In turn, this implies that the assignmentinfluences which facts and information are considered nec-essary. The set Assignment A in Figure 1 represents thesefacts which are implicitly required to fulfill the assignment.Thus, the students should elicit all facts fact ∈ A in orderto be able to complete the assignment. Accordingly, thepreparation of the virtual stakeholders should at least includethe facts necessary to answer the questions implicitly raisedby the assignment. To be perceived as authentic, the virtualstakeholders should be able to answer all these questionswith ease, plus the occasional divergent question along theway.2

On the other hand, the stakeholders’ authenticity de-creases if questions implied by the assignment cannot beanswered. Thus, to avoid this loss of authenticity the stake-holders need to be instructed uniformly on how to handlesuch questions which they do not know the correct answerto. Some inconsistencies can be avoided by instructing themto give a reasonable answer, however, also to tell the studentsthat they are not entirely sure. Otherwise, telling them torefer to other colleagues who are usually in command of ful-filling the corresponding task or providing this informationmight also be viable to at least maintain an authentic appear-ance, e.g., “You should ask Ben. He is usually in charge ofthis.” Generally, the latter option is recommendable for allinformation that were not considered during the preparation,or when dealing with multiple, different virtual stakeholderroles.

A good combination of preparation P and assignmentA would be P ⊇ A. This ensures that all stakeholdersgained enough insights into the problem domain to be ableto answer the questions that are implied by the assignment.While additional information (P \A) are optional, they mightincrease the perceived authenticity, especially for divergentquestions during the session.

General questions which are not considered to be im-portant for the assignment (questions eliciting facts whichare not in A) depend on the preparation P or the personalexperience XP of the virtual stakeholder. If they werenot covered in the preparation, it is improbable that allstakeholders answer the question in a consistent manner.Only if all virtual stakeholders happen to share the sameexperience concerning this questions, it might be answeredconsistently.

The discussed approach is different from using only oneperson who enacts the virtual stakeholder for multiple oreven all sessions. In this case, carry-over effects are to beexpected. Since a virtual stakeholder might learn what seemsto be important for the requirements engineers, the answersmight reflect these insights by becoming more direct and of

2Generally, a checklist of facts that need to be consistently elicited by thestudents can be provided to the virtual stakeholders in order to help themto get the important facts consistent throughout all session of all virtualstakeholders.

a higher quality. Thus, the results of the last sessions arepotentially much better than the results of the first session.The problem of growing experience is usually addressed byusing knowledgable faculty members [7] who are alreadyexperienced and, thus, need to hide their experience or byrelying on students of the same class who only experiencethe corresponding sessions once [1]. For virtual stakeholderswho do not possess this knowledge yet, such carry-overeffects should be avoided by employing multiple persons,one for each elicitation session.

IV. THE LECTURE

The lecture is a second semester undergraduate course,that aims at teaching students how to model a complexsystem within a group of eight to ten students. Participantsalready have basic knowledge about the fundamentals ofmodeling since this lecture is a follow-up event of a firstsemester course about principles of modeling techniques.Within the course an introduction to the Unified ModelingLanguage (UML3), a general purpose modeling languagein the field of software-development, is given. Within eachlecture a specific project in form of a textual problem def-inition is provided. The topics covered during past lecturesranged from workflows within a car sharing company to asystem of autonomous robots for the exploration of unknownterritories. The participants first have to understand and re-formulate the provided problem definition in a more concreteand complete way by creating an analysis document usingthe UML in combination with textual descriptions. Based onthis analysis document, that is used for the description ofthe starting point and the given requirements on the desiredsolution, a design document has to be created. The designdocument has to describe a plausible solution based on theiranalysis results. Up to the point when the design documentis delivered to the organizers, feedback is scarcely providedto the students.

Each group has a tutor assigned, i.e., a student whosuccessfully attended the same course priorly. The tutoris responsible for guiding their efforts, for assisting theoverall coordination of the procedures of the lecture andfor delegating open points concerning the problem definitionas well as questions concerning the UML between thestudents and the organizers of the lecture. As part of thelecture, these tutors can be considered rather as peers whoprovide feedback about the lecture on a meta level thanas stakeholders. Since they have only the same applicationdomain knowledge about the assignment as the students,they advise the students on how to employ UML and whatthey need to consider to fulfill the assignment.

Structured templates for the analysis as well as for thedesign document are provided to the students. These docu-ments define an overall structure, the specific diagrams that

3For more information about the UML see http://www.uml.org.

Table I: Abridged Table of Responsibilities and Roles.Responsibility Student Role

1 2 3 4 5 6 7 8Analysis Document x x x xDesign Document x x x x

Requirements Engineering Block x x x xPresentation of Analysis x xPresentation of Design x x

... x x x x

have to be used for the description of the specific parts andthey provide simple examples how the diagrams can be used.The purpose of these templates is to gain more formallystructured, less ambiguous and well formed documents. Bothdocuments have to be elaborated in a cooperative fashionwhere students are organized in groups of eight to tenparticipants. Predefined responsibilities are chosen by thestudents. Table I shows the different roles for a group ofeight students and the responsibilities associated with eachrole. E.g., the two students enacting the roles 1 and 2 areresponsible for the creation and presentation of the analysisdocument.

The lecture has been offered by two of the authors forfour years now. As mentioned before, detailed feedbackfrom the organizers has rarely been provided before thestudents delivered the design and analysis documents. Thus,due to the dependencies between the design and the analysisdocument inadequately understood or omitted requirementshad a major negative impact on the delivered solutions inform of the design documents.

To provide a more realistic experience for the participatingstudents, an iteration of requirements elicitation, specifica-tion and validation was introduced. This was done to initiallyevaluate the reaction of the students when being introducedto these concepts as well as the possibility to extend thelecture accordingly with more feedback cycles.

A. The Case Study: an Online Supermarket

The task at hand for the students was to model an onlinesupermarket based on requirements provided by the authors.These requirements were specified explicitly to mimic differ-ent perspectives within the assignment document. Through-out the document’s 12 pages, the level of detail changedbased on the topics covered. While the requirements for theworkflows were rather rudimentary as to be expected fromsomeone starting a new business, the restrictions for pass-words of customers were quite thorough. This was done inorder to stimulate the students’ sensibility to different rolesand their different needs within the considered company.

Apart from the requirements that were to be expected foran online supermarket, one special feature was emphasized.While customers usually have to rely on external deliverycompanies to bring the ordered goods at some point in time,customers living nearby are eligible for straight to the pointdeliveries with a scheduling period of half an hour. E.g.,fresh apples can be ordered at 9:37 AM to be deliveredbetween 1 PM and 1:30 PM on the same day. Thus, the

system has to be able to organize deliveries for differentcategories of customers and must be able to arrange time-slots for the delivery accordingly. However, the assignmentcontained only vague indications about potential problems,e.g., “If the goods cannot be delivered in time, the bill canbe reduced by up to 10%.”

B. Extension for the Requirements Engineering Sessions

During the first lecture, it was announced that an evalua-tion of the results of the individual groups would take placewith an external stakeholder. Based on the size of the groups,we decided to limit these stakeholder meetings to fourstudents from each group, since more participants wouldhave introduced more communication overhead within theindividual groups throughout these sessions. Also, to balancethe workload within all groups, not all students were exposedto this experience (36 out of 81 students).

For us, it was important to distinguish between the workinvested into the project and the efforts for the requirementssessions. Everything the students learned about the appli-cation domain was modeled as an extension to the lecture’scase study. Therefore, we introduced the role of a call centeragent, referred to as support agent, into the case study. Asupport agent acts as a negotiator between several other roleswithin the online supermarket, e.g., accounting, warehouse,and the person delivering the goods and the customer. Themain task of a support agent was to negotiate in cases ofproblems or whenever problems might arise. The enactmentof this role was the task of the employed virtual stakeholders.

After the students delivered the analysis of the case study,we coordinated dates for the meetings between the fourstudents from each group and the stakeholders. Then, twoweeks prior to these meetings, an announcement was madeto prepare the students according to their case study: “Thecompany responsible for the project you all are workingon is able to send someone over whose role within theconsidered online supermarket was not sufficiently presentin your original assignment. Thus, to avoid errors due toomission, this person will be available for an interviewsession as well as a validation session afterwards.” Thewhole requirements engineering block was planned for fourhours which were allocated as follows:

1) Requirements Elicitation: The supervisor introducedthe four students to the stakeholder and proposed to refer toeach other on a first name basis. The students were instructedto work problem-oriented only, i.e., without focussing onsolutions at all. Thus, their main objective was to elicitenough information from the stakeholder to understand whatthey do and how they do it. The students had 50 minutesto interview the stakeholder about his or her role within thecompany of the case study, including the interaction withother employees of the company. After these sessions, allparticipants including the stakeholder were asked to fill out

a questionnaire designed to evaluate the perceived successof this session.

2) Requirements Specification: Afterwards, the studentshad 100 minutes for synthesizing what they had learned dur-ing the interview. During that time, the stakeholders were notneeded. The students were told to either use a free modelingor a formal modeling approach to specify their findings. Thefocus was on discussing and specifying everything that waslearned about the workflows of the stakeholder in order topresent and validate it with the stakeholder afterwards. Briefexamples of what was produced by the students is presentedin Figures 2 and 3 (German). Afterwards, another question-naire was filled out to evaluate the common understanding ofwhat was synthesized, the students’ satisfaction concerningtheir results, and the students’ team work.

Figure 2: Examples of the synthesized models created usingUML as specification language.

3) Requirements Validation: During the last 45 minutes,the students were asked to present their specifications to thestakeholder. Therefore, the students used projectors, white-boards, or flip-charts for their presentation. The stakeholderswere asked to comment on errors they found concerningwhat they told the students beforehand during the elicitation.A last questionnaire for all participants was handed out toevaluate the success of the validation presentation in termsof found misunderstandings and understandability for the

Figure 3: Example of two groups using pen and paperor whiteboard to model and communicate their findings(enhanced for readability).

stakeholder.All questionnaires used a 5-point Likert scale (1 being

agreement, 5 being disagreement) and free text questions.These questionnaires where handed out to the 36 participat-ing students as well as to the 9 stakeholders.

Since the sessions were distributed throughout 10 days,we asked the students to not tell anyone about the contentof their session to avoid a bias for other participants.

V. CONDUCT

A. The Virtual Stakeholders

As mentioned before, each stakeholder was only involvedin one meeting in order to avoid carry-over effects betweendifferent sessions. Our approach was conducted in order toevaluate the following hypothesis: We can teach studentswithout software engineering knowledge the applicationdomain knowledge necessary to become viable virtual stake-holders.

1) Selection of Potential Stakeholders: Based on the con-straints mentioned in Section II concerning stakeholders, wechose to recruit graduate students explicitly from other facul-ties who do not have any software engineering background.We thereby explicitly introduced the semantic gap betweenthe stakeholders and the students. The potential disadvantageof missing application domain knowledge can be consideredirrelevant, since the application domain was designed to besuitably easy to understand for the stakeholders (see SectionIV-A). Also, we prepared the stakeholders accordingly towork around the potential lack of domain knowledge (seeSection V-A2).

To find suitable stakeholders, we sent an email to thepublic student list of the University of Potsdam whichreaches up to 20.000 students. The email was sent by afaculty member who was not officially involved in thelecture. Thus, we tried to avoid that the students of thelecture might suspect that we are casting stakeholders. In thisemail we searched for people without a software engineeringbackground and stated that “acting talent might help.”

From the student list, we received about 224 applications.We chose nine applications of students using the first-in-first-out principle but under the condition that each one is agraduate student with different background.

2) Preparation of the Stakeholders: To prepare the stake-holders for the role they had to enact, we invited them tendays before the first elicitation date for three hours. Theyhad no software engineering background, therefore, the roleof requirements engineering had to be explained to them inorder to motivate their role and why they need to enact it.Afterwards, we defined requirements suitable for the exten-sion of the case study as discussed in Section IV-B. Aftereverybody understood the case study, their role as a supportagent was explained and we brainstormed potential differentproblems that one could experience between ordering andreceiving goods from an online supermarket. These problemswere then synthesized into different scenarios. Essentially,the stakeholders came up with ten different scenarios ofwhat might happen. They also had to explain their individualunderstanding of these requirements by reusing them in ascenario from the perspective for their role, i.e., speaking inthe first person. This ensured a deeper understanding, createdand limited the stakeholders’ need to bring a checklist intothe elicitation sessions.

Through the articulation of own thoughts during thispreparation, we hoped for a sense of common ownershipwhich we believe to be the basis for a successful adaptionfor the stakeholders. Thus, they might enact their role moreconvincingly when telling the students their scenarios thanthey would do reciting scenarios we gave them.

The general goal of the preparation was to create acommon understanding between the stakeholders what asupport agent has to do. The assignment potentially includedquestions about other roles within the online supermarket aswell. However, the stakeholders were briefed to convincinglytell the students to redirect this question to another personfrom the company. This separation between the supportagents and other roles within the company is illustrated inFigure 4.

Still, to counteract the delay between the preparation andthe last requirements session, a summary of the preparationwas created and sent to the stakeholders as a reminder.

B. Preparation of Students

Apart from a five minute announcement about our slightchanges to the lecture, i.e., that a requirements engineering

Figure 4: The stakeholders were prepared to correctly answeras much of their role as possible in an authentic manner.

session will be conducted this year, the students received noadditional training or education than what was taught as partof the lecture anyway. This was based on considerations ofCallele and Makaroff [7] who argue that requirements engi-neering education might employ shocking or even traumaticevents in order to convey the uncertainty or the sense of risk.Such a shock was not explicitly created. We rather tried tocreate a relaxed situation by, e.g., introducing the studentsand stakeholders on a first name basis.

C. Requirements Elicitation

Due to the lack of training, the interview sessions wereonly partially structured. Nobody was assigned to ask ques-tions or write down the answers, thus, nearly everybodydid both. In one group, the interview grew more aggressiveas all of the four students individually came up with newquestions, asking all the time without letting the stakeholderfinish her sentences.4

Regarding our goal G1 – the students should experiencea semantic gap – we are convinced that the stakeholderswere quite authentic in their roles. Since they were castwithout a software engineering background, they would haveneeded additional training to close the gap that was therealready. When we asked the students whether they had to askmultiple times due to ambiguous answers of the stakeholder,they rather disagreed (x=3.58, s2=0.71). While they alsorather disagreed that they needed to reformulate a questionbased on ambiguous terms multiple times (x=3.47), theirindividual perception was quite inconsistent with a varianceof 1.34.

The perception of the stakeholders differed concerningthe semantic gap. When we asked the stakeholders whetherthey perceived that the students asked multiple times toensure that they understood [the stakeholder] correctly, theyrather agreed (x=2.22, s2=0.94). However, they also ratheragreed that these redundant questions were justified, e.g.,due to inconsistent statements (also x=2.22, s2=0.94). Weconclude that the students were able to bridge the semanticgap surprisingly good without any prior training. Generally,

4As it later turned out, this was not perceived at all by the group.However, the stakeholder graded the results of these students significantlyworse compared to how other stakeholders rated their groups in theirrespective sessions.

the students of the nine individual groups were considered asconfident (x=2.0, s2=0.25), curious (x=2.0, s2=0.5), relaxed(x=1.89, s2=0.61), intelligent (x=1.67, s2=0.25) and compe-tent (x=1.78, s2=0.44). Also, their conduct of the elicitationsession was considered rather authentic (x=2.0, s2=0.5).

Out of the nine groups, only two used the whole interviewduration without any pauses longer than 30 seconds. Threegroups hesitated multiple times for more than 30 seconds,but still continued through the whole 50 minutes. Fourgroups decided to end the interview before the 50 minuteswere over.5 The students were asked whether they requiredadditional interview sessions with the stakeholder directlyafter the interview (rather not with x=3.54, s2=1.15) aswell as after the specification stage (rather not as well withx=3.43, s2=1.07).

Generally, the students perceived the stakeholders as quiteconfident (x=1.83, s2=0.71), intelligent (x=1.94, s2=0.47),competent (x=1.69, s2=0.56) and authentic (x=1.75,s2=0.59). When asked about the authenticity of the atmo-sphere, the students also rather agreed (x=2.11, s2=0.62).These results are also visualized in Figures 5 and 6.

D. Requirements Specification

As mentioned before, the students were told to use dif-ferent modeling approaches, either informal or formal ones.The general goal was to expose them to consistency issueswhen it comes to synthesizing the information they gatheredduring the interview (G2).

As it turned out, most of the students were rather satisfiedwith the specifications they created (x=2.19, s2=0.96). Also,the students were quite confident about the correctness oftheir models (x=2.13, s2=0.46). The students were askedafter the specification whether or not they agreed about thestructures within the company before and after the specifica-tion session. While they rather agreed when the specificationsession started (x=2.25, s2=0.88), there was a significantlyimproved common agreement at the end of the specificationphase (x=1.36, s2=0.35).6 This is also visible in Figure 7.Judging from the results of the free text questions, they mostcommonly discussed about suitable representations. Whencertain details were identified as inconsistent, they usuallyagreed upon the intersection of the different interpretationsand skipped the parts they argued about.

For our goal concerning the specification, we have asignificant increase of common understanding after the

5They ended their interviews after 35, 37, 43, and 48 minutes, respec-tively.

6All groups presented a similar progress of agreement except for onegroup. For this group, a graduate student emerged quite early as the groupleader since he had the most experience. This, however, distorted theirperspective on the agreement process, since all of them stated that theiragreement was the same throughout the whole session (x=1.0, s2=0).Without their unusual agreement values, the mean of the remaining eightgroups is 2.41 (s2=0.77) before and x=1.41 (s2=0.38) after the specificationsession.

Figure 5: The stakeholders (9) and students (36) perceived each other quite similar within their respective groups.

Figure 6: The atmosphere was rated quite positively by students as well as stakeholders.

Figure 7: The agreement among the 36 students increasedsignificantly after they synthesized their interview noteswithin their respective groups.

students synthesized their notes. Additionally, most of thecomments about when and how they needed to compromisedealt with how to handle inconsistencies. Thus, we exposedmost of them successfully to inconsistencies within theirelicited requirements.

E. Requirements Validation

Although all stakeholders were instructed to question whatthe students present to them based on their real understand-ing of what they told the students, the results differed quitea lot. It was our goal to expose the students to the problemof validating specified requirements with a stakeholder (G3).Generally, the stakeholder strongly agreed that they under-stood what the students presented to them (x=1.22, s2=0.19).When asked whether they had to inquire often about whatwas presented they rather disagreed (x=3.78, s2=0.94).However, the questions they had were mainly concerning theunderstandability what was presented (x=2.67, s2=1.5). Notsurprisingly, most of the understandability questions wereasked by stakeholders for students who used UML models(x=1.67, s2=0.33, 3 groups), although the students that usedUML rather disagreed when being asked beforehand whethertheir models are quite technical (x=3.83, s2=0.15).

As a general observation, the validation stage was highlydependent on the stakeholders and their individual capacityfor providing critical feedback. The shortest validation ses-sion took only 13 minutes. After the students explained whatthey understood in ten minutes, they asked the stakeholderwhether he had any comment on what they presented. Whenhe said that his requirements were sufficiently represented

and shortly explained which aspects he liked most, thevalidation was officially finished, since pointing out minorerrors would have diminished the stakeholders authenticityand authority.

VI. RESULTS AND LESSONS LEARNED

Based on goals specified in Section I, we conclude thatour effort of teaching students the basics of requirementsengineering hands-on was partially successful.

When we asked whether the students believe that theinterview and validation session are important for [their]further studies (x=2.86, s2=1.44) or [their] professionalcareer (x=2.75, s2=1.68), only few students seemed con-vinced. Thus, the students do not seem to be sensitizedthat requirements engineering is important for their edu-cational and business career. However, they agreed thatthis experience should be repeated within [their] universitycurriculum (x=1.75, s2=0.65).

Maybe our project raised the sensitivity; however, studentsare primarily focused on providing solutions for given re-quirements. How they get the requirements does not seemto matter. This is also one of the reasons we chose to usean undergraduate course for our approach. As argued byCallele and Makaroff [7], the students have to overcome theconditioning from school, where all assignments were stableand marks were granted on the basis that you only fail if youdo not try. We are convinced that the expected results can beenforced by adding more iteration cycles of elicitation andvalidation with additional time in between for the gatheredassumptions to sink in. Two hours of specification beforethe validation seem insufficient for that to happen.

The feedback that we received shortly after each sessionas well as during a debriefing within the lecture indicatedthat the students were pleasantly surprised and gained in-sights into the problems typically encountered by require-ments engineers. The general satisfaction of the studentswas highly dependent on whether or not they got feedbackimmediately after the session or later. Due to the effortsfor supervising several sessions in parallel, two groupswere unsatisfied with the abridged feedback they received.Thus, the effort was futile in their eyes, until they receivedcorresponding feedback belatedly.

A. Evaluation of Hypothesis

We hypothesized that we could successfully employ stu-dents without a software engineering background as virtualstakeholders. Our conduct seems to be a viable proof ofconcept. The students that acted as virtual stakeholdersseemed to understand their application domain and duringthe elicitation and validation phase they were commonlyable to present their domain and its problems. Additionally,they were also able to discuss the presented results withthe students. Still, while the results concerning our teachinggoals are rather good, the stakeholders themselves did not

feel appropriately prepared according to the questionnairesthey filled out after the interview session. When askedwhether they felt confident enacting their role during theinterview, they did not agree as strongly as we hoped themto (x=2.44, s2=0.28). This might be due to the fact, thatthey were not able to use all the prepared scenarios (Didyou use many of the prepared scenarios? x=2.67, s2=0.5),while at the same time they agreed that they reached the limitof what was prepared (x=2.39, s2=1.36). In these cases, thestakeholders commented in the questionnaires that they triedto evade such questions or improvise something sensible.

In conclusion, the effort spent to prepare the stakeholdersseems to have been underestimated. While we reachedour goals, the stakeholders felt not confident enough. Thismight have been avoided by extending their preparationwith a simulated interview situation. Although all of themapproximately knew what to expect, some of them seemedto be rather unprepared for the students.

On the other hand, exposing them to a simulated interviewmight also present the expected semantic gap too explicitly.In turn, this might diminish its effect during the interviewsession. Thus, a good balance between the need for asimulated interview for the virtual stakeholders and theirgenuine perspective concerning the semantic gap needs tobe found.

B. Perceived Authenticity of Stakeholders

Most of the students admitted after the interview situationthat they were not sure whether the stakeholders were real.Although some students were not convinced, they did notwant to risk any failures in case the situation was real.

Certain aspects worked in favor for the authenticity.Firstly, by introducing new faces, i.e., students they neversaw before.7 Additionally, the students were told to dress ina business casual manner to emphasize their expectationsconcerning the interview session. Although all sessionsbegan with the introduction between the students and thestakeholder on a first name basis, several students started toaddress the stakeholder more formally after a few minutes.

On the other hand, some stakeholders gave inconsistentinformation, e.g., one of them claimed that the supportagents are reachable for 18 hours for six days a week. Fiveminutes later, she also claimed that she was the only supportagent working for this company. The stakeholders were notexplicitly briefed about such details since the assumptionwas that they would be able to answer details from theirpersonal experience in terms of what makes sense. Anothershortcoming that was perceived from the students was theneed of some stakeholders to get visual feedback fromthe supervisor whether the answers during the interviewsession were correct. To avoid irritating the stakeholders the

7The University of Potsdam is distributed over three locations. Whilesome students have to visit all three for their lectures, others spend theirstudies in one location only.

supervisors were told to nod all the time when being lookedat by the stakeholder. This was contrary to the stakeholders’briefing which stated that all facts are acceptable as long asthey are to the best of one’s knowledge and belief.

While the setting (clothing, new faces, etc.) was judgedas suitable, the perceived authenticity was mostly based onthe stakeholder’s character. Thus, the confidence with whichthey were able to tell their stories and to react the same waywhen confronted with potential inconsistencies was mostimportant. As outlined in Section V-C, the authenticity ofthe stakeholders and the atmosphere was rated quite good.

C. Potential Improvements for the Presented Setting

As argued before, the stakeholders felt that they neededto be prepared in a more detailed fashion. Also, the amountof information that could be elicited differed depending onthe additional time they spent preparing at home, rangingfrom shortly thinking about it on the way to the meetingup to three hours of online investigation and practice. Basedon the best practice, additional time has to be scheduled toallow for this kind of preparation. In a controlled setting, itis also possible to induce additional stress for the students,e.g., by telling the stakeholder to drop in after one hourdemanding them to present their results earlier than planned.The scenarios dealt with different complex situations ina simple application domain, but focused on only oneperspective of it. By introducing a second or third role, thestudents should be better able to experience inconsistent andconflicting requirements.

VII. CONCLUSION AND FUTURE WORK

We presented our approach of introducing students fromother faculties, i.e., without software engineering back-ground, to incorporate virtual stakeholders into the require-ments engineering education. We chose students without anysoftware engineering experience to deliberately introduce thesemantic gap between engineers and stakeholders which cantypically only be simulated by all other variations of virtualstakeholders, i.e., faculty members or students from the sameclass. Additionally, we provided preparation guidelines onhow to set up requirements elicitation and validation sessionswith virtual stakeholders. This also includes instructions forthese virtual stakeholders on how to behave when askeddivergent questions they were either not prepared for orforgot the answers to.

As mentioned in Section II, external students might needto be motivated financially. We cast the virtual stakeholdersand hired them according to the tariff of the university. Thus,the whole setup of embedding a requirements engineeringblock into an existing lecture was possible for 1.200 Eurowhich makes it feasible in terms of expenses and results.

To further refine the idea of employing students from otherfaculties and the corresponding guidelines, we are going torepeat the requirements engineering block of this lecture,

only this time with multiple iterations instead of only onecycle of elicitation and validation. This will allow us tomeasure the progress of the students in a more detailedmanner, and thus to further evaluate the effectiveness of ourapproach.

ACKNOWLEDGMENT

The authors would like to thank Christoph Kuhnl for supporting thepreparation and the evaluation as well as our colleagues Stephan Hilde-brandt and Thomas Vogel for supervising some of the sessions. Also, weare grateful for the participation of the stakeholders and the students. Weare also grateful for the anonymous referees’ valuable comments that ledto significant improvements of this paper.

REFERENCES

[1] D. Zowghi and S. Paryani, “Teaching requirements engi-neering through role playing: Lessons learnt,” RequirementsEngineering, IEEE International Conference on, vol. 0, p.233, 2003.

[2] L. Macaulay and J. Mylopoulos, “Requirements engineering:An educational dilemma,” Automated Software Engineering,vol. 2, no. 4, pp. 343–351, December 1995.

[3] D. Polajnar and J. Polajnar, “Teaching software engineeringthrough real projects,” in WCCCE 2004 Western CanadianConference on Computer Education, Kelowna, B.C., Canada,May 2004, pp. 83–90.

[4] A. Blatner. (1995) Role playing in education.www.blatner.com/adam/pdntbk/rlplayedu.htm. Corrected inOctober 2009, Accessed June 2010. [Online]. Available:www.blatner.com/adam/pdntbk/rlplayedu.htm

[5] R. J. Barnes, D. C. Gause, and E. C. Way, “Teaching theunknown and the unknowable in requirements engineeringeducation,” Requirements Engineering Education and Train-ing, vol. 0, pp. 30–37, 2008.

[6] M. Glinz and R. J. Wieringa, “Guest editors’ introduction:Stakeholders in requirements engineering,” IEEE Software,vol. 24, no. 2, pp. 18–20, 2007.

[7] D. Callele and D. Makaroff, “Teaching requirements en-gineering to an unsuspecting audience,” in SIGCSE ’06:Proceedings of the 37th SIGCSE technical symposium onComputer science education. New York, NY, USA: ACM,2006, pp. 433–437.

[8] N. Yusop, Z. Mehboob, and D. Zowghi, “The role of con-ducting stakeholder meetings in requirements engineeringtraining,” in Second International Workshop on the Require-ments Engineering Education and Training in conjunctionwith RE07, Delhi, India, October 2007.

[9] M. Gnatz, L. Kof, F. Prilmeier, and T. Seifert, “A practicalapproach of teaching software engineering,” Software Engi-neering Education and Training, Conference on, vol. 0, p.120, 2003.

[10] D. Rosca, “An active/collaborative approach in teachingrequirements engineering,” Frontiers in Education, Annual,vol. 1, pp. T2C/9–T2C12, 2000.