A Simulation Model for eLearning: Course Planning in the Participative Web
-
Upload
stephanie-frost -
Category
Documents
-
view
220 -
download
0
Transcript of A Simulation Model for eLearning: Course Planning in the Participative Web
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 1/22
A Simulation Model for eLearning:
Course Planning in the Participative WebCMPT 858 Term Project
Regular Session 2010-2011, Term 2
CMPT 858: Topics in Modeling and Operations Research
Instructor: Dr. Nathaniel Osgood
Due April 25, 2011
Project by: Stephanie Frost
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 2/22
A Simulation Model for eLearning:Course Planning in the Participative Web
Contents
1 Introduction & Background 2
2 AIED Literature & Research Context 4
3 Description of Simulation Model Structure 53.1 Environments and Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.2 Agents: Learner Agents, Learning Object Agents . . . . . . . . . . . . . . . . . . . . . . . . . 8
4 Model basis in iHelp, a Course Management System 11
4.1 iHelp data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3 How to Quantitatively Measure whether “Significant Learning” occurs . . . . . . . . . . . . . . 13
5 Baseline Scenario description 14
6 Sensitivity Analysis 14
6.1 Comparing Affective States with and without Learning Objectives . . . . . . . . . . . . . . . . 15
6.2 Parameter sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.3 Structural sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
7 Experiments, Interventions, ”What if” scenarios 18
8 Conclusion 19
1 Introduction & Background
Instructors face many kinds of uncertainty while preparing to teach a course: Will the students understand
the materials presented? Which topics and in what order should they be presented? How will student
individual backgrounds and interests affect the direction of the course? Can this diversity be leveraged to
create unique and meaningful learning experiences?
Following research about best teaching practices1, classrooms are increasingly being transformed from
places where students listen passively into places of activity and interaction. Teachers are perhaps further
challenged with the appearance of social media: these tools are presented as a means to increase the
degree of interaction among students, both inside and outside the classroom. But, how?
Suppose ten students all have a different comment or question about an article posted by the teacher.
The experience in this situation would be arguably different than within a class where only a single student
1In [Bain, 2004] [2], an outstanding teacher is said to be one who has made a sustained, substantial, and positive influence on howtheir students think, act and feel. Bain and colleagues performed a study to pinpoint techniques used by the best teachers to create a“natural critical learning environment”. One of many techniques is to involve students in class or small group discussions.
CMPT 858- Regular Session 2010-2011, Term 2 Page 2 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 3/22
A Simulation Model for eLearning:Course Planning in the Participative Web
shared a reaction to the same article. By studying the behaviour around the creation and re-use of Learning
Objects (i.e. anything that a student can use for their learning) we can start to see how to use technology
to scaffold the learning environment with supportive activities, suggestions or chunks of knowledge at just
the right time to try and optimize the learning environment.
In this project, we developed a simulation model to begin to explore such questions. The model repre-
sents the materials in an online course, a social network of students, and some rules about an individual
students affective state and how this affects their actions: browsing, creating and interacting with course
content. The goal of the model is to help instructors predict how students might navigate through the learn-
ing environment that the instructor is designing. The context of our work is within Artificial Intelligence in
Education (AIED) as we seek to build individualized, adaptable educational environments. Findings from
this model’s results could inform the design of future Intelligent Tutoring Systems.
We emphasize that the objective of our simulation model is not to seek to control the behaviour of
learner-agents, but, rather to learn how to best scaffold their experiences for maximum significant learning,
and to show quantitative measures for a “natural critical learning environment” [Bain, 2004] [2]. For these
measures, we present a series of measurements based on Fink’s Taxonomy of Significant Learning, dis-
cussed in section 4.3.
Our simulation model’s baseline scenario shows learning outcomes when students are given a “struc-
tureless” curriculum, that is, an empty set of learning objectives. Different intervention scenarios can provide
quantitative learning outcome measures for different assignments of learning objectives. Teachers might
ultimately decide to focus on a set of learning objectives in their class with maximum learning outcome
measures in the simulated model. We explore the balancing act between individual learning goals and
overall learning goals of the group. We also see that the knowledge artifacts created by learners can impact
the browsing experience of other students.
Although it was not possible to explore all of the questions uncovered in this modelling experience, we
can use the model structure that we have developed to give form to our future work. The model gives us
the ability to quantify a multitude of subtle questions that could not be measured in a real study, and might
be used to narrow down the important, specific questions that warrant a future real-world study.
In this report, will share the insights we gained during the model development process. Course planning
CMPT 858- Regular Session 2010-2011, Term 2 Page 3 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 4/22
A Simulation Model for eLearning:Course Planning in the Participative Web
can and should take into account the personal backgrounds and interests of each student, while providing
group activities that allow each person to discover more than they could on their own. Our work has helped
us ask how to plan the best group activities that have have great meaning at the individual level. Although
these questions are not new themselves, to ask them in a precise computational manner gives exciting
leverage for future work in AIED.
2 AIED Literature & Research Context
AIED researchers commonly strive to give their systems the ability to reason over the knowledge that a
learner is trying to absorb themselves while using the system. The term, “task domain” refers to the con-
cepts that the learner is studying; the term was introduced in [VanLehn 2006] [15], and many different
knowledge representation techniques have been used to encode task domain knowledge. In our model,
the task domain ontology is only indirectly referenced through their association with Learning Objects. Our
Learning Objects are considered agents, much like the ”Application Agents” in [Vassileva et al., 2003] [16].
In order to simulate different teaching strategies in the future, we expect that we will need to have more ex-
plicit knowledge representation of the task domain ontology. Similarly, modelling human teaching strategies
gives insight on different ways an Intelligent Tutoring System (ITS) and a learner can interact. Boulay et al.
present a survey paper in this area [du Boulay et al., 2001] [6]. Our model drastically abstracts the teaching
process as ”to assign learning objectives”.
In contrast to the deep modelling of task domain ontology and specifics of human teaching, the Eco-
logical Approach suggests that meticulous metadata is not necessary, rather, to quote from the abstract in
[McCalla, 2004] [10], “In a phrase, the [Ecological] approach involves attaching models of learners to the
learning objects they interact with, and then mining these models for patterns that are useful for various
purposes.” It is not the metadata markup that is useful, but rather how the object ends up getting used
after all. Throughout this project, we have certainly asked ourselves whether we could and should leverage
knowledge from educational psychology and instructional design to build effective eLearning environments:
What should be modelled for use in an effective eLearning environment?
Our model looks at the macro scale in terms of many learners, but work has gone into modelling a
Learner’s internal understanding of a subject, such as in[Conati et al., 2002] [4]. We ask whether under-
standing within an individual’s mind could impact decisions at the macro level, like how to structure group
activities over the course of a term. To guide future investigation about how agent-specific interactions
CMPT 858- Regular Session 2010-2011, Term 2 Page 4 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 5/22
A Simulation Model for eLearning:Course Planning in the Participative Web
might inform more effective group learner activity, we note two techniques from AIED that show how a
system could interact with an individual learner over time. First, Model Tracing [8] pre-programs a rela-
tively specific course plan and encourages the learner to follow expected behaviour in order to complete
an activity. Second, the Constraint Based Tutoring Approach [Mitrovic et al., 2001] [12] matches a stu-
dent’s solution during a tutoring session to a set of constraints that describe a correct solution. Tailored
feedback can be given to the student based on the particular constraint violated in their answer. Student
answers that do not violate any constraints are correct solutions and thus do not require hints or corrections.
Our model does not have any representation for the individual steps in a particular problem, but in future
work we are interested in asking whether a student’s experience at a fine-grained level could or should
impact the system’s responsiveness to their personalized curriculum at a broader level. Additionally, we
note that research into the concept of curriculum in an Intelligent Tutoring System prior to 1990 is surveyed
in [McCalla, 1990] [11].
3 Description of Simulation Model Structure
This section will describe the major components of the model: The network of Learners, the network of
Learning Objects, the attributes of these two types of agents, and the data sets used to measure dimensions
of significant learning experiences.
3.1 Environments and Social Networks
The key structural elements of the Main part of our model are shown in Figure 1. Our model uses two envi-
ronments: The Learning Object environment houses a population of Learning Object agents, connected by
content relations such as is-a, part-of, prerequisite, etc., discussed further in [Wasson, 1991] [18]. Content
relations are important considerations when making pedagogical decisions about what options to recom-
mend to a learner given their past experience. The Learning Object Agents reside in a continuous environ-
ment using a Spring Mass layout. We selected this configuration because we wanted the system to show
the course content on the screen such that concepts that share epistemic relationships are shown close
together.
Like traditional classrooms, some eLearning classrooms have a synchronized curriculum where all stu-
CMPT 858- Regular Session 2010-2011, Term 2 Page 5 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 6/22
A Simulation Model for eLearning:Course Planning in the Participative Web
Figure 1: - Main structural elements The learnerAgents population is associated with the learners environment
and the learningObjectAgent population is associated with the objects environment. The goal of our model is to help ex-
plore the relationship between content, knowledge of teaching processes, and individual states like Focused, Stressed,
Confused. All of these things are reflected in the creation and consumption of Learning Objects by Learners.
dents in the class are expected to be working on the same topic at roughly the same time. However, we
also wish to explore the pedagogical possibilities around a self-directed learning environment, where learn-
ers may come-and-go, starting the material at different times and working through it at different paces.
We discovered that the modelling process has helped to refine the questions around alternative curricu-
lum scheduling, and to uncover questions about pedagogical techniques. A“time synchronized” curriculum
might take the form of a series of Events in AnyLogic, where an event corresponds to an assignment of the
same Learning Objective to each student, perhaps with an associated deadline. The opposite situation is
when learners are not coordinated at all. Even at the individual level, [Vassileva et al., 1996] [17] describe
the “Locus of Control”, whether the Learner has most control over the direction of the lesson, vs. whether
the computer takes control. However, we are uncertain whether this has been explored in a quantitative
manner.
The second environment in our model represents relationships between Learners. In Figure 2, Learn-
ers are represented with circles and Learning Objects are represented with rectangles. We designed our
system to be able to represent relationships like “follows the research of”, or “is a student of”, or “holds
authority over”” although we have not yet leveraged this structure in our analyses. These types of relation-
CMPT 858- Regular Session 2010-2011, Term 2 Page 6 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 7/22
A Simulation Model for eLearning:Course Planning in the Participative Web
ships could factor into our algorithm for Learner behaviour that determines whether they choose to share
material with a particular person, or whether they would follow the recommendation of a particular person.
A study was conducted by [Daniel et al., 2008] [5] that unveiled the circumstances under which students
share materials. For example, it was found that people are mostly unwilling to share knowledge with people
they hardly know, and that people would not share knowledge in an environment where competition instead
of cooperation is encouraged and one in which the notion of “knowledge is power” is maintained. Putting
social relationships into our model could also help the system cluster students together for group activities.
Figure 2: - Two EnvironmentsWhen a Learner (circle) is ”browsing” a Learning Object (rectangle), then the Learner
icon is shown on top of the Learning Object icon. As the simulation runs, you see the circles appearing and disappearing
on top of content nodes. If a Learner creates a new artifact such as a blog entry or Twitter (www.twitter.com) message,
you will see the underlying Learning Object network grow a new node from the Learner’s current location.
CMPT 858- Regular Session 2010-2011, Term 2 Page 7 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 8/22
A Simulation Model for eLearning:Course Planning in the Participative Web
3.2 Agents: Learner Agents, Learning Object Agents
Our model has two types of agents: Learner Agents and Learning Object Agents. The Learners represent
students taking a class (or, trying to learn roughly similar things in roughly similar space). It would have
been interesting to explore sub-typing of Learners, for example: those taking a specific class vs. “Lay
Person” learners who would share interactions and be influencing the environment themselves, but not be
subject to the curriculum goals set by a teacher. We acknowledge Sterman’s advice: Model the problem,
not the system. [Sterman, 2000] [14] (p. 79) However, it remains to be explored how Lay Persons can
indeed become an important part of the Education process. Our model did not explore the concept of
Teacher agents, either. Our model considers a single course; it may be interesting to consider a more
macro scale - such as the planning of “The First Year experience” at a particular institution, with different
teachers assigning learning objectives to overlapping subsets within a population of learners.
Each Learner Agent has two sets of learning objectives: those that are “imposed” by a teacher and
those that are self-selected by the Learner. In our final analysis, we only used the first category, but think
that further exploration of both is warranted because of the strong impact of internal vs. external motivators
in an educational setting.
Figure 3: - Affective States A Learner’s current affective state is relative to a particular learning Object. Further
exploration is required around whether these should indeed be relative to a learning object, or should refer more
generally to the current learning objective, or perhaps both at the same time. For example, a person might feel confused
about the concept of subtraction, but be wildly enthusiastic about a particular YouTube video that illustrates subtraction.
This might inform the selection of the next tutoring action to be taken with the student, or, in simulation model, could
yield a high score in a significant learning measure.
CMPT 858- Regular Session 2010-2011, Term 2 Page 8 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 9/22
A Simulation Model for eLearning:Course Planning in the Participative Web
Emotion is an important part of learning, and researchers are finding ways to model emotion in computer
tutors (examples: [Woolf et al., 2007] [19] and [Afzal et al., 2007] [1]. In our model, we based the Learner
state chart on a selection from the emotions taxonomy of Baron Cohen (as presented in [ 1]) as well as the
categories in a study for emotional intelligence for computer tutors in [ 19].
The emotional states are relative to the current-topic-of-study in the environment. For example, when an
individual learner is ”Stressed”, this will be in relation to a particular topic. If we were living up to our aspi-
ration of implementing Ecological approach, whenever we add a Learning Object to an individual’s history,
we would also save a copy of the emotional state at the time. This would allow pattern-matching later on,
for example, ”Give me all of the Learner Agents who were feeling Angry-frustrated when they were visiting
this particular Learning Object”.
At each Learning Object visit, each Learner could interact with the environment in three ways, depending
on their state:
1. browse: following links, traversing the landscape by content-connection. This is achieved with the
Learner Agent’s browseTaskDomain() function.
2. warp to a Learning Object that corresponds to one of their goals (i.e. Learning Objectives).
3. create new learning object: remain stationary and produce a new learning object
The combination of activities above that a Learner takes at a given moment is impacted by their affective
state (Figure 3). For example, if a Learner is Attentive-Confident, then they will create a new learning object
and warp to the next objective. A Learner who is Bored-MentalFatigue will wander around without producing
anything. Each affective state is described below.
Attentive / Confident Default starting state - Further study is needed to more accurately state conditions
under which learners could transition to attentive confident, and we expect this would vary among
individuals. Currently, we say that a Learner remains in this state so long as learning objects aren’t
empty (!(learningObjectives prescribed.isEmpty()))
Confused / Unsure - When visiting a new Learning Object, we say that learners will become Confused /
Unsure 25% of time upon visiting a learning object.
Frustrated / Angry Learners become Frustrated/Angry if there are no learning objects in the system that
suit their learning objective. This check is performed using the expression, (getLearningObject-
CMPT 858- Regular Session 2010-2011, Term 2 Page 9 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 10/22
A Simulation Model for eLearning:Course Planning in the Participative Web
ToMatchGivenObjective(currentLearningObjectiveOfFocus)==null) and is checked after each browse-
TaskDomain() step.
Bored / MentalFatigue A Learner becomes Bored if they have completed all of their learning objectives.
We may say that a Bored person is more likely to follow the Learning Objects being created by their
friends than those created by the teacher. Learners will transition this state if the following expression
is true: if (getNewLearningObjective()==null).
Stressed Not used - However, [Daniel et al., 2008] [5] have shown that Stressed Learners are highly
unlikely to share learning objects. In the future, if we introduce time limits on each learning objective,
this state might trigger if the Learner Agent doesn’t have enough time to complete all their objectives.
At each Learning Object visit, there is a 75% probability the Learner will ”learn” the topic covered by that
Learning Object. Future versions of this model could include a function that would use a different probability
based on the learner’s current affective state.
The second type of agent in our model, the Learning Object Agent, is used to represent anything that
a person can use, interact with, or create during their studies. Examples of Learning objects include: web
pages, RSS feeds, blog entries, assignments, discussions, an intelligent tutoring system. We distinguish a
Learning Object (an artifact, and an agent in our model), from a Learning Objective (a goal), which is an
attribute of a Learner agent in the model.
When new Learning Object Agents are created, we do not know which previous Learning Object Agents
are related, and thus how to set up the Agent Connections in the system. Should the new learning object
inherit some parent attributes from the learning object that the learner is currently sitting on? These are
stochastic characteristics because we do not know what the learner is thinking or writing. However we do
have some context: We know the learners current goal and their browsing history, and have some sense of
the input they are receiving from their friends and the teacher.
A practical challenge faced during this project was the visual synchronization of our two networks. When-
ever a Learner Agent created a new Learning Object Agent, the Spring Mass Layout for Learning Objects
would update, while the Learner Agents would not move. We got this working, but only if there are not too
many Agents moving around and when the populations do not grow too quickly. For example, if Sally were
visiting ”Introduction to Algebra”, the circle representing Sally would be displayed on top of the rectangle
representing Introduction to Algebra. Suppose Sally creates a new learning object, say, she writes down
CMPT 858- Regular Session 2010-2011, Term 2 Page 10 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 11/22
A Simulation Model for eLearning:Course Planning in the Participative Web
the equation 3x = 9 and solves it. This chunk of math is a new Learning Object that gets added as a child
node of ”Introduction to Algebra”. AnyLogic recalculates the Spring Mass Layout, causing the ”Introduction
to Math” node to shift to one side, leaving Sally floating in the air. We had to decide if we should move Sally
to sit on top of the ”chunk of math” node, or, if we ought to leave her on the ”Introduction to Math” node.
Theoretically, we wanted to choose the former, but, practically, we could only get the latter to work!
Figure 4: - Online Course in iHelp Our model can take any subset of the pages within an iHelp course using a
MySQL query. The structure of the overall course is shown in the left-hand column in the Figure above. Students can
click to any topic to view the details in the larger pane on the right of the screen. Every page in the course is represented
in our model as a Learning Object agent, discussed in Section 3.2.
4 Model basis in iHelp, a Course Management System
4.1 iHelp data
Data from an online course in iHelp is used to load our model with Learning Objects. The navigationallinks within the online course are used to create content relations between the objects (i.e. agent network
connections). In Section 3.1, we discussed that content could contain different types of relations like is-a
or part-of. However, in this first version of our model, we have only an uncategorized, directionless link be-
tween learning objects. [Breuker et al., 1999] [3] showed multiple dimensions of content links at increasing
levels of abstraction. These could help a system retrieve a highly relevant learning object, for example, a
CMPT 858- Regular Session 2010-2011, Term 2 Page 11 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 12/22
A Simulation Model for eLearning:Course Planning in the Participative Web
system may know that a particular student could benefit from being provided with an example of a ”general-
ization” of a particular concept. Having content links that are labelled with these dimensions would enable
the system to retrieve the relevant learning object for the particular student in a particular context.
Figure 5 shows the MySQL query we used to load Learning Objects into our model. Luckily for us, we
could use a shortcut to map between any Learning Object and its Course Module by looking at the code for
the chat window for that Module. Options include: M1 EvolComp, M2 HwSw, M3 Internet, M4 BegXHTML,
M5 AdvXHTML, M6 ProgLang, M7 IntroJS, M8 Security, M9 JSConcepts. Therefore, we could add or re-
move chunks of Learning Objects by Module by modifying the subject codes being imported.
SELECT
DISTINCT(contentid), title, parent_contentid, chat_forumid
FROM course_content
WHERE courseid=3
AND chat_forumid IS NOT NULLAND NOT title=’Exercises’
AND NOT title=’Quiz’
AND NOT title=’Conclusion’
AND NOT title=’References’
AND NOT title=’Assignment’
AND NOT title=’Videos’
AND (chat_forumid=’M1_EvolComp’
OR chat_forumid=’M3_Internet’
OR chat_forumid=’M6_ProgLang’
OR chat_forumid=’M9_JSConcepts’
OR chat_forumid=’M4_BegXHTML’
);
Figure 5: - MySQL query to iHelp Database to obtain Learning Objects Each record in the ResultSet
corresponds to a Learning Object Agent in our AnyLogic model. This is done in the Function, initializeAgentAttributes in
AnyLogic’s Main. For cosmetic reasons, we excluded several learning objects with repetitive titles. If we were calibrating
against actual student browsing behaviour, we should put these learning objects back in.
4.2 Calibration
iHelp has valuable data about learner quiz scores, dwell times and enter/exit times on each learning object,
information about when learners posted questions in iHelp Disussions. Although our model did not make
use of this rich data, Calibrations or initial Parameterization of the model would be a fruitful direction for
future work.
CMPT 858- Regular Session 2010-2011, Term 2 Page 12 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 13/22
A Simulation Model for eLearning:Course Planning in the Participative Web
4.3 How to Quantitatively Measure whether “Significant Learning” occurs
Seeing as our goal is to help a teacher maximize the learning of their students, we had to find a way to mea-
sure these experiences. In our model, we used Fink’s Taxonomy of Significant Learning[7], which places
kinds of learning into five categories. Below, we explain each category according to Fink’s definition, and
then we describe how we have interpreted how to apply this definition within our model as a quantitative
measurement. We propose that the following significant learning experiences could be represented in a
model, and, that a teacher can arguably predict that their students will have a higher quality learning expe-
rience by optimizing these measures.
We were only able to implement one measure in our final model: Foundational Knowledge. Within the
Learner Agent population, a Statistic calculates the Average of each Learner Agent’s countFoundational-
Knowedge() function, which counts the number of the learning objects that this agent created that match
the topics covered in their learning objectives. In other words, we are saying that if a Learner creates
Learning Objects that match their learning objectives, then they are meeting the Foundational Knowledge
criteria. This is perhaps too large of an assumption because it may incorrectly give credit to a learner who,
for example, produces an article containing errors or mistaken information about the topic.
Learning How To Learn To measure whether a student is becoming a self-directed learner, or is mastering
a particular kind of inquiry (example: Scientific Method), we could use Messages in AnyLogic between
agents, and characterize types of messages (example: asking a question, stating a fact) and match
patterns. Does an experienced Learner exhibit different behaviour than a novice Learner?
Foundational Knowledge A student who understands and remembers facts will create new learning ob-
jects that embody those facts. Therefore, we can measure a simulated student’s degree of founda-
tional knowledge by counting the number of Learning Object Agents created by them that match the
given category.
Application Application learning allows new kinds of foundational knowledge to become useful. In themodel, this could be reflected in the longevity of the Learning Objects created by the Learner, and by
its usefulness to others.
Integration When students make connections between ideas, integration has occurred. Within our sim-
ulation model, we might describe an abstract pattern for taking two ideas from a student’s past (i.e.
looking at the history of Learning Objects visited by a student), and create links between these parent
CMPT 858- Regular Session 2010-2011, Term 2 Page 13 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 14/22
A Simulation Model for eLearning:Course Planning in the Participative Web
objects and a new Learning Object created by that student. We might say that this particular pattern
occurs under certain circumstances (for example, if a student has a certain level of experience with a
topic, or is in a particular state like Attentive-confident or Confused-unsure) with a certain probability.
Human dimension Is it possible to measure or predict a student’s learning about one’s self and others?
The Human Dimension is perhaps the most challenging category for a modeller. For a teacher using
this simulation model for course planning, perhaps it is enough to leave each student to ask these
questions for themselves. On the other hand, in this very young field of research, it is probably too
early to judge!
Caring When students show desire from within to pursue a particular topic beyond the requirements set
by a teacher, they are showing the ”Caring” category of learning. This can be measured by counting
the number of learning objectives in a Learners ”subscribed” list of Learning Objectives (as opposed
to the ”prescribed” objectives set by the teacher).
We started to build in a measure for Integration, but did not finish. We say that a Learner has achieved
Integration if they have created a Learning Object that covers more than one topic. Unfortunately, at this
time we have not adequately explored how multiple ideas (i.e. “Concept” objects in our model) might come
to be attached to a particular learning object. This is a fruitful area to explore the relationship between
emotions/affect, task domain knowledge and process.
5 Baseline Scenario description
Since our model is primarily a course planning tool, we wanted to have a baseline scenario that would
represent the expected behaviour of people in a learning environment without any structure, course plan,
or any external motivations from a teacher. In our model, this means that each Learner Agent’s list of
“prescribed” learning objectives would start empty. The idea is to let a teacher input several different course
plans and observe the measures for Fink’s significant learning experiences. A teacher may decide to design
a course plan that maximizes learning outcome measures in the simulation model.
6 Sensitivity Analysis
Focusing on the number of learners, the the addition and removal of learning objectives, and the starting
set of topics in the learning environment, we describe expected and unexpected results.
CMPT 858- Regular Session 2010-2011, Term 2 Page 14 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 15/22
A Simulation Model for eLearning:Course Planning in the Participative Web
6.1 Comparing Affective States with and without Learning Objectives
While keeping the number of Learners constant at 10 Learner Agents and using the same starting set of
Learning Objects, we compared the Affective States (i.e. the percentage of Learners who were Confused,
Attentive / Confident, Bored / Mental Fatigue, Frustrated / Angry) for when we put zero learning objectives
into the system as compared to when we did assign a starting set of learning objectives to each learner.
In both cases we observed, as expected, that at the beginning 100% of the learners started at the
Attentive / Confident state. However, in the case where there were zero learning objectives, the Learners
gradually all transitioned to Bored. Figure 6 shows the simulation when learners have zero learning objects
assigned and the Learners are all Bored.
Figure 6: - Snapshots through time with No Learning Objectives This figure shows the same pie chart with
screenshots taken a few seconds apart. It shows that at the start of the baseline run, all learners started in an Attentive
state, and it did not take long for them all to transition to Bored. A Learner Agent transitions to Bored when they have
zero unaccomplished Learning Objectives left (or, in this case, none assigned at all).
In the case where the Learners had been assigned learning objectives, we observed that none of the
learners were ever Bored. Rather, the entire population was divided into Confused, Attentive / Confident
and Frustrated /Angry. The percentage of Learners in each category fluctuated quite a lot. For example,
the Confused category would disappear altogether with 40% at Attentive / Confident and 60% at Frustrated
/ Angry.
We also kept an eye on the new Learning Objects created by the Learner Agents, and which topics saw
the most growth. When there were Learning Objectives assigned, we observed that this could cause one
category to be ignored by the learners altogether. For example, in Figure 7, none of the learners are paying
any attention to the chapter about Module 6 - Programming Languages. This is observable by the cluster
CMPT 858- Regular Session 2010-2011, Term 2 Page 15 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 16/22
A Simulation Model for eLearning:Course Planning in the Participative Web
Figure 7: - Learners have Learning Objectives Assigned The pie chart on the left is the same as Figure 6 on
a different run; the only change made between runs is that this one shows the results of Learner Agents having learning
objectives assigned to them.
of sand-coloured nodes shoved down in the lower right with no Learner Agents (i.e. circles) sitting on them.
6.2 Parameter sensitivity
Given the current functionality of our model, other parameters we could vary in new experiments include:
The number of learners, the starting “Learning Object Visited” of each Learner, the starting affective state
of each learner, the starting set of learning objectives, the circumstances under which Learners create new
learning objects or browse the task domain. We might even consider issues of retention and withdrawal
rates by adding/removing learners from the population.
Outputs we could measure include: the number of learning objects created, the navigation paths taken
by each learner, the “affective paths” of each learner (i.e. the order in which they transitioned across various
CMPT 858- Regular Session 2010-2011, Term 2 Page 16 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 17/22
A Simulation Model for eLearning:Course Planning in the Participative Web
states), as well as Fink’s measure for Foundational Knowledge.
Although our model was built from scratch and is a very early and rough representation of the Course
Planning problem in eLearning, we did try to identify points of leverage in the system. During the first few
runs of our model, we noticed that the average foundational knowledge output (i.e. the topic coverage by
the average learner) is tied directly to the learning objectives imposed by the teacher. We searched for
different phases in the system where different feedback loops might dominate the system, and noted two
major phases: first, as Learners are fulfilling their learning objectives, and second, behaviour after Learners
have completed their objectives and continue to exist within the system.
6.3 Structural sensitivity
We ran our simulation with each learner being given the learning objective to master Module 6 - Program-
ming Languages. We observed that as the simulation ran, zero learners were picking up any knowledge
from this chapter. This was surprising because our model had a very broad assumption that if a Learner
merely visited a learning object with that topic, that they would immediately master the topic. We quickly
arrived at the conclusion that none of the learners were even browsing to any learning objects in that cat-
egory at all, which was why Fink’s measure for Foundational Knowledge remained at zero. In other words,
if by chance a Learner Agent’s starting location was placed in a subgraph that did not contain any learn-
ing objects that matched their learning objectives, then the Learner had zero possibility of learning anything.
Each Learner Agent’s behaviour was set to follow our browse() function, which causes them to navigate
from learning object to learning object like a graph traversal. However, we realized that our graph was
actually 3 graphs: Due to our iHelp query, there were zero links between learning objects of different cate-
gories! We realized our model did not realistically replicate true learner behaviour at all. In reality, learners
using iHelp could ”warp” from Module to Module simply by clicking links in the navigation tree; they didn’t
necessarily have to use the Forward and Back links that our graph was based on in AnyLogic.
Because of this discovery, we wrote a LearnerAgent function called warpToAnyNewLearningObjectThat-
MatchesMyObjectives() that allows Learner Agents to jump across subtrees without the need for content
links between all learning object nodes in the repository. While writing this function, we uncovered yet an-
other assumption that should be further investigated: When a Learner Agent starts to follow a new Learning
objective and there are 15 possible Learning Objectives in the repository that have the same topic, how does
CMPT 858- Regular Session 2010-2011, Term 2 Page 17 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 18/22
A Simulation Model for eLearning:Course Planning in the Participative Web
the learner start? For our model, we simply select a random object.
We chose to base the Learning Object Agent network on an iHelp course, but would like to expand to al-
low the addition of Learning Objects in ways other than being created by Learner Agents. For example, the
instructor might flag a set of relevant RSS feeds and ask students to read incoming news items throughout
the course. Since we discovered that the content of the Learning Objects was a point of leverage in the sys-
tem, it would be important to perform further sensitivity analysis on a stochastic element like unpredictable
course content and its effect on Learners.
7 Experiments, Interventions, ”What if” scenarios
Although we ultimately only explored a “random, wandering learner” compared to a “Learner with objec-
tives”, we also present the other possibilities for experiments that we investigated in the earliest phases of
this project:
System vs. Learner control in learning objective negotiation [17] In our system, we could change the
proportion of “prescribed” vs “subscribed” learning objectives allowed. The system could allow more
freedom by allowing access to add or remove from the set the learner works on in their day-today
activity online. A related measure is Quality vs. Quantity in browsing: One learner may browse through
many Learning Objects, but not experience any deep learning, while another learner may browse
fewer objects but gain more value from them. In future work, this is probably the most important to put
in; some probability of absorption.
Automated Content Sequencing perhaps instead of an assignment, we could also impose a Reading
Path and restrict the Learner’s navigation – “no distractions allowed for a sec”. Would we want to
study this effect on the significant learning measures?
idea transmission across social networks agent’s opinions of other agents - [Olorunleke et al., 2003]
[13] designed a simulation to study the spread of delusion among agents. HIs findings may show
ways to improve our model’s assumptions.
Trying different teaching techniques testing out teaching techniques (assuming teaching techniques can
be represented as processes, perhaps using different Events). Should our system give learners
feedback when they make a mistake, or let them explore? Our model would need a lot of work to be
CMPT 858- Regular Session 2010-2011, Term 2 Page 18 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 19/22
A Simulation Model for eLearning:Course Planning in the Participative Web
able to explore this type of question. A simulation might help us to see if this decision causes any
emergent behaviour at the macro level across many students.
Learning object Recommendation What would happen if we gave the learners different starting sets of
learning objects? Or, different learners different sets of learning objects, but let them start to share
with each other. Can we measure different learning measures when the instructor uses a different
starting set of learning objects?
Group work Be able to test what happens when you cluster certain groups of people together with certain
characteristics or preferences and assign common learning objectives.
Single user vs. Many users Since other Learners impact the environment by adding new learning objects,
an individual agent would have a different experience than an agent within a population of many.
For example, if a person is extremely interested in a particular topic, they could flood the learning
environment with questions and articles about that topic. This will impact the browsing experience
of others, because it changes the proportion of items available within a particular topic. Contribution
rates may also be impacted as a side-effect of the changing Learning Object landscape; if the course
is going in a direction that the student is uninterested in, their rates may drop.
Time Pressure and Work Load If Learning Objectives were given a time limit, and learners were expected
to work through all objectives within a certain period of time, then we may see impact on the amount
of exploration and browsing activity from each learner. For example, with too many Learning Ob-
jectives under a tight deadline we might expect to see high levels of Learner Agents spending time
in the Stressed state, while too few Learning Objectives might see an increase in time spent in the
Bored mentalFatigue state. Allowing a teacher to adjust the number of Learning objectives and then
measuring the number of ”subscribed” Learning Objects (as opposed to ”prescribed” learning objects)
could help them select the sweet spot for their class.
8 Conclusion
We work toward a new course planning tool that will help teachers move away from a model where all stu-
dents perform the same tasks at the same time toward a learning-centred environment where a teacher can
plan, deliberately, for each student to follow a personalized curriculum. At the same time, it is important to
have some measures of standardization (example, for accreditation or certification) and we suggest that the
set of “prescribed” learning objectives in our model fills this need. Common learning objectives also allows
CMPT 858- Regular Session 2010-2011, Term 2 Page 19 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 20/22
A Simulation Model for eLearning:Course Planning in the Participative Web
for the planning of collaboration among groups of students by assigning the same Learning objective to
subsets of students but not for others. Although we did not explore this as much as we would have liked, we
have stumbled upon new ground and have made a good start. Other interesting questions a teacher might
ask are: optimum group sizes of certain types of classes. Maybe some domains (humanities, sciences,
etc.) work great with many learners and others work better with fewer.
Human tutor “action words” that are normally considered unattainable for a computer can now be consid-
ered in relation to the framework we have established: give advice advice, analogy, assert, clarify concept,
summarize, hint, explain, encourage, and so on (see [Kumar et al., 2005] [9] for a thorough list). We suggest
a few: To give advice means to identify a learner in a simulation with similar characteristics, calculate a few
likely navigation paths through the Learning Objects at hand, and then recommend the action with highest
utility (i.e. measure for significant learning experience). To give an analogy is to find a set of learning objects
with similar relationships between them (albeit easier said than done!). To assert means to have a learner
visit a Learning Object containing that fact. And so on... although these are still extremely difficult problems
in Ar tificial Intelligence, using a simulation allows us to describe them quantitatively. We continue to try
to understand what a computational model for different teaching strategies might look like, discussed in
Section 2, and whether detailed computational representations for human teaching tactics and dimensions
of epistemology are necessary for effective eLearning environments.
Throughout the development of this model, we have come to better understand the Ecological approach
and how it might be implemented. In our model, we would have to keep track of the purpose for which each
Learner made use of a particular learning object, and record that information with each Learning Object
Agent. We would also have to build in a recommendation mechanism that would compare Learner Agents
and present each with a set of Learning Objects to choose from based on the behaviour of previous similar
Learner Agents. This might help answer our question in Section 6.3, where we were only using a random
Learning Object Agent instead of a recommended one.
Our model allows us to articulate research questions in AIED that were not previously quantifiable. The
process of teaching and learning online can be explored as an experiment in an Agent-Based simulation,
using research findings from educational psychology and educational technology as a basis for the model
formation. We tackle the questions teachers face about their role in the participative web, and how they
continue to play a critical role even in a world where the teacher is no longer the primary disseminator of
knowledge, and the classroom is no longer the only place for collaboration and communication between
CMPT 858- Regular Session 2010-2011, Term 2 Page 20 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 21/22
A Simulation Model for eLearning:Course Planning in the Participative Web
students.
References
[1] S. Afzal and P. Robinson. A Study of Affect in Intelligent Tutoring. In Proceedings of Workshop on
Modelling and Scaffolding Affective Experiences to Impact Learning, Intl. Conf. on Artificial Intelligence
in Education (AIED), Los Angeles, July 2007 , 2007.
[2] Ken Bain. What the best college teachers do . Harvard University Press, Cambridge, Massachusetts,
2004.
[3] J Breuker and A Muntjewerff. Ontological modelling for designing educational systems. AI-ED 1999
Workshop on Ontologies for Intelligent Educational Systems (Le Mans, France) , 1999.
[4] C Conati, A Gertner, and K Vanlehn. Using Bayesian networks to manage uncertainty in student
modeling. User Modeling and User-Adapted Interaction , 12:371–417, 2002.
[5] BK Daniel, GI McCalla, and R Schwier. Social Network Analysis techniques: implications for informa-
tion and knowledge sharing in virtual learning communities. International Journal of Advanced Media
and Communication , 2(1):20–34, 2008.
[6] B Du Boulay and R Luckin. Modelling human teaching tactics and strategies for tutoring systems.
International Journal of Artificial Intelligence in Education , 12:235–256, 2001.
[7] L. Dee Fink. Creating Significant Learning Experiences: An Integrated Approach to Designing College
Courses . John Wiley & Sons, Inc., San Francisco, 2003.
[8] NT Heffernan, KR Koedinger, and L Razzaq. Expanding the model-tracing architecture: A 3rd gener-
ation intelligent tutor for Algebra symbolization. International Journal of Artificial Intelligence in Educa-
tion , 18(2):153–178, 2008.
[9] Vive Kumar, Jim Greer, and Gord McCalla. Assisting online helpers. Int. J. Learn. Technol., 1:293–321,
mar 2005.
[10] G McCalla. The ecological approach to the design of e-learning environments: Pur pose-based capture
and use of information about learners. Journal of Interactive Media in Education , 2004.
CMPT 858- Regular Session 2010-2011, Term 2 Page 21 of 22
8/6/2019 A Simulation Model for eLearning: Course Planning in the Participative Web
http://slidepdf.com/reader/full/a-simulation-model-for-elearning-course-planning-in-the-participative-web 22/22
A Simulation Model for eLearning:Course Planning in the Participative Web
[11] Gordon McCalla. The search for adaptability, flexibility and individualization: Approaches to curriculum
in intelligent tutoring systems . Adaptive Learning Environments: Foundation and Frontiers. Springer-
Verlag, Berlin, 1990.
[12] A Mitrovic, M Mayo, P Suraweera, and B Martin. Constraint-based tutors: a success story. Proc.
IEA/AIE-2001, Budapest, Springer-Verlag Berlin , pages 931–940, 2001.
[13] O Olorunleke. Overcoming agent delusion. In Proceedings of the second international Conference
on Autonomous Agents and Multiagent Systems , AAMAS’03, July 14-18, 2003, Melbourne, Australia,
2003.
[14] John D. Sterman. Business Dynamics . Systems Thinking and Modeling for a Complex World. The
McGraw-Hill Companies, Inc., Boston, 2000.
[15] K Vanlehn. The behavior of tutoring systems. International Journal of Artificial Intelligence in Educa-
tion , 16(3), 2006.
[16] J Vassileva and G McCalla. Multi-agent multi-user modeling in I-Help. User Modelling and User
Adapted Interaction , 2003.
[17] J Vassileva and B. Wasson. Instructional planning approaches: From tutoring towards free learning.
Proceedings of Euro-AIED’96, Lisbon, Portugal , 1996.
[18] Barbara Wasson. PEPE: A computational framework for a content planner. In S. Dijkstra, H.P.M.
Krammer, and J.J.G. van Merrienboer, editors, Instructional Models in Computer-Based Learning En-
vironments , pages 153–170. NATO ASI Series: Advanced Science Institutes Series F: Computer and
Systems Sciences, Springer-Verlag Berlin Heidelberg, 1991.
[19] B Woolf, W Burelson, and I Arroyo. Emotional intelligence for computer tutors. Supplementary Pro-
ceedings of the 13th International Conference on Artificial Intelligence in Education , 2007.
CMPT 858 R l S i 2010 2011 T 2 P 22 f 22