INCT SEC Gislaine Final

8/2/2019 INCT SEC Gislaine Final

http://slidepdf.com/reader/full/inct-sec-gislaine-final 1/11

Using Logical Sensor Network to Support Context-Aware

Learning AssessmentGislaine C.M. Rosales, Delano Beder, Joice L. Otsuka, Regina B. Araujo

Department of Computer Science – University of São Carlos

São Carlos, SP – Brazil

{gislaine, delano, regina}@dc.ufscar.br, [email protected]

Abstract. Early detection of learning issues can contribute to mitigate

problems such as poor school performance, low participation of students and

high dropout rates. Most of the existing solutions to learning assessment are

based on a post evaluation of data and does not support real-timeinterpretation and feedback. This paper presents a context-aware learning

assessment system that can provide a more precise monitoring of the student´s

learning process. The novelty of our system lies on the use of physical and

logical sensors for the accurate tracking of events that can influence the

teaching and learning process. The events captured by sensors are modeled as

contexts in ontologies. A simple and usable interface is offered to students,

tutors, teachers and course coordinators.

1. Introduction

The continuous emergence of new information and communication technologies, frominteractive, collaborative, context-aware and immersive multimedia and 3D simulations,

to multi-modal interfaces, augmented reality and broadband wireless networks, to

mention a few, makes it possible for teachers and students, to envisage new and

powerful forms of creation, distribution and consumption of content for distance

education courses. Despite these exciting technologies, making courses content more

accessible to students is often achieved by replicating information in different formats

and to different virtual spaces what generates a mass of content, version conflict and

maintenance difficulty. This way of organizing theoretical contents and exercises, which

makes up a course, makes it more difficult for both students and teachers to track

academic progress and organize activities in an orderly manner. Moreover, it is oftenobserved in distance education production environments the difficulty in the

management of information for teachers, course coordinators, instructional designers

and other managers of the process, mainly due to asynchrony of study intervals. It is

well known the benefits of indirect mediation in space and time provided by distance

education, where students can organize their own activities according to their

convenience, allowing some students to attend few disciplines in a condensed manner,

while others can attend various parallel disciplines. However, the monitoring and the

assessment of all students of a discipline/course become laborious and complex. Also,

with existing tools for assessing learning process, it is a complex task to identify

ongoing learning difficulties of each student. As a consequence, poor school

performance, low student´s participation and high dropout rates are expected.



This paper introduces a powerful system for the monitoring and assessing of the

teaching and learning process in distance education courses. The novelty of our system lies

on the use of physical and logical sensors (the latter as a software abstraction) for the

accurate tracking of events that can influence the teaching and learning process. Our approach considers not only the logical environment the user is in but also his/her

surrounding physical environment. The main characteristics of our system include: accurate

monitoring of localized events; real-time and posterior feedback to LMS´s users on the

learning process; more secure online evaluations through accurate evaluation and

monitoring of both physical locations where online evaluations will take place (e.g.,

computer laboratory) and logical environments (LMS and other computer applications

running in parallel); LMS platform independence, among others. The main goals of our

system are: to help teachers, tutors and course coordinators to better understand students’

learning difficulties and lack of motivation so that early actions can be taken; to provide a

better understand of users’ behavior patterns and their different learning styles; to help

students to better organize themselves along the course and improve performance. Similar goals have been pursued by researchers in the area, such as Chen et al. (2007a); Chen et al.

(2007b); Chen and Chen (2009); Dringus and Ellis (2005); Enright et al. (2008); Huang et

al. (2009); Otsuka (2007); Zaïane and Luo (2001). However, existing solutions focus

typically on storage and posterior data analysis, providing little or no support for real-time

feedback. They also lack support for the monitoring of special types of actions, such as

user’s interaction intention.

This paper is organized as follows: Section 2 presents the related works. Section

3 introduces our system and the use of contexts and ontologies for the learning process

monitoring. Section 4 presents a brief description of our system´s architecture. Three

case studies to be used to validate the proposed system are described in Section 5,followed by Conclusions and References.

2. Related Works

There are several interesting works that propose solutions for the monitoring and

assessment of learning in LMSs. Dringus and Ellis (2005) proposed a data mining

system that identifies common participation indicators in order to judge the entries in

the forum tool and check adjustment of the teaching and learning process during its

development. The performed analysis is based on the data mes-sages exchanged by

students during the discussion, which are stored in the LMS database. Zaïane and Luo

(2001), presents a framework based on data mining techniques to analyze web server

access logs. By tracking the students’ actions, logs are obtained and behavior patterns

are identified; they are essential because assist in the learning assessment process. The

system provides a configured interface through which the user can set restrictions for the

collection, the steps of discovery and analysis of patterns of behavior, applying filters

according to their research interest.

Enright et al. (2008) proposes an intelligent tutoring system, applying data

mining techniques that provides the teacher the possibility of detecting possible patterns

of repeated actions by students that may indicate problems with the adopted teaching

strategies; it also enables the teacher to make changes in the classroom and suggestions

for new content. A model designed to provide formative assessment support in LMSs,

based in a multi-agent architecture and the use of ontologies, is presented by Otsuka et



al. (2007). Its architecture is primarily based on two types of agents: (i) activity

monitoring agents, which collect data on student participation in each activity, according

to specified criteria, such as regular access and postings in forums, and the number of

posts; (ii) analyzer agents, which are responsible for analyzing the participationindicators collected from a particular activity (such as Discussion Forums, Chat and

Portfolio), on a knowledge base. Those agents may indicate patterns of behavior which

might infer in the occurrence of problems, alerts and quality in holdings. Chen et al.

(2007a) presents a framework that provides a communication interface based on

intelligent agents: interaction, instruction, information, assessment, and log agents. The

model is based on data analysis from databases and web logs that contain information

about the interactions of participants in the LMS. It also presents the users´ performance

in the activities, their efforts and progresses. The system sends messages automatically

to teachers when a student performance is low, which is a situation that requires more

attention and monitoring. Chen et al. (2007b) combines techniques of data mining to

identify factors of learning; they also use fuzzy analysis to identify the students'

performance during the assessment process in portfolios. The system presented by

Huang et al. (2009) supports formative assessment and self-evaluation in mobile

learning environments. Based on the answers given by students, the system performs

adaptive assessment, which is based on successes and mistakes, and also the

technological capacity of users' devices to refine the assessment by presenting issues of

levels more or less complex. The system uses access to the databases that store

information about students’ interactions in order to define the issues to be included in

the ratings.

In a similar proposal to the previous one, Chen and Chen (2009) presents a tool

for formative assessment techniques using mobile data mining, and rules of inference

based on fuzzy logic, which enables the teacher to monitor the students' learning. Both

teacher and students can access the environment using a mobile device. The system is

based on agents, such as an agent of formative assessment and feedback to staff who

work in the student’s responses analysis to define evaluation rules that are used to infer

the level of individual learning in each student. It can be observed that the presented

solutions focus on storage and subsequent analysis of data that were stored in databases

or obtained from system logs generated by users or by applications. However, the

proposed tools do not support the monitoring of special types of actions performed in

the environment, nor the contexts that may be important in the process of students’

assessment. Examples of such actions include: movements performed with the mouse,information entered and then deleted in a form and so on. Examples of context include:

different times the student performs his/her activities, actions he/she is performing in

parallel, local where activities are performed, network connection quality, server access

latency, among others. In our system, this information is collected, interpreted and

delivered, via a web-based interface, to a teacher or tutor to help him/her, to better know

each student’s educational profile. This can help not only to pinpoint problems during

the educational process, but also to create the basis for personalized education.

Moreover, it is also expected that early detection of education problems can reduce high

dropout rates in distance learning courses, low teaching quality, and low interaction

among students, teachers and tutors.



Next section introduces the logical sensor network abstraction and the support of

contexts and ontologies concepts in the modeling of our system.

3. Accurate Monitoring Using Physical and Logical Sensors

Our system makes use of the concept of sensors and sensor networks to track specific

and localized events. In our system, a sensor can be either physical or logical. A physical

sensor is a device that can be deployed in a physical environment to capture information

of interest (e.g., temperature, humidity, movement, presence of a person in an

environment). A set of physical sensors that are deployed on an area of interest (e.g., a

building, a room, a soya plantation) and can collaborate among themselves and forward

information to a recipient node (sink) is defined as a sensor network. A sensor network

can be deployed, for instance, in a room that will be used for online examinations to

evaluate and monitor the room for abnormal activities and actions (e.g., from personal

identification to detection of cell phone activity). The information (events) collected bya sensor network can be combined with information collected from digital environments

to provide a more accurate and powerful view of what is occurring. For the detection of

events in a digital environment, the concepts of logical sensor and logical sensor

network are used. A logical sensor is a software component that can capture specific and

relevant information about actions and intentions of a user in a digital environment. A

logical sensor network is defined in our work as a set of logical sensors that can

collaborate and forward information to a recipient node. A logical sensor network can

be deployed, for instance, on both: an LMS and a remote user´s device for localized and

more refined detection of users’ and LMS applications’ events, as shown in figure 1 .

Figure 1. Physical and Logical Sensors Network

3.1. Use of Physical and Logical Sensors

Examples of events that can be collected by the sensors of our system include: users’navigation path along the course resources and activities; the order an information was



typed in a form (e.g., an evaluative questionnaire); how long the user took to fill in a

text box; how many times a checkbox field was checked and unchecked; number of

mouse clicks in a forum; content types visited, and visiting frequency; time an activity

was submitted; discipline status; what students are doing and in which virtualenvironments (disciplines) they are browsing; what actions/activities are being

performed at the same time (listening to music, for example); statistical reports

(activities submitted, activities that are late, hits, posts, etc), access to a chat (including

hours of access, time remained, posted messages, etc.), sending messages to a forum

(considering the subject, message body, time of delivery, number of characters of the

message, time needed to type the message), among other events required for the desired

monitoring of the application. Examples of events generated by users and/or their

applications outside the virtual environment and collected by our system include: access

to the scroll bar while reading a text or participate in a chat, movements of the mouse

over the list of participants in a forum or other activity; if the user has oscillated by

clicking on the submit button while participating in an questionnaire activity or another

activity that involves web form, users’ parallel activities while studying (e.g., listening

to music, watching a video, browsing on other sites); among others. We identify user

events that can be collected and interpreted during the study activities, which can help

managers, such as pedagogical coordinators and teachers, to understand the teaching-

learning process. We can also identify the users´ preferences and provide support for

future implementations that allow the selection of content based on the learning style of

each user. A simple example of how physical and logical sensors can be used together to

provide a better view of the actions performed in an exam room is the deployment of

cell phone activity sensor (physical sensor) and the deployment of “content types

visited” and “activities accessed by the student” (both logical sensors) on the LMS beingused by the students at exam time. The co-relation and interpretation of the collected

data is made using context and ontologies concepts. Next sub-sections introduce these

concepts.

3.2. Context – Aware Learning Assessment

Users´ information, actions and intentions are captured by sensors and mapped into

contexts. According to Dey (1999), context is defined as any information that can be

used to characterize the situation of an entity. An entity is the focus of interest of an

application. In our system an entity can be an LMS, a user (student, tutor, teacher,

pedagogical), activities (task), and disciplines, among others. A system is context awareif it uses context to provide relevant information and/or services to users, Dey et al.

(2001). Our system is context-aware because it uses information generated by LMS´s

users, or its applications, from their access devices, which are collected by logical

sensors and interpreted to infer relevant information. Our learning monitoring system

provides an interaction interface, added to the virtual environment, as illustrated in

Figure 4, which provides additional information relevant to the user. Abowd and Mynatt

(2000) define some categories for context-aware applications, called 5Ws, which are:

who, where, when, what and why. We use this abstraction of the 5 Ws to classify the

learning contexts of our system and support the process of software development. The

who, where, when and what dimensions are mapped as classes of an ontology and can

be combined in order to determine the reason a situation occurs. The use of ontologies

allows us to explicitly define the concepts in the context of learning and its restrictions.



3.3. Use of Ontologies for Context Modeling

Our ontology represents concepts defined and proposed from an extensive research on

various models of distance education used in universities and colleges worldwide.

Terms such as coordinator, teacher, tutor, student, course, discipline, resource and

activity were established from both: the distance education models studied and concepts

often used in traditional LMS such as Moodle (Modular Object Oriented Dynamic

Learning Environment) and Sakai. The who dimension indicates the LMS user on which

the monitoring system will collect information. The profiles mapped to the who

category follow a hierarchy of monitoring, - a user with "course coordinator" profile has

the highest priority and can carry out the monitoring of her/his own events as well as

events from other users at lower levels. Students occupy the lowest level in the

hierarchy of tracking options; they can only monitor events related to their own

activities and courses. The restrictions established by the hierarchy of LMS users are

determined in the properties of context ontology. Where indicates the virtual space(s) inwhich a user is while navigating on the LMS (e.g., a course, a discipline, a study

material or an activity). Our monitoring system has a logical sensor that detects where

the user is to determine what types of information can be presented to him/her. For

instance, if the system identifies that the user is browsing for an activity, information

about that activity will be presented according to predetermined settings for the user

profile. If the user is in a discipline, another information set will be presented, and so on.

The when dimension indicates the relationship of time associated to events that are

collected by the logical sensors. From this information, relevant information can be

extracted, such as time spent on the development of activities – a user´s routine can be

created from her/his activities history. In addition, the when dimension is useful to assist

in determining preferences and possible user learning difficulties, which will consider

the temporal information combined with time that remained connected and with the

number of accesses made to materials, activities, and course. What is mapped as a set of

information concerning people, their activities and devices used to interact with the

LMS. These can be monitored by our system and have a set of rules applied to them in

order to infer what is happening. This information set that make up the what dimension

is mapped to a class of Learning ontology, as shown in Figure 2. Some examples of

subclasses used are: Access, AcademicPerformance, ConnectedTime, FeedbackTime,

Post and Avoidance. The why dimension helps to determine more complex situations,

such as the reason of a particular situation or an action performed by the user. We can

say that the dimension why is the outcome of the inferences made from the applicationof a set of rules on the information collected by the sensors to provide possible causes

and consequences for academic standing and the user´s behavior in the virtual

environment. Examples include: possible causes for a poor school performance and

reasons for absence in synchronous activities. The modeling of contexts using

ontologies based on the taxonomy of the 5W's allows us to represent knowledge about

the teaching-learning process in a satisfactory manner to provide a more refined

monitoring than provided by traditional LMSs. More complex situations can be inferred

when more contexts are considered, such as social, professional, personal and family

user´s contexts. These new contexts can also be mapped on ontologies to enhance the

results of the system inferences, but due to space restrictions, will not be discussed in

this paper. Next section presents a brief description of the proposed system architecture.



Figure 2. Partial view of the Learning Ontology

4. System Architecture

Our system architecture follows a hybrid model, as shown in Figure 3. It is hierarchical,

but the communication between some layers, such as Applications, Middleware and

Logical Sensors, follows a publish/subscribe model, instead of traditional client/server

model. On the other hand, communication between the layers LMS and Logical Sensors

follows the traditional client/server model. The proposed architecture aims to facilitate

the communication between the applications and middleware services. One of the main

services offered by the middleware is the Context Interpretation Service, composed of

an Inference System and Queries Ontologies. This service is responsible for providinghigh-level information about what is perceived by the sensors in the monitoring process

of a user's actions on his/her device and LMS. In order to increase the level of

abstraction of information to facilitate understanding of the teaching-learning process,

the service uses a knowledge base composed of ontologies and a set of rules applied to

the data collected. The system architecture has the following main components:

LMS: In the first (bottom) layer of the architecture is the LMS, which represents

the platform for storage and delivery (in the web) of digital study content. LMS

and User Device can be monitored by logical and physical sensors to capture

events that can be of interest for the learning process monitoring.

Sensor Network: The second layer consists of logical (and physical, when

required by the application) sensors that monitor users' actions. The logical and

physical sensor network can be deployed on the user´s device and on the LMS.

Middleware: A middleware is systems software that resides between the

applications and the underlying operating systems and networks, and provides

reusable services that can be composed, configured, and deployed to create

networked applications rapidly and robustly, Blair et al. (2004). Our middleware

has the following services: Learning Object Service, Context Interpretation

Service, Data Mining Service, Logic Sensors Configuration Service, Subscribe

Management Service and Content Adaptation Service.



Application: the top layer of the architecture consists of applications that make

use of our learning monitoring system.

The middleware interacts with the sensor network to get the events from the

learning environment. It then uses its interpretation service to make inferences about the

entity being monitored.

Figure 3. System Architecture showing the Context Interpretation Service

The publish/subscribe mechanism is used for the communication among

applications, modules, and services of the middleware. Subscribers are entities

interested in receiving events generated by publishers. Each new generated event is

published to all interested subscribers. A subscription performed by an application to a

sensor or multiple sensors happens every time a learning action is configured in the

LMS (a forum, a chat, a questionnaire, among other tools). Upon subscription, the

events generated by the sensors that monitor a specific action will be retrieved and sent

to the middleware. Three case studies are being developed to validate the system, which

are described in the following section.

5. Case Studies

Three case studies are being developed to validate the system: 1) Monitoring

performance and study organization (student view); 2) Monitoring of online

examinations (teacher view); and 3) Monitoring of students’ actions and progress (tutor

view) and are described below.

5.1. Monitoring Performance and Study Organization

This case study seeks to identify the main processes and event flows that can better support students in their own performance status and work organization, such as:



activities status (general and individual: late, in progress, finalized); errors versus

success in activities (general and individual); suggestions for study agenda organization

considering deadlines and exams proximity, etc. After performing the login in the LMS,

our system identifies the user and their access permissions. At this moment, other logical sensors are activated, according to the predetermined profile and settings for the

user, who will receive information of interest showed as she/he navigates the virtual

environment. The information presented is specific to the virtual space he/she is, i.e., it

refers to a course or a discipline or a particular activity that is being browsed at the time.

This case study is being implemented.

5.2. Monitoring of Online Examinations

This case study involves the monitoring of online examinations whose aim is to meet

security requirements for exams application. This study has great relevance, especially

for situations in which online exams are not applied due to the lack of securemechanisms (vulnerability present on the platforms; difficulty to manage aspects of

reliability and legitimacy of the process). In this context, our system can support the

monitoring of online exams carried out remotely. For this, a logical set of sensors is

placed in the user's device. This device can be either a computer of the institution,

located in a lab where the teacher has access only remotely, or a student's own

equipment. The logical sensors can monitor events that can jeopardize the security of

the process, such as attempts to access applications or unauthorized files, access to

unauthorized sites, access to different users´ accounts from the same device during the

test, among others. Sensors can also be deployed on the LMS to monitor students´

actions, such as the use of communication tools not allowed for this exam, access toLMS`s unauthorized resources, among others. Physical sensors that can detect cell

phone activity, film the room, even process face recognition can be used in combination

with the above mentioned logical sensors. This work is being developed with the

support of the network technical staff, teachers and students (face to face and at

distance/virtually) of our Institution.

5.3. Monitoring Students’ Actions and Progress

This case study considers the monitoring of students´ and tutors ´actions and progress

from the teachers´ point of view. The teacher can monitor students' progress in activities

and disciplines by means of a friendlier interface that presents information interpreted,

inferred from the data collected by logical sensors. This interface provides better

understanding of the students’ actions and progress in the LMS for early problems

identification. Figure 4 shows an interface prototype being developed in our laboratory.

The system adds a toolbar to the LMS, which is displayed at the bottom of the page. The

toolbar shows information obtained from logical sensors, configured for the monitoring

of forum activity, identified with the title „AS6-1 – Doubts Forum‟. In this example, a

user with the teacher profile accesses the forum activity by following the LMS standard

navigation and view complementary information. This information was added by our

monitoring system and includes: the amount of posts in this forum; the number of

participants with interactions in this activity; a chart with the amount of tutor´s and

students´ interactions. It also shows the number of participants missing the discussion.

For a more refined query, the teacher can click on the toolbar, included by the



map out new context situations in the existing ontologies and to use new rules to context

interpretation. Also, as further work, the system will deliver personalized content to

students according to their learning characteristics, which are identified by the system.

References

Abowd G. and Mynatt, D, 2000. Charting past, present, and future research in

ubiquitous computing. In ACM Transactions Computing – Human Interact (ACM

Press), Vol. 7, pp. 29-58.

Blair, G. et a. Middleware technologies for future communication networks. IEEE

Network, v. 18, n.1, p. 4 - 5, 2004

Chen, H.-M. et al, 2007a. E-Homebook System: A web-based interactive education

interface., Vol. 49, pp.160-17.

Chen, C.-M. et al, 2007b.Mining Key Formative Assessment Rules based on LearnerProfiles for Web-based Learning Systems. Proceedings of the 7th IEEE International

Conference on Advanced Learning Technologies . Niigata, pp. 584-588.

Chen, C.-M. and Chen, M. C., 2009. Mobile formative assessment tool based on data

mining techniques for supporting web-based learning. Vol. 52, pp.256-273.

Dey, A.K., 2001. Understanding and Using Context. In Personal Ubiquitous

Computing, Vol. 5, Nº. 1, pp. 4 – 7.

Dey, A.K. and Abowd, G.D, 1999. Towards a better understanding of context and

context-awareness. GVU Technical Report GITGVU- 99-22, College of Computing,

Georgia Institute of Technology.

Dey, A. et al, 2001. A conceptual framework and a toolkit for supporting the rapid

prototyping of context-aware applications. In Human-Computer Interaction Journal,

Vol.16, Nº. 2, pp.97-166.

Dringus, L. P. and Ellis, T., 2005. Using data mining as a strategy for assessing

asynchronous discussion forums. In Computer & Education, Vol. 45, pp. 141-160.

Enright, J. et al, 2008. Intelligent Functional Dependency Tutoring Tool. 16th

International Conference on Computers in Education (ICCE 2008). Taipei, Taiwan, pp.

171-172.

Huang, Y. –

M. et al, 2009. An adaptive testing system for supporting versatileeducational assessment. In Computer & Education, Vol. 52, pp.53-67.

Otsuka, J. L. et al, 2007. A Multi-Agent Formative Assessment Support Model for

Learning Management Systems. Proceedings of the 7th IEEE International Conference

on Advanced Learning Technologies. Niigata.

Zaïane, O. and Luo, J., 2001. Towards Evaluating Learners Behaviour in a Web-Based

Distance Learning Environment. International Conference on Advanced Learning

Technologies (ICALT 2001. Madison, USA, pp. 6-8.

INCT SEC Gislaine Final

Documents

Transcript of INCT SEC Gislaine Final