Demo Navigation Service (WP7)
Hans Hummel
Open University of the NetherlandsTC meeting Bolton (Jan 8, 2007)
Aims & planning
(TC Task 7.5; started June 1, 2006; lead by OUNL)• Aim: Delivering navigation service to advise next most suitable learning activity (and –path).• Status: Currently Phase 1 (proof of concept) June 2006-June 2007, including pilot at OUNL with CMS (Moodle) and service designed in PHP / MySQL.• Techniques: Predictions based on combination of collaborative filtering (matrix with finished learning activities by
other learners) and group profiles (motivation, time, interest)
• Testing: First TC pilot (!) from Oct 2, 2006 to Jan 24, 2007; started with 66 students (2x 33), currently around 160 (2x 80) students registered
• Test and evaluate the effect of a personalized recommendation service on learning behavior in learning networks
• The group with the personalized advise is expected to become more efficient, with possible indicators being:
study time (faster), results on grades (higher, faster), completion (more, faster)
Focus (one at a time) or variety (less), satisfaction (more)
Experimental design (1)
Experimental design (2)
Exp. Group
100 students
Using indirect social navigation
(collaborative filtering information) and profile
information (time, motive, interest)
to personalizerecommendation for
next ‘course’
Personalized advise
Control Group
100 students
Randomized list of ‘courses’
No advise
Effectiveness
Efficiency
Satisfaction
Efficiency and Effectiveness:• Effectiveness: Students with PRS will be more satisfied with
learning, will participate more, will get closer to completion, will get higher progress test and exam results
• Efficiency: Students with PRS will achieve the same test results and completions in less time
Variety:• Students with PRS will persevere more to finish one course at a
time, because they study content personalized to their profile• For students with PRS, the learning path will be more ‘linear’.
Assumptions
Planning Phase 1
October 06Maintain
Experiment
November 06
Data CollectionExperiment
December 06 Data Collection
Experiment
January 07 First Results
Proof of Concept
February 07 Results Proof
of Concept
March 07 Service
Development
April 07 Service
Development
May 07 Testing
June 07 First release
recommend course based on
MATRIX-> A | B | C
[available]
PRS Advise
[not available]
Check if course is part of matrix A
[not part of A]
Remove study timecheck if part of matrix B
[yes]
recommend course based on direct
subdomain interest
use personal interest in subdomain;
courses unfinished?
Get recommendation
Already finished course by user available?
Remove study motivecheck if part of matrix C
[not part of B] [not part of C]
recommend coursebased on indirect
subdomain interest[topdomain]
[no][yes] [yes] [yes]
Recommendation created
with a first prediction strategy
PROVIDE INFORMATION ABOUT PRIOR KNOWLEDGE
UPLOAD DOCUMENTS
Documentation of the recommendation service (Version: 11/23/06):
Interface / object layer: The recommendation service is clustered into two main layers. The basic communication functions are located in the interface layer. The interface layer is responsible for getting all required data from connected services and profiles.On top of this layer is the object layer located. The object layer is in charge of creating a reasonable profile for the requesting learner. This contains a collection of required profile information and suitable learner groups for recommendation.
Profile class: The Profile class is like the Position class part of the interface layer between the recommendationEngine and the connected objects in 10CC domain model. It is responsible for getting required data regarding for the profile of the learner. The function getProcessData(learnerID) submit a learner id to the actor class and returns a list called profileData. The list profileData contains required information for the recommendationEngine from the actor class.The functiongetStudiedLearningActivities(learnerID) submit a learner id to the process Log class and returns a list with finished learning activities. Already completed learning activities will be stored in the listOfExcludedNodes which is part of the learner class.
Position class: The Position class is like the Profile class part of the interface layer between the recommendationEngine and connected objects in 10CC domain model. It is responsible for getting the current position of the learner regarding to the LN. It works as an interface between the positioning service and the recommendation service. It provides data for recommendationEngine that is taking into account for the recommendations.
The function getPosition(learnerID,learningGoal) will submit a learner id and a learning goal to the positioning service. It returns a list of nodes that are stored in the array listofExcludedNode which is part of the learner class. Additional to this list could the positioning service provide some metadata. This metadata information is stored in the array listOfMetaData in the Learner class.In the ISIS pilot is no positioning class implemented, because no positioning service was available.
Learner: The Learner class gathers all required data about learners. It is used for creating a character for the requesting learner as well as input for the learner group class. It requires the Profile and Position class from the interface layer to get required data for the requesting learner. The result of the Position class could be null, but any feedback from this class is required as input for the recommendation strategy. The Profile class delivers always information to the learner class.The function getPosition(learnerId) is requesting the getPosition(learnerId) function in the Profile class to get data. The same happens with the getProfile() function. Both functions of this class provide required parameter like learnerID or a learningGoal to the classes of the interface layer.
CurrentLearner: The CurrentLearner class is an instance of the learner class. It represents the requesting learner and provides all information for this learner for the recommendationEngine.
Learner Group: The LearnerGroup class generates a list of helpful learners if possible. It gathers all required data about helpful learners regarding to the current learner. It uses the learner class to select matching learners and provide a list with people to the RecommendationEngine.The function lookForHelpfulLearners() checks the LN for learners with similar learning goals, interest or reason to study (studytime, studymotivation). The cardinality relationship describe that there could be no or more than one suitable LearnerGroup available for the current learner. Recommendation Engine: The RecommendationEngine is the heart of the recommendation service.It calculates which recommendation technique is suitable for the current learner. Therefore a specific recommendation strategy is implemented. This strategy decides which prediction technique is most suitable to cover the needs of the current learner. For a recommendation it takes into account the data given by CurrentLearner and the LearnerGroup.The functions getCurrentLearner() uses the CurrentLearner class as a starting point for the recommendation strategy. The function getSuitableLearnerGroup() look for learners with similar profile and already done learning activities. The function chooseRecommendationStrategy() uses the recommendation strategy to come up with a prediction for the current learner. The function priory’s suitable learning activities regarding to the recommendation strategy.
The function provideListOfRecommendations() delivers the output of the recommendationEngine to the learner.
The experimental Group finish more Courses.
1 614 19
2936
4354
69 7387
2 617
2936
44
6675
8495
120
0
20
40
60
80
100
120
140
time checkpoints
Com
plet
ed C
ours
es(G
roup
Lev
el)
control group 1 6 14 19 29 36 43 54 69 73 87
experimental group 2 6 17 29 36 44 66 75 84 95 120
06.10
13.10
20.10
27.10
03.11
10.11
17.11
24.11
01.12
08.12
15.12
Individuals in the Experimental Group finish more Courses successful.
0,00
0,50
1,00
1,50
2,00
2,50
time checkpoints
Avg
of c
ompl
eted
C
ours
es p
er L
earn
er(In
divi
dual
Lev
el)
control group 0,10 0,40 0,60 0,80 1,10 1,20 1,30 1,50 1,90 1,80 1,80
experimental group 0,20 0,40 0,80 1,20 1,30 1,40 1,80 2,00 2,00 2,20 2,00
06.10
13.10
20.10
27.10
03.11
10.11
17.11
24.11
01.12
08.12
15.12
The Experimental Group need less Studytime to finish Courses successful.
0
50
100
150
200
250
time checkpoints
Avg
of
Tim
e(G
rou
p L
evel
)
control group 67,5 143,4 163,2 165 195,4 202,7 211,7 194,2 185 186,4 196,5
experimental group 129,2 129,2 158,8 143,1 150,8 162,5 170,2 172 173,2 178,4 185,3
06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12
Individuals in the Experimental Group need less Studytime to finish Courses successful.
0,00
2,00
4,00
6,00
8,00
10,00
12,00
time checkpoints
Avg
of T
ime
per
Cou
rse
per L
earn
er(In
divi
dual
Lev
el)
control group 5,60 9 6,5 6,6 7,5 6,5 6,4 5,5 5 4,7 4,5
experimental group 10,80 8,1 7,6 6 5,4 5,2 4,7 4,5 4,2 4,1 3,8
06.10
13.10
20.10
27.10
03.11
10.11
17.11
24.11
01.12
08.12
15.12
Usage of Recommendation Technique
3345
71
9097
113
131 137 143151
164
0 0 2 5 7 12
3038
4759
77
0
20
40
60
80
100
120
140
160
180
time checkpoints
ontology 33 45 71 90 97 113 131 137 143 151 164
collaborative f iltering 0 0 2 5 7 12 30 38 47 59 77
06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12
Total Amount of Starters per Week
1216
25 25 26
3133
3537
40 41
1216
2124
2831
3638
4144
47
0
5
10
15
20
25
30
35
40
45
50
time checkpoints
control group 12 16 25 25 26 31 33 35 37 40 41
experimental group 12 16 21 24 28 31 36 38 41 44 47
06.10 13.10 20.10 27.10 03.11 10.11 17.11 24.11 01.12 08.12 15.12
7.5: Risks
• Proof of concept delivers no useable results
• Collaborative filtering is not properly working because of little students
• Discuss interface definition between the navigation and other services, esp. positioning• Discuss SOA design of a navigation service
Joint publications on:- Visualisation of Navigation (courses and paths) in Learning Networks (with UPF)- Profiling of Learners, integration with positioning service in Learning Networks (with L3S)
7.5: Collaboration
Links
http://elo.ou.nl(hah + OUPW)
http://moodle.psychologie.ou.nl/moodle/(demo + de&MO$)
http://dspace.ou.nl/handle/1820/830(draft article on combined approach)
Top Related