Post on 27-Mar-2015
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20041
A Multi-criteria Model to Support A Multi-criteria Model to Support Automatic Recommendation of e-Automatic Recommendation of e-
Learning Quality ApproachesLearning Quality Approaches
Nikos Manouselis, Demetrios G. Sampson
e-mail:{nikosm,sampson}@iti.gr
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20042
short overviewshort overviewQuality in e-learning
Variety of approachesNeed for selection
Recommender systemsRecommendation tasksRecommendation techniques
European Quality Observatory (EQO)Model for describing e-learning quality approachesOnline repository with quality approachesRecommendation mechanismExample scenario
conclusions - future directions
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20043
Quality in e-learningQuality in e-learningQuality: an issue of increasing importance
in e-learningSeveral approaches developed worldwide
They address quality control, assurance, and assessment dimensions of quality in e-learning
A variety of available methods, criteria, processes, instruments etc.
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20044
Variety of approachesVariety of approaches The various approaches have different priorities,
focusing on different e-learning quality aspects How can e-learning researchers and
practitioners identify and select interesting or appropriate approaches for their application scenarios?
Issues: Collection / Categorization of quality approachesRecommendation of quality approaches
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20045
Variety of approachesVariety of approaches The various approaches have different priorities,
focusing on different e-learning quality aspects How can e-learning researchers and
practitioners identify and select interesting or appropriate approaches for their application scenarios?
Issues: Collection / Categorization of quality approachesRecommendation of quality approaches
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20046
Recommender systemsRecommender systems “Any system that produces individualized
recommendations as output or has the effect of guiding the user in a personalized way to interesting or useful items in a large space of possible options” [Burke, 2002]
They help individuals in a community of users to identify items of interest from a potentially overwhelming set of choices [Resnick & Varian, 1997]
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20047
Recommendation tasksRecommendation tasks
Recommendation broadly used to support two main user tasks [Herlocker et al., 2004]:“Find Good Items”: providing users with ranked
lists of recommended items based on generated predictions
“Annotation in Context”: generating predictions for an item that the user is currently viewing
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20048
Recommendation techniquesRecommendation techniques Collaborative recommendation
Recognizing commonalities between users on the basis of commonly rated items
Demographic recommendation Based on demographic classes of users
Content-based recommendation Matching descriptions of items with user profiles
Utility-based recommendation Based on a utility function for each user
Knowledge-based recommendation Using knowledge structures for reasoning
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 20049
European Quality ObservatoryEuropean Quality Observatory European Quality Observatory (EQO) initiative
Funded by the EC eLearning ProgrammeOne of the four initiatives specifically focusing on e-learning
quality Aims:
To develop a metadata model for describing and categorizing quality approaches
To collect quality approaches and store them in an online repository
To support e-learning researchers and practitioners in identifying quality approaches suitable for their application needs
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200410
EQO ModelEQO Model Metadata Model for the description and
categorization of quality approaches for e-learning
Extending an initial model of the CEN/ISSS Learning Technologies Workshop (LTW) Quality Assurance GroupExpanding description and categorization capabilities of
initial CEN/ISSS modelAllowing for the description and assessment of a quality
approach application
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200411
Structure of ModelStructure of ModelEQO Model v.1.2Four main
categories of elements
(available from
http://www.eqo.info)
GENERAL
CONTEXT
METHOD
EXPERIENCE
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200412
EQO web portalEQO web portal Provides access to
the online repository Users can
search/contribute descriptions of quality approaches
Users can search/contribute experiences from quality approaches
http://www.eqo.info
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200413
EQO Recommendation ModuleEQO Recommendation Module Provides recommendations to the users of the EQO
web portal when searching for quality approaches in the repository
Utility-based recommender, calculating the suitability of a quality approach (QA) for given application context prioritiesMultiple criteria engagedRanking available quality approaches (Find Good Item
task)
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200414
Recommendation Life-CycleRecommendation Life-Cycle Quality Approach Contribution Phase
Expert contributing a quality approach provides its intended priorities and an initial estimation of its suitability
Users contributing an experience express how suitable they found the quality approach for priorities of their application context
Quality Approach Recommendation Phase User searching for a suitable quality approach specifies
the priorities of the intended application Recommendation module calculates the suitability of a
quality approach
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200415
Recommendation Life-CycleRecommendation Life-Cycle
QA Contribution
Priorities of QA
Suitability of QA
Expert contributing Quality Approach to
EQO repository
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200416
Recommendation Life-CycleRecommendation Life-Cycle
QA Contribution
Priorities of QA
Suitability of QA
Expert contributing Quality Approach to
EQO repository
Experience Contribution
Priorities of application
Suitability of QA
User contributing Experience from the Quality Approach to
EQO repository
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200417
Recommendation Life-CycleRecommendation Life-Cycle
QA Contribution
Priorities of QA
Suitability of QA
Expert contributing Quality Approach to
EQO repository
QA Recommendation
Priorities of intended application
User providing priorities of the
application context for which Quality
Approach should be suitable
Experience Contribution
Priorities of application
Suitability of QA
User contributing Experience from the Quality Approach to
EQO repository
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200418
Recommendation Life-CycleRecommendation Life-Cycle
QA Contribution
Priorities of QA
Assessment of QA
Expert contributing Quality Approach to
EQO repository
QA Recommendation
Priorities of intended application
User providing priorities of the
application context for which Quality
Approach should be suitable
CONTRIBUTION PHASE RECOMMENDATION PHASE
Experience Contribution
Priorities of application
Assessment of QA
User contributing Experience from the Quality Approach to
EQO repository
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200419
Recommendation Life-CycleRecommendation Life-Cycle
QA Contribution
Priorities of QA
Assessment of QA
Expert contributing Quality Approach to
EQO repository
QA Recommendation
Priorities of intended application
User providing priorities of the
application context for which Quality
Approach should be suitable
CONTRIBUTION PHASE RECOMMENDATION PHASE
Experience Contribution
Priorities of application
Assessment of QA
User contributing Experience from the Quality Approach to
EQO repository
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200420
Criteria for calculating suitabilityCriteria for calculating suitability Expressing the focus of a quality approach upon several
criteria In our case, adopted from [Frydenberg, 2002] the following:
1. Focus on Institutional & executive organization / commitment
2. Focus on Technological infrastructure3. Focus on Student services4. Focus on Instructional design and course development5. Focus on Instruction and instructors6. Focus on Program delivery7. Focus on Financial management8. Focus on Regulatory and legal compliance9. Focus on Evaluation and assessment procedures
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200421
Providing Providing search prioritiessearch priorities
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200422
RecommendationRecommendation resultsresults
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200423
Example scenario (1/3)Example scenario (1/3)
Two users A and B searching for suitable QAs
Each user has different priorities
12 QAs are stored in the repository, they are all candidate for recommendation
g
User A
User B
1 65 25
2 55 55
3 85 45
4 70 85
5 85 35
6 95 55
7 45 60
8 35 75
9 25 70
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200424
Example scenario (2/3)Example scenario (2/3)UQA for User A UQA for User B
1. QA12 0,152 1. QA4 0,094
2. QA7 0,088 2. QA7 0,082
3. QA2 0,078 3. QA6 0,077
4. QA3 0,076 4. QA2 0,076
5. QA6 0,072 5. QA3 0,073
6. QA11 0,065 6. QA9 0,070
7. QA9 0,067 7. QA12 0,069
8. QA4 0,061 8. QA11 0,068
9. QA5 0,060 9. QA10 0,059
10. QA10 0,057 10. QA5 0,056
11. QA8 0,048 11. QA8 0,047
12. QA1 0,000 12. QA1 0,000
The suitability of each QA in the EQO repository is calculated
The QAs are ranked according to their suitability
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200425
Example scenario (3/3)Example scenario (3/3)
1 2 3 4 5 6 7 8 9 10 11 12
QA #
Su
itab
ilit
y C
om
par
iso
n
User A
User B
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200426
Conclusions – Future directionsConclusions – Future directions
The recommendation module seems to be providing useful hints to the users of the EQO portal evaluation from EQO users is required
A utility-based recommender technique seems appropriate for cases similar to the EQO case study where multiple criteria are required to describe items other techniques could be also examined
The criteria for calculating suitability could be furthermore elaborated and validated from the users explore the effect of user role in the definition of criteria validation of criteria through user survey
University of PiraeusDepartment of Technology Education and Digital Systems
Centre for Research and Technology - Hellas(C.E.R.T.H.) Informatics and Telematics Institute (I.T.I.)
N. Manouselis & D.G. Sampson EDMEDIA’04, Lugano, Switzerland, 24th June 200427
Contact DetailsContact Details
Nikos Manouselis and Prof. Demetrios Sampsonnikosm@iti.gr, sampson@iti.gr
http://www.ask.iti.gr
Informatics and Telematics Institute, Centre for Research and Technology Hellas,
42, Arkadias Street,Athens,GR-15234,Greece&
Department of Technology Education and Digital Systems,University of Piraeus
150, Androutsou Street, Piraeus, GR-18534 Greece