on the COLLECTION of QUESTIONNAIRES
Transcript of on the COLLECTION of QUESTIONNAIRES
SEVAQ - European report Version 2
Appendix 2. Details of the assessments08/03/2006 Page 1 of 1
Self-Evaluation and
Quality in
E-learning
EUROPEAN REPORT on the COLLECTION of QUESTIONNAIRES
COUNTRY EFODL
SEVAQ PARTNERS BE-ODL, BLA, CESI, ISQ, NADE, Le PREAU, VDU, VIDEOSCOP
Rik Dalle
E-MAIL [email protected] This is the report of the first phase of the SEVAQ project in which the participating partners have enquired the status of existing evaluation instruments for the assessment of e-learning courses. In the first phase of the SEVAQ project we have concentrated on the use of questionnaires that are presented to the users of e-learning courses who are the learners. These questionnaires could be presented to the learners in an on-line form or in paper form. In this report we describe the results for the whole of the European contributions. We draw conclusions and remarks on the different topics of the enquiry and on the whole of the investigation. We also point to differences in the results between the different country samples. For the conclusions on each topic, we have based our results also on the country reports provided by each SEVAQ partner. For more details on the results of the enquiry, we refer to these country reports. We have to take into account that some sample were rather small, so that we have to consider the conclusions with care. The first chapter of the report provides an overview of the results from the SEVAQ mailing sent to a large number of organisations in the participating countries, while the second chapter describes the results of the assessments by the SEVAQ partners of the questionnaires sent by the respondents. CHAPTER 1. THE MAILING RESULTS
For each topic we present tables with the total of all European data and a histogram displaying the distribution per partner. The details of the scores for each country are presented in appendix 1.
SEVAQ - European report Draft
08/03/2006 Page 2 of 2
THE MAILING
Number Percentage
Number of mails sent 9170 --- Number of mail replies received 181 1,97 % Number of organisations that sent their questionnaire(s) 78 0,85 % Number of questionnaires received 152 Mean number of questionnaires sent per organisation that sent mails 1,95
The participating partners in the SERVAQ project have sent out a large number of mails to a variety of organizations that are possibly dealing with e-learning. From the 9170 mails that were sent out, not even 2 percent of answers were received and les than 1 % of the organizations sent one or more questionnaires. The number of questionnaires that were sent was a bit disappointing. The Belgian partner also organized a telephone enquiry to organizations that hadn’t replied to the mailing. For a report on the telephone enquiry we refer to …
GENERAL INFORMATION
What was the profile of the organisations that replied on the mailing?
1. Type of organisation
Type Number PercentageNo answer 24 13
Company 48 25
Non profit organisation 19 10
Educational institute 99 52
Total 190 100
SEVAQ - European report Draft
08/03/2006 Page 3 of 3
041116
10
1
12
10
611
17
01
53101540
5334
25
17
30
12
0
10
20
30
40
50
60
70
80
90
100
Number of answers
No answer Company Non profit organisation Educational instituteType of organisation
SEVAQ - TYPE OF ORGANISATION
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP In the European sample more than half of the respondents were educational institutes like universities and a quarter were companies. There are however clear differences between the countries. Some countries like Norway, France (Vidéoscop), Lithuania and Portugal have a large majority of educational respondents. In Belgium, the United Kingdom and Spain, at least 50 % of the responders are companies. At Le Préau there is an equal amount of companies and educational organisations. The educational institutions in the Préau sample don’t include universities but private educational organisations that form an important market in France. The conclusions drawn on the type of organisations are partly due to the different compositions of the networks of the partners involved, and so of the composition of their mailing lists. Some networks are merely oriented to the educational world (universities or not academic), while others are more oriented towards commercial enterprises.
2. Provider or user of e-learning
Number Percentage No answer 26 14 Provider 102 54 User 17 9 Provider and user 45 24
Total 190 100
SEVAQ - European report Draft
08/03/2006 Page 4 of 4
241116
101
8
11
21
25
23
22
10
60100730
657429
9
3
0
20
40
60
80
100
120
Number
No answer Provider User Provider and userProvider or user
SEVAQ - PROVIDER OR USER
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 78 % of the respondents are either provider or provider and user. 33 % is also a user of e-learning courses. In the Norvegian sample we found the largest share of pure providers (89 %) in a group that are mainly educational institutes. From these results we can conclude that the providers that are most represented in the enquiry are the effective target of the SEVAQ research and are also interested in the results of this project. Probably the providers are “waiting” for general quality standards for e-learning and an appropriate tool to measure quality.
3. Size of organisation
Number of employees Number Percentage No answer 29 15 < 50 64 34 50 – 200 22 12 200 – 500 19 10 > 500 56 29
Total 190 100
SEVAQ - European report Draft
08/03/2006 Page 5 of 5
3
4112
6
11
1
2
9
21
11
16
23
0
13106
10
10
5002
7
230
11
4
7
22
11
6
13
0
10
20
30
40
50
60
70
Number
No answer < 50 50 – 200 200 – 500 > 500Size of organisation
SEVAQ - SIZE OF ORGANISATION
BE-ODL N BLA N CESI N ISQ N NADE N LE PREAU N VDU N VIDEOSCOP N Conclusions and comments on the type and size of the organisations Two groups are mostly present in the results: the small organisations of less than 50 employees (34 %) and the large organisations with more than 500 employees (29 %). We can observe some differences between the countries. Small organisations of less than 200 employees are less represented in the Belgian sample (14 %). We suppose that most of the e-learning players in Belgium are large organisations, especially large companies. We see the same profiles in Spain. Videoscop (93 %) and also ISQ have a majority of large (educational) institutes (universities). At the other side, the BLA (45 %), Le Préau (36 %), NADE (39 %) and VDU (52 %) have significant proportions of small organisations (< 50) in their samples.
INFORMATION REGARDING THE QUESTIONING AND EVALUATION
1. Can the e-Learning student and trainer give feedback concerning the e-learning course?
Answers Number Percentage No answer 25 13 Yes 155 82 No 10 5
SEVAQ - European report Draft
08/03/2006 Page 6 of 6
Total 190 100
041106
121
22
12
105
28
38
28
12
04000141
0
20
40
60
80
100
120
140
160
Number
No answer Yes NoAnswers
SEVAQ - CAN LEARNER GIVE FEEDBACK?
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP It is not surprising that almost 82 % want to receive feedback from the learners. A reason for a negative answer can be that the organisation is not yet dealing with e-learning. The reasons why organisations ask for feedback can be very different: - questionnaires are made for the customer and to measure the quality of the serviced provided; - questionnaires are used to evaluate the quality of the provider and to measure the ROI; - in a process of change, feedback is used to measure the receptiveness of learners to this new way of learning; - feedback can also be part of a formal quality management programme; - to improve their system; - to feed their reflection on distance learning.
2. The form of the feedback
Form Number PercentageNo answer 38 20 A written questionnaire 76 40 An online form 93 49 Informal conversation (e.g. during coffee break) 25 13
SEVAQ - European report Draft
08/03/2006 Page 7 of 7
Telephone conversation 27 14 Other 26 14
Total 285 ---
181117
16
3
9
644
10
19
17
7
11
11
72
18
16
23
5
42323461
433208
61
4223010
41
0
10
20
30
40
50
60
70
80
90
100
Number
No answer A writtenquestionnaire
An online form Informalconversation (e.g.
during coffeebreak)
Telephoneconversation
Other
Form of feedback
SEVAQ - FORM OF FEEDBACK
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP The majority of the feedback is gathered in a structured way through a written or an online questioning. In 49 % of the organisations online feedback is being used. At Le Préau, Vidéoscop and ISQ written questionnaires are used more than online ones. This can be due to blended learning where paper versions can be used more frequently. We can conclude that online questionnaires are getting common in e-learning. Organisations can also use other channels as e-mail, videoconferencing or direct assessment by the controller of the worker. It is obvious that the respondents often use more than 1 way of evaluation. The gathering of this information is not finished yet. In the samples of Le Préau (40 %), Vidéoscop (42 %) and BE-ODL (37 %) a significant group used 2 to 4 ways of evaluation. Number of evaluation types Number PercentageNo answer 1 way of evaluation
SEVAQ - European report Draft
08/03/2006 Page 8 of 8
2 ways of evaluation 3 ways of evaluation 4 ways of evaluation
Total To be finished.
3. Use of the feedback obtained
Feedback Number PercentageNo answer 38 20 Feedback is being archived 53 28 Feedback is made available to the trainer 95 50 Feedback is used to improve the quality of the evaluation system
92 48
Feedback is used to improve the quality of the e-learning course
124 65
Other 16 8 Total 418 ---
471126
16
1
4321
20
12
65
9
552
25
24
16
9
9
9
74
18
18
17
10
15
13
82
24
32
19
11
21121810
0
20
40
60
80
100
120
140
Number
No answer Archive Available to trainer Quality of evaluation Quality of e-learning OtherUse of feedback
SEVAQ - USE OF FEEDBACK
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 65 % of the organisations are using the feedback to improve the quality of the e-learning course. The other most important objectives of feedback are giving feedback to the trainer (50 %) and to improve the quality of the evaluation system
SEVAQ - European report Draft
08/03/2006 Page 9 of 9
(48 %). These scores are very homogenuous spread over the countries. Only in Norway there is a significant high score for making the feedback available for the teacher. There are also other objectives for feedback as focussing on the outcomes of the course and the achievement of the objectives. Feedback can also be delivered to e-learning professionals like designers, tutors, authors and others. Organisations have more than one objective to assess the course: in France more than 80 % have more than one reason to let the learner assess the courses, while in Belgium this is only 46 %. These data are further collected. Number of objectives in doing evaluation Number PercentageNo answer Just one use of the feedback 2 objectives of feedback 3 objectives of feedback 4 objectives of feedback 5 objectives of feedback
Total To be finished.
4. Preceding and interim evaluation
Do the organisations work with preceding and/or interim evaluations in addition to the standard evaluation after the course? Answers Number Percentage No answer 30 16 Yes 80 42 No 80 42
Total 190 100
SEVAQ - European report Draft
08/03/2006 Page 10 of 10
341106
12
3
8
6
5
4
16
19
18
4
11
10
51
12
20
14
7
0
10
20
30
40
50
60
70
80
Number
No answer Yes NoAnswers
SEVAQ - PRECEDING / INTERIM EVALUATION
BE-ODL N BLA N CESI N ISQ N NADE N LE PREAU N VDU N VIDEOSCOP N 42 % of the organisations work with preceding and/or interim evaluations. In some samples it seems more common (BLA 50 % and NADE 57 %), while Videoscop (29 %) and BE-ODL (36 %) have lower scores on this item.
KEEPING THE ORGANISATIONS INFORMED ABOUT THE SEVAQ PROJECT
Can we keep the organisation informed about the SEVAQ project? Answers Number Percentage No answer 190 100 Yes 0 0 No 0 0
Total 190 100 100 % didn’t answer this question. After observing all European scores, we can conclude that this score is due to a technical bug.
SEVAQ - European report Draft
08/03/2006 Page 11 of 11
CHAPTER 2. ANALYSIS of THE ASSESSMENT OF THE QUESTIONNAIRES
In this chapter we describe the results of the assessments of the questionnaires of the respondents. These assessments were executed by the SEVAQ partners on the sample that each partner had collected. We used 4 main criteria with a few subcriteria: content, usability and delivery, technical specifications and organisation. The subcriteria were assessed in a range from 1 to 5: 1 = not at all 2 = a little bit 3 = average 4 = quite a lot 5 = very much In the next pages we discuss the main results per criteria. The detailed scores and graphs of the subcriteria can be found in appendix 2. We refer to “SEVAQ-e-learning-EFQM1.doc” by Jeanne Schreurs for a theoretical foundation of the set of quality criteria.
1. CONTENT
We measured 6 topics in Content criterion: 1.1. The questionnaire inquires about the matching of the learning objectives.
1.2. The questionnaire inquires about the learning level.
1.3. The questionnaire inquires about the foreknowledge of the learner.
1.4. The questionnaire inquires about the difficulty level of the e-learning course.
1.5. The questionnaire inquires about the pace of the e-learning course.
1.6. The questionnaire inquires about the availability and quality of the assignments, tests and exercises.
The results for all the SEVAQ partners together are displayed in the next table and graph.
1 2 3 4 5
1.1 33 26 27 15 6
1.2 36 33 22 13 2
1.3 56 22 16 8 5
1.4 43 31 18 14 2
1.5 45 40 11 7 4
1.6 33 18 33 14 8
SEVAQ - European report Draft
08/03/2006 Page 12 of 12
CONTENT
0
10
20
30
40
50
60
1 2 3 4 5
levels
scor
es
1.1 1.2 1.3 1.4 1.5 1.6
Conclusions and comments about the content criterion Generally, the scores are towards the low end. Only a small part of the questionnaires are concerned about the learners competencies development during the course. The best results are for 1.1 matching of the learning objectives and 1.6 availability and quality of the assignments, tests and exercises. The subcriterion with the lowest general scores is 1.5 on the pace of the course. Some mention that questionnaires are dealing with other questions like the resources of the course, its relevance and its professional breadth and depth.
SEVAQ - European report Draft
08/03/2006 Page 13 of 13
2. USABILITY AND DELIVERY
We measured 5 topics in this criterion: 2.1. The questionnaire inquires about the quality of the navigation in the course.
2.2. The questionnaire inquires about the quality of integrated communication.
2.3. The questionnaire inquires about the quality of embedded tutoring.
2.4. The questionnaire investigates whether the learning environment or e-course is motivating, stimulating or encouraging to learn (fun).
2.5. The questionnaire asks about the feedback to the end user concerning his progress, scores, etc.
The results
1 2 3 4 5
2.1 60 13 18 10 4
2.2 53 24 15 9 3
2.3 49 21 15 11 9
2.4 55 22 15 9 3
2.5 43 23 15 18 6
USABILITY
0
10
20
30
40
50
60
70
1 2 3 4 5
levels
scor
es
2.1 2.2 2.3 2.4 2.5
SEVAQ - European report Draft
08/03/2006 Page 14 of 14
Comments and conclusions about the usability and delivery The scores of the usability and delivery criteria are also towards the low end. The attention for usability is rather limited. Some remark that usability is not really a problem anymore, although I doubt about that. There are no questions found on embedded tutoring in Belgium, although especially Norway and France found a number of questionnaires that showed some interests in this matter. Maybe this has to do with the usage of embedded tutoring that is yet made in the different countries. This was also a remark that tutoring mostly concerns about human interaction and not the embedded way.
SEVAQ - European report Draft
08/03/2006 Page 15 of 15
3. TECHNICAL SPECIFICATIONS
We measured 5 subcriteria in this category: 3.1. The questionnaire inquires about the set up (installation) of the e-learning environment.
3.2. The questionnaire inquires about the session start-up, the performance and uploading of the pages.
3.3. The questionnaire inquires about the browser independency of the learning environment.
3.4. The questionnaire inquires about the availability and system stability.
3.5. The questionnaire inquires about the quality of the helpdesk facilities.
The results
1 2 3 4 5
3.1 72 10 9 10 3
3.2 58 24 11 9 2
3.3 90 11 2 1 0
3.4 70 14 8 9 2
3.5 62 18 13 6 3
TECHNICAL SPECIFICATIONS
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5
levels
scor
es
3.1 3.2 3.3 3.4 3.5
SEVAQ - European report Draft
08/03/2006 Page 16 of 16
Comments and conclusions about the technical specifications There is also a minor attention to the technical aspects of the learning management systems and the e-courses. The “best” score, although still very weak, is found in 3.2 start-up, performance and uploading of the pages. If there are some questions about technical issues, they seem to be very general.
SEVAQ - European report Draft
08/03/2006 Page 17 of 17
4. ORGANISATION
We measured 2 topics in this category: 4.1. The questionnaire inquires about the welcoming at the beginning of the course.
4.2. The questionnaire assesses the following administration topics: course registration, scheduling of the training programme, payment, and other information offered by the learning organisation.
The results
1 2 3 4 5
4.1 60 26 11 4 5
4.2 42 27 13 15 7
ORGANISATION
0
10
20
30
40
50
60
70
1 2 3 4 5
levels
scor
es
4.1 4.2
Comments and conclusions about the organisation of e-learning There is only a minor interest in organisation matters, although 4.2 about the administration topics show generally a relatively good score. Partners report that questions sometimes ask about the locations where learners go online and about the possibility to use an internet connection.
SEVAQ - European report Draft
08/03/2006 Page 18 of 18
GENERAL CONCLUSIONS CONCERNING THE ASSESSMENT OF THE PARTICIPATING ORGANISATIONS AND THEIR QUESTIONNAIRES
General conclusions concerning the Belgian examination: - The questionnaires in the European samples scored rather low in the 4 criteria
we had chosen. - The criteria in our inquiry did possibly lack some important themes or some
criteria were to general (like coaching) and need to be more detailed.. - Typical and important aspects of online learning are seldom evaluated: usability
and technical issues. Although, these matters can be significant in the success of e-learning.
- Some remarks noticed that a number of questionnaires that organizations sent where questionnaires that are also used in face to face education. These are often very general questionnaires or they ask for topics that are typical for class room teaching as questions about the quality of the location, the teacher, the catering and the hand outs. That influenced also the assessments.
Some partners have added imported recommendations for further research so that other interesting criteria can be added in the tool that has to be developed in the next stages of the SEVAQ project. I refer to the different country reports. We can conclude from this enquiry that it’s very obvious that there is a need for an evaluation tool that is created to assess the quality of e-learning. This conclusion has to be finished further.
SEVAQ - European report Draft
Appendix 2. Details of the assessments08/03/2006 Page 19 of 19
APPENDIX 1. DETAILS OF THE MAILING RESULTS General information concerning the mailing
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE
Number of mails sent 1500 2500 583 20 350 3300 359 558 9170 Number of mail replies received 26 20 11 6 15 45 44 14 181 Number of organisations that sent their questionnaire(s) 16 6 6 3 11 19 9 8 78
Number of questionnaires received 20 20 11 6 28 45 10 12 152 1. Type of organization
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 0 4 1 1 1 6 10 1 24 Company 12 10 6 1 1 17 0 1 48 Non profit organisation 5 3 1 0 1 5 4 0 19 Educational institute 5 3 3 4 25 17 30 12 99
Total 22 20 11 6 28 45 44 14 190 2. Provider or user of e-learning
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 2 4 1 1 1 6 10 1 26 Provider 8 11 2 1 25 23 22 10 102 User 6 0 1 0 0 7 3 0 17 Provider and user 6 5 7 4 2 9 9 3 45
Total 22 20 11 6 28 45 44 14 190
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006 Page 20 of 20
3. Size of organisation
Number of employees BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 3 4 1 1 2 6 11 1 29 < 50 2 9 2 1 11 16 23 0 64 50 – 200 1 3 1 0 6 10 1 0 22 200 – 500 5 0 0 2 7 2 3 0 19 > 500 11 4 7 2 2 11 6 13 56
Total 22 20 11 6 28 45 44 14 190 INFORMATION REGARDING THE QUESTIONING AND EVALUATION 1. Can the e-Learning student and trainer give feedback concerning the e-learning course?”
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 0 4 1 1 0 6 12 1 25 Yes 22 12 10 5 28 38 28 12 155 No 0 4 0 0 0 1 4 1 10
Total 22 20 11 6 28 45 44 14 190
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006 Page 21 of 21
2. In which form?
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 1 8 1 1 1 7 16 3 38 A written questionnaire 9 6 4 4 10 19 17 7 76 An online form 11 11 7 2 18 16 23 5 93 Informal conversation (e.g. during coffee break)
4 2 3 2 3 4 6 1 25
Telephone conversation 4 3 3 2 0 8 6 1 27 Other 4 2 2 3 0 10 4 1 26
Total 33 32 20 14 32 64 72 18 285 BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 1 7 3 11 Just one way of applying evaluation 13 22 7 42 2 ways of applying evaluation 5 12 4 21 3 ways of applying evaluation 3 2 0 5 4 ways of applying evaluation 0 1 1 2
Total 22 0 0 0 0 44 0 15 81
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006 Page 22 of 22
3. Use of the feedback obtained
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 4 7 1 1 2 6 16 1 38 Archive 4 3 2 1 20 12 6 5 53 Available to trainer 9 5 5 2 25 24 16 9 95 Quality of evaluation 9 9 7 4 18 18 17 10 92 Quality of e-learning 15 13 8 2 24 32 19 11 124 Other 2 1 1 2 1 8 1 0 16
Total 43 38 24 12 90 100 75 36 418 BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 4 6 1 11 Just one way of applying evaluation 8 6 2 16 2 ways of applying evaluation 3 15 2 20 3 ways of applying evaluation 3 9 4 16 4 ways of applying evaluation 4 5 4 13 5 ways of applying evaluation 0 2 0 2
Total 22 0 0 0 0 43 0 13 76
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006 Page 23 of 23
4. Preceding and interim evaluation Do the organisations work with preceding and/or interim evaluations in addition to the standard evaluation after the course? Preceding / interim evaluations BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 3 4 1 1 0 6 12 3 30 Yes 8 6 5 4 16 19 18 4 80 No 11 10 5 1 12 20 14 7 80
Total 22 20 11 6 28 45 44 14 190 KEEPING THE ORGANISATIONS INFORMED ABOUT the SEVAQ PROJECT Can we keep your organisation informed about the SEVAQ project?
BE-ODL BLA CESI ISQ NADE PREAU VDU VIDEOSCOP EUROPE No answer 22 20 11 6 28 45 44 14 190 Yes 0 0 0 0 0 0 0 0 0 No 0 0 0 0 0 0 0 0 0
Total 22 20 11 6 28 45 44 14 190
SEVAQ - European report Draft
Appendix 2. Details of the assessments08/03/2006 Page 24 of 24
1. CONTENT
1.1. The questionnaire inquires about the matching of the learning objectives.
1 2 3 4 5 BE-ODL 8 6 3 1 2 BLA 1 0 3 2 0 CESI 0 1 2 2 0 ISQ 4 0 0 0 0 NADE 6 7 8 4 0 LE PREAU 7 2 4 3 1 VDU 5 0 3 1 3 VIDEOSCOP 2 10 4 2 0 EUROPE 33 26 27 15 6
8
10
4
6
7
5
2
6
010
7
20
10
3
3
20
8
4
3
4
12
20
4
3
12
200001
30
0
5
10
15
20
25
30
35
Number
1 2 3 4 5Assessment
SEVAQ - 1.1 CONTENT - OBJECTIVES
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1.2. The questionnaire inquires about the learning level.
1 2 3 4 5 BE-ODL 14 3 1 1 0 BLA 1 1 3 1 0 CESI 0 1 3 1 0 ISQ 4 0 0 0 0 NADE 1 12 9 2 1 LE PREAU 4 3 4 5 1 VDU 8 4 0 0 0 VIDEOSCOP 4 9 2 3 0 EUROPE 36 33 22 13 2
14
104
1
4
8
4
3110
12
3
4
9
13
30
9
402
11102
5
03
00001100
0
5
10
15
20
25
30
35
40
Number
1 2 3 4 5Assessment
SEVAS - 1.2 CONTENT - LEARNING LEVEL
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1.3. The questionnaire inquires about the foreknowledge of the learner.
1 2 3 4 5 BE-ODL 11 6 1 1 1 BLA 4 1 0 1 0 CESI 3 0 1 1 0 ISQ 4 0 0 0 0 NADE 7 6 9 2 1 LE PREAU 7 6 2 2 0 VDU 8 0 0 1 3 VIDEOSCOP 12 3 3 0 0 EUROPE 56 22 16 8 5
11
4
3
4
7
7
8
12
6
1006
603
1010
9
203
11102210
10001030
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessment
SEVAQ - 1.3 FOREKNOWLEDGE
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1.4. The questionnaire inquires about the difficulty level of the e-learning course.
1 2 3 4 5 BE-ODL 11 4 2 3 0 BLA 2 2 0 2 0 CESI 2 3 0 0 0 ISQ 4 0 0 0 0 NADE 3 10 8 3 1 LE PREAU 6 3 7 1 1 VDU 7 2 1 2 0 VIDEOSCOP 8 7 0 3 0 EUROPE 43 31 18 14 2
11
22
4
3
6
7
8
4
2
30
10
3
2
7
2000
8
7
10
3
200312
3
00001100
0
5
10
15
20
25
30
35
40
45
Number
1 2 3 4 5Assessment
SEVAQ - 1.4 DIFFICULTY LEVEL
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1.5. The questionnaire inquires about the pace of the e-learning course.
1 2 3 4 5 BE-ODL 13 5 0 2 0 BLA 4 2 0 0 0 CESI 1 2 0 1 1 ISQ 3 1 0 0 0 NADE 10 11 3 0 1 LE PREAU 3 4 4 4 2 VDU 7 4 1 0 0 VIDEOSCOP 4 11 3 0 0 EUROPE 45 40 11 7 4
13
4
13
10
3
7
4
5
221
11
4
4
11
00003
4
13
20100400
00101200
0
5
10
15
20
25
30
35
40
45
Number
1 2 3 4 5Assessment
SEVAQ - 1.5 PACE
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1.6. The questionnaire inquires about the availability and quality of the assignments, tests and exercises.
1 2 3 4 5 BE-ODL 9 2 7 0 2 BLA 4 1 1 0 0 CESI 0 2 0 2 1 ISQ 0 0 0 4 0 NADE 7 8 6 3 0 LE PREAU 5 2 4 4 2 VDU 5 1 3 0 3 VIDEOSCOP 3 2 12 1 0 EUROPE 33 18 33 14 8
9
4
00
7
5
5
3
2120
8
212
7
100
6
4
3
12
002
4
3
4
01
201002
30
0
5
10
15
20
25
30
35
Number
1 2 3 4 5Assessments
SEVAQ - 1.6 ASSIGNMENTS & TESTS
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
2. USABILITY AND DELIVERY
2.1. The questionnaire inquires about the quality of the navigation in the course.
1 2 3 4 5 BE-ODL 17 0 2 0 1 BLA 3 0 1 2 0 CESI 2 0 1 2 0 ISQ 4 0 0 0 0 NADE 12 6 2 4 0 LE PREAU 5 2 6 2 1 VDU 8 1 1 0 2 VIDEOSCOP 9 4 5 0 0 EUROPE 60 13 18 10 4
17
324
12
5
8
9
00006
214
21102
6
15
02204200
10000120
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessments
SEVAQ - 2.1 NAVIGATION
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
2.2. The questionnaire inquires about the quality of integrated communication.
1 2 3 4 5 BE-ODL 14 4 1 1 0 BLA 3 3 0 0 0 CESI 2 2 1 0 0 ISQ 4 0 0 0 0 NADE 5 8 4 3 3 LE PREAU 5 4 3 4 0 VDU 9 0 2 1 0 VIDEOSCOP 11 3 4 0 0 EUROPE 53 24 15 9 3
14
324
5
5
9
11
4
320
8
403
10104
324
10003
410
00003000
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessments
SEVAQ - 2.2 INTEGRATED COMMUNICATION
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
2.3. The questionnaire inquires about the quality of embedded tutoring.
1 2 3 4 5 BE-ODL 20 0 0 0 0 BLA 4 1 1 0 0 CESI 2 0 1 0 2 ISQ 3 1 0 0 0 NADE 6 6 2 7 3 LE PREAU 5 3 3 2 3 VDU 6 5 0 0 1 VIDEOSCOP 3 5 8 2 0 EUROPE 49 21 15 11 9
20
4
23
6
5
6
3
0101
6
3
5
5
0110230
8
0000
7
202
00203
310
0
5
10
15
20
25
30
35
40
45
50
Number
1 2 3 4 5Assessments
SEVAQ - 2.3 EMBEDDED TUTORING
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
2.4. The questionnaire investigates whether the learning environment or e-course is motivating, stimulating or encouraging to learn (fun).
1 2 3 4 5
BE-ODL 15 1 3 0 1 BLA 0 3 1 2 0 CESI 3 1 0 1 0 ISQ 4 0 0 0 0 NADE 9 9 2 2 1 LE PREAU 6 4 4 1 1 VDU 9 0 2 1 0 VIDEOSCOP 9 4 3 2 0 EUROPE 55 22 15 9 3
15
03
4
9
6
9
9
1310
9
404
31002423
02102112
10001100
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessments
SEVAQ 2.4 MOTIVATION
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP 2.5. The questionnaire asks about the feedback to the end user concerning his progress, scores, etc.
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
1 2 3 4 5 BE-ODL 15 2 2 0 1 BLA 2 2 2 0 0 CESI 0 2 1 1 1 ISQ 0 0 0 4 0 NADE 8 6 3 6 1 LE PREAU 5 3 5 2 1 VDU 8 1 0 1 2 VIDEOSCOP 5 7 2 4 0 EUROPE 43 23 15 18 6
15
200
8
5
8
5
2220
6
31
7
22103
5
02
001
4
6
21
4
10101120
0
5
10
15
20
25
30
35
40
45
Number
1 2 3 4 5Assessments
SEVAQ - 2.5 FEEDBACK ON PROGRESS
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
3. TECHNICAL SPECIFICATIONS
We measured the following technical specifications: 3.1. The questionnaire inquires about the set up (installation) of the e-learning environment. 1 2 3 4 5 BE-ODL 20 0 0 0 0 BLA 2 1 1 1 0 CESI 5 0 0 0 0 ISQ 4 0 0 0 0 NADE 7 7 6 3 1 LE PREAU 5 2 1 6 2 VDU 11 0 1 0 0 VIDEOSCOP 18 0 0 0 0 EUROPE 72 10 9 10 3
20
25
4
7
5
11
18
01007
200
01006110
01003
600
00001200
0
10
20
30
40
50
60
70
80
Number
1 2 3 4 5Assessments
SEVAQ - 3.1 THE SET-UP
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
3.2. The questionnaire inquires about the session start-up, the performance and uploading of the pages.
1 2 3 4 5 BE-ODL 12 2 3 3 0 BLA 2 1 2 0 0 CESI 3 1 1 0 0 ISQ 3 0 0 1 0 NADE 7 10 5 1 1 LE PREAU 7 4 0 4 1 VDU 10 2 0 0 0 VIDEOSCOP 14 4 0 0 0 EUROPE 58 24 11 9 2
12
233
7
7
10
14
2110
10
424
32105000
30011400
00001100
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessments
SEVAQ - 3.2 PERFORMANCE
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
3.3. The questionnaire inquires about the browser independency of the learning environment.
1 2 3 4 5 BE-ODL 20 0 0 0 0 BLA 5 0 0 0 0 CESI 4 1 0 0 0 ISQ 4 0 0 0 0 NADE 19 5 0 0 0 LE PREAU 13 2 0 1 0 VDU 9 1 2 0 0 VIDEOSCOP 16 2 0 0 0 EUROPE 90 11 2 1 0
20
544
19
13
9
16
00105212
00000020
00000100 000000000
10
20
30
40
50
60
70
80
90
Number
1 2 3 4 5Assessments
SEVAQ - 3.3 BROWSER INDEPENDENCY
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
3.4. The questionnaire inquires about the availability and system stability.
1 2 3 4 5 BE-ODL 16 3 1 0 0 BLA 2 2 0 1 0 CESI 5 0 0 0 0 ISQ 0 0 0 4 0 NADE 15 4 1 2 1 LE PREAU 8 4 3 1 0 VDU 7 0 3 1 1 VIDEOSCOP 17 1 0 0 0 EUROPE 70 14 8 9 2
16
250
15
8
7
17
32004
401
10001330
01042110
00001010
0
10
20
30
40
50
60
70
Number
1 2 3 4 5Assessments
SEVAQ - 3.4 SYSTEM STABILITY
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
3.5. The questionnaire inquires about the quality of the helpdesk facilities.
1 2 3 4 5 BE-ODL 20 0 0 0 0 BLA 3 1 1 0 0 CESI 2 1 1 0 1 ISQ 4 0 0 0 0 NADE 6 5 5 4 2 LE PREAU 7 4 3 2 0 VDU 11 0 1 0 0 VIDEOSCOP 9 7 2 0 0 EUROPE 62 18 13 6 3
20
324
6
7
11
9
01105
407
01105
312
00004200
00102000
0
10
20
30
40
50
60
70
Number
1 2 3 4 5Assessments
SEVAQ - 3.5 HELPDESK FACILITIES
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
4.1. The questionnaire inquires about the welcoming at the beginning of the course.
1 2 3 4 5 BE-ODL 15 1 2 1 1 BLA 3 2 0 1 0 CESI 2 3 0 0 0 ISQ 4 0 0 0 0 NADE 7 12 4 1 1 LE PREAU 9 1 2 1 3 VDU 10 2 0 0 0 VIDEOSCOP 10 5 3 0 0 EUROPE 60 26 11 4 5
15
324
7
9
10
10
1230
12
12
5
20004
203
11001100
10001300
0
10
20
30
40
50
60
Number
1 2 3 4 5Assessments
SEVAQ - 4.1 WELCOMING
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP
SEVAQ - European report Draft
Appendix 1. Details of the mailing results 08/03/2006
4.2. The questionnaire assesses the following administration topics: course registration, scheduling of the training programme, payment, and other information offered by the learning organisation.
1 2 3 4 5 BE-ODL 11 4 3 2 0 BLA 2 2 1 1 0 CESI 2 2 0 0 1 ISQ 3 0 0 0 0 NADE 4 6 3 6 5 LE PREAU 5 5 2 3 1 VDU 7 4 1 0 0 VIDEOSCOP 8 4 3 3 0 EUROPE 42 27 13 15 7
11
22
3
4
5
7
8
4
220
6
5
4
4
31003
213
2100
6
303
0010
5
100
0
5
10
15
20
25
30
35
40
45
Number
1 2 3 4 5Assessments
SEVAQ - 4.2 ORGANISATIONAL TOPICS
BE-ODL BLA CESI ISQ NADE LE PREAU VDU VIDEOSCOP