Post on 15-Dec-2014
description
Acceptance of TEL: Key Success Factors and Reasons of Failure
PPE Summer School 2008, Ohrid, FYROM
Patrick Johnscher (TU Darmstadt)
Sebastian Kelle (Open University, Heerlen)
Steinn E. Sigurðarson (WU Vienna)
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 2
Training & Learning
Training measures can be used simply because there is no alternative…
….but it should be about good design!
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 3
Good TEL Design
… is more than just a collection of PowerPoint slides…
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 4
So why don't we design what the users want?
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 5
Why is acceptance of TEL important?
What we just learned from Dilbert: TEL design should consider the users' requirements A system that meets expectations is more likely to gain acceptance and get used – at
least if there are learning alternatives
Common misperception: "If we build it, they will come!" E-Learning is not a field of dreams: just providing technology doesn't mean learners will
use the systems At the end of the day, TEL is no end in itself: there has to be a measurable learning
outcome, and a business case has to be made
Davis (1989): "Technology Acceptance" Information technology offers the potential for substantially improving white collar
performance But performance gains are often obstructed by users' unwillingness to accept and use
available systems Because of the persistence and importance of this problem, explaining user acceptance
has been a long-standing issue in MIS research
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 62008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 6
A What-went-wrong Example: COVCELL
COVCELL Project – Effective language learning (http://www.covcell.org) Most important factor for language learning:
“learning through conversation” 88% of students 75.9% of teachers
So, interaction and communication is quite important?
COVCELL developed tools such as: User Presence and Chat Audio/Video Conferencing Collaborative Whiteboard Audio Recording
User acceptance as a basis for design requirement specifications
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 72008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 7
COVCELL: Percentage of users happy with…
14,3%
7,1%
28,6%
71,4%
0 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
…audio recording
…whiteboard
…audio/video conferencing
…user presence and chat
…using the wiki
…using the forum
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 82008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 8
COVCELL: User Ratings of…
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
…audio recording
…whiteboard
…audio/video chat
...user presence and chat
…wiki
…forum
Language LearningGeneral
Scale: 0-5
4.2 / 54.5 / 5
3.6 / 53.4 / 5
2.3 / 52.4 / 5
2.3 / 52.3 / 5
3.4 / 53.2 / 5
3.0 / 5
2.8 / 5
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 92008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 9
COVCELL: People didn't use…
71,4%
92,9%
78,6%
85,7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
…audio recording
…whiteboard
…audio/video conferencing
…user presence and chat
COVCELL: What went wrong?
Most users actually reported “I did not use it, but I would have liked to”
Most activities we mention here were “presence based”. The simultaneous presence of other users was required to engage in the activities, and use the tools
Users did not stay for long stretches of time in the system, and thus rarely were in the presence of other users
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 11
Some User Quotes
•I don't learning English with computers
•I did not like using computers because it decreased personalization with theteacher.
•I didn ´t like how many courses on line are badly organised and do not take the learner step by step through procedurs.
• I like being able to access material when it is conveinient for me.
• I liked: Being able to study even though I live far away from the Universityand when I was listening to lectures I could always stop and replay if I wasn'tsure of things I didn't like: Not being able to ask questions in class and notmeeting the other students.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 12
A Few Questions…
What is "acceptance"? Let's try some definition…
Which factors influence acceptance? A closer look at acceptance models…
How can I measure acceptance? Using acceptance models in empirical research…
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 13
Acceptance – A Definition
"Acceptance of an innovation or technology describes the positive adoption decision of users – in contrast to the rejection of an innovation or technology." (Simon 2001, p. 87)
"Accordingly, the innovation-decision process is the process through which an individual or other decision-making unit passes
1. from first knowledge of an innovation,
2. to forming an attitude toward the innovation,
3. to a decision to adopt or reject,
4. to implementation of the new idea, and
5. to confirmation of this decision." (Rogers: Diffusion of Innovations, 2003, p. 161)
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 14
Acceptance – A Definition
Technology acceptance usually consists of two separate aspects: An attitude component, comprising affective and rational aspects A behavioral component, describing the adoption of an innovation in the
form of actual observable behavior (e.g. use of a system)
Since the behavioral component is usually easy to observe (and thus easy to measure), modeling and research is mostly focused on the attitude component of user acceptance:
Affective aspects, including motivational and emotional attributes Rational aspects, including cost-benefit relations as seen from the users'
individual perspective
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 15
Acceptance Models
Frequently cited models in the literature are: Technology Acceptance Model Technology Acceptance Model 2 The Unified Theory of Acceptance and Use of Technology
Basic concept underlying user acceptance models:
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 16
Technology Acceptance Model (TAM)
Developed by Davis (1989) Main idea: Perceived usefulness and perceived ease of use are
fundamental factors influencing the user acceptance as they influence the user’s attitude towards the system.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 17
Technology Acceptance Model (TAM2)
Developed by Venkatesh and Davis (2000) Extension of the original TAM model to explain perceived usefulness and
usage intentions in terms of social influence process and cognitive instrumental processes.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 18
Unified Theory of Acceptance and Use of Technology (UTAUT)
Developed by Venkatesh et al (2003)
Consolidation of eight prominent technology user acceptance models. The eight models reviewed are: Theory of Reasoned Action (TRA) Technology Acceptance Model (TAM) Motivational Model (MM) Theory of Planned Behavior (TPB) Combined TAM and TPB (C-TAM-TPB) Model of PC Utilization (MPCU) Innovation Diffusion Theory (IDT) Social Cognitive Theory (SCT)
The UTAUT states that four constructs play a significant role as direct determinants of user acceptance and usage behavior. They are: performance expectancy effort expectancy social influence facilitating conditions
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 19
Unified Theory of Acceptance and Use of Technology (UTAUT)
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 20
Using Acceptance Models in Empirical Research
Models are so-called "Structural Equation Models" (SEM) or "Causal Models" SEM encourages confirmatory rather than exploratory modeling; thus, it is suited
to theory testing rather than theory development. There are usually two main parts to SEM:
the structural model showing potential causal dependencies between endogenous and exogenous variables,
and the measurement model showing the relations between the latent variables and their indicators.
Benefits: User Acceptance Models like TAM, UTAUT and others are widely accepted and represent
state-of-the-art in IS research Models come with hypotheses "on board"; you don't have to formulate your own Scales and question items exist and have been tested in a number of previous studies
("tried and true") Drawbacks:
Relatively high demands on data quality Parameter estimation using covariance-based techniques can be considered advanced
statistics
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 21
Acceptance Model for TEL?
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 22
Influence Factors (Exogenous Variables): Draft Classification
Technological Organizational Didactical / Pedagogical Personal Cognitive Cultural Motivational …
Others?
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 23
Now over to you…
Form up in groups of 4-5 students
Think about factors influencing the acceptance of TEL – and reasons why TEL implementation fails (possibly also solutions presented at this summerschool?)
Try to cluster the success factors / reasons of failure into categories, e.g. "organizational", "technological"
Identify connections and dependencies between the separate factors: which factors directly influence acceptance; which influence each other (indirect influence on acceptance)?
Visualize your group results as a map on a flipchart (or laptop, e.g. with conzilla)
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 24
Literature
Venkatesh, V., Morris, M.G., Davis, F.D., and Davis, G.B.: "User Acceptance of Information Technology: Toward a Unified View," MIS Quarterly, 27, 2003, 425-478.
Venkatesh, V. and Davis, F.D.: "A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies," Management Science, 46, 2000, 186-204.
Davis, F. D., Bagozzi, R. P., and Warshaw, P. R.: "User Acceptance of Computer Technology: A Comparison of Two Theoretical Models," Management Science, 35, 1989, 982-1003.
Goodhue, D. L.; Thompson, R. L. (1995): "Task-Technology Fit And Individual Performance", MIS Quarterly, Vol. 19, Issue 2, 213-237.
Rogers, E. M. (2003): Diffusion of innovations (5th ed.). New York: Free Press.
Nanayakkara, C. (2007): "A Model of User Acceptance of Learning Management Systems: a study within Tertiary Institutions in New Zealand", Educause Australasia 2007, http://www.caudit.edu.au/educauseaustralasia07/authors_papers/Nanayakkara-361.pdf.
Tai, L. (2008): Corporate E-Learning: An Inside View of IBM's Solutions. Oxford University Press, New York.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 25
Construct Definitions
Attitude: Individual's positive or negative feeling about performing the target behavior (e.g., using a system).
Behavioral intention: The degree to which a person has formulated conscious plans to perform or not perform some specified future behavior.
Computer anxiety: The degree of an individual’s apprehension, or even fear, when she/he is faced with the possibility of using computers.
Computer playfulness: The degree of cognitive spontaneity in microcomputer interactions.
Computer self-efficacy: The degree to which an individual beliefs that he or she has the ability to perform specific task/job using computer.
Effort expectancy: The degree of ease associated with the use of the system. Facilitating conditions: The degree to which an individual believes that an
organizational and technical infrastructure exists to support use of the system.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 26
Construct Definitions
Image: The degree to which use of an innovation is perceived to enhance one's status in one's social system.
Job relevance: Individual's perception regarding the degree to which the target system is relevant to his or her job.
Objective usability: A comparison of systems based on the actual level (rather than perceptions) of effort required to complete specific tasks.
Output quality: The degree to which an individual believes that the system performs his or her job tasks well.
Performance expectancy: The degree to which an individual believes that using the system will help him or her to attain gains in job performance.
Perceived ease of use: See the definition of effort expectancy. Perceived enjoyment: The extent to which the activity of using a specific system
is perceived to be enjoyable in it’s own right, aside from any performance consequences resulting from system use.
Perceived usefulness: See the definition of performance expectancy.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 27
Construct Definitions
Perception of external control: See the definition of facilitating conditions.
Result demonstrability: Tangibility of the results of using the innovation. Social influence: The degree to which an individual perceives that
important others believe he or she should use the new system. Subjective norm: Person's perception that most people who are
important to him think he should or should not perform the behavior in question.
Voluntariness: The extent to which potential adopters perceive the adoption decision to be non-mandatory.
2008-06-19 | PPESS 08 Ohrid, FYROM | Workshop: "Acceptance of TEL: Key Success Factors and Reasons of Failure" | 28