Educational Design Research: appropriate to address problems in educational practice
description
Transcript of Educational Design Research: appropriate to address problems in educational practice
![Page 1: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/1.jpg)
Educational Design Research:
appropriate to address problems
in educational practice
Tjeerd PlompProfessor Emeritus
University of TwenteEnschede, The Netherlands
1140519 ICREAMS
![Page 2: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/2.jpg)
Van den Akker (1999; director of Netherlands Institute for Curriculum Development) about research in domain of curriculum development and implementation:many ‘traditional’ research approaches such as experiments, surveys, correlational analyses, with their emphasis on description hardly provide prescriptions that are useful for design and development problems in education
2140519 ICREAMS
![Page 3: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/3.jpg)
In this presentation:
• Research functions and research designs• What is design research• Research question • Quality criteria for interventions• Generalizability in design research• New book on design research
3140519 ICREAMS
![Page 4: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/4.jpg)
Focus of research project
Dependent on research question/goalresearch may address a need
– to describe– to explain, understand, predict– to compare– to evaluate– to design/develop
Called: research functions4140519 ICREAMS
![Page 5: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/5.jpg)
Research approaches/designsto realize research functions e.g.:
• Survey: to describe, compare, evaluate• Case studies: to describe, to explain• Experiments: to explain, to compare• Ethnography: to describe, to understand• Correlational research: e.g. to describe, to
explain• Evaluation research: e.g. to determine the
effectiveness of a program5140519 ICREAMS
![Page 6: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/6.jpg)
Research designs, e.g.:
Design research: to design/develop an intervention (e.g. program, T/L strategy, process, product, system, etc) with the purpose
– to improve practice and contribute to body of knowledge
OR– to develop a theory(and possibly: to inform decision making and policy
development)Note: intervention is container term 6140519 ICREAMS
![Page 7: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/7.jpg)
What is Educational Design Research (EDR)?
Distinction between:
• Development studies aimed at research-based solutions for complex problems in educational practice
• Validation studies aimed at development or validation of a theory
Research on interventions versus research through interventions (McKenney & Reeves, 2012)
Often both angles in one study! 7140519 ICREAMS
![Page 8: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/8.jpg)
What is Educational Design Research (EDR)?
Educational design research – type development studies - isthe systematic analysis, design, development and evaluation of educational interventions with the dual aim of
• generating research-based solutions for complex problems in educational practice, and
• advancing our knowledge about the characteristics of these interventions and the process of designing and developing them.
8140519 ICREAMS
![Page 9: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/9.jpg)
What is Educational Design Research (EDR)?
Educational design research – type validation studies - isthe study of educational interventions – such as learning processes, learning environments and the like – with the purpose to develop and validate theories about such processes and how these can be designed.
9140519 ICREAMS
![Page 10: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/10.jpg)
Educational Design Research encompasses systematic educational design processes.
The reverse is not true:Not all systematic educational design can
be called research
10140519 ICREAMS
![Page 11: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/11.jpg)
11
Iterations of systematic design cycles
analysisdesign & develop
prototype
evaluationRevision: yes?
No? STOP
PROBLEM
![Page 12: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/12.jpg)
Design research methodology can be expressed as follows:
12
ID of problems with practitioners
Development of prototype solutions
Testing solutionsin context
Reflection to producedesign principles
(adapted from Reeves, UoGeorgia, Athens, USA)
Revision
140519 ICREAMS
![Page 13: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/13.jpg)
Susan McKenney, Univ of Twente:
13140519 ICREAMS
![Page 14: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/14.jpg)
Phases in Design Research• Preliminary phase:
needs and content analysis, review of state-of-art of literature => conceptualisation & design
specifications• Development or Prototyping phase
(iterative design phase + formative evaluation) <=> micro-cycles of research
• Assessment phase (semi-summative evaluation)
ALL Phases: systematic reflection and documentation (resulting in
design principles)14140519 ICREAMS
![Page 15: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/15.jpg)
On terminology:
(Educational) Design Researchstands for a ‘family’ of related research approaches
• Design studies, design experiments• Design-based (implementation) research• Development research• Participative action research• Formative research• Engineering research
(e.g. Van den Akker et al., 2006):But a number of characteristics in common!
15140519 ICREAMS
![Page 16: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/16.jpg)
Common characteristics
• Interventionist• Iterative• Process oriented• Utility oriented: merit of design
measured• Theory oriented: based upon &
contributing to• Involvement of practitioners
16140519 ICREAMS
![Page 17: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/17.jpg)
17
In general:Research goal may vary, but always:(i) design/development of intervention (ii) design principles or theory validation
Context Z:
Intervention XInput => Process
OutcomesY1, Y2, …, Yn
A closer look:
140519 ICREAMS
![Page 18: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/18.jpg)
18
Twofold yield in design research
Develop-ment studies
(i) developing a research based intervention as solution to complex problem, and (ii) constructing (re-usable) design principles
Validation studies
(i) designing learning environments with the purpose (ii) to develop and validate theories about learning, learning environments, or to validate design principles
Implemen-tation studies
(i) implementing a particular program and (ii) strategy and conditions under which implementation can happen (design principles).140519 ICREAMS
![Page 19: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/19.jpg)
Research question in design research
If research goal is development of intervention:What are the characteristics of an <intervention X> to realize outcomes Y in context Z
OR in case of developing/validating a theory (tentatively):
What is an adequate learning and teaching strategy for <topic B> in context C in order to cope with the main difficulties in the learning of this topic
19140519 ICREAMS
![Page 20: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/20.jpg)
Examples of research questions
#1 what are the characteristics of an effective
in-service programme for mathematics
teachers through which they develop the
ability to apply student-centred
pedagogical methods (Tecle, 2003)
20140519 ICREAMS
![Page 21: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/21.jpg)
Examples of research questions
#2 What is an adequate learning and teaching strategy for genetics in upper secondary biology education in order to cope with the main difficulties in learning and teaching genetics, and to promote the acquisition of a meaningful and coherent understanding of hereditary phenomena? ( Knippels, 2002)
21140519 ICREAMS
![Page 22: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/22.jpg)
Quality criteria for interventions:1. Relevance or Content Validity:• need for the intervention• based on contemporary scientific insights2. Consistency or Construct Validity3. Practicality
- can be used in settings for which developed4. Effectiveness:
- yields desired results- cost-benefit ratio
#3&4: Expected versus Actual22140519 ICREAMS
![Page 23: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/23.jpg)
Shift in emphasis on criteria
23
Stage Criteria
Preliminary
research
Emphasis mainly on relevance and consistency (i.e. validity),
less on practicality
Development or Prototyping phase
initially: consistency and practicality; increasingly practicality and gradually effectiveness
Assessment phase
practicality and effectiveness140519 ICREAMS
![Page 24: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/24.jpg)
Formative evaluation e.g.: Design
specsGlobal design
Partly developed
Completely developed
Relevance • Screening
• Focus grp
• Screening• Focus group
• Screening• Focus group
• Focus group• Micro-eva
Consistency • Screening
• Focus grp
• Screening
• Focus grp
• Screening• Focus group
• Focus group• Micro-eva
Practica-
lity
• Screening• Focus group
• Screening
• Focus grp
• Walkthrgh
• Focus grp
• Micro-eva
• Walkthrgh
• Focus grp
• Micro-eva
• Try-out
Effective-
ness
• Screening• Focus group
• Screening• Focus group
• Focus grp
• Micro-eva
• Focus grp
• Micro-eva
• Try-out24140519 ICREAMS
![Page 25: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/25.jpg)
25
Context Z:
Intervention XInput => Process
OutcomesY1, Y2, …, Yn
design principles as intervention theorylocal instruction theory
Generalizability in Design Research:Yin (2003) about case study research:striving to generalize findings to some broader theory:
![Page 26: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/26.jpg)
Generalizability in Design Research:Yin (2003) about case study research:
striving to generalize findings to some broader theory
A theory (design principles or local theories) must be tested through replications of findings in a 2nd, 3rd or more cases:
if same results in various cases, then ….the theory (= design principles or local theories) might be
accepted for a much larger number of contexts.
This replication logic is the same that underlies design research!
Analytical generalization.(compare with ‘transferability’ in qualitative research)
26140519 ICREAMS
![Page 27: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/27.jpg)
Edited by Tjeerd Plomp & Nienke Nieveen
Published by: SLO – Netherlands Institute for Curriculum Development. Enschede; 2013
Part A: an IntroductionPart B: Illustrative cases
Free available from:http://international.slo.nl/edr
27140519 ICREAMS
![Page 28: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/28.jpg)
Part B: 51 Cases of Design Research
Case Selection Tool:•Countries •Educational Sectors•Educational domains (>1)•Main aim•Educational focus•Type of intervention (>1)
See: http://international.slo.nl/edr
28140519 ICREAMS
![Page 29: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/29.jpg)
Part B: 51 Cases of Design Research
Case Selection Tool:•22 countries: CHI, SIN, IND, KOR – USA (14), NET (11)•Educational Sectors: all•Educ domains (>1): curriculum (16), learning & instruction (29), subject-related pedagogy (34), school organization (3), instructional technology (7), ICT in education (11)•Main aim: development studies (36); validation studies (14); implemtation studies (1)•Educational focus•Type of intervention (>1)
29140519 ICREAMS
![Page 30: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/30.jpg)
Part B: 51 Cases of Design Research
Case Selection Tool:•Countries •Educational Sectors•Educ domains (>1)•Main aim•Educational focus: teaching-learning methods; curriculum planning; ICT in education; school management & leadership; monitoring quality of educ; professional development; workplace learning•Type of intervention: curr unit/course (34), educ programme (12), learning task (9), monitoring system (4), assessment task (2), other (4)
30140519 ICREAMS
![Page 32: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/32.jpg)
Quotes illustrate that
many researchers in various ‘corners’
of our field belief that there is a need
for an alternative research approach
that is directly relevant for problems in
educational practice.
32140519 ICREAMS
![Page 33: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/33.jpg)
NOTE: One resq => more than one res function
Example:if aim of research isto design and develop a teaching-learning strategy
for acquiring the competency of mathematical modelling (in grade 11 & 12),
then researchers may first want to understand and carefully describe what barriers students experience with mathematical modelling.
But: there is primary research function 33140519 ICREAMS
![Page 34: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/34.jpg)
Why embark on Design Research?
Various reasons – e.g.:
1. Need to solve a complex problem in education practice for which no how-to-do guidelines are available
2. Need to elaborate and/or validate a theory (e.g. whether the theory of Realistic Mathematics Education is valid in a certain context)
34140519 ICREAMS
![Page 35: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/35.jpg)
Problems may exist at all levels - for example:
System level:Need for system for e-learning to serve specific
group of students
Institutional level:What are effective methods for collaborative
learning
35140519 ICREAMS
![Page 36: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/36.jpg)
When is Design Research Appropriate?
Eamonn Kelly (2009):• Initial state(s) unknown or unclear• Goal state(s) unknown or unclear• Operators to move from initial states to goal
states are unknown or how to apply the operators is unclear
36140519 ICREAMS
![Page 37: Educational Design Research: appropriate to address problems in educational practice](https://reader035.fdocuments.us/reader035/viewer/2022081515/568143eb550346895db070e4/html5/thumbnails/37.jpg)
Generalizability in Design Research:
So: design principles must be seen as ‘heuristic’ statements providing guidance and
direction,but do not give ‘certainties’.
When we give proper weight to local conditions, any generalization is a working hypothesis, not a conclusion!
(Lee Cronbach, 1975, cited by Tom Reeves)
37140519 ICREAMS