Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online...
description
Transcript of Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online...
![Page 1: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/1.jpg)
Temporal patterns of knowledge construction:
Statistical discourse analysis of a role-based online discussion
To appear in the International Journal of Computer-Supported Collaborative Learning
Alyssa WiseSimon Fraser University
Ming Ming ChiuState University of New York –Buffalo
I appreciate the research assistance of Choi Yik Ting
![Page 2: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/2.jpg)
Online, asynchronous forums• Can participate anywhere – no geographic limits• Can share ideas at any time – more time to think• But often disconnected, only lists of isolated ideas
Guzdial & Turns, 2000; Herring, 1999; Thomas, 2002Summaries• Connect previous ideas and develop them• But often occur at end of discussion
& Do not benefit other members De Wever et al., 2007; Schellens et al. 2005; 2007
Encourage summaries in the middle of discussions?
Motivation for the Study
![Page 3: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/3.jpg)
Knowledge Construction (KC) FrameworkGunawardena et al.’s (1997) Five-Phase Model
![Page 4: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/4.jpg)
Research Context for the Study Emerging Themes in Collaborative Learning Research
(e.g. Chiu & Khoo, 2005; Kapur, 2001; Reimann, 2009)
(e.g. Cress, 2008; Suthers & Teplovs, 2011)
(e.g. Arvaja, 2007; Stahl, 2004; Strijbos et al., 2004)
![Page 5: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/5.jpg)
Possible KC Patterns
0 5 10 15 201
2
3
4
51a
0 5 10 15 201
2
3
4
51b
0 5 10 15 201
2
3
4
52a
0 5 10 15 201
2
3
4
52b
0 5 10 15 201
2
3
4
53
0 5 10 15 201
2
3
4
54
Knowledge Construction Phase
Post Number
![Page 6: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/6.jpg)
Research Questions• What patterns characterize knowledge
construction processes during an online discussion?
• What characterizes pivotal posts that divide a discussion into distinct segments? Summaries?
• Which characteristics of a post influence the knowledge construction phase of the next post?
![Page 7: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/7.jpg)
PivotalPost
FunctionsSummary (+)…
RolesSynthesizer (+)…
Individual Control variablesGenderAge…
PostControl variables# of wordsTime of post…
Time contextWeek
![Page 8: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/8.jpg)
KnowledgeConstruction
FunctionsSummary (+)…
RolesSynthesizer (+)…
Individual Control variablesGenderAge…
PostControl variables# of wordsTime of post…
Time contextWeekSegment
![Page 9: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/9.jpg)
Methods
![Page 10: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/10.jpg)
Function
Role
Give Direction
New Idea
BringSource
Use Theory Respond Summarize
Starter X X Inventor X Importer X X Mini-me X Questioner X Elaborator X Devil’s Advocate X
Traffic Director X
Synthesizer X XWrapper X
![Page 11: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/11.jpg)
Content Analysis
Variable Inter-rater reliability ()Knowledge construction .84New Idea .65Bring in Source .92Use Theory .73Respond .98Give Direction .76Summarize .88
Unit of analysis: Post / Note / Message Objectively identified unit that its author defines
Rourke, Anderson, Garrison, & Archer, 2001Inter-rater reliability Krippendorf’s (range: -1 … 1; desired: > .67)
![Page 12: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/12.jpg)
4 types of Analytical Difficulties
• Time
• Outcomes
• Explanatory variables
• Dataset
- No missing data
Statistical Discourse Analysis
![Page 13: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/13.jpg)
Statistical Discourse AnalysisDifficulties regarding Time
Segments differ (S2 S4)
Serial correlation (p8 → p9)
Branches of notes
Strategies
Breakpoint analysis + Model Multilevel analysis (MLn, HLM)
Test with I2 index of Q-statistics Model with lag outcomes, KC (-1)
Store path: Identify prior turn
1
2
3
8
4
5 6
7
![Page 14: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/14.jpg)
ID Action Turn # Valid?Previous
TurnValid (-1)
Ana Do three times four. 1 – –Ben Three times four is seven 2 X 1 Eva Three times four is nine. 3 X 2 XJay Three times four is twelve. 4 3 X
ID Action Turn # Valid?Respondto post?
Valid (-1)
Ana Do three times four. 1 – –Ben Three times four is seven 2 X 1 Eva Three times four is nine. 3 X 1 Jay Three times four is twelve. 4 3 X
![Page 15: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/15.jpg)
Statistical Discourse AnalysisDifficulties regarding Time
Segments differ (S2 S4)
Serial correlation (p8 → p9) Multiple topics
Branches of notes (→→ )
Strategies
Breakpoint analysis + Model Multilevel analysis (MLn, HLM)
Test with I2 index of Q-statistics Model with lag outcomes, KC (-1)
Store path: Identify prior turn Vector Auto-Regression
Lag explanatory variablese.g., Valid (-1), Girl (-1) Valid (-2)
1
2
3
8
4
5 6
7
![Page 16: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/16.jpg)
Statistical Discourse AnalysisOutcome Difficulties
Ordered outcome (KC 1-5) Infrequent outcomes (00010)
Strategies
Ordered Logit / Probit Logit bias estimator
![Page 17: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/17.jpg)
Statistical Discourse AnalysisExplanatory model Difficulties
People, Groups & Topics differ
Mediation effects (X→M→Y)
False positives (+ + + +)
Strategies
Multilevel analysis Multilevel mediation tests
2-stage linear step-up procedure
![Page 18: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/18.jpg)
Results – KC PhasesKC Phase % of Posts1) Sharing Information 60
2) Exploring Dissonance
3
3) Negotiating Meaning 16
4) Testing / Modifying 4
5) Agreeing / Applying 17
![Page 19: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/19.jpg)
Results: Summaries as Pivotal PostsEach discussion averaged
1 pivotal post (2 time periods)
![Page 20: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/20.jpg)
Results - KC Patterns
0 5 10 15 201
2
3
4
51a
0 5 10 15 201
2
3
4
51b
0 5 10 15 201
2
3
4
52a
0 5 10 15 201
2
3
4
52b
0 5 10 15 201
2
3
4
53
0 5 10 15 201
2
3
4
54
Knowledge Construction Phase
Post Number
No Regressive Segments
Pivotal Posts → Distinct Segments
No Regressive Segments
Segments Skipped
KC phases
![Page 21: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/21.jpg)
Predicting Pivotal Posts
SynthesizerPivotal
PostExtensive SummaryWrapper
Role Current Post
![Page 22: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/22.jpg)
Time 2 posts ago Previous post Role Current post
Knowledge Construction
Summary
After 1st
pivotal post
New Idea (-1)after 1st pivotal
post
Respond (-2) after 1st pivotal post
Wrapper
Synthesizer
Predict Knowledge Construction
![Page 23: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/23.jpg)
KC pattern KC phase 1 KC phase 3 or 5
(Share) (Negotiate Meaning or Agree/Apply) Few KC phases 2 or 4 (Dissonance, Testing)
Pivotal post Extensive Summary often By Synthesizer or Wrapper usually
Extensive Summary Showed higher KC Elevated KC of subsequent posts
Summary of Results
![Page 24: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/24.jpg)
Teacher / Designer Assign Synthesizer Role
- Increase midway summaries and elevate KC- Simple, effective intervention
Productive online discussions do not require all phases
Researcher Empirically test Gunawardena et al’s KC model New method for analyzing online discussion
- Statistically identifies pivotal posts & segments- Test hypotheses about relationships among posts - Examine variables at multiple levels - Examine differences over Time
Implications
![Page 25: Temporal patterns of knowledge construction: Statistical discourse analysis of a role-based online discussion](https://reader036.fdocuments.us/reader036/viewer/2022062815/56816946550346895de0d4cb/html5/thumbnails/25.jpg)
Further Questions• With many choices of dimensions for the
breakpoints, which one(s) should we use?
• What do identification of same vs. different breakpoints across different dimensions tell us?
• How can we do meta-analyses of multiple data sets with somewhat different codes?
• Which analyses (qualitative and/or quantitative) might be fruitful on the same data set?