Post on 26-Dec-2015
http://www.pages.drexel.edu/~jw65/
How to run “successful” online discussion
boards: Some observations from the
coalface
Jim WatersThe iSchool at Drexel
Jim Waters, Drexel University, 2009
But Asynchronous Discussion Boards are old hat !
• Surely Chat is faster and more interactive ?
• Reflection – knowledge building discourse is not a speed sport
• Richer peer interactions
Jim Waters, Drexel University, 2009
Gauging Success
• Participation• Discourse• Knowledge Building • Consensus• Understanding
If you cannot measure it, you cannot improve it - Lord Kelvin
Jim Waters, Drexel University, 2009
Participation
Posting messages – how many ?
All messages are not equal“ I agree with Fred”“ I Like cats”“ I see you are an eagles fan”“ yeah it’s week 10 !”“ LOL”“ It is nice to see you again”
My understanding of participation is not yours
Thread length – The K-Tel approach, 25 original hits unrelated messages
Think better not more often !
Jim Waters, Drexel University, 2009
Discourse
Student-to-Student interactionsStudent-faculty interactions
Apple for teacher ?Appetite for knowledge ?
Deep threads Most participants in a threadIterations – most dyadsInteraction types
Jim Waters, Drexel University, 2009
Knowledge BuildingIs any knowledge really new ?Schrödinger's cat Is an inference new knowledge ?Content AnalysisLeast measured outcome
Jim Waters, Drexel University, 2009
Consensus
“So we are agreed , we will land the Cuban exiles in the Bay of Pigs “
“A mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when the members' strivings for unanimity override their motivation to realistically appraise alternative courses of action – Irving L. Janis, Victims of Groupthink, 1972
Jim Waters, Drexel University, 2009
Understanding
The Turing test ?ElizaInterpretationInferenceExemplificationAbstraction
Jim Waters, Drexel University, 2009
Small Group Discussion
Cohesive groupsLess cliquey behaviorNon-participation is obviousManageable size for creative projectMixed ability
The strong leaderAn army of followersThe dysfunctional group
Jim Waters, Drexel University, 2009
Research Context Analyzed online interactions in graduate
classes in information systems and information science
Course interactions via discussion board on Blackboard learning system.
15 online courses over a five year period 325 Students 21,000 messages 4000 message threads
Jim Waters, Drexel University, 2009
Research Method Collected and analyzed student messages
posted to discussion board and small group discussion
Analyzed Thread depth, thread length Cognitive content of message Patterns of message sequences
Student interactions related to outcomes
Jim Waters, Drexel University, 2009
Findings I
Are all participants created equal ? Dealing with the Divas ? Rewarding the loudest ? Leave-alone or interact ?
Harangue Encourage Flatter The dark side of rewards
Questions or self-questions ? Question design ? Small group reflections ? In what context Charlie ? How many questions ? Grading discussion board interactions , does it motivate ?
Jim Waters, Drexel University, 2009
Findings II
Cognitive Outcomes Do Social posts mean less cognitive outcomes How many social posts is acceptable Dyads, cliques and committees – what level of
group interaction is most productive ? How to deal with cliques ? Breaking up the cartels ? How truthful to be in feedback/moderation
Jim Waters, Drexel University, 2009
Participatory democracy vs. benevolent oligarchy Will an active core of students bring in
peripheral participants or exclude them ? How do we get peripheral participants to
join the circle ? Does it matter if some do not actively
participate ? I meant to ask that but…… Vicarious learners or lurkers ?
Should we moderate core participants dominating the discussion or encourage them ?
Jim Waters, Drexel University, 2009
Moderated vs. Laissez-faireCase Study III
Two sections of an IS course delivered at the same time – same basic syllabus
Same number of students (25) Selected six “identical” questions on each
section Different Instructor approach Heavy moderation vs. lightweight moderation
Jim Waters, Drexel University, 2009
Question Heavy Moderation Low Moderation
Systems Analyst as problem solver
69 74
Agile methods 96 97Project design 150 97
Requirements Analysis 96 83Fact Finding 85 90Data Modeling Practice 182 180Average 112 103
Questions and Approach (messages)
Jim Waters, Drexel University, 2009
Systems Analyst as Problem solver
Heavy Moderation Low Moderation
Total Messages 69 74
Instructor – student messages
17(24%) 0
Deep sub-threads ( 4 levels or greater)
4 6
Student messages 52 74
Student-instructor messages
25 25
Student-student messages
27 52
Jim Waters, Drexel University, 2009
Agile methods Heavy Moderation Low Moderation
Total Messages 96 97
Instructor – student messages
16(17%) 0
Deep sub-threads ( 4 levels or greater)
10 8
Student messages 80 97Student-instructor
messages52 24
Student-student messages
28 73
Deep thread messages (students)
65 44
Jim Waters, Drexel University, 2009
Agile methods Heavy Moderation Low Moderation
Total Messages 95 97
Deep sub-threads ( 4 levels or greater)
2 8
Deep threads without instructor intervention
Jim Waters, Drexel University, 2009
Project design Heavy Moderation Low Moderation
Total Messages 150 97
Instructor – student messages
44(30%) 0
Deep Threads 18 9
Student messages 106 97Student-instructor
messages53 23
Student-student messages
53 74
Deep thread messages (students)
78 46
Jim Waters, Drexel University, 2009
Goals objectives and scope Heavy Moderation Low Moderation
Total Messages 150 97
Deep sub-threads ( 4 levels or greater)
3 9
Deep threads without instructor intervention
Jim Waters, Drexel University, 2009
So does providing feedback help ? Moderated Un-moderated
Agile methods (Words) 8823 21203
Agile methods (Messages) 80 97
Project design (Words) 12227 20211
Project Design (Messages) 106 97
SA problem solver (Words) 5220 15714
SA problem solver (messages) 52 74
Tot messages 238 268
Tot words 26270 57128
Average words/student message 110.38 213.16
What about question design ?
Jim Waters, Drexel University, 2009
Posing questions - general Sometimes a committed , motivated and
interested group with decent moderation will be inert.
Even within the same niche of the same domain some questions just work better than others
Good question design is not trivial even for domain experts
Does the question connect to student experience, real or vicarious
Is the question relevant to the course Does the question represent a well-structured
single knowledge domain goal
Jim Waters, Drexel University, 2009
Good, bad or average?
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
Jim Waters, Drexel University, 2009
And the Winner is
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
Jim Waters, Drexel University, 2009
GoodI want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
150 posts (2nd best thread) Several sub-threads extremely deep (7 or 8 levels) Question is level 130% of messages were Instructor to Student29% of messages were student to instructorCritique, feedback, support and facilitationWell-placed faculty moderation, nudges rather than cattle prodsWell-bounded but open-ended problem, students define problemInitial high level of misunderstanding of task (goals vs. objectives) despite material having been formally covered alreadyStudents negotiated the meaning of the task collaborativelyDeliberate pitching as cooperative
Jim Waters, Drexel University, 2009
AverageI would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
85 posts (Mean was 91)
Moderate sub-thread depth (mostly 3 or 4 levels)
31% were messages from Instructor to students
20% were messages from students to instructor
Well-placed faculty moderation, focus on challenging assumptions.
Reasonably open-ended problem
Far less cooperative inter-student activity
Not pitched as a cooperative activity
Students not answering a common question, but question is defined
Jim Waters, Drexel University, 2009
Bad
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
46 posts
Limited sub-thread depth - mostly 2 (question then single response)
45% were messages from Instructor to students
37% were messages from students to instructor
18% were student-student messages
Faculty intervention much more critical (didactic)
Five questions in one, one open-ended 4 bounded
Very little cooperative inter-student activity
Not pitched as a cooperative activity
One overarching common question
Jim Waters, Drexel University, 2009
Final Words: Value of online discussion
this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions
I was moved to comment on how refreshing the lack of competition in the Communications for the online classes seemed to me. It was a discussion and a sharing of experiences
Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback
No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to..I've never done this before.
I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.
Jim Waters, Drexel University, 2009
Worrying or not ?I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.
Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback
Stopped posting after week 2Little opportunity to get drawn inWhat do you do if someone will not contribute ?
Concerned about how posts interpretedMost posts were responded toMost posts positive feedback but no added contentMany posts Social rather than task-orientedRarely posted thread starters
Jim Waters, Drexel University, 2009
Conclusions I Tentative findings Deep engagement can be encouraged by
thought-leaders Useful thought-leader behaviors highly
context dependent Interaction behaviors interact Faculty feedback improves some forms of engagement but perhaps at the expense of student-student interactions ?
Finite tank of engagement ?
Jim Waters, Drexel University, 2009
Conclusions II Question design can be crucial to
engagement Discussion needs to be framed as
collaborative not competitive Students recognize core participants,
should we alter our behaviors to fit ? Committed engagement is possible So is isolation and inertia ! Sense of community is not automatic
Jim Waters, Drexel University, 2009
Related References Waters, J. '
Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008.
Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007.
Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006.
Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot
Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Manua, Hawaii, January 2005, 2005, p. 3b.
A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm
Jim Waters, Drexel University, 2009
About Me I am a Doctoral Candidate at the iSchool at Drexel University, Philadelphia
(USA), graduating Summer 2009. My principal research interests lie in
Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI. I am currently employed (2005 - 2008) as a Research Assistant on an IMLS funded project "Toward a Model Curriculum for the Management of Digital Information".
I received a BA in Psychology at Warwick University, UK (1979), an MSc in Occupational Psychology at the University of Hertfordshire, UK (1991) and a MS in Information Systems at the College of IS&T at Drexel University (2002). Prior to my advanced academic studies I enjoyed a substantial career in Systems Design, Management and IS Consultancy.
Email: jw65@drexel.edu