Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2

8/7/2019 Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2

http://slidepdf.com/reader/full/achieving-trans-active-cmc-brant-knutzen-icel-2010-v2 1/22



Abstract

A common failing of e-learning instructional design is when online discussion learning activities

devolve to become just a “public assignment”: each student posts once and no peer interaction is

achieved. This study shows how instructors can successfully design an e-learning environment to

increase student engagement and achieve transactive communication, where a discussion develops

as participants respond to teach other: giving feedback, taking positions, and bringing in evidence to

socially negotiate new knowledge.

Despite the widespread recognition of transactive communication as a goal of Computer-Mediated

Conferencing (CMC), online discussions often merely consist of individual answers to a question

posed, without any interaction (or even recognition) of the work of other students. A graphical

depiction of this learning activity would look like a "lawn", with short individual blades of grass -- no

branching, no vertical building, none of the "bushiness" of a highly transactive discussion. This paper

develops new quantitative and qualitative ways to measure discussion transactivity, and examines the

instructional design that supports and encourages this type of learning activity.

This paper examines a case study of CMC transactivity at a university using 1-to-1 computer access(each student has a computer on their desk) in a blended teaching environment in Hong Kong.

Four variables were examined for their effect on successful student engagement in online discussions:

Teacher facilitation of social grouping of students

Class time to initiate online discussion interaction

Setting open-ended, challenging topic questions that encourage discussion and debate

Assessment system that reinforces production and peer interaction

The students in the case study used the Moodle e-learning environment, and English as the medium

of instruction. The study took place in a tertiary institution with graduate students in the faculty of

Education, using a blend of face-to-face and e-learning instructional methods. When the instructionalenvironment was set up to support all four discussion design conditions, production and interactivity

both increased dramatically and transactive CMC was achieved.

The results of this study indicate that in order to achieve highly productive and transactive online

discussions in a blended 1-to-1 teaching environment, instructors should incorporate a specific set of

design conditions. When these instructional design conditions are successfully incorporated, the

potential benefits of social constructivism as an instructional design paradigm can be realized within a

blended educational environment.

Key terms : transactivity, social negotiation, discourse, educational dialogue



1.0 Introduction

There is widespread recognition among academic researchers of the value of collaborative groups in

the effective construction of knowledge. This efficacy has been found to be linked to the process that

learners utilize in working on the task together (Fischer et al 2002), and the ways that they socially

negotiate arguments and argument sequences (Leitão 2000; Voss & Dyke 2001). Biggs captured the

educational value of discussion when he stated: "Good dialogue elicits those activities that shape,

elaborate, and deepen understanding" (1999 p. 5).

The final ingredient in this recipe for effective knowledge construction is the method by which they

build on the contributions of their fellow learners, which forms the basis for our working definition of

transactivity (Berkowitz & Gibbs 1983).

Studies in this domain define a transactive discussion as a learning process which involves

understanding the ideas of learning partners, comparing their ideas to one's own, then socially

negotiating a common ground upon which a solution is collaboratively built (Teasley 1997). This

development of a shared understanding is dependent upon the reaction to the ideas of others through

elaboration, questioning, and taking opposing positions (Azmitia & Montgomery 1993).

If we accept that transactivity is a key theoretical construct for measuring collaboration, how can we

describe it in easily grasped graphical and quantitative ways? Once we can measure it, how can we

formulate the instructional design conditions which consistently result in more productive learning

activities?

This paper develops a simple method for measuring transactivity, and examines a case study which

demonstrates how to develop highly productive online discourse activities.

1.1 Transactivity

One aspect of transactivity is strictly objective and quantitative, requiring little judgment on the part of

researchers: discourse productivity. In this study we defined the term production to be the average

number of posts per discussion participant. This term is purely quantitative, as it merely requires

counting the number of posts in a discussion, and dividing the total by the number of participants n :

Production = Total number of discussion posts / n

Researchers in the domain of educational psychology and Computer Supported Collaborative

Learning have developed an extensive categorization of transactive contributions in discourse (Joshi

& Rose 2007). Berkowitz & Gibbs (1983) termed each contribution a transact , and some examples

from their coding scheme are listed in Table 1:

Table 1. Types of transacts and their concrete examples from Berkowitz and Gibbs (1983)

Transact Example

Feedback Request Do you understand or agree with my position?

Dyad Paraphrase Here’s a paraphrase of a shared position.

Competitive

Juxtaposition

I will make a concession to your point, but also reaffirm part

of my position.

Extension Here’s a further thought or an elaboration offered in the

spirit of your position.



Because this study does not seek to identify the key segments of discourse which predict a learning

outcome, we chose to lump all contributions to a discussion which fit any transactive category in this

coding scheme into a simple category: feedback . This coarse level of discourse coding granularity

permits all "dialogue acts" (Stolcke et al 2000) which are judged to be a response to the ideas of a

collaborative learning partner to be simply categorized into a single type. Any responsive transact

was therefore coded as feedback in this study, and the term interactivity was defined as the meannumber of feedback posts per participant:

Interactivity = Total number of feedback posts / n

While the initial basis of a transactive discussion is a quantity of posts in the context, it is only in the

considered response to learning partners that collaboration can take place. While you can have

production without interactivity, you cannot have interactivity without production. Therefore we chose

to define transactivity as the product of quantity and responsiveness, or Production times Interactivity:

Transactivity = Production × Interactivity

An additional way to quantitatively describe a discussion is to measure the level of group activity. The

same way an oral discussion with only one person in it cannot have interactivity, the interactivity level

of an online discussion can be measured by the average number of posts per discussion topic. This

measures the level of group activity:

Group Activity = Total number of discussion posts / number of topics

1.2 Prior Research

In prior research the principal researcher conducted a case study on the CMC production of

adolescent students in a 1-to-1 computer access environment using the Moodle LMS at an English-

medium international secondary school in Hong Kong (Knutzen 2010). In that study evidence was

found that four instructional design conditions had to be met in order to create a learning environment

where transactive online discourse would occur:





When these design conditions were met, student engagement increased dramatically and the

discourse production increased by an order of magnitude, from an average of 1 per student to over 10.

1.3 The current study

The purpose of the current study is to extend this prior research on the instructional design of online

discussions in a blended learning environment. Based on our experience and previous research in

the field, we predicted that when the four instructional design variables met the conditions developed

in the prior study, student engagement and discussion production in the current study would increase

dramatically. Furthermore, we predicted that further data collection and analysis would reveal a

radical improvement in other indicators of a transactive discussion, such as interactivity and group

activity.

Our predictions can be summarized in the two hypotheses:

H1. Redesign of instructional design variables to meet the four conditions specified would

have a significant positive effect on production in the resulting online discussions.



H2. Redesign of instructional design variables to meet the four conditions specified would

have a significant positive effect on transactivity in the resulting online discussions.

To facilitate this examination of the impact of the instructional redesign, the researchers built on

previous research in this field to develop simple quantitative definitions of production, interactivity, and

transactivity. In order to create a graph which visually illustrates the comparison between the student

work products in the two classes, the researchers developed a new information visualization

technique for displaying production and interactivity in an online discussion.

The development of this new graphical technique to represent contributions to a discussion forum

combines the bush-like illustration of social networking and concept sharing in the FlowerGarden data

visualizations of Lantin & Judelmann (2006), with the vertical user paths of the PeopleFlowers data

portraits of Xiong & Donath (1999). By using elements of each the researchers hoped to better

represent the interactivity component which is key to the present analysis of transactivity in a

discussion.

The researchers developed an information visualization (see example in Figure 1) where each

participant in a discussion topic is represented along a horizontal axis, and the height of the graphdenotes the time since the discussion began. Each post along that user timeline is indicated by an

icon which represents the participant, and the interaction is clearly delineated by an arrow which

points from the feedback post back to the post to which it is responding. This new type of data portrait

clearly illustrates the "bushiness" of a highly transactive discussion, and is thus named the

BushGraph.

Figure 1. An example BushGraph illustrating the dynamics of a transactive online discussion.



2.0 Method

2.1 Sample

This case study documents the instructional redesign of an existing blended post-graduate course

titled "eLearning Strategies and Management" at a university in Hong Kong. The "before re-design"

condition was taught by the second author, it had 33 participants and is labeled Class #1, while the"after re-design" condition was taught by the first author, it had 20 participants is labeled Class #2.

2.2 Procedure

The two authors collaborated on the instructional design of the discussion component of Class #2,

and all other design variables were controlled as much as possible. Both classes were taught face-to-

face in the same room using the same blended learning environment: laptops arranged 1-to-1 with

seats around large tables which sat up to 12 students. Both classes used the Moodle LMS to support

the e-learning activities. The course content in both classes remained virtually the same: the lecture

materials, reading assignments, and discussion topic questions were carried over to the new course

design. The only intentional design difference was the modification of the online discussion activities

so that the design variables met the four conditions specified:





2.2.1 Design Variable #1 - Teacher facilitation of social grouping of students

In the initial class session of Class #2, the first author encouraged the students to form small groups

of three to five students for the first oral discussion, and to move to sit close together to facilitate

conversation. This same self-selected grouping was then applied to the online discussions: one

student from each group would create a discussion topic (usually by copying the topic questions), and

then each student in the group would reply to that post in order to form a small group discussion

within the forum.

2.2.2 Design Variable 2 - Class time to initiate online discussion interaction

For each in-class discussion based on the previous lecture segment and the current topic questions,

class time was given for the groups to develop oral arguments, on average about 5 minutes. Next,

the online discussion with matching topic questions was opened on the LMS, and more class time

given (5 - 10 minutes) for the students to take that discussion online, and develop at least their initial

position statements on the topic at hand. The interaction is termed "semi-synchronous" because the

initial posts are put up in real-time, and often the students look around at their learning partner'sactivities and get a few response posts up before time is called for the class to move on to the next

lecture topic.

2.2.3 Design Variable 3 - Setting open-ended, challenging questions that encourage discussion and

debate

Most of material in the course could be easily adapted to create open-ended discussion topic

questions and sub-topics. These types of questions are excellent for basing a Question & Answer (Q

& A) discussion on, as the students can adopt various perspectives and argumentative positions.

Although the lecture materials, reading assignments, and discussion topic questions were all virtually

identical between Class #1 and Class #2, there was more effort made in Class #2 to recap the issue



at hand: restate the topic questions, link to issues from the lecture material, and include thought-

provoking graphics which could be incorporated into the discussion.

2.2.4 Design Variable 4 - Assessment system that reinforces production and peer interaction

The students in Class #2 were trained in the same peer-marking participation-based marking system

developed in the 2007 study, where the initial Summary posts are rated by groupmates as a 6 (out of

10 possible), and every Q & A post thereafter is rated a 10. This marking system uses no subjective

evaluation of content, as the overall mark is completely based on the quantity of contributions to the

discussion, and the arithmetic mean calculated by Moodle and stored as the final grade for the

learning activity.

On every discussion forum this marking system was restated, specifying exactly what was expected

for each type of post, and how it was to be rated. In addition, the instructions specified that the

subject line of the post should indicate the type of post and the expected rating. This helped the

learning partners identify the first post from each participant and mark it with the specified starting

grade of 6.

For this study the discussion instructions requested that the students produce a fourth type of post:

Reflection. This type of contribution was designed to be the culmination of the socially negotiated

knowledge produced by the learning activity, answering the questions "What do you think about this

new material? Has the exposure to other people's opinions changed your viewpoint?". If the summary

post was the trunk of the tree of knowledge, and the Q & A posts were the branches, the Reflection

posts are meant to be the fruit, where the students think critically about what they have learned.

2.3 Measures

All of the student posts in the five examined discussions were coded as either a stand-alone

contribution, or as a feedback response to a previous post. The student work produced in the onlinediscussion learning activities were quantitatively analyzed to produce a data-based comparison of the

student work product between Class #1 and Class #2, where:

Production = Total number of discussion contributions / n

Interactivity = Total number of feedback posts / n

Group Activity = Total number of posts per discussion topic / n

Transactivity = Production × Interactivity

3.0 Results

3.1.1 Results - Discussion #1 (Class #1)

Table 2. Example of discussion #1 instructions from Class #1

Discussion instructions:

Post your comments here.



The discussion instructions provided for Class #1 are extremely minimal. They do not restate thetopic questions from the lecture, link back to pertinent issues, provide graphics to illustrate the topic,or list sub-topic questions.

Table 3. Record of group activity produced by the students on discussion #1 (Class #1)

(Note: student names blocked to protect privacy)

The record of group activity for Class #1 on Discussion #1 primarily shows that there was no group

formation: each student posted in their own discussion topic, and only three posts were technically

replies. Because the learning activity instructions did not direct the formation of small groups, most

students (particularly in Asia) are very hesitant to post replies to another student, as this may be

interpreted as a confrontational challenge. Instead, each student will start their own discussion topic,

and often not read a single post from another student.



The BushGraph in Figure 2 representing Discussion #1 activity by Class #1 is very flat, more of a

"lawn of grass" with individual posts rather than the "bushiness" characteristic of a transactive

discussion. Less than half of the students (0.42) participated in the discussion, and there was not a

single reply responding to another post! Even the three posts which are posted as replies do not

actually respond to the previous post in that discussion topic, but are stand-alone posts makingindependent statements, and were not coded as feedback.

Figure 2. The BushGraph representing discussion #1 (Class #1). Note: the three top-level posts shown are

not responses to the previous posts in the same topic discussion, so they were not coded as feedback.

http://tlcserver.ln.edu.hk/BrantKnutzen/Research/ICEL_2010/Fig2-Discussion1Class1.png



3.1.2 Results - Discussion #1 (Class #2)

Table 4 shows an example of the comprehensive instructions which Class #2 were given for all online

discussions. They specifically state what the students are expected to do, how much time they will be

given for oral discussion, and the topic questions from the lecture. In addition, the peer-marked

participation-based assessment system is methodically laid out in a series of steps, so the students

know what to do and what sort of grade they will get if they participate in the discussion. Also note the

deadline provided (two weeks from the start date), which is essential for successful moderation and

management of the learning activity.

Table 4. Example of the same discussion #1, but using the progressive peer-marking participation-based

system for Class #2 specified in the instructions. Note: the terms LMS (Learning Management System) and VLE

(Virtual Learning Environment) are interchangeable in this context.

Discussion Instructions:

VLE Comparison Discussion

As the students present their information about the selected VLEs, take notes.

Afterward, the class will have 5 - 10 minutes to discuss and compare them.

Address the following questions in your discussion:

1. What are the common factors that seem to be present in all VLEs?

.

2. Is one system obviously better than another system?

.

3. Who are the various VLEs designed for (administrators, teachers, or students?)

.

4. Are there any specific issues special to Hong Kong students?

--------------------------------------------------------------------------------

(This discussion is peer assessed using a 6 / 10 / 10 / 10 system)

Each table group should Create a Discussion Topic and post:

1. A summary of the material from the lecture, the resources, and their research online. (Subject line:

First post = 6) 2. Read other people's summaries and reflections. What questions do you have for them? (Subject

line: Q & A = 10) 3. Read the questions that have been posted to your group. Do some research, and construct your

answer to the question. (Subject line: Q & A = 10) 4. Reflections on summary posts. What do you think about this new material? Has the exposure to

other people's opinions changed your viewpoint? (Subject line: Reflection = 10)

Each student should put up a minimum of 4 posts. The rating grades will be averaged into a single

Participation grade for the forum.

The more you post, the better your grade, but be SURE to keep the quality upwith thoughtful posts.

Include pictures if you feel they illustrate the concept.

Note: participation on this discussion must be posted by Oct 12, 6:30pm (posts after that date cannot be

rated)



Table 5 shows the record of group activity for Discussion #1 produced by Class #2, and the effect of

the procedure creating small self-selected groups based on physical seating proximity. Each small

group of 3 to 5 students forms a nexus around which a discussion focus develops. Groupmates are

recognized as learning partners, and the social barriers of responding to each other are reduced.

Table 5. Record of group activity produced by the students on discussion #1 (Class #2).

(Note: student names and pictures blocked to protect privacy)

The BushGraph in Figure 3 qualitatively illustrates the "bushiness" of a discussion which has achieved

transactivity. Reply posts which give feedback to previous posts are indicated with an arrow pointingback to the post they are responding to.

Figure 3. The BushGraph representing discussion #1 (Class #2)




Table 6 shows the quantitative descriptive statistics comparing discussion #1 between the two

classes. The Class #2 discussion is far more productive and transactive, reflecting the higher student

motivation and engagement with this learning activity.

Table 6. Quantitative descriptive statistics comparing discussion #1 between classes (based on Figures 2 and 3)

3.2 Results - Discussion #2 (comparing Class #1 to Class #2)

Figures 3 and 4 show how the BushGraph provides a quick qualitative overview of the discussionactivity, and a sense of the transactivity achieved. Again, Class #2 achieves a much higher vertical

dimension, reflecting the layers of responsive feedback developed as the learning partners socially

negotiate an understanding of the topic. Two of the members of the ManU group found the

interactivity available in the "4 in Group" group to be more rewarding than their own group, and

actually posted more in that discussion topic than their own!


Class #1 #2

Production 0.42 3.40

Interactivity 0.00 1.75

Group Activity 1.27 11.3

Transactivity 0.00 5.95





Table 7 quantifies the huge difference in student engagement with the discussion learning activity

between the two classes. The production for Class #2 is nearly 14 times higher than that of Class #1.Some of the difference in motivation levels is due to the fact that only Discussion #3 was assessed for

Class #1, while for Class #2 every discussion activity contributed to the overall grade.

Table 7. Quantitative descriptive statistics comparing discussion #2 between classes (based on Figures 4 and 5) Class #1 #2






Figure 6 displays the BushGraph generated by the most productive discussion for Class #1:

Discussion #3, where they achieved an average of 2.03 posts per student. This jump in production is

undoubtedly due to the fact that this discussion was the only discussion learning activity worth points:

about 3% of the total course grade. This discussion posed some of the key issues of the entire e-

learning course, and there were no specific answers available in the extant literature, so the students

were required to post their own opinions. Where there is no established single correct answer, the

topic is open to debate. Even here the lack of student groupings limited the interactivity to 0.03, so

the overall transactivity score was only 0.01.





(Note: long horizontal graph broken and displayed in two parts)

Figure 7 shows the BushGraph for the learning activity of Class #2 generated by Discussion #3. This

was also the most productive discussion for Class #2, with an average of 5.56 posts per student. The

students responded to the challenging and open-ended topic, and at this point in the course (Session

6 of 10) they had grown comfortable with their learning parters. The 4G group in particular was highly

productive and interactive in their discussion topic, generating 45 replies. This group led the early

exploration and development of possible answers on this discussion, closely watched by the other

groups. The principal researcher also played an active moderator role, putting up 7 posts in their

discussion topic to praise work, answer questions, and highlight questionable logic.






Table 8 highlights the value of interactivity and group activity in producing a transactive discussion.

Even though Class #1 achieved 37% of the production of Class #2, their interactivity on this

discussion was only 10%, and only 6% of the group activity, compared to that of Class #2. The

resulting transactivity score for Class #1 was less than 4% that of Class #2.


Class #1 #2









Figure 8 shows that Class #1 is starting to overcome the lack of teacher-supported groupings, and

develop their own learning partner groupings. This discussion generated the highest level of

interactivity for Class #1 (0.55 per student), perhaps due to the localization of the topic (about

technology use in Hong Kong) making the topic more approachable and interesting.

Figure 8. The BushGraph representing discussion #4 (Class #1).






Figure 9 shows that Class #2 also found this topic interesting and approachable, with a production

level of 6.16 per student. Group activity was very high as students felt comfortable and made multiple

posts to bring in outside resources to the discussion. Table 9 sums up the quantitative gap between

the Class #1 activity and Class #2 on Discussion #4, where the overall Transactivity score of Class #2

was 33 times that of Class #1!


Class #1 #2












Figure 11 shows how the BushGraph yields valuable insight into the tremendous differences between

the group dynamics in the discussion topic groups: "4G" has a leader who tends to dominate, "5-3-2"

actually has very little interaction, "5 Girls" is primarily a conversation between two students, and the

"4 in Group" discussion is a wild melee with posters responding to a wide range of other participants.


Class #1 #2





4.0 Analysis

The visual representation of the BushGraphs clearly shows the qualitative difference between the

discussions of Class #1 and Class #2, on multiple dimensions. The vertical dimension is much higher

for all Class #2 graphs compared to Class #1, reflecting the higher productivity and group activity.

The proliferation of arrows in the Class #2 graphs reflects the much larger quantity of feedback posts

between users, which is the most direct measure of interactivity. The grouping of students results in a

series of discussion foci with three to five participants, which serves as the nexus around which forms

the "bush" of replies characteristic of a transactive discussion.

Table 11 summarizes the quantitative descriptive statistics for the five examined discussions. These

statistics state the case even more unequivocally than the BushGraphs: on all measures the

discussions of Class #2 score dramatically higher than Class #1. The transactivity scores are

exponentially higher for Class #2, reflecting the higher student motivation to produce work being

multiplied by the degree of interactivity in the discussions.



Table 11. Summary of the quantitative data comparison between Class #1 and Class #2.

Class #1

Transactivity Score

Discussion # Production Interactivity Group Activity (Production × Interactivity)

1 0.42 0 1.27 0.002 0.36 0.03 1.09 0.01

3 2.03 0.42 1.26 0.85

4 1.18 0.55 1.95 0.65

5 0.55 0 1 0.00

Marg. Means 0.91 0.20 1.31 0.30

Class #2

Transactivity Score

Discussion # Production Interactivity Group Activity (Production × Interactivity)1 3.4 1.75 11.3 5.95

2 5 2.8 20 14.00

3 5.56 4.15 22.6 23.07

4 6.15 3.45 24.6 21.22

5 6.25 3.55 25 22.19

Marg. Means 5.27 3.14 20.70 17.29

Another issue that must be addressed with the peer-marking, participation-based rating of discussion

posts is the reliability and validity of this assessment method. How consistent are the gradesproduced by this method, and how well do they correlate with grades on teacher-assessed learning

activities?

Table 12 shows that the correlations, with few exceptions, are quite high between the gradesproduced by the five examined discussion learning activities. This indicates that the peer-markedparticipation-based discussion grading method has an acceptable level of test-retest reliability.

Table 12. Correlations between grades produced by discussions in Class #2.

Discussion # 1 2 3 4 5

1 0.387 0.637** 0.666** 0.667**

2 0.387 0.656** 0.308 0.542*

3 0.637** 0.656** 0.527* 0.836**

4 0.666** 0.308 . 0527* 0.578**

5 0.667** 0.542* 0.836** 0.578**

Marg. Means 0.589 0.476 0.664 0.520 0.656

* significant at the p < .05 level (2-tailed)** significant at the p < .01 level (2-tailed)



We surmise that the variability in these correlations is due to the differences in the content being

examined in the discussion, and the topic questions employed. Different types of discussions appeal

to different types of students, which alters their learning activity behavior, and the resulting patterns of

grades produced. Another factor which may play a role in reliability is the varying student attendance

for the real-time in-class interaction.

The reflective journal assignment is a collection of all the reflection posts from the discussion forums,

which the student then re-examines and reflects on in the light of later learning and meta-cognition.

This essay assignment is teacher-assessed with a weight of 5% of the final course grade. The final

project is the design and implementation of an e-learning course by each student based on their

uniquely selected learning context. It is a coherent collection of online learning activities which are

designed to meet their stated course learning outcomes, and is teacher-assessed with a weight of 50

percent of the final course grade.

Table 13. Correlations between discussion grades and the teacher-assessed learning activities and course total

grades in Class #2.

Discussion # Reflective Journal Final Project Course Total

1 0.631* 0.608* 0.735*

2 0.791* 0.757* 0.856*

3 0.829* 0.919* 0.845*

4 0.615* 0.675* 0.622*

5 0.793* 0.885* 0.838*

Marginal Means 0.732 0.769 0.779

* significant at the p < .01 level (2-tailed)

As table 13 shows, the correlations between the peer-marked participation-based grades on the five

examined discussions and these teacher-assessed learning activities are very strong (range from

0.608 to 0.919), with every single one significant at the p < .01 level. This would indicate that the

grades produced by the peer-marked participation-based discussion forums are definitely measuring

the same attributes of student behavior as the teacher-assessed learning activities. The final project

is produced by a series of seven formative stages with feedback, and it is the culmination of all the

learning activities in the course. Because this assignment is thoroughly examined and scored by the

instructor on multiple scales based on an exacting rubric, the high correlation with the discussion

grades (marginal mean of .769) indicates a very high validity for interpreting this participation-based

grading method as an accurate measure of student performance.

Table 13 also shows a high degree of correlation between the individual discussion grades and the

course total grades (marginal mean of .779), each correlation significant at the p < .01 level. Because

the Class #2 curriculum had over 50 assessed learning activities, these correlations indicate that the

individual discussions have a very high validity for interpreting this participation-based grading method

as an accurate indicator of overall student performance in the course.

As a final analysis of the validity of the participation-based grading method, we investigated the

relationship between the teacher-assessed final project learning activity and the transactivity

characteristics of the discussions. Given that there is a strong correlation between the grades of the

five examined discussions and the project grades, the correlation between these coefficients and thedescriptive statistics of each discussion could prove illuminating.



Table 14 shows the correlations between the grades correlation coefficients and the discussion

descriptive statistics. They range from .61 to .82, which indicate a very high level of correlation. We

interpret these correlations to indicate that as a discussion reaches higher levels of production,

interactivity, group activity, and the resulting transactivity, there tends to be a higher correlation

between the discussion grades and the project grades.

The most important factor would appear to be interactivity: as students respond more to each other in

a discussion, the resulting discussion grades correlate more strongly with the teacher-assessed

project grade.

Table 14. Correlations between discussion descriptive statistics and the relationship between project grades

and discussion grades

Discussion Descriptive Statistic Correlation between elements

Production 0.61

Interactivity 0.82

Group Activity 0.65

Transactivity 0.76

Given the assumption that a teacher-assessed activity (particularly a final project worth 50% of the

final grade) is highly accurate in measuring student performance, this 0.82 correlation would therefore

indicate that as interaction in a discussion goes up, the learning activity becomes more accurate in

predicting student performance.

5.0 Discussion

The primary aim of this study was to confirm and extend prior research on the effect of instructional

design on discussion transactivity. We also sought to extend Xiong's PeopleGarden (1999) data

portrait to better graphically illustrate discussion interactivity.

We did find evidence supporting our first hypothesis that the redesign of the instructional design

variables to meet the conditions developed in study #1 would have a significant positive effect on the

production in the resulting online discussions. In addition, we found evidence to support our second

hypothesis: the redesign had a significant positive effect on several measures of transactivity,

including interactivity, group activity, and transactivity as a calculated product of discussion productionand interactivity. The peer-marked participation-based scoring method was found to have high

reliability and validity, with strong correlations to teacher-assessed learning activities and the course

total grade.

There were some limitations to the study. The sample size is small and almost exclusively Asian

students. Replication and extension of this study with a larger sample size in other cultural settings

would provide additional perspectives with greater predictive validity. Replication would also address

a possible additional factor in the observed increase in discussion production and transactivity:

differences in instructor teaching style and emphasis on the importance of online discussions in a

blended learning environment.

Further directions for research in this area could seek to determine a causal direction: is the highly

interactive discussion environment conducive to the creation and demonstration of higher levels of



understanding, or are students with higher levels of understanding more capable and willing to create

interactive discourse?

6.0 Conclusion

The results of this study indicate that in order to achieve highly productive and transactive online

discussions in a blended 1-to-1 teaching environment, instructors should incorporate a specific set ofdesign conditions:





When these instructional design conditions are successfully incorporated, the potential benefits of

social constructivism as an instructional design paradigm can be realized within a blended educational

environment.

7.0 References

Azmitia, M. and Montgomery, R. (1993) "Friendship, transactive dialogues, and the development of

scientific reasoning", Social Development , Vol. 2, No. 3, pp 202-221.

Berkowitz, M. and Gibbs, J. (1983) "Measuring the developmental features of moral discussion",

Merrill-Palmer Quarterly, Vol. 29, pp 399-410.

Biggs, J. (1999) "What the Student Does: teaching for enhanced learning", Higher Education

Research & Development , Vol. 18, No. 1, pp 57.

Fischer, F., Bruhn, J., Gräsel, C., and Mandl, H. (2002) "Fostering collaborative knowledge

construction with visualization tools", Learning and Instruction, Vol. 12, pp 213-232.

Josh, M. and Rose, C. (2007) "Using Transactivity in Conversation for Summarization of Educational

Dialogue", Proceedings of the SLaTE Workshop on Speech and Language Technology in Education.

Knutzen, K.B. (2010) "Achieving Productive Computer-mediated Conferencing", Presented at the 3C

Teaching and Learning Innovation Symposium, Polytechnic University, Hong Kong

Lantin, M. and Judelman, G. (2006) "FlowerGarden: An Interactive Visualization of Concept-Sharing",

Proceedings of the CHI 2006 Conference on Human Factors in Computing Systems.

Leitão, S. (2000) "The potential of argument in knowledge building" Human Development, Vol. 43, pp

332-360.

Stolcke, A., Ries, K., Coccaro, N., Shriberg, J., Bates, R., Jurafsky, D., Taylor, P., Martin, R., Van

Ess-Dykema, C. and Meteer, M. (2000) "Dialogue act modeling for automatic tagging and recognition

of conversational speech", Computational Linguistics, Vol. 26, No. 3, pp 339 –373.

Teasley, S. D. (1997) "Talking about reasoning: How important is the peer in peer collaboration?", In:

L.B. Resnick, R. Säljö, C. Pontecorvo and B. Burge, Editors, Discourse, tools and reasoning: Essays on

situated cognition, Springer, Berlin (1997), pp 361 –384.

Voss, J.F. and Van Dyke, J.A. (2001), "Argumentation in Psychology", Discourse Processes, Vol. 32,

No. 2/3, pp 89-111.

Xiong, R. and Donath, J. (1999), "PeopleGarden: creating data portraits for users", Proceedings of the

12th annual ACM symposium on User interface software and technology.

Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2

Documents

Transcript of Achieving Trans Active CMC Brant Knutzen ICEL 2010 v2