Wang, S.-L., Hsu, H.-Y., Lin, S. J., & Hwang, G.-J. (2014). The Role of Group Interaction in Collective Efficacy and CSCL

Performance. Educational Technology & Society, 17 (4), 242–254.

242 ISSN 1436-4522 (online) and 1176-3647 (print). This article of the Journal of Educational Technology & Society is available under Creative Commons CC-BY-ND-

NC 3.0 license (https://creativecommons.org/licenses/by-nc-nd/3.0/). For further queries, please contact Journal Editors at ets-editors@ifets.info.

The Role of Group Interaction in Collective Efficacy and CSCL Performance

Shu-Ling Wang1*

, Hsien-Yuan Hsu2, Sunny S. J. Lin

3 and Gwo-Jen Hwang

1

1Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology,

Taiwan // 2Department of Leadership and Counselor Education, University of Mississippi, Mississippi, USA //

3Institute of Education, National Chiao-Tung University, Taiwan // shuling@mail.ntust.edu.tw //

suhy0914@gmail.com // sunnylin@mail.nctu.edu.tw // gjhwang.academic@gmail.com *Corresponding author

(Submitted June 8, 2013; Revised December 25, 2013; Accepted March 14, 2014)

ABSTRACT Although research has identified the importance of interaction behaviors in computer-supported collaborative

learning (CSCL), very few attempts have been made to carry out in-depth analysis of interaction behaviors. This

study thus applies both qualitative (e.g., content analyses, interviews) and quantitative methods in an attempt to

investigate the role of interaction behaviors (i.e., cooperative processes, cognitive involvement) in group

motivation (i.e., collective efficacy) and group performance in CSCL. A total of 35 college students generated

387 cooperative processes and 421 cognitive ideas during the collaborative process over six weeks. The results

indicated that the quality of interaction behaviors was positively related to CSCL performance. High

performance groups were more involved in complex cooperative processes, and expressed more cognitive ideas

at both high and low cognitive levels than low performance groups. For the role of interaction in collective

efficacy, the quantity rather than quality of interaction behaviors played a more critical role in constructing

collective efficacy. High efficacy groups had more cooperative processes and cognitive ideas than low collective

efficacy groups. The interviews also revealed that group interaction played a mediating role in constructing

collective efficacy. For the role of collective efficacy in group performance, the results also show that the high

efficacy groups performed better than the low efficacy groups. Implications and suggestions for future research

are also provided.

Keywords Group interaction, Collective efficacy, Computer-supported collaborative learning (CSCL), Cooperative process,

Cognitive involvement

Introduction The development of technology has been found to have positive effects on student learning. Particularly, research suggests that collaborative learning with computer technology (e.g., CMC) or computer-supported collaborative

learning (CSCL) has gained great attention in the educational context, because it provides students with more flexible ways of acquiring information (Palmieri, 1997; Wang & Lin, 2007). In a meta-analysis of 122 studies with technology in CSCL (hereafter, CSCL is used for all terms regarding collaborative learning with technology), the results in general support that collaborative learning promotes students’ use of strategies, positive attitudes and achievement (Lou, Abrami, & d’ Apollonia, 2001). However, research also suggests that students may behave differently in their interaction, such as in online discourse (Caspi, Chajut, Saporta, & Beyth-Marom, 2006; De Laat

& Lally, 2003; Häkkinen & Järvelä, 2006; Liu, Chung, Chen, & Liu, 2009; Salovaara & Järvelä, 2003), and cognitive activities (De Laat & Lally, 2003; Hurme, Palonen, & Järvelä, 2006; Ke, 2013). For example, research has proposed students’ behavioral problems in CSCL discussion, including difficulties in connecting online comments to course concepts, providing assertions without evidence, being less critical online (Murray, 2000), and generating less high quality discourse regardless of education level (Lipponen, Rahikainen, Hakkarainen, & Palonen, 2002; Wang & Hwang, 2012). To better understand students’ interaction behaviors, this study thus attempts to apply qualitative

analyses to students’ authentic interaction behaviors occurring in CSCL, which should provide a more comprehensive understanding of students’ interaction behaviors, and along their role in CSCL performance. In addition to group interaction, research has indicated that group motivation, namely, collective efficacy, has positive effects on group performance in various areas, including schools, organizations, and sports (Bandura, 1997; Goddard, 2001; Klassen & Krawchuk, 2009; Schunk, Pintrich, & Meece, 2008; Wang & Lin, 2007). However, very

few attempts have been made to examine the role of collective efficacy in CSCL performance. This study attempts to fill this gap. In addition, research has shown that collective efficacy is critical in collaborative learning, so there is a need to learn more about its construction. Bandura (1997) proposes the theory of collective efficacy and also suggests that group interaction is very likely to influence the construction of collective efficacy. As research

243

suggested, CSCL environments containing less social clues, such as body gestures, and facial expressions, result in various interaction behaviors as compared to traditional environments (Wang & Hwang, 2012). Group interaction may play a particular role in collective efficacy in the CSCL environment. Hence, this study also attempts to investigate the role of interaction behaviors in constructing collective efficacy in the CSCL environment.

Theoretical background

Group interaction (cooperative processes, cognitive involvement), group performance and collective efficacy

Recent research has shown that interaction plays a significant role in CSCL (Hooper, 2003; Ke, 2013; Jung, Choi, Lim, & Leem, 2002; Puntambekar, 2006; Van Drie, Van Boxtel, Jaspers, & Kanselaar, 2005), possibly due to the text-based features and flexibility of the online environment, which makes interaction more convenient and conducive to deep thought, but may also be less focused or even uncontrollable. For example, although research suggests that online written discussion requires more reflection, and thus may develop deeper thought (Serçe, et al., 2011; Wiley & Schooler, 2001), other research suggests that students tend to be passive, less effective, and less

critical in online discussion (Chiu & Hsiao, 2010; Murray, 2000; Walther, Loh, & Granka, 2005), and produce limited cognitive quality during group interaction in CSCL (Wang & Hwang, 2012). Since the effect of group interaction during discussion in CSCL is not clear, the role of interaction behaviors in CSCL is still in need of further analysis. This study therefore attempts to investigate the quality of interaction behaviors, such as cooperative processes and cognitive involvement, along with their roles in CSCL performance.

With respect to cooperative processes, according to Hertz-Lazarowitz (1995), the quality of cooperative processes depends on the actual interaction behavior that occurs in the three stages of learning: input, process and outcomes. As Hertz-Lazarowitz (1995) suggested, simple cooperative processes occur in the input stage such as giving resources, or in the outcome stage such as providing answers without any further discussion, which are considered as low quality interaction. On the other hand, high quality cooperative processes, such as complex cooperative processes, take place in the process stage, for example, engaging in the process of producing a product. Recent research has

indicated that students in online text-based discussion produce relatively more initiatives but fewer responses (Strømsø, Grøttum, & Lycke, 2007), which is in accordance with Hertz-Lazarowitz’s simple and complex cooperative processes, respectively. This study thus attempts to further explore the role of cooperative processes, such as simple and complex processes, in the CSCL environment. In addition, research has suggested that cognitive involvement is important for learning in computer-mediated

communication (Henri, 1992). A recent study indicates that online text construction could be enhanced by students’ metacognition (Yeh & Yang, 2011). Moreover, research also indicates that students who use higher level cognitive strategies have better performance (Garavalia & Gredler, 2002; Wang & Wu, 2008). Willoughby, Wood, McDermott and McLaren (2000) also suggest that, when a group is able to share sophisticated strategic information, the group members are more likely to make contributions to promoting knowledge. This study therefore also strives to explore the quality of cognitive involvement (defined as cognitive ideas applied in learning) and its role in CSCL

performance. Moreover, research indicates that interaction increases student motivation (Jung et al., 2002; Markett, Arnedillo Sánchez, Weber, & Tangney, 2006) and positive interaction promotes group motivation (Johnson & Johnson, 1998). In particular, research suggests that group interaction is more likely to affect collective efficacy (Bandura, 1997, 2000). Jung and Sosik (2003) further illustrate that group members with more frequent interaction tend to develop

stronger collective efficacy in the classroom context. Therefore, in addition to examining the role of cooperative processes and cognitive involvement in group performance, this study also investigates the role of these interaction behaviors in collective efficacy in the CSCL environment. Collective efficacy and performance

According to Bandura (1997), collective efficacy is defined as a group’s shared beliefs in its conjoined capabilities to perform the sequences of action required to achieve designated goals. In other words, collective efficacy is concerned with the performance capability of a group as a whole. Research shows that collective efficacy has a significant effect on group functioning, especially on levels of effort, persistence and achievement (Bandura, 1997, 2000;

244

Myers, Feltz, & Short, 2004; Stajkovic, Lee, & Nyberg, 2009). Research also indicates that collective efficacy is positively correlated to group performance in schools, organizations, and sports (Bandura, 1997; Chow, 2009; Goddard, 2001; Hodges & Carron, 1992). Two meta-analysis studies indicate that the relationship between collective efficacy and group performance is significantly positive (Gully, Incalcaterra, Joshi, & Beaubien, 2002; Stajkovic et

al., 2009). For example, a meta-analysis investigating 6,128 groups with 96 studies shows that collective efficacy is significantly related to group performance (r = .35) (Stajkovic et al., 2009). This study thus hypothesizes that collective efficacy should have similar effects on CSCL performance. Research questions

What is the role of interaction behaviors (i.e., cooperative process, cognitive involvement) in CSCL

performance? What is the role of interaction behaviors (i.e., cooperative process, cognitive involvement) in collective

efficacy in the CSCL environment? What is the role of collective efficacy in CSCL performance?

Methodology Measures

Questionnaire The scale of “collective efficacy” consisted of 8 items (e.g., I believe that my group can understand the most difficult part of this course) and has proven to be very reliable and valid in previous studies (Wang & Lin, 2007). The questionnaire used a 7-point Likert scale. The results of the reliability test indicated that it was very reliable, with an alpha of .926. All items were loaded with a factor loading over .40.

The networked system

Figure 1. The NetPorts system (Wang & Lin, 2007)

This study used the networked portfolio system (NetPorts, Figure 1), which has been proven to be reliable and

applicable (Wang & Lin, 2007). The system was designed with a system management module which allows teachers

to easily monitor student progress. In addition, the NetPorts system provided an online questionnaire module to

collect students’ data, and also a non-synchronous online chat room that allowed students to discuss their group

assignment with their group members. Moreover, students were able to upload homework, process peer assessments,

propose ideas and view records of self-reflection, peer feedback and ideas using NetPorts.

245

Participants

There were 35 students generating 387 interaction processes and 421 cognitive ideas over six weeks in this study.

The selected participants were students enrolled in a mandatory course entitled “Educational Measurement and

Evaluation” in the Teacher Education Center of a research university in Taiwan. The Teacher Education Center offers

courses for students from various majors who consider the teaching profession as their future career. Of these 35

participants, 10 were from Mathematics, 15 from Foreign Languages, 3 from Science, 3 from Biology, 2 from Civil

Engineering, and 2 from Computer Science.

Procedures

Group assignment

The group assignment was to design test questions for a high school mid-term examination based on the group

members’ academic domain. For example, students who were majoring in Biology would write the test items for the

mid-term Biology exam. All groups needed to discuss and decide on the content of units and question format (e.g.,

multiple choice, short answer, true-false questions, etc.), as well as design the test items of the exam in accordance

with Bloom's two-way specification table, which also placed demands on their cognitive effort. The design of the

mid-term exam was not a simple but an ill-structured task, and should encourage more group interaction and

cognitive involvement during the process. Finally, the group assignments were evaluated and graded by the

instructor based on the scoring rubrics. Thus, every group would receive a score based on their performance of the

assignment (i.e., group performance).

Grouping

Research has suggested that small group size encourages group interaction (Abrami et al., 1996), so each group only

consisted of 2-3 students from the same major, since they needed to complete a group assignment which was based

on their academic domain. If only two students were from the same major, there were only two students in the group.

Process

Figure 2. The group discussion board

The students were requested to discuss their group projects using the NetPorts system over a period of six weeks.

The students could discuss their projects in the non-synchronous online chat room through Netports. Students could

User name

Group discussion board

Student registration information

Log out

News

New posts Time

Subject

Group member

id

246

only discuss their projects with their group members. They could initiate or reply to a topic issue posted on their

group discussion board through NetPorts (see Figure 2). Students’ networked discussion would be considered as

their participation score (15%) for their final grade, a form of participation evaluation common in instructors’

grading systems. All students had to fill out the questionnaire of “collective efficacy” through the NetPorts system,

and the system would then allow them to upload their group projects.

After completing the collaborative task, researchers also interviewed students from two groups with high quality

interactions, and two groups with low quality interactions. These students were interviewed individually. The

interview questions consisted of students’ perceptions of group interaction and their influence on collective efficacy,

as well as the factors influencing group interaction.

This study used both quantitative and qualitative methods for data analysis.

Quantitative method

Statistical techniques were used to analyze the data. Traditionally, a t-test is used to assess the statistical significance

of differences between groups. However, according to Coe (2002), “statistical significance” in statistics, the

likelihood of the difference between two groups, could just be an accident of sampling, and other researcher has also

suggested that Cohen’s effect size, which is independent of sample size, can provide practical significance for better

data interpretation (Cohen, 1988). Although there were 387 interaction processes and 421 cognitive ideas for data

analysis, there were only 13 groups in this study, for which it was difficult to achieve traditional statistical

significance among groups. Thus, the Cohen’s effect size, which is not affected by the sample size, should be more

appropriate for the data analysis of this study. In addition, the recent publication manual of the APA (American

Psychological Association) (2009) has claimed that “it is almost always necessary to include some measure of effect

size in the results section…such as a Cohen’s d value” (p. 34). Thus, in addition to the statistical significance test,

this study also reported the practical significance using the Cohen’s d test for data analysis. According to Cohen

(1988), the effect size d = 0.2 is considered as a small effect size, d = 0.5 as medium, and d = 0.8 as large. It is

important to note that, in this study, only the large effect size (d ≥ .8) is considered as having practical significance in

our data analysis, while small and medium are not.

In addition, although our groups consisted of either two or three members (according to their academic domain),

which might be suspected as having an effect on group interaction, after checking all the quantitative results, our

results showed no difference between the groups with two members and those with three members, as well as no

difference between individual (i.e., group interaction data/members) and group level analysis. Since this study places

emphasis on computer-supported collaborative learning, the results are presented at the group level of analysis.

Qualitative methods

In addition to the quantitative analysis for questionnaire and hypothesis, this study also applied qualitative methods,

such as content analysis of online discussion and interviews to explore more in-depth information. For example,

researchers have suggested that content analysis involving both numeric and interpretive data analysis provides a

more meaningful analysis of group discussion (Hara, Bonk, & Angeli, 2000; Henri, 1992; Woo & Reeves, 2007), as

well as that interviews help gain insight into participants’ experiences (Turner, 2010). In this study, two content

analyses were used to analyze the CSCL discourse, one for the students’ cooperative processes, and the other for the

quality of cognitive involvement. The analysis of the cooperative processes was based on Hertz-Lazarowitz’s (1995)

three phases of interaction (input, process and outcomes). Students’ interaction behaviors (i.e., posted messages) only

involved in the input phase (e.g., giving resources), or in the outcome phase (e.g., providing answers) were

considered as simple cooperative processes, while interactions engaged in the process of achieving a cooperative

goal, such as engaging in discussions, were perceived as complex cooperative processes. Each posted message to

other group members only counted as one cooperative process (i.e., a simple or complex process). Therefore, the

sum of the simple and complex processes was the total number of cooperative processes of an individual. The total

cooperative processes of each group were the total cooperative processes of all group members.

247

In addition to cooperative processes, the quality of the students’ cognitive involvement (i.e., the quality of cognitive ideas) was also analyzed. Each idea was considered as one unit to be analyzed. The assessment of students’ cognitive ideas during the CSCL discussion was based on Bloom’s (1956) taxonomy of educational objectives. The taxonomy consisted of six cognitive levels, ranging from knowledge, comprehension, application, analysis, synthesis to evaluation. Research suggests that knowledge and comprehension are viewed as low-level cognitive skills, whereas application, analysis, synthesis and evaluation are considered as high-level cognitive skills (Swart, 2010). Thus, student group discussion ideas categorized as knowledge and comprehension were coded as low-level cognitive ideas, whereas ideas comprising application, analysis, synthesis and evaluation were coded as high-level cognitive ideas. It is important to note that our study placed more focus on the role of task -related interactions in CSCL, so only task-related (or cognitive-related) interactions were evaluated for cognitive involvement and cooperative process, while off-task discussions (i.e., discussions not related to task content or cognitive-unrelated messages) were not analyzed in our content analysis. Moreover, the interviews were also transcribed and coded using content analysis. Two graduate students in the Department of Education served as the raters for this study. The inter-rater reliability was calculated by the Cohen’s Kappa. The results indicated that the inter-rater reliability was good with .826 for the simple processes, .873 for the complex processes, .887 for low cognitive involvement, and .868 for high cognitive involvement. This demonstrated that content analysis was a reliable measure for interaction behaviors in our study. Results The descriptive statistics of content analysis for cooperative process and cognitive involvement were shown in Table 1. Taking the average of the scores from the two raters, the descriptive statistics showed that there were a total of 387 cooperative processes (M = 29.77, SD = 11.80) and 421.5 cognitive ideas (M = 32.42, SD = 16.59) generated by the 13 groups. Our analysis also showed that all variables analyzed in our study were approximately normally distributed.

Table 1. Descriptive statistics of cooperative processes and cognitive ideas

X SD Max Min Total

Cooperative processes 29.77 11.80 48.00 10.00 387

Complex processes 4.77 5.08 16.50 0.00 62

Cognitive ideas 32.42 16.59 59.00 11.00 421.5

High cognitive ideas 6.65 5.69 16.50 0.00 86.5

Low cognitive ideas 25.77 12.49 44.00 7.00 335

Examples of content analysis for the cooperative processes were shown in Table 2. As previously stated, a simple cooperative process refers to a group member’s posted messages in either the input or the output phase, without any on-task discussion, while a complex cooperative process refers to group members’ messages engaged in the on-task discussion, usually with a discussion thread. For example, a group member replying to others with, “I fully agree with your viewpoint, I think we should make the test questions in accordance with every educational objective,…” is considered as a complex cooperative process. The pair t-tests showed a significant difference (t = 8.16, p < .01) between the simple (M = 25, SD = 9.15) and complex cooperative processes (M = 4.77, SD = 5.08); that is, students had significantly more simple cooperative processes than complex ones.

Table 2. t-test and examples of content analysis for cooperative processes

Examples of cooperative process Mean SD t

Simple process

I found the following test item examples on the Internet, please refer to the attached files (only input phase, no discussion). I have completed the short-answer questions for our projects, so I have done my work, please just add it in (output phase, no discussion).

25.00 9.15 8.16**

Complex process

I fully agree with your viewpoint, I think we should make the test questions in accordance with every educational objective,…so I will check every test question and its corresponding educational objective. (process phase, with discussion thread)

4.77 5.08

**p < .01

248

In addition, for the content analysis of cognitive involvement, Table 3 showed both students’ low and high cognitive

ideas in their group discussion. For example, one student posted the message “For the items you constructed, the

most problematic issue is that you used a multiple choice format to assess students’ cognition of evaluation level.

Multiple-choice could be adequately used to measure lower level cognition but not higher level cognition…”, which

was considered as analysis level of cognition, and coded as a high cognitive idea. The pair t-tests showed a

significant difference (t = -6.83, p < .01) between low (M = 25.77, SD = 12.49) and high cognitive ideas (M = 6.65,

SD = 5.69); that is, students generated significantly more low cognitive ideas than high cognitive ideas.

Table 3. T-test and examples of content analysis for cognitive ideas

Cognitive

Levels

Examples of cognitive levels in content analysis M SD t

Low

cognitive

ideas

Knowledge

Comprehension

According to the textbook, making short-answer

questions for the test is the easiest.

I think making a good test should consider 4 sources

that influence quality: reliability, validity, referencing,

and objectivity. Among these, reliability and validity are

of higher priority for quality test construction.

25.77

12.49

-6.83**

High

cognitive

ideas

Application

Analysis

Synthesis

Evaluation

In constructing a multiple choice item, the plausibility

(seeming likely to be true for test takers) of each choice

is my priority concern. Thus, the items that I constructed

are listed in the following….

For the items you constructed, the most problematic

issue is that you used a multiple choice format to assess

students’ cognition of evaluation level (in Bloom’s

taxonomy). Multiple-choice could be adequately used to

measure lower level cognition, but not higher level

cognition. Therefore, I suggest that you construct some

“subjective scored” items such as essays.

Short-answer questions cannot be used for complex or

high-level learning outcomes, so I suggest integrating

the short-answer questions into the authentic

assessment.

Your designed test seems not to be in accordance with

the rule of the “two-way specification table”. Based on

the rule, the item construction should consider both the

educational objective and content, so the ratio of item

constructions should be …..

6.65

5.69

**p < .01

The results for the study’s research questions are showed as follows: What is the role of interaction behaviors (i.e., cooperative process, cognitive involvement) in CSCL performance? To better understand the role of interaction behaviors in CSCL performance, we divided the groups into high and low performance clusters based on the scores of group assignment. The scores of 13 groups ranged from 63.00 to 87.00 with mean of 74.77 and SD of 7.64. The high performance cluster comprised groups of performance scores in the

249

approximate top 40%, while the low performance cluster consisted of groups of scores in the bottom 40%. For both high and low performance clusters, our analyses showed that simple/complex processes and low/high cognitive ideas were nearly normally distributed. The t-test results indicated a significant difference between the high and low performance clusters (p < .01). For the role of cooperative process in group performance, as shown in Table 4, both the t-test (t = -2.36, p < .05) and Cohen’s d of 1.50 showed that the higher performance groups engaged in more complex cooperative processes than the low performance groups, but there was no difference in the simple cooperative processes between these two groups. That is, the high performance groups had more complex cooperative processes than the low performance groups. For the role of cognitive involvement in performance, although the t-test results only showed a significant difference for low cognitive ideas, the Cohen’s d values showed practical differences for both low (Cohen’s d = 2.29) and high level (Cohen’s d = 1.01) cognitive ideas between the low and high performance groups. Our results also indicated that the correlation between high cognitive ideas (r = .62) and performance, as well as low cognitive ideas (r = .84) and performance were both significant at the .05 level. In other words, both high and low cognitive ideas were significantly related to CSCL performance. It seems reasonable that students not only need to apply high level cognitive ideas but also low level ideas to achieve a better outcome, because a low cognitive level of information processing, such as knowledge and comprehension, is the basis of knowledge construction. To better understand students’ total cognitive involvement (the sum of the low and high level cognitive ideas) in learning, our t-test (t = -3.38, p < .01) and Cohen’s d (d = 2.13) results also indicated a significant difference in the total cognitive involvement between the low and high performance groups. In other words, the high performance groups had more cognitive involvement in group discussion than the low performance groups.

Table 4. The comparison of interactions between high and low achieving groups

High Ach Low Ach t Cohen's d

Mean SD Mean SD

Simple processes 27.60 10.84 24.80 10.19 -0.42 0.27

Complex processes 8.20 6.37 1.40 0.89 -2.36* 1.50

*

Total cooperative processes 35.80 15.31 26.20 10.04 -1.17 0.74

Low cognitive ideas 37.60 6.67 17.60 10.40 -3.62**

2.29*

High cognitive ideas 10.30 6.95 4.60 3.97 -1.59 1.01*

Total cognitive ideas 47.90 12.96 22.20 11.04 -3.38**

2.13*

t value: *p < .05.

**p < .01.; Cohen’s d:

*large effect size

What is the role of interaction behaviors (i.e., cooperative process, cognitive involvement) in collective efficacy in the CSCL environment? To better understand the role of interaction behaviors in collective efficacy, students were also divided into high (top 40%) and low (bottom 40%) collective efficacy clusters. The data of both high and low collective clusters appeared approximately normally distributed in our analysis. The result of a Cohen’s d value of 1.36 showed a practical significance between high and low collective efficacy clusters, and the t-test results also almost achieved a significant difference between these two clusters at the 0.05 level (t = -2.14, p = .06.). For the cooperative process, as shown in Table 5, the Cohen’s d value of 1.44 indicated a large effect size in the simple cooperative process between low and high efficacy groups, but no difference in the complex cooperative process between these two groups. The results also revealed that there were practical significant differences (Cohen d = 1.31) in the total cooperative processes (the sum of the simple and complex cooperative processes). That is, the high collective efficacy groups had more simple and total cooperative processes than the low collective efficacy groups. For the role of cognitive involvement in collective efficacy, the Cohen’s d values of 1.30 and 1.18 revealed a large effect size for low cognitive ideas as well as total cognitive ideas, and .46 was close to the medium effect size for high cognitive ideas between the low and high efficacy groups.

Table 5. The comparison of interactions between high and low collective efficacy groups

High CE Low CE t Cohen's d

Mean SD Mean SD

Simple processes 30.80 9.84 18.90 6.27 -2.28 1.44*

Complex processes 5.10 6.77 3.20 3.27 -0.57 0.36 Total cooperative processes 35.90 13.05 22.10 7.20 -2.07 1.31

*

250

Low cognitive ideas 31.70 12.46 17.40 9.27 -2.06 1.30*

High cognitive ideas 8.10 5.98 5.80 3.63 -0.74 0.46 Total cognitive ideas 39.80 18.06 23.20 10.08 -1.78 1.18

*

Cohen’s d: *large effect size

The interview analysis also showed the relationship between group interaction and collective efficacy. In the

interviews with the two high quality interaction groups, the students reported that their groups had high confidence in

achieving their group goal through members’ involvement in the group discussion. On the other hand, the students in

the low interactive groups reported that they did not have confidence in completing the group project due to their

team members’ low interaction and commitment in the group discussion. Interestingly, in the low interactive groups,

the interviews also found that even though the students initially perceived themselves or their team members as

having high capability or high efficacy (i.e., self-efficacy) beliefs, without interaction, the group members eventually

came to hold low collective efficacy, believing that their groups could not complete the project with good quality.

That is, group interaction played an important role in forming collective efficacy.

Additionally, the students of the high quality groups reported that they had a good leader who played an important

role in their group interaction, such as assigning the work, monitoring the process, as well as actively promoting

group discussion, which thus enhanced their group’s efficacy beliefs. In other words, leadership may play an

important role in collective efficacy, but needs to go through group interaction, which in turn leads to increased

collective efficacy.

What is the role of collective efficacy in CSCL performance?

For the role of collective efficacy in collaborative performance, the Cohen’s d value of 1.22 showed a large effect

size in achievement between the low (M = 69.20, SD = 4.44) and high collective efficacy groups (M = 77.60, SD =

8.56). In other words, the Cohen’s d value showed that high collective efficacy groups perform better than low

collective efficacy groups.

Conclusions and discussion

In summary, for the role of interaction behaviors in group performance, the results show that the quality of

interaction behaviors is positively related to CSCL performance. The higher performance groups engage in more

complex cooperative processes, and also appear to express more cognitive ideas at both high and low cognitive

levels than the low performance groups. For the role of interaction behaviors in collective efficacy, our results

indicate that the quantity instead of the quality of interaction plays a more substantial role in constructing collective

efficacy. Our results show that groups with more cooperative processes, and those applying more cognitive ideas

tend to have higher collective efficacy. The interview results show a similar finding that collective efficacy is more

likely conveyed by group interaction. For the role of collective efficacy in collaborative performance, our results

indicate that groups with higher collective efficacy demonstrate better academic performance than low collective

efficacy groups in CSCL.

Consistent with our hypothesis, the results show that group interaction is important for students’ CSCL performance.

Particularly, our results demonstrate that high performance groups are more involved in group processes rather than

only in the input or output stage. Accordingly, a study of the effects of interaction on collaborative concept mapping

also demonstrates that complex cooperative processes have positive effects on group performance (Chiu, Huang, &

Chang, 2000). Additionally, other researchers also indicate that group members engaging more in group processing

develop better understanding (Puntambekar, 2006) and have better academic achievement (Hooper, 2003; Kutnick,

Ota, & Berdondini, 2008; Stamovlasis, Dimos, & Tsaparlis, 2006). Indeed, students who are more involved in group

processes should gain more feedback from in-depth discussion to improve their group productivity. To promote

complex cooperative behavior, Kreijn, Kirschner and Jochems (2003) have suggested that positive interdependence

is essential to promote interaction. Teachers may need to provide tasks (e.g., ill-structured tasks) that foster positive

interdependence, which require all team members to contribute efforts to achieve their common goal. In addition, the

task difficulty should also be taken into account; if the designed task is too simple for the team members, they may

complete it individually and provide the final product in the output phase. Moreover, our interviews indicate that

251

leadership facilitates collaborative process, such as assigning the work, monitoring the process, and promoting group

interaction. Therefore, teachers may consider to assign the appropriate leaders to the groups in need in order to

promote collaborative process.

In addition, in accordance with other research findings that interaction has influences on motivation (Markett,

Arnedillo Sánchez, Weber, & Tangney, 2006; Kutnick, Ota, & Berdondini, 2008), our results further indicate that

group interaction is also important in constructing collective efficacy in the CSCL environment. However, our study

shows that the quantity rather than the quality of interaction plays a more critical role in forming collective efficacy.

As previously mentioned, our results show that both complex processes and high cognitive ideas are far less (4-5

times) than simple processes and low cognitive ideas, respectively. As a result, students’ collective efficacy beliefs

are less likely to be affected by very few interactions, such as complex processes or high cognitive ideas in this study.

Both these results consistently show that collective efficacy cannot be developed without sufficient interaction. Our

study further indicates that groups with more interactions, such as cooperative processes or more cognitive ideas,

have higher collective efficacy. However, Chow’s (2009) recent study in sports shows different findings, suggesting

that collective efficacy may not be influenced by individuals’ frequent interactions with proximal members or group

communication density. This difference is possibly due to the fact that our study examines students’ actual

interaction behaviors occurring in CSCL, while Chow’s (2009) study using social network analysis is based on every

member’s general perception of their interactions with each other. The investigation of students’ authentic group

interactions should help to make our study more legitimate and persuasive in understanding the role of group

interaction in the construction of collective efficacy. In addition, it is important to note that the quantity of interaction

is particularly important in constructing collective efficacy in the CSCL environment, which in general lacks social

presence, gesture, or expression, so students need more interactions to construct their collective efficacy. Teachers

may particularly need to encourage more interactions in online environments in order to promote students’ collective

efficacy in collaborative learning.

Our results based on content analysis indicates the importance of group interaction in the construction of collective

efficacy, and the interview results not only validate these findings but also provide more in-depth information, such

as revealing the mediated role of group interaction in constructing collective efficacy. For example, our results

indicate that leadership exerts its influence on collective efficacy through group interaction. The results also further

reveal that the groups benefit from high efficacy (i.e., self-efficacy) members only when these members are involved

in group interaction. In other words, the factors influencing collective efficacy may possibly need to go through

group interaction. These aforementioned findings are consistent with Bandura’s (1997) viewpoint; that is, group

interaction plays an important role in developing collective efficacy. Future research on the factors that affect

collective efficacy should consider the possible mediated effect of group interaction between the factors and

collective efficacy.

The limitation of this study is that several factors that may influence group interaction are not taken into account. For

example, research suggests that heterogeneous group composition is more conducive to collaborative learning (Webb

& Palincsar, 1996), but the group composition of our study was more homogeneous than heterogeneous, because our

task was designed for the students to develop an exam based on their academic domain. Although such group

composition and task design was more realistic in practice, and might help the students to develop instructional skills

for their future career, the homogeneity of the group composition might have some influences on the group

interactions. In addition, since our study only focused on the interaction behaviors in an asynchronous environment,

the role of synchronous features in group interaction is still unknown. Thus, future studies should take these variables

into account, and investigate their influences on group interaction in the CSCL environment. Moreover, a recent

study also indicates a strong relationship between collective efficacy and team performance, but further suggests that

the relationship is likely moderated by collective efficacy dispersion (i.e., heterogeneous collective efficacy within

teams) (Chow, 2009). Thus, the role of collective efficacy beliefs dispersion may need to be further investigated in

the CSCL environment in order to better understand the relationship between collective efficacy and CSCL

performance. Furthermore, our interviews also reveal two possible factors influencing group interaction, such as

group members with high capability and off-task social-emotional discussion; that is, group members with high

capability tend to devote more resources in group interaction, and off-task social-emotional discussions also

influence task-related group interaction. These two factors may need to be further examined in future research.

Finally, future research should also explore other interaction styles or behaviors (e.g., helping behaviors) in order to

better understand their effects on computer-supported collaborative learning and achievement.

252

Acknowledgements

This study is supported in part by the National Science Council of the Republic of China under contract numbers

NSC 99-2511-S-011-009-MY3 and NSC 102-2511-S-011 -007 -MY3.

References

Abrami, P. C., Chambers, B., Poulsen, C., De Simone, C., d’ Apollonia, S., & Howden, J. (1996). Classroom connections:

Understanding and using cooperative learning. New York, NY: Harcourt Brace.

Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.

Bandura, A. (2000). Exercise of human agency through collective efficacy. Current Directions in Psychological Science, 9(3), 75-

78.

Bloom, B. S. (1956). Taxonomy of educational objectives: Handbook of cognitive domain. New York, NY: McKay.

Caspi, A., Chajut, E., Saporta, K., & Beyth-Marom, R. (2006). The influence of personality on social participation in learning

environments. Learning and Individual Differences, 16(2), 129-144.

Chiu, C. H., Huang, C. C., & Chang, W. T. (2000). The evaluation and influence of interaction in network supported collaborative

concept mapping. Computers & Education, 34(1), 17-25.

Chiu, C. H., & Hsiao, H. F. (2010). Group differences in computer supported collaborative learning: Evidence from patterns of

Taiwanese students’ online communication. Computers & Education, 54(2), 427-435.

Chow, G. (2009). Social interaction and collective efficacy dispersion: A social network analysis. (Unpublished master’s

dissertation). Michigan State University, Michigan, USA.

Coe, R. (2002). What is an “Effect Size”? A guide for users. Research Capacity Building, 4, 1-15.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence.

De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a

networked learning community. Instructional Science, 31(1-2), 7-39.

Garavalia, L. S., & Gredler, M. E. (2002). Prior achievement, aptitude, and use of learning strategies as predictors of college

student achievement. College Student journal, 36(4), 616-625.

Goddard, R. D. (2001). Collective efficacy: A neglected construct in the study of schools and student achievement. Journal of

Educational Psychology, 93(3), 467-476.

Gully, S. M., Incalcaterra, K. A., Joshi, A., & Beaubien, J. M. (2002). A meta-analysis of team efficacy, potency, and performance:

Interdependence and level of analysis as moderators of observed relationship. Journal of Applied Psychology, 87(5), 819-832.

Häkkinen, P., & Järvelä, S. (2006). Sharing and constructing perspectives in web-based conferencing. Computers & Education,

47(4), 433-447.

Hara, N., Bonk, C. J., & Angeli, C. (2000). Content analysis of online discussion in an applied educational psychology course.

Instructional Science, 28(2), 115-152.

Hertz-Lazarowitz, R. (1995). Understand interactive behaviors: Looking at six mirrors of the classroom. In R. Hertz-Lazarowitz

& N. Miller (Eds.), Interaction in cooperative groups (pp.102-119). New York, NY: Cambridge University Press.

Henri, F. (1992). Computer conferencing and content analysis. In A. R. Kaye(Ed.), Collaborative Learning through Computer

Conferencing: The Najaden Papers (pp. 117-136). New York, NY: Springer.

Hodges, L., & Carron, A. V. (1992). Collective efficacy and group performance. International Journal of Sport Psychology, 23(1),

48-59.

Hooper, S. (2003). The effects of persistence and small group interaction during computer-based instruction. Computers in human

Behavior, 19(2), 211-220.

Hurme, T. R., Palonen, T., & Järvelä, S. (2006). Metacognition in joint discussions: An analysis of the patterns of interaction and

the metacognitive content of the networked discussions in mathematics. Metacognition and Learning, 1(2), 181-200.

Johnson, D. W., & Johnson, R. T. (1998). Cooperative learning and social interdependence theory. In R. S. Tindale, L. Heath, J.

253

Edwards, E. J. Posavac, F. B. Bryant, Y. Suarez-Balcazar, E. Henderson-King & J. Myers (Eds.), Theory and research on small

groups (pp. 9-35). New York, NY: Plenum Press.

Jung, I., Choi, S., Lim, C., & Leem, J. (2002). Effects of different types of interaction on learning achievement, satisfaction and

participation in Web-based instruction. Innovations in Education and Teaching International, 39(2), 153-162.

Jung, D. I., & Sosik, J. J. (2003) Group potency and collective efficacy: Examining their predictive validity, level of analysis, and

effects of performance feedback on future group performance. Group & Organization Management, 28(3), 366-391.

Ke, F. (2013). Online interaction arrangements on quality of online interactions performed by diverse learners across disciplines.

Internet and Higher Education, 16, 14-22.

Klassen, R. M., & Krawchuk, L. L. (2009). Collective motivation beliefs of early adolescents working in small groups. Journal of

School Psychology, 47(2), 101-120.

Kreijns, K., Kirschner, P. A., & Jochems, W. (2003). Identifying the pitfalls for social interaction in computer-supported

collaborative learning environments: A review of the research. Computers in Human Behavior, 19(3), 335-353.

Kutnick, P., Ota, C., & Berdondini, L. (2008). Improving the effects of group working in classrooms with young school-aged

children: facilitating attainment, interaction, and classroom activity. Learning and Instruction, 18(1), 83-95.

Lipponen, L., Rahikainen, M., Hakkarainen, K. & Palonen, T. (2002). Effective participation and discourse through a computer

network: Investigating elementary students’computer supported interaction. Journal of Educational Computing Research, 27(4),

355-384.

Liu, C. –C., Chung, C. W., Chen, N.-S, & Liu, B. –J. (2009). Analysis of peer interaction in learning activities with personal

handhelds and shared displays. Educational Technology & Society, 12(3), 127-142.

Lou, Y., Abrami, P. C., & d’ Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review

of Educational Research, 71(3), 449-521.

Markett, C., Arnedillo Sánchez, I., Weber, S. & Tangney, B. (2006). Using short message service to encourage interactivity in the

classroom. Computers & Education, 46(3), 280-293.

Murray. B. (2000). Reinventing class discussion online. APA Monitor on Psychology, 31(4), 54-56.

Myers, N. D., Feltz, D. L., & Short, S. E. (2004). Collective efficacy and team performance: A longitudinal study of collegiate

football teams. Group Dynamics: Theory, Research, and Practice, 8(2), 126-138.

Palmieri, P. (1997). Technology in education. . . Do we need it? ARIS Bulletin, 8(2), 1-5.

Puntambekar, S. (2006). Analyzing collaborative interactions: divergence, shared understanding and construction of knowledge.

Computers & Education, 47(3), 332-351.

Salovaara, H., & Järvelä, S. (2003). Students' strategic actions in computer-supported collaborative learning. Learning

Environments Research, 6(3), 267-284.

Schunk, D. H., Pintrich, P. R., & Meece, J. L. (2008). Motivation in education: Theory, research, and applications (3rd ed.).

Englewood Cliffs, NJ: Prentice Hall.

Serçe, F. C., Swigger, K., Alpaslan, F. N., Brazile, R., Dafoulas, G., Lopez, V. (2011). Online collaboration: Collaborative

behavior patterns and factors affecting globally distributed team performance. Computers in Human Behavior, 27(1), 490-503.

Stajkovic, A. D., Lee, D., & Nyberg, A. J. (2009). Collective efficacy, group potency and group performance: Meta-analyses of

their relationships and test of a mediation model. Journal of Applied Psychology, 94(3), 814-828.

Stamovlasis, D., Dimos, A., & Tsaparlis, G. (2006). A study of group interaction processes in learning lower secondary Physics.

Journal of Research in Science Teaching, 43(6), 556-576.

Strømsø, H. I., Grøttum, P., & Lycke, K. H. (2007). Content and processes in problem based learning: A comparison of a

computer mediated and face to face communication. Journal of Computer Assisted Learning, 23(3), 271-282.

Swart, A. J. (2010). Evaluation of final examination papers in engineering: A case study using bloom’s taxonomy. IEEE

Transactions on education, 53(2), 257-264.

Turner, D. W. (2010). Qualitative interview design: A practical guide for novice investigators. The Qualitative Report, 15(3), 754-

760.

Van Drie, J., Van Boxtel, C. A. M., Jaspers, J., & Kanselaar, G. (2005). Effects of representational guidance on domain specific

reasoning in CSCL. Computers in Human Behavior, 21(4), 575-602.

254

Walther, J. B., Loh, T., & Granka, L. (2005). Let me count the ways: The interchange of verbal and nonverbal cues in computer-

mediated and face-to-face affinity. Journal of Language and Social Psychology, 24(1), 36-65.

Wang, S. L., & Lin, S. J. J. (2007). The effect of group composition of self-efficacy and collective efficacy on computer-supported

collaborative learning. Computers in Human Behavior, 23(5), 2256-2268.

Wang, S. L., & Wu, P. Y. (2008). The role of feedback and self-efficacy on web-based learning: The social cognitive perspective.

Computers & Education, 51(4), 1589-1598.

Wang, S. L., & Hwang, G. J. (2012). The role of collective efficacy, cognitive quality, and task cohesion in computer-supported

collaborative learning. Computers & Education, 58(2), 679-687.

Webb, N. M., & Palincsar, A. S. (1996). Group processes in the classroom. In D. C. Berliner & R. C. Calfee (Eds.), Handbook of

educational psychology (pp. 841–873). New York: Simon & Schuster.

Wiley, J., & Schooler, J. (2001). The mental web: Pedagogical and cognitive implications of the net. In C. Wolfe (Ed.) Learning

and Teaching on the World Wide Web (pp. 243-257). New York, NY: Academic Press.

Willoughby, T, Wood, E., McDermott, C., & McLaren, J. (2000). Enhancing learning through strategy instruction and group

interaction: Is active generation of elaborations critical? Applied Cognitive Psychology, 14(1), 19-30.

Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. The Internet

and Higher Education, 10(1), 15-25.

Yeh, H.-C., & Yang, Y.-F. (2011). Metacognitive process in online text construction. Educational Technology & Society, 14(3),

82-101.