Investigating the Technical and Social Challenges Faced by Large Collaborating
Groups
Pedro Antunes, Gustavo Zurita, Nelson Baloian
Abstract. Our main research goal is qualitatively investigating the influence of large
groups on decision-making. We set up an interpretative case study of three large groups,
with 42, 44 and 48 participants, accomplishing a “choose” task with no right answer. The
investigated factors considered information overload, awareness, task involvement, and
collaboration support. Furthermore, we considered these impacts on two main components
of the group task: discussion and consensus. The results indicate that the participants
understand and deal with several trade-offs, notably between stress and immediacy, and
between feedback and noise. The participants avoid information overload using multiple
data reduction and attitudinal strategies. This study contributes to understand the various
trade-offs faced by large collaborating groups.
Keywords: Large Group Collaboration, Group Decision-Making.
1. Introduction
Research on group collaboration has traditionally favoured small groups. A basic search in
SCOPUS for articles from 2007 to 2016 with the keywords “collaboration” and “small
group”, “collaboration” and “large group” and “collaboration” and “crowd”; returned about
374227, 4381 and 64168 papers, respectively, for the field of “Ccomputer science” and
“Ssocial sciences”. Similar search in WoS for articles from same period or years, with also
same topics “collaboration” and “small group”, “collaboration” and “large group” and
“collaboration” and “crowd”; for the research areas “Computer sciences” and “Social
sciences and other topics”; returned 18, 4 and 22 papers; and for research area “Education
and educational research”, returned 94, 7 and 26 papers respectively. Alike results were
obtained with Google Scholar, considering the same keywords specified above, period of
years, and for all subject areas; returned 135, 51 and 134 papers. Although this type of
search is merely circumstantial, with many false positives and negatives, it gives a
reasonable indication of the breadth of research associated with each topic. Interestingly,
the set of papers returned by searching for crowd collaboration has been mainly published
between 2011 and 2015, which also indicates that increasing the group size is a recent
endeavour.
One reason explaining the lack of studies with large groups is that traditional Group
Support Systems (GSS) pose many technical and logistical problems to evaluation [1].
However, the emergence of a new generation of GSS, where we may include Wikipedia,
Facebook, LinkedIn, Doodle, Dropbox, Twitter, Zoho, Google Docs, and Google Maps,
among others, brought new opportunities to research large group collaborations. All these
systems rely on the Web infrastructure to support collaboration and are popular, widely
accessible, and always available [2]. Thus, many of the traditional logistical problems have
been suppressed.
Another reason is that the Web infrastructure is fostering new types of organizations, work
arrangements and collaborations. Projects like Amazon Mechanical Turk challenge
traditional conceptions about organizational culture, sense of belonging, roles and
attributions, responsiveness and remuneration, just to mention a few [3].
In this paper we report on a case study with three groups of 42, 44 and 48 participants. The
main purpose of this study was obtaining insights about the technical and social challenges
faced by large groups performing a collaborative task.
The data collection was obtained from three samples. In each sample a group was requested
to fulfil an urban design assignment. To successfully complete the assignment, the group
members had to walk around a city area, finding design ideas to improve the city living,
sharing their ideas with the others, and then converging into a list of the ten best ideas.
The samples used different collaboration tools, which allowed reducing some biases caused
by analysing collaboration supported on one particular tool.
Considering the research goals, the adopted data analysis methodology was founded on the
principles of qualitative data analysis, where qualitative inquiry and serendipitous findings
are privileged over quantitative assessments [4]. The qualitative data was gathered through
a survey.
The paper is organized as follows. In the next section we provide a brief overview of the
related work. Section 3 describes in more detail the research methodology. Section 4 details
the case study. Section 5 presents the obtained results. Finally, Sections 6 and 7 discuss the
obtained results and present some conclusions from this research.
2. Related Work
The literature related with large group collaboration is dispersed through several research
fields including group decision-making, computer supported collaborative work, social
media, and also the combinations of these fields, which is the specific case of group support
systems. Even though the number of research studies specifically focussed on large groups
seems to be scarce, we have been able to identify several factors that seem to be influenced
by group size. Table 1 provides a summary of the literature review, noting the different
factors that were identified, the observed effects of group size found when studying those
factors, and the restricted context where the effects were analysed.
Table 1. Summary of factors affecting large group collaboration reported in the related
literature.
References
Factors Effects Context
[5] Behavioural cascading
Technology mMitigates coordination costs
Ordinary public goods game, and in a public goods game with punishment
[6] Participation Increases with more participants
Only when participants know each other
[7] Communication Decreases Learning[8] Critical mass Task fails with insufficient
number of contributorsCrowdsourcing
[9][10]
Quality of outcomes Decreases because of coordination costs
Crowdsourcing
[11] Production blocking Decreased Brainstorming[12] Synergy Increased Brainstorming[13] Motivation Decreases because of perceived
marginal individual and anonymous contributions
Group decision-making
[14] Awareness Decreased Manufacturing organization[15] Social loafing Increases Brainstorming[14, 16] Coordination costs Increase Manufacturing organization /
Visual constructs on larger displays[14] Coordination
mechanismsPreferred over collaboration mechanisms
Manufacturing organization
[16] Coordination strategy Moderation preferred over distributed work
Visual constructs on larger displays
[17] Usability Decreases because of problems related with resource utilization, group dynamics, social interactions, display resolution, and visualization
Interactions with Tabletop shared-display Groupware
[18] Perceived utility Increases Access to experts[19] Production blocking Decreased in both, face to face
and computer based support meetings
Face to face large oral, and computer based meetings
[19] Evaluation apprehension
Decreased in both, face to face and computer based support meetings
Face to face large oral, and computer based meetings
[19] Participation Decreased more in face to face meetings, compared with computer based meetings
Face to face large oral,and computer based meetings
[20] Generation of ideas Enhance individual novel ideas Individual brainstorming followed of access to the ideas of others
Fowler and Christakis [5] hypothesized that online technology helps mitigating the negative
effect of group size on collaboration because of increased behavioural cascading, which
magnifies particular attitudes, e.g. being helpful and punishing wrongful interactions. Based
on simulation experiments, Siegel [6] found that participation rates tend to increase with
size for a particular group structure: the village (or clique), in which every participant
knows each other. However, Shaw [7] studied a highly related factor, communication, and
found out that communication was higher for small groups than larger groups. This study
was conducted in a specific context: Computer-Supported Collaborative Learning (CSCL).
Schenk [8] reports that group size is important to achieve critical mass: large collaborative
tasks may fail if an insufficient number of individuals cannot be guaranteed to contribute to
the task. This effect was found in the context of crowdsourcing, where the participants
contribute to a common goal, often as volunteers, but working in silos. In some cases the
participants are not even aware of each other because some central entity coordinates the
production of results [21].
Goodchild [9] notes that, as the group size increases, the quality of outcomes may decrease.
It has been suggested that such loss in quality may be attributed to coordination problems
[10].
Early studies on electronic brainstorming found out that large groups would get better
results than smaller groups because they would be less affected by production blocking
[11]. Subsequent research found out that electronic brainstorming is actually affected by a
combination of positive and negative factors, but as group size increases the positive factors
become more significant [12].
Still in the context of electronic brainstorming, synergy was also identified as a factor
positively influenced by large decision-making groups [12].
Beyond the heavily studied electronic brainstorming niche, it has been reported that large
electronic groups offer lower motivation to complete a task, because technology factors like
anonymity tend to exacerbate the perception of marginal individual contributions [13].
Team size is considered the main explanation for the social loafing phenomena consistently
observed in electronic groups [15]. Furthermore, it has been observed that smaller groups
have more awareness about the team members [14].
It has been suggested that Steiner’s productivity model [22], in which productivity
decreases with group size because of coordination costs, also applies to computer supported
collaborative work [14, 16]. Sando et al. [16] report some experiments where large groups
tended to rely on a central person to perform the task with others just providing advice, as a
strategy to reduce coordination costs. Bradner [14] observed that large teams prefer
coordination to collaboration technology, while smaller teams show a preference for the
opposite.
Ryall [17] suggested that large group interactions may also be affected by usability
problems. These problems may be related with resource utilization, group dynamics, social
interactions, display resolution, and visualization.
White et al. [18] made an experiment with a synchronous Q&A system where the users’
perception of group size was investigated. They explicitly asked about the perceived system
utility. The obtained results indicate that utility increases with group size. The authors
suggest this increase may be related with having more experts to answer questions.
According to [19]{Lindblom, 2014 #29} , many factors influence the outcome of meetings,
as por example, technology, task type, individual characteristics of the participants, and size
of groups. Specifically, they found that production blocking and evaluation apprehension
are lower with “very” large groups exchanging oral comments both in a face to face (37
and 38 participants), and a computer based meetings (33 and 35 participants). Although,
participation with comments were lower in face to face meetings. Participation was
measured by the total number of comments, the number of relevant comments, and the
number of words per comment.
In [20], it is explained that a number of studies on electronic brainstorming have found that
large electronic groups can facilitate the number of ideas generated relative to control
groups of similar numbers of solitary performers. Thus far there is no clear evidence for the
basis of this facilitative effect. The most likely explanation is that group members benefit
from exposure to the wide range of ideas in large groups. Therefore the authors, designed to
assess the role of number of ideas and number of folders on individual idea generation and
to eliminate some alternative interpretations for the group size effect. The results indicated
that only the number of ideas factor was important for facilitating idea generation.
The table above highlights two gaps in the existing body of knowledge. In particular, we
note that information overload has not been studied. Information overload, i.e. the
perceived inability to process information in an efficient and timely manner, can be
overemphasized by digital media [23] and more so in a collaborative context [24]. Team
awareness in large groups has only been alluded to in the extant literature [14], and it has
generally been restricted to group awareness, i.e. the perception of who belongs to the team
members. However, the phenomena of awareness can addresses richer information such as
context awareness and activity awareness [25-27], for which we do not know much about
in a large group context.
3. Research Approach
This research adopts an interpretive case study approach. According to Klein and Myers
[28], the interpretive approach assumes that knowledge of reality is gained through the
analysis of meanings that people assign to a phenomena, through social constructions and
interaction with people, tools and other artefacts. We find this research lens particularly
adequate to an exploratory study of large collaborating groups: large groups may have
different views of what is at stake, may engage with the group in multiple ways and
different contexts, and may use time, space and tools in very different ways. Of course
these arguments do not deny the value brought by quantitative research, but since
knowledge about large group collaborations is relatively scarce and the difficulties
associated with experimental research with very large groups and time and space
dispersion, gathering qualitative data can be a good strategy to explore the area and develop
posterior quantitative studies. Next we provide additional details about the adopted
approach.
3.1. Conceptual Framework and Research Questions
Commensurate with the adopted qualitative approach, we define “large group” as a concept
referring to tenths or maybe hundredths of participants. This expresses the conceptual
distinction between a “small group”, having less than a tenth, and a “very large group”,
which may have thousands or more participants. We prefer not operationalizing this
concept as a variable, since we are not considering measuring the effects of varying group
size, which would require using interval or absolute scales. Still, the “large group” concept
supports a clear operational definition of the concept being studied, can be easily
operationalized in an interpretive case study, and no less important, it can be easily
explained to the case participants.
We hypothesise that a large group may affect the way groups accomplish a task and
consider two main task constituents where the participants may perceive such effects:
discussion and consensus. This selection is not comprehensive but mainly pragmatic. In
particular, we are excluding the early stages of a decision process were the participants
typically gather information in a divergent way, or generate ideas like in brainstorming
sessions. Our intention is to delimit the study to the stages where groups must start
consolidating information and converging towards a solution, which we perceive as
potentially being more affected by a large group constituency. Furthermore, the research
literature on group decision-making also notes that convergence is usually more difficult to
achieve than divergence, manly because the participants must go beyond information
sharing and communication towards other cognitive phenomena such as persuasion and
conflict management. The adopted qualitative approach seems particularly adequate to such
a complex scenario.
The influence of a large group on the task may be exerted along multiple factors. In our
study we consider four factors: information overload, awareness, task involvement, and
collaboration support. We define information overload as the participants’ perceived
inability to handle the flows of information generated by the group. Information overload is
a natural candidate for this study. Since group decisions necessarily involve information
exchange, not only with the purpose to share knowledge and ideas but also to coordinate
individual contributions and to build consensus, we expect that a large functional group will
generate more information and hypothesise that the group participants may feel overloaded
with so much information. We are excluding dysfunctional groups from the study, where
the participants may not collaborate, may engage in conflicting behaviours, or may not be
able to accomplish the assigned tasks.
We define awareness as the perception of the group contributions to the task, including
awareness of who belongs to the group, what contributions to the task are submitted by the
group members, what messages supporting discussion and consensus are shared by the
group, and also awareness of the group dynamics as the task enfolds. We selected
awareness as a factor to consider in the study because research on collaboration support has
been reporting its impact on group behaviour. Especially in remote collaboration, where the
communication technology may accommodate fewer cues about who is participating in the
group and what they are doing, building collaboration awareness may be challenging. For
obvious logistic reasons, we expect large group collaborations to mainly rely on
communication technology and hypothesise that large groups may have significant issues
finding what, how and when contributions are brought to the group.
Task involvement refers to commitment to the tasks goals, perceived attachment to the
group, and willingness to contribute to the group. Prior research refers that large groups
tend to contribute less to the group, since the participants perceive their contributions to be
diluted and can downgrade their efforts while being unnoticed. We hypothesise that this
phenomenon will also be present in our study and so it is natural to include it in the
conceptual framework.
The last factor that we consider is collaboration support. In our research context, we define
it as the perceived contribution of the several tools used by the group when accomplishing
the task. Often collaboration support is operationalized as a variable measuring different
levels of support, for instance using an ordinal scale with two categories, lack of support
and tool support. However, as with the other considered factors, we explicitly avoided
using variables and scales, since the study is not focussed on measuring the impact of
different values assigned to the task outcomes. Instead, we seek to obtain qualitative
insights about the tools adopted by the group and their perceived impact on the group
process.
Of course more factors could have been considered for this study. Other possible factors,
which have already been mentioned in the literature review, would be synergy, production
blocking, motivation, social loafing, coordination costs, etc. Many qualitative studies adopt
a grounded approach and let researchers find evidence in the rich data acquired from the
study participants. This is usually possible because the process of inquiry is open to
surprises and flexible enough to capture different types of insights, for instance using the
interview as a data collection instrument. But this is usually possible because the study
designs tend to trade off data richness with a small number of contacts. In our case, the
number of contacts suggests a research design with less information richness but capturing
insights from a larger number of contacts, for instance using the survey as a data collection
instrument. This explains why only four factors have been considered in the conceptual
framework (Figure 1).
The research questions are directly derived from the elements discussed above. We
consider one initial research question about the relationship between large groups and the
group task:
RQ1: What was the perceived impact of group size on the task?
Besides establishing the relationship between the two main concepts under analysis, large
groups and group task, we will also use this question to validate the applicability of the
conceptual framework: If the gathered data does not reveal the participants’ perceptions
about information overload, awareness, task involvement, and collaboration support, then
the framework should be rejected for being inadequate to guide data gathering process.
The following questions are directly related with the various framework elements
previously described. Regarding the tasks constituents, we have:
RQ2 (discussion): How the participants discussed ideas?
RQ3 (consensus): How the participants reached consensus?
Regarding the factors affecting the group tasks, we have:
RQ4 (information overload): The participants felt information overload during the task?
RQ5 (awareness): The participants were aware of the group members’ contributions to the
task?
RQ6: (task involvement): The participants felt involved in the task?
RQ7 (collaboration support): What was the perceived contribution of collaboration support
to the task?
Figure 1. Conceptual framework
3.2 Data Collection Instrument
The adopted instrument is the survey of participants based on a questionnaire administered
electronically two days after the group has completed the task. Considering the adopted
interpretive approach, all selected questions were open-ended. The questions were
formulated to address the research questions in a contextualised way and using simple
language. The list of questions was then randomised:
Q1: Did you feel information overload during the task?
Q2: What was the impact of group size on the task?
Q3: Could you perceive the others’ comments?
Q4: Did you feel involved in the task?
Q5: What do you think about the collaboration support?
Q6: What did you do to discuss ideas?
Q7: How did you reach consensus?
To the list above, we added some more specific questions with the purpose to capture more
detailed information from the participants:
Q3.1: How were you aware of the others’ work?
Q5.1: What did you do to collaborate?
Q6.1: What ideas have you discussed most?
Q7.1: How did you select the best ideas?
The data collected from this questionnaire was complemented with snapshots of the data
shared by the participants using the adopted software tools. These snapshots allow
corroborating the participants’ comments about collaboration support (Q5) and the group
task (Q6 and Q7).
3.3 Sample
The study involved students from an undergraduate course in information systems divided
in two samples. Both samples were subject to a standardized set of research conditions. The
same task was assigned to both groups. There were no shared members between groups.
The samples had approximately the same group size, gender composition and age: One
group had 48 students (26 male; average age was 22.8); another group had 44 students (25
male, average age was 22.3); and another group had 42 students (XXXXXX). The students
were taking the same undergraduate course at the same faculty. All participants were
proficient with computers and knowledgeable about using collaborative tools, and therefore
the question regarding collaboration support was not ambiguous to them. These conditions
were checked with pre-test questionnaires.
However, the samples were different in one particular contextual factor: the primary
collaboration tool that was used. We decided to recommend each group to adopt different
tools as the primary collaboration mechanism. Our objective was not studying tool use as
an independent variable, or even comparing which tool performs better. Instead, the goal is
to increase data richness through diversity while avoiding analytical bias caused by
anchoring the study on one single collaborative tool.
3.4. Group Task
The research literature suggests that the type of task should be made explicit in studies
involving group decision making, since it can be considered an intervening factor [29]. The
McGrath’s typology of tasks [30] defines four main types of tasks, generate, choose,
negotiate, and execute. Within the choose type, McGrath makes the further distinction
between tasks with correct answers and tasks with no right answer. For this study we
selected the choose task with no right answer. The main reason is that the lack of a correct
answer stimulates discussion, while choosing also requires the participants to reach
consensus. This ensures that both group task constituents defined in the conceptual
framework are addressed by the study.
The task was set up as a combination of remote asynchronous activities and face-to-face
synchronous activities, the former done at home or on the move, and the latter done in
assembly rooms. This combination aims to avoid biases caused by restricting the study to a
synchronous or asynchronous collaboration mode.
We arbitrarily specified that the task should be completed in a period of one week. The
main reason was giving the participants sufficient time to discuss the task, to settle rules
and procedures, and to develop strategies for converging towards a group choice.
The assigned task was to collaboratively identify community problems in a delimited urban
area, proposing innovative solutions based on information technology and assigning
priorities. The task was not restricted to any particular type of ideas. Examples were given
to the groups before the study, when the groups were instructed about the task. Given
examples included improving local businesses, community services, and improving social
interaction and responsibility.
The task instructions specified minimum levels of participation. Each student should
suggest two innovative solutions as a minimum requirement for participating in the group.
Group participation was also a minimum requirement for being approved in a course
assignment.
The students were encouraged to suggest problems and solutions, and to discuss and give
their opinions about the others’ suggestions and comments. The groups were also instructed
to generate a consensus list with the ten best solutions. The specific task instructions
indicated:
“1) you must work on a specific urban area of Santiago; 2) you must use tool [A / B] in collaborative
mode to share and select problems and ideas; 3) the problems and ideas may be commented and
supplemented with text, sketches, and photos, as much as necessary to emphasize their importance; 4)
the whole work must be accomplished collaboratively; 5) you may take a picture of the place or
context where you identify a problem; 6) the list of the 10 best ideas must be accepted by consensus;
and 7) you have one week to accomplish the task.”
No instructions were given about what type of technology to use (e.g. mobile phones of
laptops). Neither a certain type of convergence mechanism for selecting the best ideas was
indicated or recommended. The definition of coordination and consensus mechanisms had
to be resolved by the groups.
3.5 Data Analysis
To analyse the collected data, we adopted the traditional qualitative coding process [4].
This is a three-stage process where, initially, commonalities in the data are captured in
“descriptive” codes to clearer capture the essential attributes of the phenomenon. Next, as
more data and codes are available, “interpretive” codes are abstracted from the idiographic
confines of the concrete incidents to help understand what is going on “behind” the data.
And lastly, inferential “pattern” codes, now abstract of space and time and etic to the
substantive range of the research, are conceptualized; they are explanatory and often
predictive. Since the study has an exploratory nature, only the two first stages of qualitative
coding were done. The Nvivo tool supported the coding process.
4. Results
4.1 Initial comments
In all groups, the activities started with a face-to-face meeting. The participants relied on
these meetings to discuss the given instructions and devise collaboration strategies.
After the initial meeting, the participants started to asynchronously work on the task,
walking around the designated city area in their spare times, identifying problems and
opportunities, while sharing ideas and comments with the others, and jointly creating the
required list of priorities. Most participants took notes and pictures while on the move using
mobile phones.
During the time assigned to accomplish the task, the groups organized sessions where they
all met face-to-face again. During these sessions, they redefined the collaboration strategies
devised on the initial meeting, changed work conventions, and made several changes on the
way they used the collaboration support tools (more on that later). In particular, the
convergence procedures necessary to successfully complete the task were discussed and
settled face-to-face.
All 134 students that participated in the study answered the questionnaire. The open
questions generated a corpus with about 56.022 words and 272.435 characters (without
spaces), which indicates that the students carefully considered the questions. In the
following, we present the results according to the research questions.
4.2 Perceived impact of group size
Overall, the major impact of group size perceived by the participants was related to
coordination problems, since 54 participants (about 40% of the cohort) mentioned them.
Within the list of coordination problems, the participants emphasized in particular the lack
of experience handling large groups and problems with time management. As some noted,
“the majority was accustomed to work in much smaller groups” and “never before I have
faced collective work involving so many people”.
We also found evidence of impact of group size on the discussion and consensus
components of the task. The participants noted multiple problems discussing ideas, in
particular, confusing information brought in from many sources, lost information because
of deficient access control, difficulties communicating, lack of control over the decision
process, lack of moderation, lack of motivation mechanisms, and lack of rules. Regarding
reaching consensus, it was noted that consensus requires improved communication, is a
slow process, there were too many items to converge to, and there were initial difficulties
establishing the work process.
On a more positive note, the participants found multiple benefits of group size on task
involvement, noting in particular the good communication/interaction, ample
brainstorming, and especially improved quality through consensus. In particular, a large
number of participants (14) referred that the discussion was improved with critical
comments, new points of view, and not only more ideas but also better integration of
others’ comments in suggested ideas. As one participant explicitly noted, “the results are
much richer”. One participant noted the participants felt some social pressure towards
completing the task.
Some participants established the relationship between group size and awareness: The
decision process benefited from improved feedback and contributed to develop a holistic
view of the problem. In the words of one of them, “the opinions from the other participants
allowed improving the proposals made by each one of us, using some sort of feedback”.
However, a negative impact was found on information overload. The participants noted the
diminishing returns, caused by repetition and information losses, having to process too
many contributions, difficulties following all contributions, and also difficulties following
rules of conduct. More than one participant mentioned each one of these contributors to
information overload. One of them noted in particular that “it becomes more complex to
maintain a certain order and conventions about how to do things, a moderator is necessary
with more capacity to bring cohesion and unity to the work”.
Few impacts on collaboration support were identified by the participants, which were
divided between the positive and negative sides. On the positive side, the participants
referred to having a faster decision process and better information sharing. On the negative
side, they referred to difficulties managing the collaboration tools.
Considering that this question was fundamentally asked to check if the participants would
spontaneously bring forward the set of factors we considered as potentially affecting the
group task, we were content that all elements specified in the conceptual framework were
raised. The main surprise was the significant importance given by the participants to
coordination, a factor that is subdued under collaboration support in the conceptual
framework. Future research should definitely promote this factor and make it explicit in the
framework. In Table 2 we provide a quantitative summary of the responses related to the
perceived impact of group size on the task.
Table 2. Perceived impact of group size.
Benefits Drawbacks
Coordination 54
Discussion 37
Task involvement 19 1
Information overload 2 15
Collaboration support 4 1
Awareness 3
4.3 Information overload
Our inquiry about the participants’ perception of information overload raised four
considerations. The first one is that the participants had multiple problems processing
information, a category that was mentioned by 36 participants. They found it hard to
analyse so much information in detail, commented about having stress while receiving so
many comments, the time spent reading them, missing new comments, forgetting
information, dealing with repetition, and losing interest on the task. One of the participants
noted, “the owner of an idea collapsed with so much information and comments and would
forget to add all suggestions in [her/his] own idea”. Other noted there was “too much
information, very unclear, and unstructured”. And another made an interesting comment
that the group was “complexifying the problem”.
Our second consideration concerns the group interaction. The participants raised several
interaction problems, such as the dependence on others’ feedback, dealing with peak
activity times, and dealing with coordination issues. One participant noted “the ideas had to
receive feedback with other opinions, since this was the only way to really measure the real
impact of the idea and its social acceptance”.
It was particularly interesting to find out that the participants adopted several strategies to
avoid information overload. Some of the mentioned strategies were just scanning the first
words of an idea or comment, using alternative communication channels (e.g. chat and
mail) to avoid saturating the primary one, and relying on an emergent leader to coordinate
the contributions.
Still, the participants perceived several positive factors from large group collaborations. It
was noted the process had democratic value, the communication was immediate, and there
was value in receiving diverse opinions and explanations, as well as imagination.
4.4. Awareness
Feedback was mainly perceived as constructive. Of the twenty participants pointing out
constructive feedback, nine explicitly referred that feedback promoted quality through
“truthful comments”, which either “helped completing ideas” or “raised valid
interrogations”, and five referred that feedback promoted collaboration.
However, the participants also noted they often could not follow all feedback. In particular,
two participants said, “it was difficult to follow the order of the discussion”, and “although
it was possible to follow all comments, there was too much information, which did not
favour total attention to all comments”.
Some participants also complained about the quality of feedback. Information in messages
was often repeated, noisy and without significant contents. As one participant commented,
“there were many comments made with the sole objective to be present, that, essentially
saying ‘I like it’, and which would did not contribute to the discussion”. On the other hand,
repetition was also perceived as contributing to reinforce ideas, which allowed the
participants to naturally converge towards certain ideas.
4.5 Task involvement
The participants reported increased task involvement caused by increased participation and
fluid collaboration. The increased participation was mainly caused by the process of
elaborating ones’ and others’ contributions, and also interest in the others’ propositions. As
mentioned by one participant, “many ideas got my interest and I felt that I wanted to be
involved more in them”. Interestingly, three participants noted they felt peer pressure
towards contributing to the task: “the constant participation of my colleagues and the
actualisation of ideas generated, in a certain way, a pressure to keep a proposal up to date to
the colleagues’ opinions”.
The perceived fluid collaboration was mainly caused by the constant revising of
contributions. As one participant noted, “the communication becomes fluid, since everyone
sees what you are doing and you feel well regarded by the group”.
Some participants reported they felt special responsibility for the shared task and therefore
assumed more central roles, such as facilitating the discussion or configuring tools for
specific goals. Though many participants indicated they did not like the format of their
participation: they lacked time to process information, noted the lack of commitment of
some colleagues, lacked feedback to some of their ideas, and the feedback was indirect.
Some participants explicitly indicated that having others deleting their contributions
reduced their commitment to the task. Finally, some participants explained their marginal
participation to the task completion as caused by the large number of participants. As one
participant said, “one feels aside when our idea is moved to the end”.
4.6 Collaboration support
The participants perceived the hybrid collaborative approach as better than face-to-face
meetings, mainly because of better access to shared information and better time/space
management when off mettings. However, the initially suggested collaboration support
tools were considered inadequate for collaboration. The participants complained especially
about the lack of control over individual contributions, which could be freely deleted by
any participant. Other negative comments hint that the tools made it difficult to associate
comments to their authors, did not log data modifications, did not support the rules agreed
on the first meeting, and were not democratic. In both groups, some time after starting to
work on the task, the participants decided to have another face-to-face meeting where they
discussed the collaboration support and eventually decided to adopt other tools and new
participation rules. An example of a rule that was settled in one these meetings was
structuring the participants’ contributions in two different categories: ideas and comments.
Another example was the adopted principle of always respecting the person taking the
initiative for proposing an idea.
4.7 Group task: Discussion and consensus
The participants ended up adopting multiple channels and tools to discuss ideas, which
included opportunistic encounters, face-to-face group meetings, one-to-one phone calls, and
also using chat tools for multi party discussions.
The face-to-face meetings were primarily focussed on defining the discussion rules,
including how to behave in the group and what tools to use and how to use them.
The participants structured the discussion by separating ideas from comments and agreed
on having a primary communication channel for sharing them, while using other channels
for chatting about the ideas and clarifying the others’ views. The main reason for separating
the discussion in ideas and comments was that it allowed easily deciding in favour or
against an idea, and it allowed to focus the discussion on the original idea.
The participants adopted multiple data reduction strategies, such as only focussing on the
most interesting or the most important ideas (those with more comments), and constraining
the discussion to pros/cons and to agreements/disagreements. One participant referred that
“I’ve gone to the extremes, commenting all ideas that seemed to me as either very good or
very bad”.
The participants also adopted several strategic attitudes towards preserving the discussion,
which included “being constructive by only submitting relevant contributions”, focussing
on rational criteria, identifying flaws in ideas and commenting on them, and showing
mutual respect and avoiding personal attacks.
Regarding consensus building, both groups initially adopted the voting strategy to converge
on the ten best ideas. However, this strategy was later on perceived as erroneous because,
even though it was fast and could be automated with software tools, it did not allow
justifying the selections. One participant referred that “consensus should go beyond
voting”. The consensus strategy that was lately adopted by one group consisted in ordering
the list of ideas according to the number of received comments, and relying on a moderator
to maintain the list (using a colour scheme). The other group adopted a similar strategy that
consisted in prioritising ideas according to comments in favour or against made by all
participants. In both groups, the later strategies were perceived as much more effective, as
they allowed elaborating ideas and justifying the participants’ preferences.
6. Discussion
The collected data provides some interesting insights on how the participants perceived the
task in the context of large group collaboration. In particular, the participants seemed well
aware of the trade-offs brought by information overload, which in the one hand bring too
much information to process but, in the other hand, bring diverse opinions, imagination and
also immediacy to the group interaction. Considering these trade-offs, the groups were able
to develop successful strategies to avoid information overload, which emphasised scanning
information for relevant contributions and avoiding saturating the primary communication
channel with secondary interactions. It is interesting to note that the strategies operate at
both channel ends, at the origin by refraining from sending data and at the destination by
aggressively filtering out irrelevant data. Although having a leader/facilitator to coordinate
contributions may also reduce information overflow, from the collected data it seems that
the leader/facilitator was mainly responsible for implementing the collaboration rules that
were decided by the group in the face-to-face meeting, and not to constrain the participants’
contributions.
The collected data also indicates that the participants perceived the trade-offs between
having too much or too few feedback, and promoted some balance between the two
extremes. For instance, repetition contributed to increase awareness about the participants’
preferences for certain ideas. However, the participants also complained that the quality of
feedback could be improved, notably by decreasing the number of repeated comments and
ideas. All in all, there seems to be a significant relationship between information overflow
and awareness, although information overflow appears as a negative factor and awareness
appears as a positive factor affecting the task.
Task involvement and awareness also seem to be highly related. The participants noted that
increased task involvement was caused by increased participation and fluid collaboration,
which in turn are promoted by awareness information regarding what ideas the participants
prefer and their opinions about them. This emphasis on collaboration fluidity is quite
interesting because once again it suggests that the different factors that were investigated,
information overload, awareness and task involvement, affect each other and indeed have to
be carefully balanced by the participants.
The gathered data provide some insights about the checks and balances done by the groups
to develop collaboration fluidity. An interesting one was the special responsibility felt by
some participants, and recognised by the others, to the shared task. This lead them to work
for the benefit of the others, e.g. reconfiguring the collaboration tools to better serve the
group or reminding them about the deadlines. Though it did not lead them to lead the group
or to monopolise the discussion. As one participant noted, “it was not my intention, but
found myself in the role of task coordinator, and therefore 100% involved in the
discussion”.
Another contributor to task involvement, which has been already mentioned, was the
overall constructive attitude. The collected data provide some glimpses on how that attitude
was constructed. One aspect was the perceived peer pressure towards contributing to the
task, even though some of them perceived that their contributions were marginal. One
participant noted, “part of being involved was communicating and collaborating with the
others”. Another noted, “being able to implement my own idea and having rapid feedback
creates a felling of involvement and commitment to the idea”. Though it was also
interesting to note that the participants were committed to change their ideas based on the
received feedback: “after uploading my idea, I received many comments, which helped me
to see things that I had not though before, or which I did not thought were relevant, and
because of that feedback cycle my idea took more sense”.
Analysing the collected data on collaboration support, the most interesting observation is
the flexibility the groups revealed adopting new collaboration strategies and tools. Though
the decisions had to be made face-to-face.
Regarding the group task, we note that the discussion of ideas involved multiple data
reduction strategies, which in particular constrained the discussion to focus on the most
important ideas and to divide the contributions to binomial categories, e.g. in favour and
against, or agree and disagree. The consensus processes were interesting because of the
need to change the initial strategies. The groups started the task with the perspective that
voting was the best strategy. Actually, both groups implemented different voting
procedures, one by voting on the best 3 ideas and the other by prioritising the ideas with
more positions in favour. However, after going through the process, the groups decided to
require the participants to justify their positions. As one participant noted, “majority voting
is not very good if you have a real project but yes, it is very efficient”. We coded comments
about the change in the convergence strategy from 24 participants, and the data suggests
that the decisions were made considering that relevance and feasibility could not be
assessed with the initial procedure.
Looking more generally to the conceptual framework, we emphasise again the strong
interrelationships and delicate balance between information overload, awareness and task
involvement. It seems that such balance may be easily disrupted by small changes in the
task or the technology. Perhaps another factor could be identified to unify these three
factors. The study hints that collaboration fluidity could be such a factor. Furthermore, the
relationships between these three factors and the task also seem complex. In particular, it
seems that the type of task affects the participants’ attitudes and behaviours that are
strongly related with information overload, awareness and task involvement. Some
contributors like the perception of value brought by diverse opinions, attitudes towards
preserving the discussion, and responsibility for the shared task seem to depend on the task
characteristics, which emphasises discussion and consensus building. But on the other
hand, we could not establish clear relationships between the studied factors and the two
task components, discussion and consensus, since the factors seem to indifferently apply to
them.
Regarding collaboration support, what stood out from the case was the groups’ flexibility
adopting different tools and the dependence on face-to-face meetings to make significant
strategic decisions.
Another issue that could be raised is the relationship between the task and time. We gave
one week to accomplish the task, which gave them plenty of time to experiment different
collaboration strategies, to fail, and finally to adopt winning solutions. Besides, this
extended period of time allowed some participants to emerge as group facilitators. It is
unclear if in a shorter period of time the groups would be willing to adapt, or instead would
go with the first strategy, which was inadequate.
This study contributes to understand various trade-offs that large collaborating groups face,
some of them with implications to collaboration support. Information overload is as much
as a threat as an opportunity, so the challenge is to find the right balance. Information has to
be organised in a way that keeps the participants constantly involved, aware and focussed
on the most important pieces of information. Updates have to be carefully done to avoid
stress and to preserve context and respect for the others’ opinions. For instance, a constant
flow of new comments keeps the discussion in pace and increases peer pressure. However,
often the participants allude to lacking time to process information or, the other side of the
coin, feeling aside by the lack of comments.
The participants’ understanding of awareness seems to be more global than local. For
instance, repetition can be globally understood as consensus and locally understood as
noise. The participants use multiple communication channels with different goals, often
with the intention to avoid cluttering the primary communication channel.
Collaboration support should allow groups to flexibly define how information is aggregated
and prioritised, and possibly should allow experimenting different arrangements. The
technology should foster refining the initial contributions based on incremental feedback.
Often the most obvious or efficient collaboration processes are not the ones preferred by the
participants, which seem to favour other criteria such as democratic participation. Though
the biggest challenge is that it seems that strategic changes still have to be negotiated face-
to-face.
7. Conclusions
.
References
[1] P. Antunes, V. Herskovic, S. Ochoa, and J. Pino, "Structuring Dimensions for
Collaborative Systems Evaluation," ACM Computing Surveys, vol. 44, 2012.
[2] G. Mark, "Large-scale distributed collaboration: Tension in a new interaction order," in
Computerization movements and technology diffusion: From Mainframes to
Ubiquitous Computing, M. Elliot and K. Kraemer, Eds., ed Medford, NJ: Information
Today, Inc., 2008, pp. 289-310.
[3] M. Bernstein, J. Brandt, R. Miller, and D. Karger, "Crowds in Two Seconds: Enabling
Realtime Crowd-Powered Interfaces," in Proceedings of the 24th annual ACM
symposium on User interface software and technology, Santa Barbara, California,
USA, 2011.
[4] M. Miles and A. Huberman, Qualitative Data Analysis. Thousand Oaks, California:
SAGE, 1994.
[5] J. Fowler and N. Christakis, "Cooperative behavior cascades in human social
networks," Proceedings of the National Academy of Sciences, vol. 107, pp. 5334-5338,
2010.
[6] D. Siegel, "Social Networks and Collective Action," American Journal of Political
Science, vol. 53, pp. 122-138, 2009.
[7] R. Shaw, "The relationships among group size, participation, and performance of
programming language learning supported with online forums," Computers &
Education, vol. 62, pp. 196–207, 2013.
[8] E. Schenk, "consultezTowards a characterization of crowdsourcing practices," Journal
of Innovation Economics, vol. 1, pp. 93-107, 2011.
[9] M. Goodchild and A. Glennon, "Crowdsourcing geographic information for disaster
response: a research frontier," International Journal of Digital Earth, vol. 3, pp. 231-
241, 2010.
[10]A. Kittur and R. Kraut, "Harnessing the wisdom of crowds in wikipedia: quality
through coordination," in Proceedings of the 2008 ACM conference on Computer
supported cooperative work, San Diego, CA, USA, 2008, pp. 37-46.
[11]R. Gallupe, A. Dennis, W. Cooper, J. Valacich, L. Bastianutti, and J. Nunamaker,
"Electronic Brainstorming and Group Size," Academy of Management Journal, vol. 35,
pp. 350-369, 1992.
[12]A. Dennis and M. Williams, "A meta-analysis of group side effects in electronic
brainstorming: More heads are better than one," International Journal of e-
Collaboration, vol. 1, pp. 24-42, 2007.
[13]L. Chidambaram and L. Tung, "Is Out of Sight, Out of Mind? An Empirical Study of
Social Loafing in Technology-Supported Groups," Information Systems Research, vol.
16, pp. 149-168, 2005.
[14]E. Bradner, "Team size and technology fit: participation, Awareness, and rapport in
distributed teams," IEEE Transactions on Professional Communication, vol. 48, pp.
68-77, 2005.
[15]O. Alnuaimi, L. Robert, and L. Maruping, "Team Size, Dispersion, and Social Loafing
in Technology-Supported Teams:
A Perspective on the Theory of
Moral Disengagement," Journal of Management Information Systems, vol. 27, pp. 203-230,
2010.
[16]T. Sando, M. Tory, and P. Irani, "Impact of Group Size on Spatial Structure
Understanding Tasks," in IEEE Pacific Visualisation Symposium, Hong Kong, China,
2011.
[17]K. Ryall, C. Forlines, and C. Shen, "Exploring the Effects of Group Size and Table
Size on Interactions with Tabletop Shared-Display Groupware," in Proceedings of the
2004 ACM conference on Computer supported cooperative work, Chicago, Illinois,
USA, 2004, pp. 284-293.
[18]R. White, M. Richardson, and Y. Liu, "Effects of community size and contact rate in
synchronous social Q&A," in Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems, Vancouver, BC, Canada, 2011.
[19]T. Lindblom, M. Aiken, and M. Vanjani, "Electronic Facilitation of Large Meetings,"
Communications of the IIMA, vol. 9, p. 3, 2014.
[20]P. B. Paulus, N. W. Kohn, L. E. Arditti, and R. M. Korde, "Understanding the group
size effect in electronic brainstorming," Small Group Research, p. 1046496413479674,
2013.
[21]D. Brabham, "Crowdsourcing as a Model for Problem Solving," The International
Journal of Research into New Media Technologies, vol. 14, pp. 75–90, 2008.
[22]I. Steiner, Group Process and Productivity: Academic Press, 1972.
[23]D. Bawden and L. Robinson, "The dark side of information: overload, anxiety and
other paradoxes and pathologies," Journal of Information Science,, vol. 35, pp. 180-
191, 2009.
[24]A. Ferreira and P. Antunes "Tackling information overload in electronic
brainstorming," in Proceedings of the 2008 Conference on Group Decision and
Negotiation Coimbra, Portugal, 2008.
[25]C. Gutwin and S. Greenberg, "A Descriptive Framework of Workspace Awareness for
Real-Time Groupware," Computer Supported Cooperative Work, vol. 11, pp. 411-446,
2002.
[26]J. Bardram and T. Hansen, "Context-Based Workplace Awareness," Computer
Supported Cooperative Work, vol. 19, pp. 105-138, 2010.
[27]J. Carroll, M. Rosson, G. Convertino, and C. Ganoe, "Awareness and teamwork in
computer-supported collaborations," Interacting with Computers, vol. 18, pp. 21-46,
2006.
[28]H. Klein and M. Myers, "A Set of Principles for Conducting and Evaluating
Interpretive Field Studies in Information Systems," MIS Quarterly, vol. 23, pp. 67-93,
1999.
[29]J. Fjermestad and S. Hiltz, "An assessment of group support systems experimental
research: Methodology and results," Journal of Management Information Systems, vol.
15, pp. 7-149, 1999.
[30]J. McGrath, Groups: Interaction and performance. Englewood Cliffs, NJ: Prentice-
Hall, 1984.
Top Related