Exploring Connectivist Massively Open Online Course (cMOOC) Microblogging Data through a Student Assessment Lens

Laura Gogia

Abstract

In 2014 six Virginia Commonwealth University (VCU) faculty designed and

implemented an eight week-long summer pilot course called Digital Engagement:

Living the Dreams, Digital Investigations and Unfettered Minds, or UNIV 200:

Inquiry and the Craft of Argument. While most of the course took place on open

blogging platforms, an associated Twitter community emerged around the hashtag

#Thoughtvectors. Intentionally initiated by course instructors, #Thoughtvectors

Twitter activity was captured in a Twitter Archiving Google Spreadsheet (TAGS) and

informally shared with all participants through the course website. This study explores

the TAGS data through the lens of student assessment. It finds that while it is possible

to create dashboards for student assessment from TAGS data, optimal use might

require adapting archiving spreadsheets to capture data specific to formalized

connected learning objectives and activities.

Study Purpose

This study was an initial step in a research agenda that aims to advance understanding

of how to best document connected learning in formal higher education settings.

Research questions included:

• What data are captured by Twitter Archiving Google Spreadsheets (TAGS)?

• How might data collected in this format be used to describe the microblogging

behaviors of learning communities, their subgroups, and individual participants?

• Is it possible to develop a dashboard for student assessment from data captured in

Twitter Archiving Google Spreadsheets? If so, what are its limitations?

Methods

A Twitter Archiving Google Spreadsheet (TAGS; Hawksey, 2013) was used to capture

#Thoughtvectors activity from the Twitter Application Programming Interface (API).

The spreadsheet was monitored and maintained from the first use of the

#Thoughtvectors hashtag until one month after the formal course (UNIV 200) had

ended.

The data captured in TAGS were transferred to Microsoft Excel where duplicates and

errors were removed. Participants were sorted into four subgroups for more detailed

analysis. A comprehensive quantitative content analysis of the captured data was

performed to extract information related to mentions, retweets, and links. Descriptive

statistics were generated through Microsoft Excel and social network analysis

performed with the addition of NodeXL (Social Media Research Foundation, 2014).

4. Others480 other individuals

tweeted or were

mentioned along with the

#Thoughtvectors hashtag

3. Open Participants

35 non-students and

over a dozen VCU

staff formally engaged

in at least some UNIV

200 learning activities.

Of these, 28

participated in

#Thoughtvectors-

related Twitter

activity..

2. Instructors#Thoughtvectors was

facilitated by six

faculty who possessed

a total of eight Twitter

accounts.

1. Students95 VCU students

enrolled in UNIV 200;

of those, 68 were

identified as

participating in course-

related Twitter

activities. activity.

TheThoughtvectors

Community

What’s Possible with TAGS Data?

12%

31%

32%

25%

Who's Tweeting?

Students Instructors Open Participants Others

24%44%

14%

73%

40%

76%56%

86%

27%

60%

Are Tweets Retweets?Yes No

42%

12% 13% 4% 12%

18%

17%27% 39% 26%

28%

33%

34% 22% 31%

12%

37%26%

35% 30%

Who’s Being Mentioned?Students Instructors Open Participants Others

30%

57% 61% 62% 56%

70%

43% 39% 38% 44%

Do Tweets Include Links?

Yes No

Developing a Hypothetical Assessment StrategyConclusions

1. It is possible to use TAGS data to develop dashboards for the assessment of

student microblogging behaviors.

The data collected from TAGS are appropriate for descriptive statistics and data

visualizations consistent with formative and summative assessments of group and

individual student performance.

2. It is better to tailor data archiving spreadsheets to the pedagogical objectives of

the microblogging activity.

In this case researchers hoped to capture the incidence of student “connection-

making” because of the pedagogical importance placed on making connections

across space, time, and disciplines in connected learning environments.

Information regarding mentions, links, and retweets was available through TAGS,

but it had to be mined from text cells in a time-consuming content analysis. Such

an analysis would not be feasible in an authentic, real-time educational context of

similar size. Ideally archiving spreadsheets could be designed to mine much of this

information automatically.

3. If assessment strategies around connection-making are to be implemented,

instructors must send clear messages to their students regarding the usefulness

of microblogging as a connection-making tool.

Individuals in the #Thoughtvectors community tweeted in very different ways.

Heterogeneity in student use might have been related to the lack of consistent and

formalized pedagogical messaging around the use of Twitter for class-related

activity. If microblogging is to be seen as a powerful learning tool that affords (1)

connection of ideas across space, time, and disciplines; (2) strategic navigation

through distributed discourse; and (3) signal amplification; and if students are to

practice using it as such, explicit pedagogical messaging around these goals should

be in place before the students’ ability to perform such tasks are assessed.

Next Steps

• Gaining a better understanding of how students and faculty use social media tools in

connected learning environments.

• Building evidence to suggest certain digital practices or patterns of practice (e.g.

mentioning, linking) promote the sort of digital engagement, digital literacies, and

connection-making valued in connected learning environments.

• Developing archiving spreadsheets specific to the purpose of documenting student

learning in connected learning environments.

• Developing meaningful, scalable, and flexible assessment systems for connected

learning that may incorporate the data from digital artifact archiving spreadsheets

similar to or based on TAGS.

Student A

#Tweets: 85

#Mentions: 66

#Links: 11

#Retweets: 64

In-Degree Centrality: 5

Out-Degree Centrality: 17

Betweeness Centrality: 3110

Student B

#Tweets: 24

#Mentions: 5

#Links: 12

#Retweets: 6

In-Degree Centrality: 5

Out-Degree Centrality: 9

Betweeness Centrality: 1108

Student C

#Tweets: 0

#Mentions: 0

#Links:0

#Retweets: 0

In-Degree Centrality: 1

Out-Degree Centrality: 0

Betweeness Centrality: 0

Characterizing Individual Student Participation through TAGS Data Analysis

Student Twitter Activity

Whole Community

Twitter Activity