Students’ use of Web 2.0 tools in higher education:Good practice in assessment and academic integrity

An ALTC priority project 2009-2011

E Assessment Webinar26 May 2010

Project team

Jenny Waycott (project manager), Faculty of Medicine, Dentistry and Health Sciences, University of Melbourne.

Celia Thompson, School of Languages and Linguistics, University of Melbourne.

Margaret Hamilton, School of Computer Science and IT, RMIT University.

Joan Richardson, School of Business Information Technology, RMIT University.

Kathleen Gray (project leader), Faculty of Medicine / Department of Information Systems, University of Melbourne.

Rosemary Clerehan, Faculty of Medicine, Nursing and Health Sciences, Monash University.

Judithe Sheard, Faculty of Information Technology, Monash University.

Project background

The future of teaching & assessing “academic writing”

Chodorow

(2000, p.91)

• “the form and substance of scholarly communications will change over time, so that it will be difficult to trace the historical flow of the work”

• “a free-flowing stream of scholarly discourse will reduce the role of scholarly authority in the progress of research”

• “the roles of individual authors will be obscured in the electronic environment”

The future of teaching & assessing “academic writing”

O’Reilly & Battelle

(2009, p. 2)

“One of the fundamental ideas underlying Web 2.0 [is] that successful network applications are systems for harnessing collective intelligence ... a large group of people can create a collective work whose value far exceeds that provided by any of the individual participants”

The future of teaching & assessing “academic writing”

Kakutani

(2010,

paras 13-14)

“jump to the summary, the video clip, the sound bite — never mind if context and nuance are lost in the process; never mind if it’s our emotions, more than our sense of reason, that are engaged; never mind if statements haven’t been properly vetted and sourced”

“tweet and text one another during plays and movies, forming judgments before seeing the arc of the entire work”

“power-search for nuggets of information that might support their theses, saving them the time of wading through stacks of material that might prove marginal but that might have also prompted them to reconsider or refine their original thinking”

What do you think?

“The assessment of student web 2.0 authoring is [.......................] for learning and teaching in Australian universities”.

Project aims

A collegial approach to addressing the challenges of university assessment 2.0:

1. Survey and interview Australian teaching academics (September 2009)

2. Convene a national roundtable (November 2009)

3. Field-test good practice guidelines (February to June 2010)

4. Produce and share resources (July 2010 ff)

1. What goes on in assessment 2.0?

The subject teaching context

1. What discipline or professional degree/s are students enrolled in when they complete this unit of study?

2. At what level/s is this unit of study?

3. How many students were enrolled in this unit of study the last time it ran?

4. When did you first use this assignment in more or less its present form in this unit of study?

The subject teaching context

DisciplineNumber of

responses

Humanities/Society & Culture 16

Education 15

Information technology 11

Management and commerce 6

Health 5

The subject teaching context

Level of study Number of responses

Bachelor or honours degree 35

Masters degree 10

Graduate certificate or diploma 6

The subject teaching context

Number of students

enrolled in unit

Number of responses

Less than 50 21

50-100 10

101-200 9

More than 200 7

The subject teaching context

Year assignment began Number of responses

2009 19

2008 13

2007 12

2006 5

2005 or earlier 3

1. What goes on in assessment 2.0?

The mechanics of the assignment :

• What Web 2.0 activities do students do in this assignment?

• How long are students given to complete the assignment?

• How long do students take to complete this assignment?

• Where do students complete the assignment?

The mechanics of the assignment

Type of Web 2.0 activity Number of responses

wiki writing 32

blogging/microblogging 31

social networking 17

audio/video podcasting 16

virtual world activities 12

social bookmarking 11

The mechanics of the assignment

Time given to complete

assignment

Number of

responses

More than 1 month 42

1 month or less 14

1 week or less 2

1 day or less 1

The mechanics of the assignment

Estimated time to complete

assignment

Number of

responses

11-20 hours 21

01-10 hours 17

21-30 hours 10

31-40 hours 7

Less than 1 hour 3

More than 40 hours 1

The mechanics of the assignment

Where students complete assignment Number of

responses

Off campus elsewhere (e.g., at home during

independent study time)52

On campus but out of class 25

On campus in class 16

Off campus while undertaking fieldwork or

workplace learning7

1. What goes on in assessment 2.0?

The point of the assignment :

• What are the intended learning outcomes for this assignment?

• How much does this assignment contribute towards students’ overall mark for this unit of study?

• Is it compulsory for students to do this assignment to pass the subject/unit of study?

The point of the assignment

Intended learning outcomes Number of

responses

Generic or graduate skills or attributes 35

Specialised knowledge or skills required in a

discipline or profession29

Foundation knowledge or skills preparatory to

a discipline or profession28

The point of the assignment

How much the assignment is

worth

Number of responses

01-10% 7

11-20% 11

21-30% 9

31-40% 6

41-50% 9

51-60% 2

61-70% 0

71-80% 3

81-90% 2

91-100% 4

The point of the assignment

Is it compulsory for students

to do this assignment?

Number of

responses

Yes 49

No 13

1. What goes on in assessment 2.0?

The marking process:

• Who marks the assignment?

• What techniques are used to mark the assignment?

• What kind of feedback do students receive?

The marking process

Who marks the assignment? Number of

responses

Marked by one staff member 40

Marked by more than one staff member 17

Marked by other students 8

Self-marked by the student/s responsible 7

What techniques are used to mark the assignment? Number of

responses

Comments as well as marks provided 41

Rubric used 33

Marked in stages 22

Equal marks shared by everyone in a student group 15

Verification of identity of students submitting work 13

Plagiarism checking tools used (e.g., Turnitin) 12

Blind marking (i.e., student work is de-identified) 3

Automated analysis or grading of student work 2

What feedback do students receive? Number of

responses

Grades in the form of a number or letter 44

Confirmation

(confirmation that work is of acceptable standard)43

Explanation

(recommendations for bringing work up to standard)41

Correction

(flagging of specific shortcomings with student work)39

Elaboration

(supplementary information to extend understanding)25

Diagnosis

(analysis of what may have led to shortcomings or

misconceptions in student work)

22

2. What would good practice look like?

When university students are asked to demonstrate their learning using this form of web 2.0 authoring, what academic standards, and assessment and reporting practices are essential or desirable?

Proceedings of national roundtable:

http://web2assessmentroundtable.pbworks.com/

2. What would good practice look like?

Mashing up:

affordances of Web 2.0

stages in the assessment cycle

academic assessment policies

2. What would good practice look like?

Affordances:Ensuring an appropriate fit between what Web 2.0 activities

entail and what assessment is trying to achieve

• Open publishing

• Communication styles and texts

• Personal identity and experience

• Co-creation, collaboration, crowdsourcing

• Content management

Affordances

Open publishing:

• Student work can be made easily accessible to an audience of peers for mutual benefit including reviewing and rating.

• Review and assessment of student work from outside the university can be invited or anticipated.

Affordances

Communication styles & texts

• Web 2.0 assignments can involve frequent short pieces of work employing conversational language and combining audio, video, images & text.

• Feedback can be exchanged rapidly, using rating or ranking systems, informal rejoinders, audio, video, images, icons.

Affordances

Personal identity and experience: • Students’ online identity can be

different from the student who is recognisable in class.

• Students’ social or cultural experiences of web authoring can influence the work they produce for assessment.

• Reflection and self-reflection about the idea of identity are prompted by the need to create and express an online identity.

Affordances

Co-creation, collaboration, crowdsourcing:

• Group work can scale between a small closed group and a large free-to-join learning community

• Individual contributions to group work can (sometimes) be distinguished.

• Groups can work on large, complex tasks.

Affordances

Content management

• Students’ assessable work may consist of remixing web content from diverse sources.

• Students’ work may be posted on several host sites. Work posted on one site may be syndicated by others and tracked back.

• Students can control the content they produce for assessment in accordance with terms of service, end user agreements or other governance policies of host sites.

2. What would good practice look like?

ProcessesSupporting individual and organisational learning throughout the cycle of assessment activities

Design

Implement

MarkFeedback

Review

Processes

Design rationale• Student learning can’t be assessed with as much effectiveness, reliability

or validity by using any other type of assignment.

• Students strive to achieve excellence more than they would in some other type of assignment.

• Staff manage the assignment related workload more sustainably than with some other type of assignment.

• Another reason ....

Processes

Implementing how?

• Explain the timing, weighting and criteria

• Show and discus exemplary student work

• Explain academic attribution and citation practices that are expected

• Provide opportunities to practice and show learning based on formative assessment, before submitting work for summative assessment

• Other teaching techniques?

Processes

Doing the marking

• Rubrics

• Reference to standard levels of attainment and grade descriptors

• Cross-marking and moderation in a teaching team

• Systematic checking for plagiarism or collusion

• Other marking techniques...

Processes

Giving feedback

• Confirmation whether the work is of acceptable standard or not

• Correction of specific shortcomings in the work

• Explanation of what would improve the standard of the work

• Diagnosis of factors that have helped or hindered learning

• Elaboration, i.e. additional advice or resources to extend learning

• Other types of feedback....

Processes

Review and monitor using...

• Student feedback about this assignment

• Input from relevant professional or industry advisors

• Longitudinal evaluation of student performance in this assignment

• Academic peer review in learning and teaching forums

• Other methods of continuous improvement....

2. What would good practice look like?

PolicyAssessment that is safe and fair for students and staff• disability

• access to IT services or equipment

• appropriate conduct

• identity and privacy

• academic honesty and integrity

• special consideration

• moral rights and copyright

3. What works and doesn’t work in real subject teaching settings?

18 subjects @ 5 universities during Semester 1, 2010:

Blogging

Criminal Law

Cultural Studies

Media Studies

Social bookmarking Education

Social networking Japanese

Photo and video

sharing

Communication Design

Economics

Work Integrated Learning

Virtual worldsBusiness

Chinese

Wiki writing

Accounting

Education

Science

Information Technology

Italian

Combined Web 2.0

tools

Document Management

Information Technology

We acknowledge contributions by ...Project Advisory Group• Matthew Allen, Bill Anderson, Greg Battye, Robyn Benson, Tracey Bretag, Jenny Buckworth,

Denise Chalmers, Geoffrey Crisp, Leitha Delves, Bobby Elliott, Jacqui Ewart, Glenn Finger, Tom Franklin, Merrilyn Goos, Scott Grant, Ashley Holmes, Christopher Hughes, David Jones, Marj Kibby, Adrian Kirkwood, Mark Lee, Catherine McLoughlin, Beverley Oliver, Kaz Ross, Alison Ruth, Royce Sadler, Mary Simpson, Arthur Winzenried, Katina Zammit, Lynette Zeeng.

Project Reference Group• Michael Abulencia, Robyn Benson, John Benwell, Marsha Berry, Marilys Guillemin, Laura

Harris, Deborah Jones, Gregor Kennedy, Shaun Khoo, George Kotsanas, Lauren O’Dwyer, Jason Patten, Emma Read, Julianne Reid, Gordon Sanson, Cristina Varsavsky.

Project Field-testing Group• Matthew Absolom, Anne Davies, Cathy Farrell, Scott Grant, Terry Hallahan, Michael

Henderson, John Hurst, Ramon Laboto, Warren McKeown, Michael Nott, Kerry Pantzopoulos, Michele Ruyters, Sukunesan Sinnappan, Michael Smith, Sandra Smith, Robyn Spence-Brown, Elizabeth Stewart, John Terrell, Jenny Weight, Lynette Zeeng

ALTC Support for this project has been provided by the Australian Learning and Teaching Council Ltd.

(www.altc.edu.au), an initiative of the Australian Government Department of Education, Employment and Workplace Relations.

The views expressed in this presentation do not necessarily reflect the views of the Australian Learning and Teaching Council, or the views of individual contributors apart from the project team.

We invite you to join in and extend the discussion

• Moodle: www.groups.edna.edu.au/course/view.php?id=2146

• Blog: http://web2assessment.blogspot.com

• Bookmarks: http://www.citeulike.org/tag/assessment20

• Workshops 2010-2011

@ HERDSA, ATN Assessment, ASCILITE, ACE

• Feedback: http://www.zoomerang.com/Survey/WEB22APCVU3JP7

References

• Chodorow, S. (2000). Scholarship & scholarly communication in the electronic age. Educause Review, 35(1), 86-92. Retrieved 28 November, 2007 from http://www.educause.edu/ir/library/pdf/ERM001B.pdf

• Kakutani, M. (2010, 17 March). Texts without context. [Book review]. New York Times. http://www.nytimes.com/2010/03/21/books/21mash.html?ref=books

• O’Reilly, T., & Battelle, J. (2009). Web Squared: Web 2.0 Five Years On. Special Report for the Web 2.0 Summit, 20-22 October , San Francisco CA. Retrieved October 1, 2009 from http://assets.en.oreilly.com/1/event/28/web2009_websquared-whitepaper.pdf