Teaching sustainable online research practices across the curriculum: The Q6C Solution Sarah Read...

Teaching sustainable online research practices across the curriculum: The Q6C Solution

Sarah Read (English)Kate Deibel (CS / Education)

Tim Wright (History)University of Washington

Computers & Writing 2009: Ubiquitous and Sustainable Computing 2

Computers & Writing 2009

Goals: Transcend traditional boundaries

Academic disciplines K-12 and higher education Online and offline Organizers, attendees, and presenters School, work, and play

Develop a sustainable perspective on lifelong computing and communication


Today’s Plan

Framing Q6C Within Conference Intent A Taste of Application What is Q6C?

Development Overview

Using Q6C For the Teacher For the Students (and Teacher)

Conclusion Discussion and workshop


The Issue


The Research Scenario

You are in a college geology course and have been assigned to brief the class on the Crandall Canyon Mine disaster. Answer the question:

Was the cause of the mine collapse geological, and what was it?

First Hit: Why is this not a good source?

The Answer

Authoritative Source and Author


Questions raised by Q6C

How do researchers know when to stop looking for sources?

What do experienced researchers in a field know about when to stop that newcomers don’t know?

How can newcomers to a field be supported to do research with the savvy of experienced researchers?


Q6C: Applied to history


Symptoms of a Serious Problem

In regards to online sources, students... …stop too soon when evaluating a source …fail to keep a skeptical frame of mind …tend to focus on surface/superficial details …trust the top search results …reject Wikipedia outright …accept Wikipedia outright …fail to transfer critical skills across domains


An Information Literacy Skill The Goal:

Teach students to be savvy, critical consumers of Internet sources for both academic and non-academic purposes

Challenges for Teaching What skills should be taught? How can those skills best be conveyed? What is needed to promote skill transfer

across disciplines / outside of the classroom?


Today’s Plan

Framing Q6C Within Conference Intent A Taste of Application What is Q6C?

Development Overview

Using Q6C For the Teacher For the Students (and Teacher)

Conclusion Discussion and workshop


Previous Work and Research

Different rules and expectations for student writing exist across disciplines

Internet poses challenges and opportunities for student research

Students report IT confidence but overestimate source judging abilities

Students report difficulties and overload in conducting research

Checklists for evaluating reliability of Internet sources

McCarthy (1987)

Sorapure et al. (1998)

Kvavik (2005)

Head & Eisenberg (2009)

Many authors


The ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is

knowledgeable and credible? If the author's qualifications aren't listed…

Sponsorship What does the URL tell you? The URL

ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org), …

Currency How current is the site? How current are the site's links? If many

of the links no longer work, the site may be too dated for your purposes.

Excerpt from Hacker’s A Pocket Manual of Style (2008)


Pitfalls of the ChecklistAuthorship Is there an author? You may need to… Can you tell whether the author is



ending often specifies the type of group hosting the site: commercial (.com), educational (.edu), nonprofit (.org)…




INACCURATE: .org has never been restricted to only nonprofits









Not all domains are regulated

Domains reflect only general purposes and not specific pages









Ignores complexity of web authorship

Encourages the usage of titles, degrees, and symbols of authority to determine credibility









Suggests recent data as being more reliable

Update frequency will vary by the type of site


Criticisms of the Checklist

Inherent problems Emphasis on surface features over content Simplistic yes/no questions with no guidance Erroneous indicators of credibility

Students fail to develop information literacy skills and critical practices Meola (2004) Helms-Park & Stapleton (2006)

Need for better evaluative methods to develop sustained, transferable skills Sidler (2002)


Developing Q6C

Intent: Develop tools to support instructors in teaching web

literacy skills

Guiding Principles: Emphasize the process of source evaluation, not the

end product Recognize disciplinary differences Promote student metacognition and transfer of skills

across and outside academia

Approach: Discuss source evaluation across the curriculum Develop a working model of the evaluation process


Discussions about Source Evaluation

Research Literature Ayers (2006) Britt & Aglinskas (2002) Head & Eisenberg (2009) Helms-Park & Stapleton (2006) Hunt & Hunt (2006) Kvavik (2005) Lorenzo & Dziuban (2006) Meola (2004) Oblinger & Oblinger (2005) Sidler (2002) Sorapure et al. (1998) Sullivan and Porter (1997) Thompson (2003) Wineburg (1991,1991,1999)

Source Evaluation Tools Barker & Kupersmith (2009) * Beck (2009) * Cohen & Jacobson (2009) * CARS / CAFÉ – Harris (2007) * Big6 – Eisenberg & Berkowitz

(2001) Hacker (2008) * TAP – Johnson & Lamb (2007) * CSU How to… – Lederer * Rampolla (2007) * Smith (1997) *

* Checklist variant

Cross-disciplinary discussions: Sarah, Kate and Tim


Q6C: Modeling Source Evaluation

Question

Categorize

Critique Rhetorically

Characterize Authorship

Contextualize

Corroborate

Conclude


Q6C: The Start

Question

Categorize



Contextualize

Corroborate

Conclude

Maintain a skeptical frame of mind

Ask questions relevant to your research, purpose, and discipline


Q6C: The End

Question

Categorize



Contextualize

Corroborate

Conclude

Is the source credible?

Is the source useful for your research goals?


Q6C: The Middle

Question

Categorize



Contextualize

Corroborate

Conclude

Different dialogues to engage in with a potential source

Features and relative importance shaped by research task and discipline

Not necessary to do all components nor in any order


Question

Q6C: The Cloud

Categorize

Contextualize

Corroborate

Conclude

Characterize

Authorship


Repeat as necessary


Simple Instantiation of Q6C

Categorize

Series of posts

on Blogger


Most recent post was 2001

Conclude

Too old to be useful

Question

Topic: Information / opinion on recent U.S. policy towards net neutrality


More Complex Instantiation of Q6C

Question

Conclude

Likely to be useful

CharacterizeAuthorship

He is a Swiss citizen

Categorize

Many posts on

ScienceBlogs

Conclude

Useful but biases exist

Corroborate

Provides links to citations


Part owner of a European

ISP

Topic: Information / opinion on recent U.S. policy towards net neutrality


Relevant posts in 2008-

09


Summary of Q6C

Working model of source evaluation process Definite start and end points Iterative and integrative middle

Emphasizes research as a process Only concerns one element of all of research

Not specific to particular disciplines Domain knowledge, heuristics, and criteria need to

be supplied by instructors Provides a common language for discussion

Dirty little secret… not limited to just online sourcesBut how do you use it in teaching?


Classroom Applications of Q6C

Two approaches for using Q6C:1. Using Q6C to inform teaching practice and

assignment and lesson design (Sarah)

2. Infecting students to the practice and habit of source evaluation via the Q6C model (Tim)

First Hit: Why is this not a good source?

The Answer

Authoritative Source and Author

Why is this not a best source?

Authoritative Source

Scientific SourcePartial

Answer

Wrong genre? But current: 07-08

Where to go from here?

Purpose tangential to the research question

Is there a better source?

Aha! The most credible and the most useful source!

Peer Reviewed

Directly relevant to research question

How to get a student (or a newcomer) to this “best” source?

Finally, a complete answer relevant to the research scenario

Visualizing the Q6C Process Question: Was the cause of the mine collapse geological, and what was it? Scenario: College-level geology course

1st hit: MSHA web page


Government agency relevant to issue (.gov)

Contextualize

Not a scientific source

Conclude

Part of the story, but not citable

Corroborate

BUT: What sources does it cite?2nd hit: Berkeley

labAnnual report


.edu / Scientific lab

Categorize

Genre: annual report (not peer reviewed)

Purpose: tangential to research question

Conclude

Best source: relevant and credible

3rd hit: UT seismology

scientific report

Contextualize

Student Reflections:

Corroboration is king (at least in this research scenario)

Q6C as a heuristic for lesson/assignment design

1. Identify a research scenario relevant to course context

2. Identify the tacit research practices of an experienced researcher in that knowledge domain: think Q6C.

3. Construct a scaffolded lesson or assignment to teach explicitly the domain knowledge necessary for critical source evaluation.

4. Plan for reflective writing or discussion to promote metacognition about research practices.

OUTCOMES: 1. students locate more useful and credible sources2. begin to learn how to approach any new research scenario.

Other choices when planning a Q6C lesson or assignment

Prepare an archive to anticipate the research process or set students loose?

How are the research process and domain knowledge weighted in assignment evaluation?

Explicitly teach the meaning of Q6C terms, or prompt students to invent their own during a reflective process?


Questions raised by Q6C

How do researchers know when to stop looking for sources?

What do experienced researchers in a field know about when to stop that newcomers don’t know?

How can newcomers to a field be supported to do research with the savvy of experienced researchers?


Q6C: Applied to history


Website analysis 1.0 beta


Website analysis 1.0

http://www.pbs.org/crucible/

http://www.smplanet.com/imperialism/toc.html

Spanish-American War 1.0

Spanish-American War 2.0


Q6C Lessons Learned

Students already do low-level evaluation Web checklists don’t help much Thinking contextually with corroboration

and purpose in mind does Students can benefit from using Q6C

explicitly


Q6C for Students: Next Steps Design and re-design assignments with

Q6C Use Q6C in course design,

implicitly/explicitly Monitor and analyze results Tweak Repeat Infect others with the Q6C madness


Ongoing Work

Continued classroom applications Modern American Civilization From 1877 (Tim) The rhetoric of Writing in the Workplace (Sarah)

Further research and development Refinement of the Q6C working model Connect to literature on cognitive apprenticeship

and metacognition (e.g., Scardamalia & Bereiter (1983))

Dissemination and partnership building Are you interested in using Q6C in your teaching?

Talk to us.


Thank You!

We wish to acknowledge the following: Center for Instructional Development and

Research Practical Pedagogy Our students Suzzallo Espresso The Internet You

QUESTIONS?


Extra Slides

Descriptions of each Q6C Component Suggestions for Teaching With Q6C Suggestions for Students Using Q6C


Q6C: Question

Maintain a skeptical frame of mind Ask questions relevant to your research


Q6C: Categorize

In the context of your research, is this a primary, secondary, or tertiary source?

What type of site is it (website, blog, wiki, database, etc.)?


Q6C: Critique Rhetorically

What do the authors’ choice of words, tone, font, display format, images, genre, and argumentative strategies tell you about the intended audience and the credibility and reliability of this site? (‘Read’ the site.)


Q6C: Characterize Authorship Identify who created the content, when

they created it, and for what purpose. Single or multiple authors? Committee?

Institution? Critic? Expert? Unknown? Other?


Q6C: Contextualize

Place the information collected in conversation with your existing experience and body of knowledge.

Does it fit? How?


Q6C: Corroborate

Assess how the content compares to other sources.

Is the content consistent, complementary, or contradictory?


Q6C: Conclude

Is the source credible? Is the source useful for your research

goals? If not, find a new source, repeat Q6C.


Suggestions for Teaching With Q6C

Remember that subject-area experts automatically perform the Q6C process, whereas novices need to consciously perform each step when learning how to assess a source's credibility and usefulness.

Identify which components of Q6C you want to emphasize and scaffold the assignment’s research process so that students learn new skills incrementally.

Teach that research is about a process, not about a product.

Construct assignments that engage your students in authentic research practices for your target discipline.



Tips for designing assignments and activities:

Make explicit the outcomes of the assignment and encourage reflection to help move students to the meta-cognitive level.

Consider choosing research topics that you are not an expert in so that you can share the discovery process with the students.

Perform the assignment yourself or with a colleague prior to class in order to anticipate student responses.


Suggestions for Students Using Q6C

Q6C is a heuristic and not a checklist. You do not need to answer every question for every source. Some sources will require only asking one component of Q6C; other sources will require repeated application of Q6C.

Remember that there are two questions you should ask for every source: Is it credible? Is it useful? Answers to these questions are not absolute: a source can still be useful but still of dubious reliability.


Suggestions for Students Using Q6C

When you think you have found a credible source, perform one last check by applying Q6C a little further. A review of Characterize Authorship, Contextualize, and Corroborate can reveal important, overlooked details.

Practice using Q6C in other classes and daily readings. As you gain experience, you will begin to automatically use Q6C to evaluate all kinds of information.

Remember: Research is a process, not a product.

Teaching sustainable online research practices across the curriculum: The Q6C Solution Sarah Read...

Documents

Transcript of Teaching sustainable online research practices across the curriculum: The Q6C Solution Sarah Read...