R U There? Looking for those Teaching Moments in Chat TranscriptsFrances Devlin, John Stratton and Lea Currie

University of KansasALA Annual Conference - Washington, D.C., June 24, 2007

R U There? Looking for those Teaching Moments in Chat TranscriptsFrances Devlin, John Stratton and Lea Currie

University of KansasALA Annual Conference - Washington, D.C., June 24, 2007

0

200

400

600

800

1000

Counts

1.1

1.2

1.3

1.4

2.1

2.2

2.3

2.4

2.5

3.1

3.2

3.3

3.4

3.5

3.6

3.7

4.1

4.2

4.3

5.1

5.2

5.3

ACRL Standards/Indicators

40%

40%

11%

5%

3%

25%

70%

0% 1%

3%

0%

10%

20%

30%

40%

50%

60%

70%

80%

1 2 3 4 5

ACRL Standards

KU

Ellis 2004

Introduction

The University of Kansas Libraries implemented its chat reference service in 2003 to provide research assistance to students, staff, and faculty. To further extend hours of service, KU Libraries established partnerships with three other state academic institutions in 2004 and created the Kansas Academic Cooperative Chat Service. Usage statistics have shown that chat developed into a popular service used primarily by students to get online library assistance. Chat transcripts generated as a result of the service provide a plethora of materials worth further examination. Based on the ACRL Information Literacy Competency Standards for Higher Education (hereafter ACRL Standards), our evaluation of transcripts provided insights into whether librarians were teaching students to determine what information they needed, how to access it, use it ethically, and to evaluate the information critically.

ACRL Information Literacy Competency Standards for Higher Education

Standard One: “The information literate student determines the nature and extent of the information needed.”

Questions Based on Performance Indicators:1.Does the reference interview allow the student to articulate the information need? Is the

student allowed to identify key terms that describe the topic?2.Does the student have the opportunity to identify the types of information he/she needs?3. Is the student given instruction on how to locate information using a variety of formats

and library services?4.Does the student have the opportunity to revise or refine his/her topic?

Standard Two: “The information literate student accesses needed information effectively and efficiently.”

Questions Based on Performance Indicators:1. Is the student instructed to use the most efficient and effective approaches to finding the

needed information?2.Does the student have the opportunity to construct a search strategy?3.Does the librarian suggest a variety of formats and search strategies for retrieving

information?4. If necessary, does the librarian help the student revise the search strategy?5.Does the librarian instruct the student on how to save and organize the information (e.g.,

email, save, print or use RefWorks/Endnote)?

Standard Three: “The information literate student evaluates information and its sources critically and incorporates selected information into his or her knowledge base and value system.”

Questions Based on Performance Indicators:1.Does the librarian give the student the opportunity to comment on the information he/she

is finding?2.Does the librarian instruct the student on how to evaluate the information they are

finding?3.When applicable, does the librarian refer the student to other services and

technologies?4. Is the student given the opportunity to compare the new knowledge with prior

knowledge?5. Is there an exchange between the student and the librarian concerning what impact the

new information is having on their value systems?6. Is the student referred to other subject specialists?7.Does the student appear to expand their search to a variety of sources and services?

Standard Four: “The information literate student, individually or as a member of a group, uses information effectively to accomplish a specific purpose.”

Questions Based on Performance Indicators:1.Does the librarian instruct the student on how to use the information appropriately?2.Does the student have the opportunity to discuss successes and failures in finding the

desired information?3.Does the librarian take the opportunity to refer student to other workshops or technology

available through the libraries?

Standard Five: “The information literate student understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.”

Questions Based Performance Indicators:1. Is the student instructed in the ethical use of information?2.Does the librarian instruct the student on policies related to the use of information?3.Does the librarian instruct the student on how to document information appropriately?

Analysis and Findings

1. Standards One and Two were most often addressed through chat interactions.

2. In evaluating Standard Three - students were given opportunity to comment on information, but little evidence they were instructed on how to evaluate it.

3. Standard Four was observed even less, since librarians do not often get to see final product; i.e., paper or assignment.

4. Standard Five was not often observed; i.e., only when students had queries about copyright or citing references in papers.

5. KU study comparable to Francoeur & Ellis (2001), Ellis (2004) and Smyth (2003).

Conclusions

1. Chat reference at KU has been very successful in meeting Standards One and Two of ACRL Standards.

2. Breakdown began with Standard Three – librarians often asked if students were finding information, but were not consistently instructing them on evaluating or questioning validity of information.

3. When evaluating Standard Four, librarians missed opportunities to ask more probing questions about how students were planning to use information.

4. Standard Five was rarely seen – the best way to teach students on ethical use of information may be at the end of chat session; e.g., ask if they know how to cite their sources and offer information on plagiarism.

5. Authors agree that there is a need to develop guidelines for librarians to use in teaching information competency through chat reference.

Methodology

• Approximately 1,955 chat transcripts from years 2005 & 2006 were reviewed.

• Using the five ACRL Standards, questions were developed for each performance indicator (see left).

Figure 1: Summary of transcripts. Counts represent number of times ACRL Standards/Indicators were observed in the set of transcripts analyzed (1,955).

Figure 2: Comparison of KU findings with Ellis (2004).

• To review chat transcripts, KU developed questions for each Performance Indicator under the ACRL Standards; while Ellis looked for occurrences of the five ACRL Standards being applied.

• KU reviewed 1,955 transcripts and Ellis reviewed 170 transcripts.

• Findings were similar, with some differences noted below:

• KU observed Standards One & Two applied equally (40%); Ellis’ data indicated a significant difference between Standards One and Two (25% and 70% respectively).

• KU observed Standard Three applied 11% of the time; Ellis’ data was 0%.

• Both studies revealed that Standard Three (evaluating information) may require more time with student. Chat may not be conducive to lengthy interactions needed to teach Standard Three.