Assessing Your One-Shot Instruction Session

Kathy Stroud, University of Oregon

What Do You Want to Assess?

1) Assessment of students learning 2) Evaluation of instructional design/delivery

1) Peer review2) Self-evaluation3) Summative Assessment

Different Types of Assessment

Summative Assessment* Formative AssessmentAlways occurs after the instruction has taken place.

Occurs at any point during instruction

Quantitative or Qualitative Quantitative or qualitativeCan provide information on worth or value of instruction, instructional design, and efficacy of instruction

Provides feedback to the instructor during the session so they may review, reiterate, restate material and adjust to achieve the objectives.

*Summative assessment of student learning can be used as formative feedback to inform the instructor

Questions to Consider When Developing an Assessment Tool

Who wants to know and why? You do, to improve your instruction

What questions are you trying to answer? Are the students understanding the material and applying it? How can I improve my instruction?

What Kind of Data do you need? What are the Workshop Learning Objectives?

This will vary based on what your teaching (generally limit to 3 main things in 50 minutes) How will your learners exhibit what they have learned? What will you do with the data collected?

I’m assuming this is for your personal use. The Library is not mandating collection of this type of assessment data. If doing a study where you will publish/share the results broadly you may need an IRB review.

How much time do you have available for the assessment? Very little if in-class

How much time/incentive is the Faculty willing to give you outside of the session.

Types of Assessment Tools

Likert (happiness scale) One-minute paper Pre-test only Post-test only Pre and post-test Free response question Worksheet completed during library instruction Viewing student portfolios/assignments (use assessment rubric) Pre-test, post-test, and post-post-test

Levels of AssessmentLevel 1: Reaction Level 2: Learning Level 3:

BehavioralLevel 4: Results

Did they like it? Did they get it? Did they apply it? Does it matter?Measures what learners think of the session:• Delivery style• Expected content• Areas of confusion• Useful content

Measures whether content understood

Measures whether they actually used what they were taught

Measures whether the instruction session made a difference in overall performance

• Likert or “happiness scale”

• Multiple choice• One-minute paper

(what liked, what confusing, etc.)

• Knowledge based tests (multiple choice, fill in the blank) mapped to specific session objectives

• Performance tests (complete task)

• One-minute paper(e.g, what did you learn, how would you….?)

Use rubric to grade “real life” product that arises from the course. (examples: bibliography, research paper, portfolio)

May look at total GPA, retention rates, increased information literacy performance at end of year or graduation.

Example Reaction Evaluations

Likert Scale On a scale of 1 to 5 how useful was this presentation To what extent do you feel you have learned from this workshop? Was the presentation clear?

One-minute paper Paired Questions:

What did you learn today that will help you with your assignment/Identify one thing from this session that confuses you.

What is the most useful thing you learned in this session?/What questions do you still have that weren’t answered?

What resources would you recommend to others? What was the muddiest point?

Levels of AssessmentLevel 1: Reaction Level 2: Learning Level 3:

BehavioralLevel 4: Results

Did they like it? Did they get it? Did they apply it? Does it matter?Measures what learners think of the session:• Delivery style• Expected content• Areas of confusion• Useful content

Measures whether content understood

Measures whether they actually used what they were taught

Measures whether the instruction session made a difference in overall performance

• Likert or “happiness scale”

• Multiple choice• One-minute paper

(what liked, what confusing, etc.)

• Knowledge based tests (multiple choice, fill in the blank) mapped to specific session objectives

• Performance tests (complete task)

• One-minute paper(e.g, what did you learn, how would you….?)

Use rubric to grade “real life” product that arises from the course. (examples: bibliography, research paper, portfolio)

May look at total GPA, retention rates, increased information literacy performance at end of year or graduation.

Example Learning Evaluations – Open Ended PaperThese should be closely tied to the objectives of the instruction session. Below are a few examples.

What is the difference between an indexing/abstracting database like ____ and a full-text database like ____?

Which kind of database was more useful to your research and why? When would you search the catalog and when would you search a different

type of database? How can you tell if an article is from a scholarly source or from a popular

magazine? How do you know if a website is suitable for academic work?

Example Activities and Evaluation Rubrics

Task Below Proficient Proficient Above ProficientWrite a few phrases describing the topic of your paper. Underline the key concepts.

Offers no phrases/viable keywords, or phrases that don’t contain key concepts and that do not clearly connect to the assignment.

Offers two or more clear phrases/viable keywords with key concepts that coherently describe assignment focus.

Offers three or more richly descriptive phrase/viable keywords with clear key concepts that capture the assignment focus.

List at least three key concepts related to your topic

Offers fewer than three concepts or concepts that are largely unrelated to the topic.

Offers three or more key concepts related to topic

Offers three or more key concepts that relate to the topic in creative, interesting ways.

List some synonyms for your key words.

Offers fewer than three synonyms or synonyms that are unrelated to the key words

Offers three or more synonyms that accurately relate to the key words.

Offers three or more synonyms that insightfully relate to the key words.

Search the library catalog for two books that could be used as sources for your paper. Provide the Title, Location, Call Number, and whether it is available for checkout.

Identifies books that are inadequate sources for the topic, may not correctly indicate whether they are available for check-out, and may not give the correct location and call number.

Identifies two books that are adequate sources for the topic, correctly indicates whether they are available for check-out, and gives correct location and call number.

Identifies two or more books that are excellent sources for the topic, correctly indicates whether they are available, and gives correct location and call number.

Use the database(s) we discussed to find two articles that will help with your research. List the database used, the title of the article, and the year it was published [may ask for more citation information if you think it is necessary]

Does not locate articles appropriate to topic or does not use database(s) discussed in class.

Locates two articles adequate for the topic using one of the databases discussed in class

Locates two or more articles clearly appropriate to his or her topic using two or more databases discussed in class.

Levels of AssessmentLevel 1: Reaction Level 2: Learning Level 3:

BehavioralLevel 4: Results

Did they like it? Did they get it? Did they apply it? Does it matter?Measures what learners think of the session:• Delivery style• Expected content• Areas of confusion• Useful content

Measures whether content understood

Measures whether they actually used what they were taught

Measures whether the instruction session made a difference in overall performance

• Likert or “happiness scale”

• Multiple choice• One-minute paper

(what liked, what confusing, etc.)

• Knowledge based tests (multiple choice, fill in the blank) mapped to specific session objectives

• Performance tests (complete task)

• One-minute paper(e.g, what did you learn, how would you….?)

Use rubric to grade “real life” product that arises from the course. (examples: bibliography, research paper, portfolio)

May look at total GPA, retention rates, increased information literacy performance at end of year or graduation.

Levels of AssessmentLevel 1: Reaction Level 2: Learning Level 3:

BehavioralLevel 4: Results

Did they like it? Did they get it? Did they apply it? Does it matter?Measures what learners think of the session:• Delivery style• Expected content• Areas of confusion• Useful content

Measures whether content understood

Measures whether they actually used what they were taught

Measures whether the instruction session made a difference in overall performance

• Likert or “happiness scale”

• Multiple choice• One-minute paper

(what liked, what confusing, etc.)

• Knowledge based tests (multiple choice, fill in the blank) mapped to specific session objectives

• Performance tests (complete task)

• One-minute paper(e.g, what did you learn, how would you….?)

Use rubric to grade “real life” product that arises from the course. (examples: bibliography, research paper, portfolio)

May look at total GPA, retention rates, increased information literacy performance at end of year or graduation.