Meeting Notes University Studies Advisory Committee

March 19, 2014

Participants Curry Guinn Von Yeager Martin Posey Paul Townend Linda Siefert Steven Emslie Mahnaz Moallem Tammy Hunt Phil Furia Jennifer Horan Notetaker: Lea Bullard

Discussion Notes

1. Presentation: General Education Assessment Results 2010-2013 for Critical Thinking and Written Communication (Linda Siefert)

a. Questions and comments: i. We do not have a performance benchmark for all students ii. Do you do analysis by students’ major? iii. Some departments don’t do course numbering by sophistication level; sometimes,

courses are numbered according to a theme iv. Providing the rubrics to the students and tailoring assignments to them could be

construed as “teaching to the test” v. There may be pedagogical reasons for not telling, too specifically, students what they are

supposed to do. b. What other information would be helpful to collect and analyze for assessment?

i. Ask for the students’ major ii. Get data from upper-division courses for CT iii. Are there plans to assess the capstone courses for CT?

c. What recommendations for improvement could the USAC begin considering? i. We need to look at some ways to drive improvement while students are at the university ii. Holding CTE workshops tends to bring together the same group of people each time so

those alone may not be enough; some ways to get more participants: 1. 10 minute roadshows to departments, alerting to issues and pointing to

resources (can prioritze to hit some of the units that may not be thinking about it as much, and hit the ones that will be undergoing assessment in those areas soon; would need to gather resources prior to this)

iii. Within each department, identify which faculty teach CT and WC

Critical Thinking and Written Communication

Curriculum Matrix that aligns the 17 components of University Studies with the 8 UNCW Learning Goals

General Education Assessment Schedule ◦ First Round (ending this semester): assessed all

Learning Goals and all components at least once in 3 years ◦ Next Round: to be determined; need USAC input

Results for Composition, FA, SBS, IDP, AIL and WI

WC1 Context and Purpose for Writing WC2 Content Development WC3 Genre and Disciplinary Conventions WC4 Sources and Evidence WC5 Control of Syntax and Mechanics

Written Communication has been assessed 4 times since Spring 2010: ◦ Spring 2010 – Basic Studies: Composition, Fine Arts,

Social and Behavioral Science ◦ Sprint 2011 – Basic Studies: Composition and

Interdisciplinary Perspectives ◦ Spring 2013 – University Studies: Aesthetic,

Interpretative, and Literary Perspectives and Writing Intensive

Spring 2010 Basic Studies ◦ Composition: ENG 201 ◦ Fine Arts: FST 210, MUS 115, PSY 105

Spring 2011Basic Studies ◦ Composition: ENG 201 ◦ Interdisciplinary Perspectives: WGS 210 ◦ 300-level pilot

Spring 2013 University Studies ◦ AIL: FST 110, MUS 115, THR 121 ◦ WI: ACG 445, NSC 415, SEC 372

Dimension SP10 Comp

SP10 Fine Arts

SP10 Social

Behavioral

SP11 COMP

SP11 IDP

SP13 US AIL

WC1 Context and Purpose 100% 78.9% 94.7% 76.6% 75.0% 76.4%

WC2 Content Development 94.6% 69.7% 94.7% 67.2% 68.7% 75.5%

WC3 Conventions 82.1% 75.0% 84.2% 78.1% 75.0% 72.6%

WC4 Sources and Evidence 82.1% 52.6% 89.5% 77.8% 50.0% 76.8%

WC5 Syntax and Mechanics 75.0% 77.6% 78.9% 84.4% 81.2% 83.8%

Percent of work products scored 2 or higher.

Dimension % 2 and above % 3 and above SP11 300-level Pilot

WC1 Context and Purpose 78.5% 48.5% WC2 Content Development 75.8% 30.3%

WC3 Conventions 87.9% 39.4% WC4 Sources and Evidence 93.9% 42.4%

WC5 Syntax and Mechanics 97.0% 63.6% SP13 Writing Intensive 400-level Course

WC1 Context and Purpose 87.7% 44.4% WC2 Content Development 86.4% 32.1%

WC3 Conventions 87.7% 30.9% WC4 Sources and Evidence 80.2% 43.2% WC5 Syntax and Mechanics 90.1% 46.9%

Scores for Written Communication are, in general, high compared to other LGs.

IRR for Written Communication is good and improving over time.

Out-of-class assignments score higher than in-class assignments.

Analysis of assignments indicates that they usually match all of the dimensions.

Scores are not increasing to desired levels.

Results for FA and AIL History SBS and UHIB

CT1 Explanation of Issues CT2 Evidence: Analysis AND Question

Viewpoint CT3 Context and Assumptions CT4 Student’s Position CT5 Conclusions and Related Outcomes

Critical thinking has been assessed 4 times since Spring 2010: ◦ Spring 2010 – Basic Studies: Fine Arts and Social

and Behavioral Sciences ◦ Fall 2010 – Basic Studies: History ◦ Fall 2012 – University Studies: Understanding

Human Institutions and Behaviors ◦ Spring 2013 – University Studies: Aesthetic,

Interpretive and Literary Perspectives

Spring 2010 Basic Studies ◦ Fine Arts: MUS 115 ◦ Social and Behavior Science: PSY 105, SOC 105

Fall 2010 Basic Studies ◦ History: HST 102, 103, 201

Fall 2012 University Studies ◦ UHIB: ANT 207, COM 160, PSY 105

Spring 2013 University Studies ◦ AIL: ENG 230, FST 110, THR 121

Dimension SP10 Basic Studies

Fine Arts

SP10 Basic

Studies Social

Behavioral Science

FA10 Basic

Studies History

FA12 University Studies UHIB

SP13 University Studies

AIL

CT1 Explanation of Issues 95.0% 59.8% 73.7% 50.5% 66.7% CT2 Evidence (Holistic) 72.5% 64.0% CT2a Evidence: Analysis 72.8% 40.8% 72.2% CT2b Evidence: Question Viewpoint

40.9% 25.6% 48.6%

CT3 Context and Assumptions NA 40.3% 59.2% CT3a Assumptions 21.4% 47.2% CT3b Context 22.3% 54.2% CT4 Student’s Position NA 51.1% 58.2% 25.3% 64.3% CT5 Conclusions and Outcomes NA 37.0% 59.6% 18.0% 56.9%

Percent of work products scored 2 or higher.

Scores for Critical Thinking are, in general, the lowest of all rubrics.

IRR for Critical Thinking is, in general, the lowest of all rubrics.

Analysis of assignments indicates that ◦ they often do not include directions/prompts that

indicate the need for certain dimensions of critical thinking (like questioning the viewpoints of authors, examining context and assumptions, etc.);

◦ they sometimes do not address the component SLOs. This indicates that there is a lack of common

agreement on what critical thinking is that seems to effect student performance.

What other assessment information would be helpful to the USAC? ◦ Written Communication: ?? ◦ Critical Thinking: data from upper-division courses

other?? What recommendations for improvement could the

USAC begin considering? ◦ Written Communication: CTE workshops and faculty

learning and teaching community for Writing Intensive courses other??

◦ Critical Thinking: promote a better understanding of the dimensions of CT and how to create assignments that promote critical thinking by… other??