4/16/07

Assessment of the Core – Science

Charlyne L. WalkerDirector of Educational Research and Evaluation, Arts and Sciences

4/16/07

Assessing the Major

SACS requires us to assess all degree programs bachelors through doctorate as part of Institutional Effectiveness

FIU has been assessing our degree programs since 1999-2000

Recent changes in SACS has moved us toward more direct measures

Departments assess programs in a variety of ways Sample IE document

4/16/07

Science Core Category Description

Our technologically dependent world requires an understanding of the processes that led us here.

Learning the basic concepts and ideas of scientific fields provides contact with not just those fields but with how science is done.

4/16/07

Science Core Category Description

In these courses students study the scientific method through examination of the foundational theories of modern scientific thought.

Students apply scientific principles and theories to problem solving, evaluate scientific statements, and incorporate new information within the context of what is already known.

Emphasizing the essential connection between theory and experiment, the hands-on laboratory experience provides the context for testing scientific theories.

Students will be able to describe the scientific method through examination of the foundational theories of modern scientific thought.

4/16/07

Foundations of Science

Students will be able toApply scientific principles and theories to problem

solving, Evaluate scientific statements, Interpret new information (e.g. scientific data, journal

articles, media reports of scientific findings, etc.) within the context of what is already known, and

Apply the scientific method by conducting experiments to test scientific hypotheses.

4/16/07

Objectives and Measures

There should be objectives for the courses in course syllabi

As objectives are developed, we may be asking for you to include them into course syllabi

As measures are developed, you might be asked to include them into course syllabi to let students know what measures are being used

This is not a “short term” solution for the 2010 SACS visit

The measures must be on going as part of SACS requirements beyond 2010

4/16/07

Types of Direct Measures Embedded questions

Set of locally developed questions intended to measure specific student learning outcomes

Placed within tests of all sections of the same course Used to track cognitive or skill development through a

program, sets of embedded questions, with each expecting a higher level of proficiency than the previous, may be used across sequential courses

Papers or short answer questions using rubrics – a rubric is used to standardize the grading of work

4/16/07

Types of Direct Measures

Locally developed exams - exams produced by faculty within a discipline

Pre/Post Test - a locally developed test that measure students’ incoming and post program levels of knowledge, skills, behaviors and attitudes to measure students’ gains

You can find more types of measures at http://oeas.ucf.edu/alc/dir_measures.htm

4/16/07

Examples of Objectives and Measures from UCF PSC 1121 Physical Science - Demonstrate an understanding of the

scientific method. In particular, to demonstrate an ability to assess which claims presented as "scientific" are indeed scientific and not pseudoscientific.

Measures: 7.1 PSC 1121 Physical Science - A pre-test and a post-test will be

given. A gain of 30 % will be considered satisfactory, a gain of 50 % will be considered well above satisfactory, and a gain of 65 % or more will be considered outstanding. The gain is defined as the net change in the average score divided by the average of the pre-test. If full tests cannot be given, few questions will be sufficient.

7.2 PSC 1121 Physical Science - In each exam, questions will be embedded to enable an assessment of the extent to which this objective is met. The questions may be multiple choice, long answer, or quantitative problems. An overall score of 70% on these questions will be considered acceptable.

4/16/07

Examples of Objectives and Measures from UCF GLY 1030 Geology and its Applications - To demonstrate an

understanding of multifaceted scientific terminology and an ability to apply the definitions involved. 

Measures:  19.1 GLY 1030 Geology and its Applications - A set of 8 tightly-written

definitions, several involving at least 3 characteristics, will be presented and illustrated in class, in conjunction with graded homework assignments. A 7-question quiz (or quiz section) will follow, testing knowledge of precisely what is included in each definition. Each quiz question will require more than one answer; and no partial credit will be given, in order to stress the importance of learning the entire definition for application later in the course. Because any error or omission can render an answer incorrect, a score of at least 4 out of 7 on these questions by 100% of students who earn a "C" or higher in the course will be considered acceptable. A similar, but smaller, set of questions will be included on a pretest in Spring 2007 to establish a baseline for use in subsequent years.

4/16/07

Examples of Objectives and Measures from UCF GLY 1030 Geology and its Applications - To demonstrate an

understanding of multifaceted scientific terminology and an ability to apply the definitions involved. 

Measures:  19.2 GLY 1030 Geology and its Applications - A subsequent test will include an

embedded set of 7 questions that require simultaneous application of all the characteristics of each definition from measure 19a that is tested. For example, determining the number of minerals in a collection described in the question-set requires consideration of whether to count: types or pieces, natural and/or synthetic materials, and crystalline and/or noncrystalline materials. In addition, knowledge of the basic mineral compositions of the igneous rocks is needed. Again, no partial credit will be given, in order to promote the learning of knowledge as a cohesive whole, rather than as a set of disconnected fragments. Because any error or omission can render an answer incorrect, a score of at least 4 out of 7 on these questions by 100% of students who earn a "C" or higher in the course will be considered acceptable. A similar, but smaller, set of questions will be included on a pretest in Spring 2007 to establish a baseline for use in subsequent years.

4/16/07

Examples

UCF General Education Science Foundation

http://iaaweb.ucf.edu/assessment/visitor/view_2006_07.asp?program_ID=1240

UCF Assessment Plans for 2006

http://iaaweb.ucf.edu/assessment/visitor/y67.asp?group=visitor