Educator Preparation Unit Assessment Handbookand Human Ecology; College of Food, Agricultural, and...

Educator Preparation Unit

Assessment Handbook

Our vision is to serve as the Epicenter of Eminence for Educator Preparation (E3P)

Office of Educator Preparation

185 Arps Hall 1945 N. High Street Columbus, OH 43210 614-292-2581 [email protected]

September 2014

mailto:[email protected]

4

Table of Contents

The Ohio State University……………………………………………………………………………………………………………………….4

Educator Preparation Unit & System Overview………………………………………………………………………………………5

Conceptual Framework of Educator Preparation Unit…………………………………………………………………………….6

Programs………………………………………………………………………………………………………………………………………………..7

Conceptual Framework Crosswalk…………………………………………………………………………………………………….8-18

Assessments…………………………………………………………………………………………………………………………………………19

Transition Gates……………………………………………………………………………………………………………………………………20

Unit Assessments and Data Administration……………………………………………………………………………………..21-23

Fairness, Accuracy, Consistency, and Bias…………………………………………………………………………………………….24

Data Collection, Analysis, and Evaluation……………………………………………………………………………………………24

Key Assessments Descriptions………………………………………………………………………………………………………………25

5

The Ohio State University Historical Context The roots of The Ohio State University go back to 1870 when the Ohio General Assembly established the Ohio Agricultural and Mechanical College. The new college was made possible through the provisions of the Land-Grant Act, signed by President Lincoln on July 2, 1862. The first classes were taught in 1873, with the institution receiving its present name in 1878, the year of its first graduating class. The University’s land-grant status has a significant impact yet today on how it views its responsibility for outreach. The Institution is one of 13 public universities in Ohio. There are 14 colleges, 175 undergraduate majors and over 12,000 courses offered per year. With an enrollment of 63,058 students, of which 49,466 are undergraduates, Ohio State boasts over 500,000 living alumni. There are four regional campuses: Lima, Mansfield, Marion, and Newark, and the Agricultural Technical Institute in Wooster. Vision The Ohio State University will be the world’s preeminent public comprehensive university, solving problems of world-wide significance. Mission We exist to advance the well-being of the people of Ohio and the global community through the creation and dissemination of knowledge. Core Goals Four institution-wide goals are fundamental to Ohio State’s mission and future success: Teaching and Learning: to provide an unsurpassed, student-centered learning experience led by engaged, world-class faculty and enhanced by a globally diverse student body. Research and Innovation: to create distinctive and internationally recognized contributions to the advancement of fundamental knowledge and scholarship and to solutions of the world’s most pressing problems. Outreach and Engagement: to establish mutually beneficial partnerships with the citizens and institutions of Ohio, the nation, and the world so that our communities are actively engaged in the exciting work of The Ohio State University. Resource Stewardship: to become the model for an affordable public university recognized for financial sustainability, unsurpassed management of human and physical resources, and operational efficiency and effectiveness.

6

The Ohio State University Educator Preparation Unit The 47 educator preparation programs of the unit are offered on the main campus and at the four regional campuses. These programs reside in the College of the Arts & Sciences; College of Education and Human Ecology; College of Food, Agricultural, and Environmental Sciences; and College of Social Work. Each college also house non-educator preparation programs; that is, programs which do not prepare individuals to work in P-12 schools. The Dean of the College of Education and Human Ecology serves as the head of the unit. The Unit Assessment System Overview

The purpose of the Assessment System of the Educator Preparation Program at The Ohio State University is to collect, compile, analyze and maintain information in an effort to manage and improve candidate, unit, and program performance. The assessment system is designed to facilitate execution of the mission and goals of our Conceptual Framework, standards for school professionals in Ohio, and Specialized Professional Association standards of the respective programs. Together, these provide common unit outcomes that are assessed through candidate key assessments, surveys and Institutional data. The assessment system helps to ensure that candidates are ready to transition through stages of their program, and that our program completers have the professional knowledge, skills and dispositions outlined in our Conceptual Framework as well as in applicable State and Specialized Professional Association standards.

Operationally, the Assessment System has been developed and continues to evolve as the needs of its constituents change. The design of the Assessment System is focused on providing utility and value for all involved by interfacing and operating with the following objectives in mind:

The Assessment system should be a continuously capable and competent enterprise through regular review and evaluation, with the goal of improving educator programs. All decision-support information and products (reports, documents, assessments, data, etc.) should be accessible and understandable by our professional and partner community. Stakeholders should be involved with the ongoing development and evolution of the system. The Assessment System should provide for consistency of management of programs while affording maximum flexibility for program needs.

7

CONCEPTUAL FRAMEWORK OF THE OSU EDUCATOR PREPARATION UNIT

VISION

To serve as the epicenter of eminence for educator preparation (E3P)

MISSION

Our mission is to prepare highly effective educators who teach, lead, and serve. These highly effective educators will apply research-based practices that support academic and social development of all learners and engage in ongoing professional development.

PHILOSOPHY

The philosophy of The OSU Educator Preparation Unit is based upon a reciprocal relationship between theory and practice, using research to inform our programs. Drawing from multiple disciplines and methodologies, the Educator Preparation Unit focuses on educational processes across diverse P12 learners. We believe that educational research, practice, and policy constantly evolve and that highly effective educators lead and advocate in their respective fields.

PURPOSE AND GOALS

Our purpose is to support and enhance the development of educational leaders who engage in and implement research-based practices to support and advance P12 educational progress. Our programs prepare candidates for initial licensure and engage experienced educators who aspire to advance their practice. Candidates at all levels gain knowledge, skills and dispositions as critical thinkers, problem solvers, communicators, and collaborators. The goals of The OSU Educator Preparation Unit provide direction for developing and aligning the curriculum, instruction, field experiences, clinical practices and assessments. Candidates shall have a commitment to the following:

1) Acquiring the knowledge, skills and dispositions to interact effectively with all students in diverse learning environments;

2) Supporting practices with foundational and current research and theory; 3) Providing learning environments that support the development of all students; 4) Developing and executing objectives, based on continuous assessment, that support

student learning; 5) Practicing integrity and ethical behavior; and 6) Engaging in professional development.

December 6, 2013

8

Programs that are part of the Educator Preparation Unit *Early Childhood (Grades P-3) Undergraduate/Graduate *Middle Childhood (LA, MA, Sc., and SS) (grades 4-9) Undergraduate/Graduate *AYA Language Arts (Grades 7-12) Undergraduate/Graduate *AYA Mathematics (grades 7-12) Undergraduate/Graduate *AYA Science: Chemistry, Biology, Earth Science, Physics, Chem/Physics, Chem/Bio, Chem/Earth,

Bio/Physics, Bio/Earth, Earth/Physics, Integrated Sciences: Undergraduate/Graduate *AYA Social Studies (grades 7-12) Graduate

Multi-age Âgricultural Science Education (grades 4-12) Undergraduate ^Drama (grades P-12) Graduate ^Family and Consumer Science (grades 4-12) Undergraduate/Graduate *Foreign Language: Arabic, Chinese, German, Hebrew, Italian, Japanese, Latin, Russian: Graduate.

Spanish, French (Grades K-12) Undergraduate/Graduate *Health (P-12) Undergraduate Întegrated Business (grades 4-12) Undergraduate/Graduate ***Intervention Specialist: Early Childhood (grades P-3), Mild to Moderate (grades K-12), Moderate to

Intensive (grades K-12): Undergraduate +Music Education (Grades P-12) Undergraduate **Physical Disability Specialist: Hearing Impaired, Visually Impaired (grades K-12) Graduate *Physical Education (Grades P-12) Undergraduate *TESOL (new) Undergraduate +Visual Arts (grades P-12) Post-baccalaureate

Administrators *Principal: Grades PK-6, grades 4-9, grades 5-12 (Graduate) *Superintendent (Graduate)

Endorsements Âdaptive Physical Education (Graduate) ^Career-Based Intervention (Graduate) *Computer Technology (Graduate) Online Êarly Childhood Generalist (grades 4-5) Undergraduate/Post baccalaureate ^Literacy Specialist (Post-baccalaureate) (dormant) ^Middle Childhood Generalist (LA, MA, Sc, SS) (grades 4-6) Undergraduate/Post baccalaureate ^P-6 Mathematics Specialist (grades P-6) Graduate ^Prekindergarten Special Needs (grades Pre-K) Undergraduate ^Reading (grades K-12) Graduate ^Teacher Leader (new) (Graduate) *TESOL (Graduate)

Professional Pupil Services Ôrientation and Mobility (Graduate) +School Audiologist (Graduate) +School Counselor (Graduate) *School Psychologist (Graduate) ^School Nurse (Graduate) +School Social Worker (Graduate) +School Speech-Language Pathologist (Graduate)

Career Technical (Formally Route B) ^Career Technical Workforce Development (grades 4-12) non-baccalaureate

* SPA reviewed license (18 + 8 UG/Grad combo) + Recognized by national accreditor (6) ^ State Reviewed (14) Total Programs = 46 Reports for SPA or state review: 75

9

Conceptual Framework Crosswalk with Unit Key Assessments

Mission Philosophy Purpose Preservice Survey

RESA survey edTPA

Admission Dispositions

Pedagogy Field

Dispositions Field

Pedagogy ST

Dispositions ST NSSE

OSU Survey

Principal Survey

"prepare highly effective educators"

"focuses on educational processes across diverse P12 learners"

"knowledge, skills and dispositions as critical thinkers, problem solvers, communicators and collaborators"

Q 3, 4, 5, 8, 19, 21, 32, 36, 37, 38, 41

Q 3, 4, 5, 8, 19, 21, 32, 36, 37, 38, 41

Rubric 2, 9, 12, 13

- Row D, G Row F, H, I, J Row D, G Row F, H, I, K

Q 11, 12, 14, 53, 75

- Q 4, 5, 6

"educators will apply research-based practices"

"is based upon reciprocal relationship between theory and practice, using research to inform our programs."

"educational leaders who engage in research-based practices to support and advance P12 educational progress."

Q 1, 44, 45 Q 1, 44, 45

Rubric 10

- Row E - Row E -

Q 26, 27, 28, 29, 30, 87, 88

- Q 9, 10

"support academic and social development of all learners"

Q 2, 9 thru 15, 31

Q 2, 9 thru 15, 31

Rubric 3, 6, 7, 8, 15

- Row D, G Row G Row D, G Row G - -

Q 3, 11, 12, 13, 15, 16, 18 thru 24

"highly effective educators who teach, lead, and serve."

"knowledge, skills, and dispositions as critical thinkers, problem solvers..."

Q 6, 7, 17, 18, 29, 30

Q 6, 7, 17, 18, 29, 30

Rubric 1, 4, 5, 11, 14

- Row A, B, C, H, I

- Row A, B, C, H, I

Row J, L - - Q 2, 7


"highly effective educators lead and advocate in their respective fields"

Q 16, 20 Q 16, 20 - Row B, C - Row A thru E - Row A thru E, M

Q 74 - Q 14

"engage in ongoing professional development"

"engage experienced educators who aspire to advance their practice."

Q 22, 26, 28, 49

Q 22, 26, 28, 49

- - - - - Row N -

Future Ed. Plans AND OSU influence

Q 1, 8, 17

10

CF Crosswalks

Crosswalk: Admission to Program Professional Dispositions Evaluation Rubric1

Conceptual Framework Goal OR

NCATE Standard

Statement from CF Mission

Statement from CF Philosophy

Ro

w A

Ro

w B

Ro

w C

Ro

w D

CF Goal 5: Practicing integrity and ethical behavior; and "highly effective educators who teach, lead, and serve."

"highly effective educators lead and advocate in their respective fields"

X X

NCATE 1g Not Applicable Not Applicable X

1 The green columns indicate the row number on the evaluation forms.

11

Crosswalk: Pedagogy Field Experience Evaluation Form

Conceptual Framework Goal OR NCATE Standard OR

OSTP Standard



Statement from CF Purpose Ro

w A

Ro

w B

Ro

w C

Ro

w D

Ro

w E

Ro

w F

Ro

w G

Ro

w H

Ro

w I

CF Goal 1: Acquiring the knowledge, skills and dispositions to interact effectively with all students in diverse learning environments;




X X

CF Goal 2: Supporting practices with foundational and current research and theory;




X

CF Goal 3: Providing learning environments that support the development of all students;


X X

CF Goal 4: Developing and executing objectives, based on continuous assessment, that support student learning;

"highly effective educators who lead”


X X X X X

NCATE 3B: Clinical Practice allows candidates to use information technology;

Not Applicable Not Applicable Not Applicable X

NCATE 3C: Assessments are conducted by candidates; Clinical practice include assessment of student learning; Clinical practice include opportunity to work with students from diverse populations

Not Applicable Not Applicable Not Applicable

X X

NCATE 4A : Design, implementation & evaluation of curriculum and experiences

Not Applicable Not Applicable Not Applicable X X X

OSTP 1: Teachers understand student learning and development and respect the diversity of the students they teach.

Not Applicable Not Applicable Not Applicable X X

OSTP 2: Teachers know and understand the content area for which they have instructional responsibility


OSTP 3: Teachers understand and use varied assessments to inform instruction, evaluate and ensure student learning.


X

OSTP 4: Teachers plan and deliver effective instruction that advances the learning of each individual student.

Not Applicable Not Applicable Not Applicable X X X X X

OSTP 5: Teachers create learning environments that promote high levels of learning and achievement for all students.


12

Crosswalk: Professional Dispositions Field Experience Evaluation Form


OSTP Standard



Statement from CF Purpose R

ow

A

Ro

w B

Ro

w C

Ro

w D

Ro

w E

Ro

w F

Ro

w G

Ro

w H

Ro

w I

Ro

w J

CF Goal 1: Acquiring the knowledge, skills and dispositions to interact effectively with all students in diverse learning environments



X X

CF Goal 3: Providing learning environments that support the development of all students


X

CF Goal 5: Practicing integrity and ethical behavior "highly effective educators who teach, lead, and serve."

X X X X X

CF Purpose: “communicators and collaborators" Not Applicable Not Applicable Not Applicable X X

NCATE 1C Professional and pedagogical knowledge and skills for teacher candidates


NCATE 1D Student learning for teacher candidates Not Applicable Not Applicable Not Applicable X

NCATE 1G Dispositions for all candidates Not Applicable Not Applicable Not Applicable X

NCATE 3C Candidates development/demonstration of KSD to help all students learn


NCATE 4A Design, implementation, & evaluation of curriculum and experiences


NCATE 4D Experiences working with diverse students in P-12 schools Not Applicable Not Applicable Not Applicable X


Not Applicable Not Applicable Not Applicable

X





OSTP 6: Teachers collaborate and communicate with students, parents, other educators, administrators and the community to support student learning.


13

Crosswalk: Pedagogy Student Teacher Evaluation Form


OSTP Standard



Statement from CF Purpose Ro

w A

Ro

w B

Ro

w C

Ro

w D

Ro

w E

Ro

w F

Ro

w G

Ro

w H

Ro

w I

CF Goal 1: Acquiring the knowledge, skills and dispositions to interact effectively with all students in diverse learning environments;




X X

CF Goal 2: Supporting practices with foundational and current research and theory;




X

CF Goal 3: Providing learning environments that support the development of all students;


X X

CF Goal 4: Developing and executing objectives, based on continuous assessment, that support student learning;

"highly effective educators who lead”


X X X X

X

NCATE 3B: Clinical Practice allows candidates to use information technology;


NCATE 3C: Assessments are conducted by candidates; Clinical practice include assessment of student learning; Clinical practice include opportunity to work with students from diverse populations


NCATE 4A : Design, implementation & evaluation of curriculum and experiences




OSTP 2: Teachers know and understand the content area for which they have instructional responsibility


OSTP 3: Teachers understand and use varied assessments to inform instruction, evaluate and ensure student learning.



Not Applicable Not Applicable Not Applicable X X X X X



14

Crosswalk: Professional Dispositions Student Teacher Evaluation Form


NCATE Standard OR

OSTP Standard

Statement

from CF

Mission

Statement

from CF

Philosophy

Statement

from CF

Purpose Ro

w A

Ro

w B

Ro

w C

Ro

w D

Ro

w E

Ro

w F

Ro

w G

Ro

w H

Ro

w I

Ro

w J

Ro

w K

Ro

w L

Ro

w M

Ro

w N

CF Goal 1: Acquiring the knowledge, skills and dispositions to interact effectively with all students in diverse learning environments



X X

CF Goal 3: Providing learning environments that support the development of all students


X

CF Goal 5: Practicing integrity and ethical

behavior


X X X X X X

CF Goal 6: Engaging in professional

development

"engage in ongoing professional development"

"engage experienced educators who aspire to advance their practice."

X

CF Purpose: “communicators and collaborators"

Not Applicable Not Applicable

Not Applicable

X X

CF Philosophy: "highly effective educators lead and advocate in their respective fields"


Not Applicable

X

CF Purpose: "knowledge, skills and dispositions as critical thinkers, problem solvers”


Not Applicable

X

NCATE 1C Professional and pedagogical knowledge and skills for teacher candidates


Not Applicable

X

NCATE 1D Student learning for teacher candidates


Not Applicable

X

NCATE 1G Dispositions for all candidates Not Applicable Not Applicable

Not Applicable

X

15


NCATE Standard OR

OSTP Standard

Statement

from CF

Mission

Statement

from CF

Philosophy

Statement

from CF

Purpose Ro

w A

Ro

w B

Ro

w C

Ro

w D

Ro

w E

Ro

w F

Ro

w G

Ro

w H

Ro

w I

Ro

w J

Ro

w K

Ro

w L

Ro

w M

Ro

w N

NCATE 3C Candidates development/ demonstration of KSD to help all students learn


Not Applicable X

NCATE 4A Design, implementation, & evaluation of curriculum and experiences


Not Applicable

X X X

NCATE 4D Experiences working with diverse students in P-12 schools


Not Applicable

X X



Not Applicable

X



Not Applicable X



Not Applicable X

OSTP 6: Teachers collaborate and communicate with students, parents, other educators, administrators and the community to support student learning.


Not Applicable

X X X

OSTP 7: Teachers assume responsibility for professional growth, performance and involvement as an individual and as a member of a learning community.


Not Applicable

X X X

16

Assessment System by which Candidates are Regularly Assessed The assessment system uses data from candidates’ course, program and unit assessments as well as from survey data from candidates, alumni, employers, and P-12 partners to better ensure that unit operations facilitate candidates’ learning. Additionally, data from unit and Institutional databases provide information that is used to manage and improve unit operations.

Use of Assessments to Monitor Candidates Key assessments in required courses and fieldwork, clinical and internship experiences are used to demonstrate the candidate’s attainment and competency in our Conceptual Framework, State and Specialized Professional Association Standards. These as well as other data points are used to monitor candidates as they transition through their programs. Candidates are monitored through four transition points (figure 1):

1. Admission to their program;

2. Admission to student teaching/internship course 3. Completion of their student teaching/internship or capstone experience. 4. Completion of their program.

At each transition point, assessment data are used to provide evidence that candidates are prepared for the next stage. Applicant data are used to determine readiness for admission to a particular program. The criteria for this assessment include: a minimum overall GPA, and a minimum GPA and grades in content courses, standardized test scores, and dispositional analysis.

17

Ohio State Educator Preparation

Transition Gates

A general description of the Transition Gates for each program in the Educator Preparation Unit Gate I: Entrance to program Requires a

completed application

disposition evaluation

3.0 GPA (if an undergraduate and admitted to the University before fall of 2014, GPA of 2.75 is needed)

standardized assessment of one of the following: o ACT 21 (minimum 21 subscore in Reading, Mathematics, and English) o SAT 490 (minimum 490 subscore Reading, Mathematics, and English) o Pearson NES 220 (minimum 220 subscore Reading, Mathematics, and English) o GRE scores of verbal 145, quantitative 144, or writing 3.5 (if admitted to the

University before fall of 2014, no standardized assessment is required) o NOTE: Acceptable scores can be a combination of the above. For example a 21 in

math and English, and a 220 in the NES. NOTE: In addition to the items above, there may be program-specific requirements Gate II: Acceptance to Student Teaching/Internship Requires a

completed ST/Intern form

background check; 3.0 GPA

Modules I: Ohio-specific standards and requirements

Module II: Dyslexia (all programs except Early Childhood, Middle Childhood and Intervention Specialist)

acceptable score on applicable Ohio Assessments for Educators tests

completed advanced field placement pedagogy and disposition forms NOTE: In addition to the items above, there may be program-specific requirements Gate III: Recommendation for Licensure Requires a

completion of the program

completed student teaching pedagogy and disposition forms

completed application

background check

official program sheet

18

Unit Assessments and Data Administration

Unit Assessment

When collected

Data Source

Who collects data

How data are collected

Who summarizes the data

How are data summarized (tables, graphs spreadsheet)

Who analyses the data

Data analysis dissemination

How often are data summarized and analyzed

How data are used

Entrance to program Disposition Evaluation

Application to admission

Pre-candidate

Programs*

TK20 Program admission committees

During committee meetings

Program admission committees

Program faculty meetings

Twice per year Understand applicant pool and reflect on admission process

Pre-admission GPA


SIS Programs TK20 Program admission committees




Understand applicant pool and reflect on admission process

Standardized assessment


SIS Programs TK20 Program admission committees Office of Educator Preparation (OEP)

During committee meetings Charts

During committee meetings OEP and Ohio Board of Regents (OBR)

During committee meetings Ohio Performance Report Card (OPRC)

Twice per year OAE - annually

Understand applicant pool and reflect on admission process OPRC – is shared with UTEC, Ed Prep Express Newsletter, Open Forum, Associate Deans meetings

Background check

Prior ST/Intern

Candidate Agency Spreadsheet, TK20

OEP Chart OEP University Teacher Education Council (UTEC)

Annually Understand applicant pool, understand volume of workload, reflect on any problems with the process

Pre-ST GPA Prior ST/ Internship

SIS Programs SIS

Modules I Prior ST/ Internship

Candidate OEP ITunes U OEP Chart OEP UTEC, Ed Prep Express, Open Forum Meetings

Annually

Module II (if applicable)

Prior ST/ Internship

Candidate OEP ITunes U OEP Chart OEP UTEC, Ed Prep Express, Open Forum Meetings

Annually

Ohio Assessments for Educators tests

Prior ST/ Internship

Candidate Programs, OEP

Pearson OEP Charts/Graphs OEP Programs OBR

UTEC, Ed Prep Express, Open Forum Meetings, OPRC

Twice per year

19

Unit Assessment

When collected

Data Source

Who collects data







How data are used

advanced field placement pedagogy and disposition forms

Summative for advanced field placement

University supervisor

OEP TK20 OEP Charts OEP UTEC, Ed Prep Express, Open Forum Meetings

Annually

student teaching pedagogy and disposition forms

At mid-term and summative for ST

University supervisor

OEP TK20 OEP Charts OEP UTEC, Ed Prep Express, Open Forum Meetings

Annually

Pre-service survey

At the end of student teaching

Candidate Ohio Board of Regents (OBR)

Qualtrics Ohio Board of Regents OEP

Table

OEP UTEC, Ed Prep Express, OPRC

Initial data each semester Summative is annual

Part of state funding equation UTEC- reflects on program strengths and weaknesses.

Principal Survey

At the end of the internship

Candidate OBR TK20 OEP Charts OEP UTEC, Ed Prep Express, OPRC

Summative is annual

Part of state funding equation UTEC- reflects on program strengths and weaknesses.

National Survey of Student Engagement

3 weeks before graduation

Candidate OSU Office of Institutional Research (OIR)

Qualtrics Ohio Board of Regents OEP

Table

OEP University, College, Department and program level

Every three years

Identify aspects of the undergraduate experience inside and outside the classroom that can be improved through changes in policies and practices more consistent with good practices in undergraduate education.

Undergraduate and Master's/Professional Pre-Graduation Surveys

Prior to graduation

Candidate Office of Student Life

Qualtrics Office of Student Life

Charts, graphs OIR Reflect on areas relevant to the education licensure

Background check

When candidate applies

Candidate Agency NA Charts OEP UTEC, Ed Prep Express, Open Forum Meetings

Annually

completion of the program

Completion of program

Program Licensure Coord.

Program sheets

OEP Charts OEP OBR

UTEC, Ed Prep Express, Open Forum

Annually

20

Unit Assessment

When collected

Data Source

Who collects data







How data are used

Meetings, OPRC

Completed licensure application

When candidate applies

Candidate Ohio Department of Education (ODE)

ODE Safe Account

OEP Charts OEP OBR

UTEC, Ed Prep Express, Open Forum Meetings, OPRC

Annually

Employer Survey

Third year of teaching

Principal ODE Qualtrics ODE Charts ODE OPRC Annually When sufficient return rate is obtained, data will be used to inform programs.

Value-added data

Years 1 to 5 of teaching

P-12 students

ODE ODE Excel ODE OEP

OPRC, UTEC Annually

RESA Third or fourth year of teaching

Completer

ODE Pearson ODE Excel ODE OPRC, UTEC Annually Effectiveness of program, preparation for the RESA

*Each program has a responsible party, which may be the advisor, faculty member, or program manager.

21

Fairness, Accuracy, Consistency, and Bias of Assessment Instruments

Content validity is demonstrated within the alignments of the candidate proficiencies with the professional and state standards as found in the appendices of the respective frameworks. As professional and state standards are revised, alignments are analyzed and realignments are made.

Conducting validity and reliability studies on all of the assessments is a long term goal. Inter-rater reliability studies on the various assessment instruments are conducted as assessment instruments are revised using matched-pairs and matched triads with data gathered.

Each instrument will follow this pattern - Content validity, five experts from across the state to evaluate the instruments using a content validity rubric. Rubrics will be reviewed for percentage of exact agreement and adjacent agreement for each rubric item and a content validity ratio will be calculated. Construct validity: ratings given will undergo a factor analysis. Concurrent validity: multiple regression analysis to check for correlations between candidates’ performance on the instruments and other performance and knowledge assessments that could be potentially related to the evaluation instruments under investigation. Test-retest reliability: a statistical analysis to examine if a correlation exists between the ratings candidates receive on the evaluation instruments. Cronbach’s Alpha test and/or Kuder-Richardson test will be used to measure internal consistency reliability. Finally, for the inter-rater reliability portion of the study, evaluators will participate in the inter-rater reliability portion of the study and select a representative sample of 10% of the total. After the forms have been completed, we will the percentage of exact agreement and adjacent agreement for each item on the evaluation instruments.

Data Collection, Analysis, and Evaluation

Data are collected, analyzed, and maintained through the Ohio State Office of Educator Preparation using a combination of internal data systems and third-party platform providers. Assessment deployment for each semester is determined by the “Matrix” crosswalk aligning course to assessment.

Appendix X graphically summarizes the major third party reporting milestone deadlines. In addition to the major third party deadlines, the Office of Educator Preparation collects candidate assessment data at the end of every semester and standardized test performance information upon notification of availability by the provider.

Appendix Z outlines the assessment instruments, decision points, internal and external assessors, data collection schedules, and processes for data collection for each of the assessment items used to assess candidate performances.

Data may be summarized at any point in time through the web application. More specifically, candidate data collected for any one semester are used to make decisions at the end of that semester to determine whether candidates are eligible/ready to move to the next transition point. Those indications are then recorded into the system for documentation and aggregation. This can be done manually or through automation if the course and assessment are encoded in TK20.

Data regarding the management and operations of the unit are collected during summer semester each year. At the end of an academic year, candidate and unit management and operations data are compiled into a Unit Profile Report and candidate data are compiled into respective Program Profile Reports for unit and program decision making at the Annual Educator Preparation Assessment Review.

22

Key assessment descriptions

Acceptance to Program: Dispositions form: This assessment is used as part of the application process. Pre-candidates complete the application. Admission teams evaluate applicants using all of the criteria, including the Application to Program Dispositions Form.

Ohio Assessments for Educators: The OAE tests are required for licensure. There are content assessments and pedagogy assessments. Each license has specific test requirements. OAE tests are required to be taken and passed before student teaching.

Module I: Ohio-Specific standards and requirements - this module is designed to meet the OBR

specific requirements such as the Ohio Operating Standards, Ohio Professional Development

Standards, etc. The module is required to be completed prior to student teaching and candidates must earn at least 80% on each assessment.

Module II: Module II: Dyslexia - this module is designed to address the OBR requirements for

dyslexia for candidates that do not take the 12 hour reading core (all licenses except ECE, MCE, and IS). The module is required to be completed prior to student teaching and candidates must

earn at least 80% on each assessment.

Field Placement Forms: Pedagogy and Dispositions: These forms are used at the end of the Field Placement as a summative assessment for the field placement and a formative assessment for student teaching. The pedagogy form has general teaching skills and the dispositions form includes dispositions needed for preservice teachers at this level.

Student Teaching Forms: Pedagogy and Dispositions:: These forms are used at the midpoint and end of student teaching. These forms are similar to the Field Placement forms, but are the next developmental level (with appropriate areas added). The pedagogy form has general teaching skills and the dispositions form includes dispositions needed for a beginning teacher.

Preservice and RESA Survey: This survey is given by the Board of Regents to our student teaching completers and to practicing teachers in their third or fourth year (when they undergo the RESA process). This survey includes a variety of NCATE oriented questions and questions unique to the needs of Ohio trained teachers.

Principal Survey: This survey is given by the Board of Regents to our principal completers. This survey includes a variety of ELCC-oriented questions and questions unique to the needs of Ohio trained principals.

Employer Survey: This survey is given annually by the Board of Regents to principals (employers) of our completers.

NSSE: The National Survey of Student Engagement (NSSE) collects information at hundreds of four-year colleges and universities about student participation in programs and activities that institutions provide for their learning and personal development. The results provide an estimate of how undergraduates spend their time and what they gain from attending college. Ohio State participates every three years.

OSU Graduate Outcomes Survey: The Office of Institutional Research and Planning annually administers a survey for OSU baccalaureate graduates one year after graduation. The survey asks about career outcomes and about how their experience at Ohio State contributed to those outcomes.

Educator Preparation Unit Assessment Handbookand Human Ecology; College of Food, Agricultural, and...

Documents

Transcript of Educator Preparation Unit Assessment Handbookand Human Ecology; College of Food, Agricultural, and...