Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda...
-
Upload
silvester-hoover -
Category
Documents
-
view
217 -
download
1
Transcript of Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda...
![Page 1: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/1.jpg)
Assessment Systems as Learning Communities
AACTE/NCATE Institutional Orientation
April 3, 2008
Hilda Rosselli, DeanWestern Oregon College of
AACTE/NCATE Institutional Orientation
April 3, 2008
Hilda Rosselli, DeanWestern Oregon College of
![Page 2: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/2.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 2
A Recursive Process for Candidates
TeacherWork
Sample
Candidates AssessPK-12 Student Learning
Candidates ReviewResults of TWS
Candidates Reflect and Plan Next Steps
Assessment Systems as Learning Communities
Candidates Teachtheir Students
![Page 3: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/3.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 3
A Recursive Process for Faculty
Faculty AssessCandidate Learning
Faculty ReviewResults of Key Assessments
DARE
Faculty TeachCandidates
Faculty ReviewAnd Refine
![Page 4: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/4.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 4
• Small public liberal arts college (5,000 students)
• Historically known for its teacher preparation programs (1856)
• License around 300 teachers per year
30+ faculty in teacher preparation Undergrad, MAT (two formats), SPED,
and MSinEd NCATE accredited for over 50 years
Context
![Page 5: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/5.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 5
Connecting Teaching & Learning• Home of Teacher
Work Sample Methodology
• Del Schalock, TRI mentor to the College
• Leadership role in Oregon teacher preparation
![Page 6: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/6.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 6
![Page 7: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/7.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 7
Foci for Faculty
Refinement of prompts and rubrics Inter-rater reliability and validity for
scorers of TWSM Systematic linkage of TWSM elements to
state and national standards Formalized rules for passage through
gates based on candidate performance; Easy access to aggregated data for use in
reporting, research and program improvement
![Page 8: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/8.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 8
Our collective tasks… Maximize existing data already being
collected; Select the most compelling types of data for
aggregation; Document the rubrics and decision making
rules for each assessment; Revisit alignment with CF, INTASC, TSPC,
and WOU proficiencies; Create a predictable cycle of compilation,
analysis, and review; Develop relational databases to support
deeper levels of analysis;
![Page 9: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/9.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 9
Our collective tasks… Capitalize on existing and emerging
technologies to support the system; Engage staff, faculty, administrators, and
advisory boards throughout the process; Synchronize efforts with any institutional
assessment efforts when possible; Support faculty engagement in scholarship
related to assessment; Institutionalize a culture in which data are
regularly reviewed and used for program verification and improvement
![Page 10: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/10.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 10
Process• Work Groups
Dispositions, Diversity, TWSM, Portfolios, Policies, Data Management System, Follow up Studies, Field Experiences
• Leadership Team• College Days• Communication• Partnership w/Computing Services and
Registrar
![Page 11: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/11.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 11
BASIC PARAMETERS OF A DATA MANAGEMENT SYSTEM FOR THE
INITIAL LICENSING OF TEACHERS AT WOU
PERFORMANCE/ MEASUREMENT FOCUS
GATE I Program
Admission
GATE II
Short-Term Supported Teaching
Experiences
GATE III
Admission to Long-Term Student
Teaching
GATE IV Recommendation for INITIAL Licensure
Knowledge/understanding (CBEST, ORELA, PRAXIS, GPA, coursework)
X
X
X
Skills/performance in school settings (Formative)
X
X
Dispositions/interview data
X X X X
Impact on student learning (TWSM)
X X
ELECTRONIC INFORMATION MANAGEMENT SYSTEM
Candidate ID/demographics
X
Placement contextual ID/demographics
X X X
Exit surveys
X
Follow up (Alumni & Employer surveys)
2 years after
![Page 12: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/12.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 12
Flowchart
NCATE Domain
Program Proficiency
Element Assessment
Data RuleYes
No Flag
![Page 13: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/13.jpg)
Leads to
Tests
Allows us to answer
Raises questions
about
leads to
estaablishes
raises the need for
raises questions
forces
invites
RESULTS
IN
COE Data
Management
System
Quality
Assurance
Program
Accountability
Conceptual
Framework
Current
Practices
Current
Policies
ASSUMPTION
Knowledge Skills
Dispositions
informs and aligns
with a College's
Past
"Tweaking"
forces
review of
Revisions to
Conceptual
Framework
New
Practices
New
Policies
Specific
Assignments
Ajustments
within
programs
Adjustments
Across
Programs
Increased
focus
New clerical
responsibilities
New Issues
Confidentiality
Flags
Role of
Portfolios
Finetuning
TWSM
rubrics
Reliability &
Validity of
Assessment
Protocals
What we
can say
about our
graduates
What
changes
need to
occur in
programs
Data
Reporting
Research
Program
Improvement
Diversity Dispositions
Impact on
student
learning
Observation
forms
Proficiencies
Deaf Ed
Quality
Assurance and
Program
Accountability
![Page 14: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/14.jpg)
DISPOSITIONS
PROFESSIONAL
KNOWLEDGE
PEDAGOGICAL
KNOWLEDGE
CONTENT
KNOWLEDGE
8.Philosophy,
Best Practice,
& Reflection
9.Professional
Commitment &
Dispositions
10. Collaboration,
Partnerships &
Leadership
2. Human
Development
& Learning
3. Diversity &
Exceptionality
4. Assessment
& Instruction
5. Classroom
Climate
7.Technology
6. Literacy Dev.
&
Communication
1.Content
Knowledge
& Pedagogy
NCATE
DOMAINS
WOU
PROFICIENCIES
Depth & Breadth*
Integration 1b
Alignment 4b
Curriculum Modific.-3f, 3a
ASSESSMENT ELEMENTS
Reflects strong link with impact on
student learning as reflected in TWSM.
Constructive Communication 2.1
Multi-Disciplinary Teaming 4.3
Assistance & Support
Learning Community
Active Participation 4.3
Professional Behavior 5a
Support Learning
Community 5b
Professional Communication 2g
Self-direction ?2.4?, 4a
Reflective Practitioner* 3.2
Philosophy of Education
Educational Research* 3d
Responsibility* 3.2
Inclusion*-1f,2c, 4a
Awareness
Advocacy
Learning Theories*
Human Development Models 4a
Applications 3a, 3d
Benchmarks & Assessments
Literature
Language Development
2nd Language* -2b, 2e
Cultural Inflluence
Facets of Literacy
Literacy Programs
Assessment Tools 4a,b
Goals & Standards
Planning--1a-g, 2j, 3c,e,
5a, c, d
Implementation*
Time Management 1g,2h,i
Classroom Assistants*2k
Human Dignity*-2a,e
Physical Environment
Student Behavior*-2b,d,e
Classroom Climate 5c
Social & Ethical Issues*
Operations & Concepts 1e
Designing Learning Environments
Evaluating/Selecting Software*
Assessment Evaluation
Undergraduate Program Gate 4
Recommendation for Initial Licensure
Professional Practice
GATE 4
Recommendation for Inital Licensure
2 Work
samples
Completion
of Student
Teaching
Y
N
Summative
Evaluationapproval
by both
Tests
Required
Coursework
Y
N
RegistrarApproval /
maintain
GPA
Y
N
Field Book
EvaluationsScore of
proficient
Y
N
Praxis Passing
Disabilities* - 2c, 2e, 2g
Human Dignity*-2a,e
Classroom Assistants*2k
Integration*-1f,2c, 4a
Graduation and
Recommendation
for Initial
Licensure
![Page 15: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/15.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 15
TWS as a Basis for Reflective Learning Community Dialogue
Candidate with Students Candidate with Candidates Candidate with Mentor Teacher Candidates with Faculty Faculty with Mentor Teachers Faculty to Faculty
![Page 16: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/16.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 16
A Continuous Teaching CycleA Continuous Teaching Cycle
PreAssess
Post Assess
Analyze and reflect
Plan
Instruct
![Page 17: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/17.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 17
D.A.R.E.
Data Analysis Review Event-held annually
Review what we are learning about our students, our programs, ourselves…
Use what we are learning to make informed decisions about our programs…
Celebrate our efforts and our community.
![Page 18: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/18.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 18
Why look at data at all?
Focus on what we know about our students
Focus on what is happening to our students while they are with us
Focus on our students after they leave us Plan based on what we learn Share information within and across areas Ask questions of ourselves Fine-tune our assessments and processes
![Page 19: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/19.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 19
Format for presenters
What question/s was investigated and who were the participants?
What process was used to investigate the question/s?
What were the findings? What implications do you see
resulting from the findings? Next steps (if applicable)
![Page 20: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/20.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 20
Typical D.A.R.E. agenda
TWSM Protocol Changes-Adele Schepige COE Faculty Survey on Inclusivity-Chloe Myers MS in Ed Alumni Surveys-Linda Stonecipher Candidate Exit Surveys-Connie Wildfang CF Walk the Talk-Mark Girod Clinical Formative Assessments-Sue Thompson Employer Satisfaction and Alumni-Hilda
Rosselli
![Page 21: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/21.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 21
Beyond Teacher Prep Table Talks
WOU Admission Surveys-David McDonald WOU Faculty Climate Surveys-Hilda Rosselli WOU Freshmen Retention Rates-IR COE Enrollment Management Results-Tiffany
Smith HPE Student Survey II-Jessica Henderson National College Health Assessment-Peggy
Pedersen WOU Advising Survey-Karen Sullivan-Vance RC/RCD Follow Up Data-Hank Bersani
![Page 22: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/22.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 22
Entering Candidate Data
C B S T M T H
7 0 . 06 5 . 06 0 . 05 5 . 05 0 . 04 5 . 04 0 . 03 5 . 0
F a l l 2 0 0 5 U n d e r g r a d u a t e C o h o r t
C B E S T M a t h S c o r e s
n = 4 6
1 0
8
6
4
2
0
S t d . D e v = 9 . 2 0
M e a n = 5 2 . 8
N = 4 6 . 0 0
C B S T W R T G
8 0 . 07 0 . 06 0 . 05 0 . 04 0 . 0
F a l l 2 0 0 5 U n d e r g r a d u a t e C o h o r t
C B E S T W r i t i n g S c o r e s
n = 4 6
4 0
3 0
2 0
1 0
0
S t d . D e v = 8 . 7 5
M e a n = 4 4 . 3
N = 4 6 . 0 0
![Page 23: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/23.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 23
Table 7.1Formative Evaluation Component: Selected Items
Teacher Candidate Performance over TimeUndergraduate Program: Fall 2005 Cohort
Mean Scores & Standard Deviation (sd)
2.983.14 3.12
2.903.14 3.103.12 3.16 3.16
1
1.5
2
2.5
3
3.5
4
UG Fall 2005Gate: Term II
UG Fall 2005Gate: Term III
UG Fall 2005Gate: Term IV
Planning for InstructionItem A
Planning For InstructionItem D
Engaging StudentsItem C
(.44) (.55)(.53) (.45) (.45) (.51) (.39) (.46) (.37)
N = 50 Teacher Candidates from the Fall 2005 Undergraduate Cohort
Score Rubric1 = Needs Improvement2 = Beginning3 = At Required Level4 = Exceeds Required Level
Clinical Ratings by Item Across Gates
![Page 24: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/24.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 24
Longitudinal Data TrendsUse Standards/Benchmarks for Long-term Curriculum
Planning
76%
78%
94%
80%
93%
64%
98%
100%
100%
24%
23%
6%
20%
8%
36%
3%
0%
0%
0% 50% 100% 150%
SP03 (N=97)
F03 (N=69)
W04 (N=50)
SP04 (N=67)
F04 (N=51)
W05 (N=42)
SP05 (N=38)
F05 (N=47)
W06 (N=40)
Poor-Fair
Good-Excellent
![Page 25: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/25.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 25
MSEd Alumni Perception of Program Impact on Content Pedagogy
4
8
13
11
1
1 = not at all 2 = slightly 3 =moderately
4 =significantly
No Response
Rating
n = 37
![Page 26: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/26.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 26
Role of External Consortium Previews prototypes Reviews results presented at DARE Recommended that faculty address
all items with a combined score of less than 80% for Good to Excellent.
Reviews faculty reflections and recommendations
Includes LAS representation
![Page 27: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/27.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 27
2007 Regional Accreditation
“The leaders in the College of Education seek feedback on their performance from a wide variety of sources--and the annual Data Analysis Review Event provides an opportunity for all faulty to become engaged in the process of analyzing and interpreting the results. Significant policy and procedural changes have resulted from the Events held during the past three years.”
![Page 28: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/28.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 28
2007 NCATE comment
“The WOU Teacher Work Sample methodology has long provided an effective way to monitor candidates’ ability to analyze the teaching/learning context, set goals, plan and assess instruction, adapt instruction to meet individual needs of students, and reflect on their own work. The process has remained basically the same, while methods of assessment have changed, each based on perceived need, and with careful thought toward excellence.”
![Page 29: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/29.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 29
2007 NCATE comment
“The COE actively involves tenure-track, term, and adjunct faculty in improving programs for the purpose of ensuring that candidates are prepared to help students learn.”
![Page 30: Assessment Systems as Learning Communities AACTE/NCATE Institutional Orientation April 3, 2008 Hilda Rosselli, Dean Western Oregon College of Education.](https://reader035.fdocuments.us/reader035/viewer/2022062422/56649ead5503460f94bb4ee9/html5/thumbnails/30.jpg)
April 3-6, 2008 H.Rosselli (AACTE/NCATE) 30
Closing Thought
“Among the factors that appear to be critical to a profession-wide change process, several seem particularly important:
• coherence in standards, • clarity about curriculum, and • commitment to act based on knowledge
about what appears to be effective.” p. 479
Preparing Teachers for a Changing World