Evaluation at the Defense Language Institute Dr. Thomas S. Parry
description
Transcript of Evaluation at the Defense Language Institute Dr. Thomas S. Parry
![Page 1: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/1.jpg)
Evaluation at the Defense Language InstituteDr. Thomas S. Parry
Directorate of Evaluation and StandardizationDefense Language Institute
Bureau for International Language CoordinationBudapest, Hungary
May, 2006
![Page 2: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/2.jpg)
Evaluation can make you rethink what you have been doing!
Why Evaluation?
![Page 3: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/3.jpg)
What is Evaluation?
The process of finding the value or amount of something—to determine it’s
significance or quality.
-- Webster’s New World Dictionary
![Page 4: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/4.jpg)
DLI Facts
• Average daily resident attendance: 3,079• Languages taught: 22• Number of students taught annually
through all Continuing Education programs: 5,000
• Highest enrollment languages:
ArabicChineseKorean
Persian-FarsiRussianSpanish Serbian-Croatian
![Page 5: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/5.jpg)
• 0515: Physical Training
• 0630: Breakfast
• 0730: Formation with Service Unit
• 0755: Class start
• 1530: Class end
• 1600: Military Training• Cumulative:
– 6 hours of class instruction
– 45 minute special assistance or enhancement training
– 3 hours of homework plus weekend assignments
– 2-3 hours of study hall (M-Th)
– Developing & sustaining basic warrior-linguist skills, common task training, and physical readiness
Typical DLI Student Day
![Page 6: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/6.jpg)
Basic Course
Language FY06 Student Load Faculty
Category IV Languages (63 weeks)Category IV Languages (63 weeks) ArabicArabic 1165 1165 272 272 KoreanKorean 833 833 216 216 ChineseChinese 558 558 126 126 JapaneseJapanese 3535 10 10
2591 (71%) 6242591 (71%) 624 Category III Languages (47 weeks)Category III Languages (47 weeks) DariDari 87 87 24 24 PashtoPashto 134 134 34 34 Persian FarsiPersian Farsi 248 248 68 68 RussianRussian 144 144 58 58 Uzbek, Hindi, Uzbek, Hindi, Urdu, & 6 other langUrdu, & 6 other lang 211211 9494
824 (23%) 278824 (23%) 278Category II Languages (34 weeks)Category II Languages (34 weeks) GermanGerman 2727 88 27 (1%)27 (1%) 88Category I Languages 26 weeks)Category I Languages 26 weeks) Spanish, French,Spanish, French, Italian, PortugueseItalian, Portuguese 196196 5656
196 (5%) 196 (5%) 56 56
Totals Totals 3638 966 3638 966
DLIFLC
Student Load by DifficultyResident Courses at Presidio of Monterey
94% of students taking CAT III & IV Languages
![Page 7: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/7.jpg)
DLI Evaluation Goals
• Ensure student and teacher accountability• Provide for continuous academic program
improvement– Faculty and staff development– Curriculum development and implementation
• Gauge our return on investment• Ensure quality of life and support services
are conducive to learning
![Page 8: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/8.jpg)
Teachers Evaluate Students
Students are held to high levels of accountability for their learning
![Page 9: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/9.jpg)
Teachers EvaluateStudents
• Teachers evaluate student performance in the classroom through:– Participation in classroom activities– Teacher-developed chapter/unit tests– Periodic counseling– Diagnostic assessments– Learning style assessment– Performance tests– Oral Proficiency Interviews (OPI)– Defense Language Proficiency Tests (DLPT)
• Listening• Reading
![Page 10: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/10.jpg)
Students Evaluate
Teachers Program of Study Quality of Life
![Page 11: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/11.jpg)
Evaluation at DLI is Bi-directional
Teachers Evaluate Students
Students Evaluate Teachers
![Page 12: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/12.jpg)
Teacher Evaluation
• Teacher evaluation at the DLI includes:– Complete student evaluation of the teacher– Scheduled and unscheduled classroom observations by
the supervisor– Periodic reviews of performance objectives with the
supervisor
• The combination of these three teacher evaluation elements provides an accurate assessment of the quality and effectiveness of foreign language instruction
![Page 13: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/13.jpg)
Teacher Evaluation
• Each teacher is evaluated on classroom performance by all the students he/she teaches
• A personal performance profile is created for each teacher by tracking his/her performance throughout each of their courses
Teacher evaluations by students have implications for merit pay and rank advancement
![Page 14: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/14.jpg)
Interim Student Questionnaire
• Administered via computer to all students enrolled in 47- and 63-week language courses 60% of the way through the program
• Takes roughly 60-90 minutes to complete• Survey sections include:
– Program Evaluation (PE)– Teacher Evaluation (TE)– Quality of Life (QL)
ISQ
![Page 15: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/15.jpg)
End-of Course Student Questionnaire
• Administered via computer at the end of language courses to all graduating students
• Takes roughly 60-90 minutes to complete
• Survey sections include:– Program Evaluation (PE)
– Teacher Evaluation (TE)
– Quality of Life (QL)
ESQ
![Page 16: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/16.jpg)
Analysis and Reporting
• Evaluation feedback from ISQ/ESQ delivered within two working days from survey administration– Deans
– Chairs
– Team leaders
– Individual teachers
• Special monthly reports provided to service commanders and individual units via separate questionnaires
• Annual summaries for each instructor sent to Provost, Vice Chancellors and school Deans
![Page 17: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/17.jpg)
ISQ/ESQ Response Scale
• Items are rated from 1-4 with an option for no opinion:
0 No opinion
1 Strongly disagree
2 Disagree
3 Agree
4 Strongly agree
![Page 18: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/18.jpg)
Teacher Evaluation (TE)
• Class preparation and quality of instruction
• Ability to teach language and related concepts effectively
• Adjustment of teaching strategies
• Ability to provide a variety of instructional activities
Sample Criteria
![Page 19: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/19.jpg)
Teacher Evaluation (TE)
• Active use of foreign language in classroom
• Ability to provide challenging instruction
• Counseling and assistance offered
• Feedback given to students
Sample Criteria
![Page 20: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/20.jpg)
Team and Individual Instructor TE Analysis
![Page 21: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/21.jpg)
Teacher Improvement
![Page 22: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/22.jpg)
Program Evaluation (PE)
• Testing and grading
• Team teaching
• Homework
• Curriculum and supporting materials
• Overall evaluations
Sample Criteria
![Page 23: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/23.jpg)
Quality of Life (QL)
• Barracks, family housing and child care
• Quality of food, mess hall and related services
• Medical and dental care
• Chaplain and community services
• Military support
• Sport and recreation opportunities
• Transportation and travel office services
Sample Criteria
![Page 24: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/24.jpg)
Quality of Life (QL)
• Equal Opportunity– Prejudice– Discrimination– Sexual harassment
• Military– Training– Duties– General quality of life
Other Areas of Concern
![Page 25: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/25.jpg)
PE and QL Analysis
![Page 26: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/26.jpg)
Serious Issues
• Depression
• Suicide threats
• Sexual assaults and harassment
• Fraternization and other misconduct
Immediate turnaround; sent directly to Command Group for action
![Page 27: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/27.jpg)
Evaluation of Continuing Education Programs
• ESQ questionnaires distributed world-wide to training sites and include:– Video tele-training sites– Mobile training team deployments– Language Teaching Detachments– Command Language Programs– Special, customized programs
• Questionnaires completed and returned to DLI via on-site program managers
• Analysis and feedback provided to instructor’s school and training site within two weeks
![Page 28: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/28.jpg)
Conclusion
• DLI evaluation practices: – Are well established—honed through years of
practice– Are rigorous and thorough– Bring about faculty and program improvement– Establish baselines for student/teacher
accountability– Give us cause to rethink what we are doing and
consider how we might do things better
![Page 29: Evaluation at the Defense Language Institute Dr. Thomas S. Parry](https://reader035.fdocuments.us/reader035/viewer/2022062806/56814f80550346895dbd32b9/html5/thumbnails/29.jpg)
Evaluation at the Defense Language Institute
Questions?
Discussion?